Is Generative AI Logical?

 

 

 

The French have a marvelous phrase, “Remise en question”, to call into question. At the beginning of the twentieth century, that phrase was indeed very much the fact. That fact produced great science but, of course, horrible politics. By assuming that electrons were not simple planetary spheres whirling around an atom core, but clouds of probability, science produced quantum physics, the computer revolution, and the generative AI of the 21^st century. But what is a likely development of AI? The following discusses the philosophical foundations of generative AI and therefore its advantages and practical limitations.

    

 

Philosophical Foundations of Generative AI 

 

Ludwig Wittgenstein founded the science of linguistics. The Wittgensteins were part of an Austrian Habsburg civilization that produced such notables as Freud, Klimt, and Bruckner. According to Janik and Toulmin, “Ludwig Wittegenstein’s (grandly named) Tractatus Logico-Philosophicus…intimately and consciously related to…the critique of language and society…” ¹ Then, as now, there was a lot of criticism about everything.

 

Wittgenstein set out to do for language what the German scientist, Heinrich Hertz, did in The Principles of Mechanics. He turned mechanics into a perfectly mathematical system involving the universal primitives of, “time, space, and mass as objects of experience.” ²  Wittgenstein’s effort was not successful; for in Philosophical Investigations he concluded that people flexibly use languages in different social ways that depend upon context, a word that is very often used in the 21^st century and is fundamental to the transformer architecture of generative AI.

 

But as Bain and Festa of Oregon State note, “There is seemingly a gap between (AI’s) ability to model language and the desired behavior we would expect of such a model.” ³ What generative AI lacks is syntax, or a set of rules that determine how languages, and therefore words, are put together. Putting a thought into linguistic form, a sentence in any language must contain a subject and a verb, such as a implies b. The consequences of generative AI’s bottom-up vector driven approach are both a fundamental lack of top-down logic and a lack of innate morality.  As the question was broached in high school, what distinguishes a human from other animals? Here, it is language, syntax, logic and therefore communication.

 

Noam Chomsky is a famous linguist and professor emeritus of MIT. In a 3/8/23 NYT article titled, “The False Promise of ChatGPT”, he writes as one who believes in human nature and top-down universals. This is an important article, also for its consideration of scientific explanation. Professor Chomsky does not like generative AI.

 

1)     “The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating the…most probable answer to a scientific question.” A model doesn’t simply get “smarter” because it has more data. To cite an example of statistical pattern matching, “Whereas humans are limited in the kinds of explanation we can rationally conjecture, machine learning systems can learn both that the earth is flat and that the earth is round. They trade merely in probabilities that change over time.”

 

2)    “On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information; it seeks not to infer brute correlations among (vector) data points but to create explanations (What is going on here?)….When linguists seek to develop a theory for why a given language works as it does, they are building consciously and laboriously an explicit version of the grammar that the child builds instinctively and with minimal exposure to information. The child’s operating system is completely different from that of a machine learning program. Indeed, such programs are stuck in a prehuman or non human phase of cognitive evolution.” They don’t, like living animals (like Fido), know what goals to seek.

 

3)    The author offers this example about true science, which is about logical explanation. The science, merely of correlation, is a “pseudoscience” because, as we learned, “correlation is not causation.”

 

a)    I hold an apple in my hand.

 

b)    The apple will fall if I open my hand (that is correlation and prediction; that is as far as generative AI can go).

 

c)    The apple will fall if I open my hand because (of the force of gravity or the curvature of space-time). These are explanations (about the universe or ordinary life, and very elegant ones at that). Furthermore, each separate theory has mountains of empirical backup data. The universe may be complex; but the reason, as Hertz notes, why scientists continually seek fundamentals is to simplify the theory.

 

4)    Chatbots lack morality. This is why so many people have been grappling with the question of “controlling” chatbots. “Note, for all the seemingly sophisticated thought and language, the moral indifference born of unintelligence. Here, ChatGPT exhibits something like the banality of evil:  plagiarism and apathy and obviation. It summarizes the standard arguments in the literature by a kind of super-autocomplete, refuses to take a stand on anything (that really matters), pleads not merely ignorance but lack of intelligence and ultimately offers a ‘just following orders’ defense, shifting responsibility to its creators (where it belongs anyway).”

 

Ultimately, generative AI lacks logic. As we noted, it lacks syntax and therefore the constraints of logic. It also lacks the logic of simple arithmetic (+,-,x,¸). If this is true, then there are real problems in using chatbots in big decisions.

 

Apple Computer has published a paper that shows chatbots lack the ability to do simple arithmetic. Samy Bengio is the senior director of AI at Apple. Bengio et al note this in, “GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models”:

 

·      The GSM8K benchmark is widely used to assess the mathematical reasoning of models on grade-school-level questions….we introduce GSM-Symbolic, an improved benchmark created from symbolic templates that allow for the  generation of a diverse set of questions….Our findings reveal that LLMs exhibit notable variance when responding to different instantiations of the same question….we investigate the fragility of mathematical reasoning in these models and demonstrate that their performance significantly deteriorates as the number of clauses in a question increases. We hypothesize that this decline is due to the fact current LLMs are not capable of genuine logical reasoning; instead, they attempt to replicate the reasoning steps observed in their training data. When we add a single clause that appears relevant to the question, we observe significant performance drops (up to 65%) across all state-of-the-art models.

 

But the current demonstrated ability of the Perplexity chatbot to do arithmetic and calculus is probably due to a logical add-on, rather than to a fundamental change in the chatbot’s bottom-up and agnostic nature. The Perplexity “Pro-Search” feature uses top-down reasoning strategies to a point, when bottom-up AI then takes over.

 

Communicated scientific research is more clearly constrained by a statistical top-down concept: the null hypothesis, H₀.

 

Advantages and Practical Limitations   

 

“Correlation is not causality.” In generative AI, correlation is probably enough to produce usable products (people also having habits). But generative AI, lacking logic, cannot be more superior than humans to pull out causality from a complex world, to also determine, using Mervyn King’s words, “What is going on here?”

 

The editor of Scientific American wrote, “…(what) I enjoy most…is learning how scientists do their work - the inspirations the questions, the insights, the collaborations, the, ‘hmm, that’s strange’ moments.” ⁴

 

Consumer Purposes

 

Our Use:

 

But the chatbot is a very useful focused information source. It is also useful to research extensions of the theories we learned in school. In both senses, derived from the data on the internet (and not other chatbots), it is like the Great Library of Alexandria in the ancient world of 2^nd to 3^rd century B.C., a universal library housing between 40,000-400,000 scrolls. If you can imagine yourself as a researcher entering the library. What you will find there is what you can make of it, depending on what questions you ask. The library, itself, doesn’t act as an oracle.

 

Consider the following exchange we had with the Perplexity chatbot on 1/4/25:

 

Us: Can you give me a language where to complete a thought, there is no verb? An one to one correspondence between an object and its name doesn’t count.

 

Perplexity: After citing sources: While there are no known natural languages that completely lack verbs, there is an artificial language called Kelen that was designed to function without traditional verbs…However its important to note that the classification of Kelen as truly verbless is debated among linguists…it’s worth noting that this structure is not found in natural human languages. All known natural languages use verbs in some form, suggesting that verbs play a crucial role in human communication and cognition.

 

We can imagine trying to research this question, unaided.

 

General Use:

 

·      Correlation is not the structure of causality. For social science purposes, a model correlation of less than .66 is not useful for theory. But people also having habits, pure correlation might be useful for agentic AI; because agents might be helpful for how people act most of the time, on the computer. It would be interesting to also ask how computer agents might be connected to life, where both the (hopefully) real facts you select and ethics matter.

·      Chatbots are currently used in academia to produce, from original thoughts and research of course, rough first drafts of an essay. They are also useful sources for new theories, materials and interesting relationships.

·      Census Bureau research on 11/28/23, regarding the economic adoption of generative AI indicates that only 3.8% of all businesses currently use the technology; but in information and technology, it is used by 13.8% of all businesses.⁵ On the other hand, a private company online commercial survey found that 39 percent of U.S. population age 18-64 used generative AI in 8/24, an adoption even faster than the adoption of the internet.⁶ These findings can all be true, but for different populations. Here, we consider the population of large (S&P 500) companies and of smaller companies.

 

Commercial Use in Large Companies

 

For large companies: Return on Investment >> Cost of Capital, the data must be in good shape (FAIR), and there must be a causal logic that justifies millions of dollars of expenditures.

 

A Goldman Sachs 8/5/24 study indicates that AI has, “…some cool applications…But there isn’t this killer application that’s consuming a lot of capacity right away….there’s a race to see who can build the best foundational model (general models that can be applied to many applications). That race isn’t going to slow down anytime soon. From an ROI perspective, if you look at it over the next one or two years, maybe the ROI isn’t great. But if you have a return stream of 20 years (Wall Street?) associated with your building the best tech stack today (costing hundreds of billions of dollars), then certainly you could justify the investment.”

 

We will comment on this later.

 

Smaller Companies

 

Text-based AI packages can provide management with a more detailed look, that is more easily implemented in a smaller company. For instance, agentic AI can help determine what a marketing force is doing in a sales territory.

 

Speed of Adoption

 

In both cases, AI adoption is likely to take time. In the larger companies, the overall company data must be in shape and implementation takes time. In the smaller companies, management may not see AI as beneficial.

 

Cost of Resources

 

According to the 12/20/24 WSJ, OpenAI’s latest model, GPT-5 requires a six month training run costing about 500 million dollars in computing costs alone. By increasing the data (which the models are running out of) or the parameters (neurons) in the model, it is hoped, “…it will make fewer mistakes…(and is supposed to get) smarter….Across the industry, a debate is raging over whether improvements in AIs is starting to plateau. Sutskever…declared at a recent AI conference that the age of maximum data is over…. ‘You can even go as far as to say that data is the fossil fuel of AI.’ And that fuel was starting to run out.”

 

Perhaps generative AI should be used to augment, rather than replace, human capabilities. It makes no sense to pour hundreds of billions of dollars into AI chips, data centers and electric grids in the hope of a “smarter” AI.

 

Climate Change

 

Coping with a changing world requires clear logic, a implies b, that is, C0₂  growth implies average temperature growth. To consider temperature growth “politically determined” is to ignore objective reality and is to make a very large mistake. We have mentioned a major philosophical problem with generative AI, a lack of syntax, that is likely to affect the scope of its applicability. From the standpoint of society, the excessive cash flow spent on AI would be better spent on clearing the brush in California, building sea walls in Miami and Manhattan.

 

Large fires in California, massive tornados in the Midwest, and floods in Miami. That is the future, if the climate becomes more extreme.

      

 

 

Footnotes

 

 

1. Allan Janik and Stephen Toulmin; “Wittgenstein’s Vienna”; Simon and Shuster; New York, NY; 1973; p. 93.

 

2. Heinrich Hertz; “The Principles of Mechanics”; London: Macmillan; 1899; p. 26, Introduction.

 

Professor Hertz, in the Introduction, also first states the essentials of “models” and “independent variables”, concepts that are commonplace today. He was noted for enabling radio transmissions by Maxwell’s electromagnetic waves. U.S. alternating current also oscillates at a rate of 60 (Hz) per second on the grid.

 

3. Robert Bain et al.; “Wittgenstein’s Philosophy of Language The Philosophical Origins of Modern NLP Thinking”; oregonstate.edu; p. 7.

 

4. Scientific American; editorial; January, 2025; p.4.

 

5.. Cory Breaux and Emin Dinlersoz; “How Many U.S. Businesses Use Artificial Intelligence?”; U.S. Government Census Bureau website; 11/28/23.

 

6. Alexander Bick et al.; “The Rapid Adoption of Generative AI”; Federal Reserve Bank of St. Louis; 9/20/24.