Artificial Intelligence

Artificial intelligence (AI) is the project of making machines do things that, when people do them, we call intelligent: recognizing a face, understanding a sentence, proving a theorem, driving a car, playing chess.

11 frameworks, from 1956 to today · Artificial Intelligence · Computer Science

still a working view ended · each leads to its entry below

Overview

Artificial intelligence (AI) is the project of making machines do things that, when people do them, we call intelligent: recognizing a face, understanding a sentence, proving a theorem, driving a car, playing chess1. What is at stake is not just automation but a question about the nature of thought itself. If a machine can reason, perceive, and learn, then intelligence may be a kind of computation rather than a special substance1. That idea has driven the field through cycles of optimism and disappointment, and it has produced techniques that now sit quietly inside search engines, translation apps, medical imaging, and the language models that write and answer questions. This is the story of how those techniques were invented, argued over, abandoned, and revived.

Symbolic AI and the Dartmouth founding

The field's conventional founding is a 1956 summer workshop at Dartmouth College, organized by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon213. McCarthy coined the term "artificial intelligence" for the proposal24. The early researchers shared a conviction: intelligence is symbol manipulation15. A computer could be given a formal language in which to represent facts and rules, and a program could search through possible inferences to reach conclusions6.

This approach, later called symbolic AI or "good old-fashioned AI," produced striking early results17. In 1956 Allen Newell, J. C. Shaw, and Herbert Simon demonstrated the Logic Theorist, a program that proved theorems from Whitehead and Russell's Principia Mathematica, and in one case found a proof more elegant than the book's81. Their General Problem Solver (1957) tried to model human problem-solving as means–ends analysis: compare the current state with the goal, find the difference, and apply an operator that reduces it91. The programs worked on toy domains but revealed a core difficulty: the space of possible moves grows explosively, and general search without domain knowledge is hopeless1.

The countermeasure was to encode knowledge. John McCarthy's Lisp (1958) gave AI its own programming language, suited to manipulating symbolic expressions. In 1959 Arthur Samuel's checkers program learned to improve its play by adjusting weights on features of board positions, an early demonstration of machine learning. The 1960s also brought perceptrons: Frank Rosenblatt's 1958 device and its software versions, which learned to classify simple patterns by adjusting connection weights. The perceptron was a physical machine, the Mark I, and it captured the imagination of the press.

The first winter and the knowledge turn

Optimism outran results. In 1969 Marvin Minsky and Seymour Papert published Perceptrons, a rigorous analysis showing that single-layer perceptrons could not compute certain simple functions, such as exclusive-or, and that scaling them up was not straightforward. The book is often blamed for freezing neural-network research, though the deeper cause was that the symbolic paradigm dominated funding and prestige. In 1973 the British government's Lighthill Report concluded that AI had failed to deliver on its grand promises, and funding in Britain collapsed103. The period is remembered as the first "AI winter."10

The response was the expert system11. Instead of general reasoning, researchers encoded the specialized knowledge of human experts as rules11. Edward Feigenbaum at Stanford led the push. MYCIN, developed in the 1970s, diagnosed bacterial infections and recommended antibiotics using several hundred if–then rules, and in evaluations it performed comparably to human physicians. XCON, deployed at Digital Equipment Corporation from 1980, configured computer orders and saved the company substantial sums. Expert systems were a commercial boom in the early 1980s, and companies built "knowledge engineering" teams to interview experts and write rules.

The boom exposed the limits. Rules were expensive to elicit, brittle outside their narrow domain, and unable to handle uncertainty gracefully7. A rule that says "if fever and rash then measles" does not say how confident to be, and real diagnosis involves probabilities. The second AI winter arrived in the late 1980s as the expert-system market collapsed.

Uncertainty, probability, and Bayesian networks

The problem of reasoning under uncertainty had a mathematical answer that the field initially resisted. In the 1980s Judea Pearl developed Bayesian networks: graphical models in which nodes represent variables and directed edges represent conditional dependencies, so that a large joint probability distribution can be factored into manageable local pieces12. A Bayesian network for a medical domain can compute the probability of a disease given observed symptoms without enumerating every combination of variables12. Pearl's 1988 book Probabilistic Reasoning in Intelligent Systems made the case that probability was the right calculus for AI, against a tradition that had treated uncertainty as a nuisance. The approach spread into diagnosis, speech recognition, and later into machine learning, where graphical models became a standard tool.

Connectionism and the backpropagation revival

Meanwhile the neural approach returned. Connectionism is the view that intelligence emerges from networks of simple units connected by adjustable weights, rather than from explicit symbols. In 1986 David Rumelhart, Geoffrey Hinton, and Ronald Williams popularized backpropagation, an algorithm for training multilayer networks: compute the error at the output, then propagate it backward through the layers to adjust each weight in proportion to its contribution to the error. The key insight was that the chain rule of calculus lets a network with hidden layers learn internal representations that a single-layer perceptron could not. A network trained on examples of handwritten digits could learn to recognize new ones. The 1989 demonstration by Yann LeCun and colleagues of a convolutional network reading zip codes showed the approach could work on real problems.

Connectionism and symbolic AI competed for decades. Symbolists argued that networks could not represent structured knowledge or perform logical inference; connectionists argued that symbols had to be grounded in perception and learning. The debate was never fully settled, and it resurfaces today as neuro-symbolic AI, which combines neural learning with symbolic reasoning, for example by using a neural network to parse a scene and a symbolic planner to reason about it131.

Reinforcement learning and the agent framework

A third strand grew from the study of how agents learn by trial and error. Reinforcement learning formalizes this: an agent takes actions in an environment, receives rewards, and learns a policy that maximizes long-run reward14. The framework rests on the Bellman equation and dynamic programming, developed by Richard Bellman in the 1950s, and on the temporal-difference methods explored by Richard Sutton in the 1980s. Sutton and Andrew Barto's 1998 textbook consolidated the field. A landmark demonstration came in 1992, when Gerald Tesauro's TD-Gammon learned to play backgammon at a strong level by playing against itself, using temporal-difference learning and a neural network to evaluate positions. Reinforcement learning later became central to game-playing AI and to the training of large language models.

Behavior-based robotics

Robotics took a different turn. Classical AI robotics built a world model, planned a path, and executed it, which failed in messy real environments. In the late 1980s Rodney Brooks argued for behavior-based robotics: build robots from simple reactive layers, each responding directly to sensor input, with no central world model. His robot Genghis, a six-legged walker built in 1989, coordinated leg movements through simple reflexive rules and could climb over obstacles. The approach, sometimes called "Nouvelle AI," claimed that intelligence could emerge from the interaction of simple behaviors with the world, and it influenced later work on autonomous vehicles and drones.

Statistical learning theory and the probabilistic turn

By the 1990s the field had shifted toward statistics1. Statistical learning theory asks how a learner can generalize from finite data, and it supplied the mathematical scaffolding: the Vapnik–Chervonenkis (VC) dimension measures the capacity of a class of functions, and the bias–variance tradeoff explains why a model that fits training data too closely fails on new data. Vladimir Vapnik and Alexey Chervonenkis developed the core theory in the 1960s and 1970s; Vapnik's support vector machines, popularized in the 1990s, found wide use1516. The probabilistic turn also brought hidden Markov models into speech recognition and Bayesian methods into spam filtering and text classification. Machine learning became a discipline in its own right, with its own conferences and journals, and the boundary between AI and statistics blurred.

Deep learning

The decisive change came when neural networks got deep and data got big. Deep learning refers to networks with many layers, trained on large datasets with backpropagation and accelerated by graphics processing units (GPUs), which turned out to be well suited to the matrix arithmetic of neural nets. In 2012 Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton won the ImageNet image-classification competition with a deep convolutional network, cutting the error rate dramatically and convincing the field that depth mattered. The ingredients were not new—convolutional networks dated to the 1980s, and backpropagation to the 1980s—but scale and computation made them work. Deep learning spread to speech recognition, translation, and game playing. In 2016 AlphaGo, built by DeepMind, defeated the Go champion Lee Sedol using deep networks combined with reinforcement learning and Monte Carlo tree search, a milestone because Go had long resisted brute-force search.

Large language models

The most recent turning point came from language. In 2017 a team at Google published the transformer architecture, which replaced recurrent networks with attention mechanisms that let a model weigh the relevance of every word in a sequence to every other. Transformers trained on vast text corpora produced large language models: systems with billions of parameters that learn statistical patterns of language and can generate, translate, summarize, and answer questions. GPT-3, released by OpenAI in 2020, showed that scaling up produced capabilities that smaller models lacked, and subsequent models became widely used tools. These systems descend from connectionism and statistical learning, and they raise open questions about reasoning, reliability, and how much of intelligence is captured by prediction from data.

Where the field stands

Today AI is a mix of traditions. Deep learning dominates perception and language; reinforcement learning drives game-playing and control; Bayesian methods handle uncertainty; symbolic methods persist in verification, planning, and knowledge representation; and neuro-symbolic hybrids try to combine them. The old debates—symbols versus networks, hand-crafted knowledge versus learned representations—have not been resolved so much as reframed by scale. The open questions are the ones the field began with: what it means for a machine to understand, how to make systems that generalize beyond their training, and whether the computational account of mind is the right one.

References

  1. Artificial intelligence (introduction, Computationalism and functionalism, History, Symbolic AI and its limits, State space search, Reasoning and problem-solving), Wikipedia
  2. Dartmouth workshop (introduction, Background), Wikipedia
  3. History of artificial intelligence, Wikipedia
  4. Dendral (History), Wikipedia
  5. Philosophy of artificial intelligence, Wikipedia
  6. Physical symbol system, Wikipedia
  7. Symbolic artificial intelligence, Wikipedia
  8. Logic Theorist, Wikipedia
  9. General Problem Solver, Wikipedia
  10. AI winter, Wikipedia
  11. Expert system, Wikipedia
  12. Bayesian network (introduction, History), Wikipedia
  13. Neuro-symbolic AI, Wikipedia
  14. Reinforcement learning, Wikipedia
  15. Vapnik–Chervonenkis theory, Wikipedia
  16. Support vector machine (History, introduction), Wikipedia

The frameworks of Artificial Intelligence

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1956-1990

    Symbolic artificial intelligence

    How can machines manipulate symbols to reason?

    About Symbolic artificial intelligence
    Symbolic AI, also called classical or logic-based AI, is the first coherent framework for building intelligent machines. It addresses a fundamental question: can intelligence be reduced to the manipulation of symbols according to formal rules? Its core claim is that thinking is a kind of computation over explicit, human-readable representations. This framework treats knowledge as a set of facts and rules, and reasoning as a search through a space of logical possibilities. Symbolic AI is distinct from later frameworks because it assumes that intelligence requires an explicit, internal model of the world, not just pattern recognition or statistical correlation.
    To understand Symbolic AI, a reader should first know that artificial intelligence aims to create systems that exhibit intelligent behavior. It also helps to have a basic sense of formal logic, such as the idea that statements can be true or false and that rules can connect them.
    The core curriculum for Symbolic AI begins with its central unit of analysis: the symbol. A symbol is a token that stands for something in the world, like 'cat' or 'temperature'. The framework's first teaching move is to understand knowledge representation, the problem of how to encode facts about the world in a machine-usable form. The canonical methods are logic (using formal languages like first-order predicate calculus), production rules (if-then statements), semantic nets (graphs of concepts and their relations), and frames (structured objects with slots for properties). The second move is search, the process of exploring possible sequences of actions or inferences to reach a goal. The third move is automated reasoning, using rules of inference to derive new conclusions from known premises. A key technique is resolution, a rule for proving theorems by refutation. The fourth move is planning, the task of finding a sequence of actions to achieve a goal, often using a means-ends analysis that compares the current state to the desired state. The fifth move is knowledge-based systems, also called expert systems, which encode domain-specific expertise in a knowledge base and use an inference engine to apply rules. The sixth move is symbolic machine learning, which includes methods like version space learning, decision tree learning (e.g., ID3), and inductive logic programming to acquire rules from examples. The seventh move is automated theorem proving, the use of a computer to prove mathematical theorems. The eighth move is constraint satisfaction, solving problems by finding values for variables that satisfy a set of constraints. The ninth move is ontologies, formal specifications of the concepts and relationships in a domain. The tenth move is explanation, the ability of a system to show the chain of reasoning that led to a conclusion.
    Symbolic AI emerged from the confluence of several intellectual currents in the mid
    The 13 concepts its tower teaches
    • Automated Reasoning
    • Constraint Satisfaction
    • Frames
    • Inference Engine
    • Knowledge-Based Systems
    • Logic
    • Means-Ends Analysis
    • Planning
    • Production Rules
    • Resolution
    • Semantic Nets
    • Symbol
    • Symbolic Machine Learning

    Fly to its tower in Nousfarer

  2. 1970-1990

    Expert system

    How can computers encode expert knowledge as rules?

    Fly to its tower in Nousfarer

  3. 1975-Present

    Evolutionary Computation

    How can algorithms evolve solutions through selection?

    Fly to its tower in Nousfarer

  4. 1985-Present

    Bayesian network

    How can machines reason under uncertainty?

    Fly to its tower in Nousfarer

  5. 1985-Present

    Reinforcement learning

    How can agents learn from reward and punishment?

    Fly to its tower in Nousfarer

  6. 1986-2000

    Behavior-based robotics

    How can simple reactive behaviors produce intelligent action?

    Fly to its tower in Nousfarer

  7. 1986-Present

    Connectionism

    How can networks of simple units learn representations?

    About Connectionism
    Connectionism offers a radical answer to the question of how machines can exhibit intelligent behavior: intelligence emerges from the collective activity of many simple, interconnected processing units, not from the manipulation of explicit symbols or rules. In artificial intelligence, this framework models cognitive processes as patterns of activation spreading across a network of neuron-like nodes, with learning occurring through the adjustment of connection strengths between them. This stands in sharp contrast to Symbolic AI, which treats intelligence as rule-governed symbol manipulation, and to Behavior-Based AI, which rejects internal representations altogether. Connectionism instead proposes that the architecture itself—the pattern of connections and their weights—is the locus of knowledge and the engine of learning.
    A first course in connectionism for artificial intelligence builds around a core set of concepts that define how these networks are structured, how they learn, and what they can represent. The curriculum begins with the artificial neuron as the fundamental processing unit, a simplified model of a biological neuron that computes a weighted sum of its inputs and applies an activation function to produce an output. These neurons are organized into network architecture, typically arranged in layers: an input layer, one or more hidden layers, and an output layer. The activation function—often a sigmoid or step function—determines whether a neuron fires, introducing the nonlinearity essential for learning complex patterns. Learning itself is driven by learning rules, procedures that adjust connection weights to reduce error. The most influential of these is backpropagation of error, which computes the gradient of the error function with respect to each weight and propagates corrections backward through the network. This process relies on gradient descent, an optimization method that iteratively moves weights in the direction that minimizes error. The network's ability to generalize from training data to unseen examples is captured by the concept of generalization, while the risk of memorizing noise rather than signal is known as overfitting. A network's representational power is described by its representational capacity, which grows with the number of hidden units and layers. The overall approach is often called parallel distributed processing, emphasizing that knowledge is not stored in a single location but is distributed across many connections. Two key phenomena illustrate the framework's strengths: graceful degradation, where damage to parts of the network leads to gradual rather than catastrophic loss of function, and pattern completion, where a partial input can reconstruct a stored pattern. Finally, the perceptron convergence theorem provides a foundational result: a single-layer perceptron can learn any linearly separable function, but its limitations—famously, its inability to learn XOR—motivated the shift to multilayer networks.
    Connectionism's intellectual history in artificial intelligence is marked by distinct waves, with the second wave (roughly 1986 to 1995) serving as the local dominant period. The first wave began in 1943 when Warren McCulloch and Walter Pitts showed that networks of simple threshold units could compute logical functions. Frank Rosenblatt's perceptron (1958) added a learning rule, sparking early excitement. But the 1969 book Perceptrons by Marvin Minsky and Seymour Papert rigorously analyzed the perceptron's limitations, particularly its inability to handle nonlinearly separable problems like XOR. This critique, combined with the dominance of Symbolic AI, led to a sharp decline in connectionist research funding and activity—the first 'AI winter' for the approach. The term 'connectionist model' was revived in a 1982 paper by Jerome Feldman and Dana Ballard, but the true renaissance came with the publication of Parallel Distributed Processing (1986) by David E. Rumelhart, James L. McClelland, and the PDP Research Group. This two-volume work introduced backpropagation of error as a practical learning algorithm for multilayer networks, demonstrated its power on a range of cognitive tasks, and articulated the philosophical implications of distributed representation. John Hopfield's 1982 work on Hopfield networks provided crucial mathematical foundations for understanding network dynamics and memory. During this second wave, connectionism was seen by many as a revolutionary alternative to Symbolic AI, sparking intense debates about whether it represented a fundamental shift in psychology and artificial intelligence. Key figures like Rumelhart, McClelland, and Geoffrey Hinton (who co-authored the influential 1986 backpropagation paper) drove this period. The framework coexisted with and was later supplemented by Probabilistic AI (from around 1988), which introduced Bayesian networks and probabilistic graphical models as alternative ways to handle uncertainty. Deep learning, beginning around 2006, can be seen as a third wave that scaled up connectionist ideas with deeper architectures, larger datasets, and more powerful hardware, leading to breakthroughs in image recognition, natural language processing, and game playing. Connectionism's legacy in AI is enduring: it established the core idea that learning from data through distributed representations is a viable path to intelligence, a principle that underpins much of modern machine learning.
    The 14 concepts its tower teaches
    • Activation Function
    • Artificial Neuron
    • Backpropagation of Error
    • Generalization
    • Graceful Degradation
    • Gradient Descent
    • Hidden Layers
    • Learning Rules
    • Network Architecture
    • Overfitting
    • Parallel Distributed Processing
    • Pattern Completion
    • Perceptron Convergence Theorem
    • Representational Capacity

    Fly to its tower in Nousfarer

  8. 1990-Present

    Statistical learning theory

    How can a learner generalize from finite data?

    Fly to its tower in Nousfarer

  9. 2005-Present

    Neuro-symbolic AI

    How can neural learning and symbolic reasoning be combined?

    Fly to its tower in Nousfarer

  10. 2012-Present

    Deep learning

    How can deep networks learn hierarchical representations?

    Fly to its tower in Nousfarer

  11. 2017-Present

    Large language model

    How can models learn language from vast text?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Symbolic artificial intelligenceExpert systemEvolutionary ComputationBayesian networkReinforcement learningBehavior-based roboticsConnectionismStatistical learning theoryNeuro-symbolic AIDeep learningLarge language model

Beyond the field

Where Artificial Intelligence's ideas came from in other fields, and what they led to there: the strongest 12 of 95 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Artificial Intelligence is a planet in Nousfarer

Its city has a tower for each of these 11 frameworks. Land, walk in, and climb one concept at a time.

Land on Artificial Intelligence

Related fields

Fields its ideas are linked to

More in Artificial Intelligence

See where Artificial Intelligence sits on the Map of Human Knowledge