Game Playing Ai
Game playing AI is the branch of artificial intelligence concerned with building programs that play games well—chess, checkers, Go, poker, backgammon, and many others.
8 frameworks, from 1950 to today · Game Studies & Design · Games
- Classical Game Tree Search 1950-Present
- Self-play 1959-Present
- Knowledge-Based Chess AI 1965-1997
- Brute-Force Search and Special-Purpose Hardware 1975-2005
- General Game Playing 2005-Present
- Monte Carlo Tree Search 2006-Present
- Counterfactual Regret Minimization 2007-Present
- Deep reinforcement learning 2013-Present
Overview
Game playing AI is the branch of artificial intelligence concerned with building programs that play games well—chess, checkers, Go, poker, backgammon, and many others. It matters for two reasons. First, games are unusually clean testbeds: the rules are fixed, the outcomes are measurable, and a win or loss is hard to argue with. Second, the methods developed to win games have repeatedly turned out to be general-purpose tools for reasoning under uncertainty, search, and learning. Much of what became modern AI planning and decision-making was first worked out on a game board.
The Analytical Engine and the First Question
The story begins before computers existed. In the 1770s, Wolfgang von Kempelen built a chess-playing automaton called the Mechanical Turk, which toured Europe and beat notable opponents. It was an illusion: a human operator hid inside. The episode matters because it framed the question—could a machine play chess?—decades before anyone could build one.
In 1843, Ada Lovelace, writing about Charles Babbage's Analytical Engine, argued that a machine could only do what it was instructed to do and could not originate anything. Chess became the standard counterexample in the debate over machine intelligence. The idea that a machine might play well without being explicitly told every move would not be taken seriously until the mid-twentieth century.
Chess as the Founding Problem, 1940s–1960s
The modern field starts with Claude Shannon's 1950 paper "Programming a Computer for Playing Chess" and Alan Turing's contemporaneous work on a chess program (written on paper, since no machine could run it). Shannon posed the central problem: chess has roughly 10^120 possible games, far too many to examine exhaustively1. His answer was a search tree—a branching structure of possible moves and replies—pruned by an evaluation function, a numerical score estimating how good a position is for one side. The program would look a few moves ahead, score the resulting positions, and choose the move leading to the best score.
This is classical game tree search, and it set the template. The key refinement came in 1956 with the minimax algorithm formalized by John McCarthy and others: each player is assumed to play optimally, so a move is valued by the worst outcome the opponent can force. In 1958, Allen Newell, Cliff Shaw, and Herbert Simon demonstrated a chess program at Carnegie Mellon that could play a recognizable game. Around the same time, Arthur Samuel's checkers program (1959) introduced a crucial idea: the program could improve its evaluation function by playing against itself, a precursor of self-play learning23.
The 1960s brought alpha-beta pruning, which skips branches of the search tree that cannot affect the result, making deeper search feasible. The theoretical basis was worked out by several researchers, and the technique became standard. By the late 1960s, chess programs could beat casual players but not strong ones.
The Deep Blue Era, 1970s–1990s
Through the 1970s and 1980s, progress came from two directions: faster hardware and better search. The dominant approach remained brute-force search with hand-tuned evaluation functions, and the field split into competing schools. Some researchers, notably at MIT and Carnegie Mellon, believed raw speed and deep search would win; others argued that chess required human-like knowledge and pattern recognition. The debate was real and shaped funding and research directions.
The decisive event was Deep Blue, IBM's chess machine, which defeated world champion Garry Kasparov in a six-game match in 1997 (after losing a match in 1996)24. Deep Blue combined specialized hardware with alpha-beta search and an evaluation function tuned by grandmasters. It searched roughly 200 million positions per second. The win was a landmark, but it also marked the end of an era: brute-force search had conquered chess, and the approach did not obviously generalize to harder games.
The Wall: Go and the Limits of Search
Go, the ancient Chinese board game, became the next target. It resisted search-based methods for decades. The branching factor—the number of legal moves at each turn—is around 250, compared with about 35 in chess, and positions are hard to evaluate because influence is diffuse and long-range. Hand-crafted evaluation functions performed poorly. Through the 1990s and 2000s, the best Go programs were weak amateurs.
The breakthrough came from a different direction: Monte Carlo tree search (MCTS)5. Instead of evaluating positions with a formula, MCTS plays out many random games from a position (called rollouts) and uses the win rate to estimate value. The method was developed in the mid-2000s, with influential work by Rémi Coulom (2006) and others, and it transformed computer Go. By 2010, MCTS programs could play at strong amateur level. The idea was powerful because it replaced hand-crafted knowledge with statistical sampling—a shift from "what do experts know?" to "what does the data say?"
General Game Playing and the Rules of the Game
A parallel thread asked a different question: could a program play any game, given only its rules? General game playing emerged in the early 2000s, with the Stanford project led by Michael Genesereth and a annual competition (the General Game Playing Competition, starting in 2005). Programs received a formal description of a game's rules in a language called GDL (Game Description Language) and had to play without game-specific tuning. The challenge forced researchers to build general search and learning methods rather than hand-crafted heuristics. It remains an active benchmark, though general players still lag behind specialized ones.
Learning to Play: Reinforcement Learning and Self-Play
The 2010s brought a convergence. Deep reinforcement learning—combining deep neural networks with reinforcement learning, where an agent learns by trial and error from rewards—began to be applied to games6. The key insight was that a neural network could learn an evaluation function and a policy (a strategy for choosing moves) directly from experience, without hand-coding.
The landmark was AlphaGo, developed by DeepMind, which defeated Lee Sedol in 201678. AlphaGo combined MCTS with deep neural networks trained on human games and then refined by self-play—playing against itself to improve7. In 2017, AlphaGo Zero removed human data entirely, learning from scratch through self-play and surpassing the earlier version. The same year, AlphaZero generalized the approach to chess and shogi, beating the best existing programs79. The method was not new in principle—self-play and search had been around for decades—but the combination with deep learning made it work at scale.
Counterfactual regret minimization (CFR), developed by Martin Zinkevich and colleagues in 2007, addressed a different class of games: those with imperfect information, where players cannot see each other's cards or private states. Poker is the canonical example. CFR is an iterative algorithm that computes strategies minimizing regret—the difference between the outcome achieved and the best possible outcome in hindsight. It converges to a Nash equilibrium, a strategy profile where no player can improve by changing strategy alone10. CFR and its variants (such as Monte Carlo CFR) powered programs like Libratus (2017) and Pluribus (2019), which beat top human poker players. The approach is fundamentally different from search-based methods because it reasons about hidden information and mixed strategies.
The Present: Hybrid Methods and Open Questions
Today, game playing AI is less a single field than a toolkit. MCTS remains central for games with large search spaces. Deep reinforcement learning and self-play are standard for training agents. CFR and its descendants dominate imperfect-information games. Classical search still matters where speed and precision are needed, and general game playing continues as a test of generality.
The open questions are substantial. How do you build agents that transfer skills across games rather than mastering one? How do you handle games with continuous action spaces or real-time constraints? How do you make learned strategies interpretable and safe? And the oldest question remains: what does game-playing skill tell us about intelligence itself? The field has answered Shannon's challenge many times over, but each answer has opened new questions about what it means for a machine to play—and to think1.
References
- Shannon number, Wikipedia
- Artificial intelligence in video games (History), Wikipedia
- Arthur Samuel (computer scientist) (Computer checkers (draughts) development), Wikipedia
- Deep Blue (chess computer), Wikipedia
- Monte Carlo tree search, Wikipedia
- Deep reinforcement learning, Wikipedia
- AlphaGo, Wikipedia
- Computer Go, Wikipedia
- AlphaZero, Wikipedia
- Nash equilibrium, Wikipedia
The frameworks of Game Playing Ai
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1950-Present
Classical Game Tree Search
How can a program choose good moves by searching ahead?
- 1959-Present
- 1965-1997
Knowledge-Based Chess AI
How much expert knowledge does a chess program need?
- 1975-2005
Brute-Force Search and Special-Purpose Hardware
Can raw speed and deep search beat human expertise?
- 2005-Present
General Game Playing
Can one program play any game given only its rules?
- 2006-Present
Monte Carlo Tree Search
Can random playouts estimate the value of a position?
- 2007-Present
Counterfactual Regret Minimization
How do you find optimal strategies when information is hidden?
- 2013-Present
Deep reinforcement learning
Can neural networks learn to play games from experience alone?
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Brute-Force Search and Special-Purpose Hardware grew out of Classical Game Tree Search
Brute-force search with special-purpose hardware was a direct scaling of Shannon's search-tree approach, using alpha-beta pruning and faster machines to search deeper.
Carried by Claude Shannon.
Deep reinforcement learning influenced by Self-play
AlphaGo and AlphaGo Zero combined deep neural networks with self-play, reviving Samuel's idea of learning by playing against oneself and scaling it with deep reinforcement learning.
Carried by DeepMind; AlphaGo (2016).
Deep reinforcement learning influenced by Monte Carlo Tree Search
AlphaGo combined Monte Carlo tree search with deep neural networks for position evaluation and move selection, making MCTS a core component of deep reinforcement learning game agents.
Carried by DeepMind; AlphaGo (2016).
Self-play influenced by Classical Game Tree Search
Arthur Samuel's checkers program used minimax-style lookahead and improved its evaluation function by playing against itself, establishing self-play as a learning method within the classical search framework.
Carried by Arthur Samuel.
Knowledge-Based Chess AI grew out of Classical Game Tree Search
Knowledge-based chess AI grew out of classical game tree search by adding hand-crafted chess knowledge to guide and evaluate the search, as in the programs of the 1960s and 1970s.
General Game Playing influenced by Classical Game Tree Search
General game playing took the classical search-tree framework and generalized it, requiring programs to search and evaluate from formal rules alone rather than game-specific heuristics.
Carried by Michael Genesereth.
Monte Carlo Tree Search grew out of Classical Game Tree Search
Monte Carlo tree search replaced hand-crafted evaluation functions with statistical rollouts, defining itself against the knowledge-intensive search that had dominated game AI.
Carried by Rémi Coulom.
Brute-Force Search and Special-Purpose Hardware rival of Knowledge-Based Chess AI
Through the 1970s and 1980s the knowledge-based school and the brute-force school competed over how to build strong chess programs, a debate that shaped funding and research directions.
Deep reinforcement learning influenced by Counterfactual Regret Minimization
Counterfactual regret minimization provided equilibrium-finding methods for imperfect-information games that were later combined with deep learning approaches in poker and other domains.
Beyond the field
Where Game Playing Ai's ideas came from in other fields, and what they led to there: the strongest 12 of 31 links.
Classical Game Tree Search influenced by Minimax theorem in Game Theory
The minimax theorem proved by von Neumann in 1928 supplied the value-backup rule that classical game tree search implements. Early chess program designers cited it as the theoretical basis for choosing moves in two-player zero-sum games.
Carried by John von Neumann.
Classical Game Tree Search influenced by Zero-Sum Games and Minimax Theorem in Game Theory
Von Neumann's minimax theorem for zero-sum two-person games gave chess and checkers programmers the formal justification for backing up values by alternating max and min. Shannon's 1950 paper on chess-playing programs and the early minimax search programs built directly on this result.
Carried by John von Neumann; Claude Shannon.
Self-play influenced by Search-and-evaluation engine in Computer Chess
Self-play requires an engine to play both sides, and the search-and-evaluation engine supplied that opponent. Samuel's checkers work pitted his evaluation-learning program against versions of itself using the same search.
Carried by Arthur Samuel.
Self-play influenced by Adversarial game-tree search in Planning And Search
Self-play in checkers and chess was run by playing the same adversarial game-tree search against itself, so the search procedure is the mechanism that generates the training games. Samuel's 1959 checkers program used self-play on top of minimax search.
Carried by Arthur Samuel.
Self-play influenced Neural self-play in Chess
AlphaGo Zero and AlphaZero built directly on the classical self-play method used in TD-Gammon and earlier game-playing programs, replacing handcrafted evaluation with neural networks trained by self-play. The idea of improving a player by playing against itself passed from this tradition into neural self-play.
Carried by AlphaGo Zero (2017); AlphaZero (2017).
Self-play influenced AlphaZero in Computer Chess
AlphaZero's training loop is self-play: the system generates its own games and learns from them. The general self-play idea, used in checkers and other game programs from the late 1950s, supplied the training scheme that AlphaZero scaled with deep networks and MCTS.
Carried by Arthur Samuel.
Self-play influenced AlphaGo in Life And Death
AlphaGo trained its policy and value networks by self-play, a technique established in Samuel's 1959 checkers program and in TD-Gammon. The 2016 Nature paper describes reinforcement learning from games the system played against itself.
Carried by Arthur Samuel; Gerald Tesauro.
Self-play influenced Self-play reinforcement learning in Computer Go
AlphaZero's self-play reinforcement learning built directly on the long tradition of self-play training in game-playing AI, training a network solely by playing against itself from random initial play.
Knowledge-Based Chess AI influenced by Adversarial game-tree search in Planning And Search
Knowledge-based chess programs kept adversarial game-tree search as their skeleton and added hand-crafted evaluation knowledge on top of it. The alpha-beta search of the 1960s remained the engine into which chess heuristics were inserted.
Knowledge-Based Chess AI led to Engine-assisted play and analysis in Abstract Strategy Games
Engine-assisted play grew out of knowledge-based chess AI, culminating when Deep Blue's 1997 match win over Kasparov shifted the field toward brute-force and later statistical engine analysis.
Carried by Deep Blue; IBM.
Brute-Force Search and Special-Purpose Hardware grew out of Search-and-evaluation engine in Computer Chess
Brute-force search and special-purpose hardware grew out of the search-and-evaluation engine by trading evaluation knowledge for raw speed and depth. The same move generation and value backup were retained while the evaluation was simplified.
Brute-Force Search and Special-Purpose Hardware grew out of Adversarial game-tree search in Planning And Search
Brute-force chess hardware was a direct scaling-up of adversarial game-tree search: the same alpha-beta minimax algorithm, pushed to greater depth by special-purpose chips. The later approach is the earlier search method carried to its hardware limit.

Game Playing Ai is a planet in Nousfarer
Its city has a tower for each of these 8 frameworks. Land, walk in, and climb one concept at a time.
Land on Game Playing AiRelated fields
Fields its ideas are linked to
- Planning And Search 7 links
- Computer Chess 4 links
- Computer Go 3 links
- Reinforcement Learning 2 links
- Chess 2 links
- Life And Death 2 links
- Draughts 2 links
- Game Theory 1 link
- Poker 1 link
- Fuseki Opening 1 link
- Go 1 link
- Game Theory 1 link
More in Game Studies & Design
See where Game Playing Ai sits on the Map of Human Knowledge