Game Playing Ai

Game playing AI is the branch of artificial intelligence concerned with building programs that play games well—chess, checkers, Go, poker, backgammon, and many others.

8 frameworks, from 1950 to today · Game Studies & Design · Games

still a working view ended · each leads to its entry below

Overview

Game playing AI is the branch of artificial intelligence concerned with building programs that play games well—chess, checkers, Go, poker, backgammon, and many others. It matters for two reasons. First, games are unusually clean testbeds: the rules are fixed, the outcomes are measurable, and a win or loss is hard to argue with. Second, the methods developed to win games have repeatedly turned out to be general-purpose tools for reasoning under uncertainty, search, and learning. Much of what became modern AI planning and decision-making was first worked out on a game board.

The Analytical Engine and the First Question

The story begins before computers existed. In the 1770s, Wolfgang von Kempelen built a chess-playing automaton called the Mechanical Turk, which toured Europe and beat notable opponents. It was an illusion: a human operator hid inside. The episode matters because it framed the question—could a machine play chess?—decades before anyone could build one.

In 1843, Ada Lovelace, writing about Charles Babbage's Analytical Engine, argued that a machine could only do what it was instructed to do and could not originate anything. Chess became the standard counterexample in the debate over machine intelligence. The idea that a machine might play well without being explicitly told every move would not be taken seriously until the mid-twentieth century.

Chess as the Founding Problem, 1940s–1960s

The modern field starts with Claude Shannon's 1950 paper "Programming a Computer for Playing Chess" and Alan Turing's contemporaneous work on a chess program (written on paper, since no machine could run it). Shannon posed the central problem: chess has roughly 10^120 possible games, far too many to examine exhaustively1. His answer was a search tree—a branching structure of possible moves and replies—pruned by an evaluation function, a numerical score estimating how good a position is for one side. The program would look a few moves ahead, score the resulting positions, and choose the move leading to the best score.

This is classical game tree search, and it set the template. The key refinement came in 1956 with the minimax algorithm formalized by John McCarthy and others: each player is assumed to play optimally, so a move is valued by the worst outcome the opponent can force. In 1958, Allen Newell, Cliff Shaw, and Herbert Simon demonstrated a chess program at Carnegie Mellon that could play a recognizable game. Around the same time, Arthur Samuel's checkers program (1959) introduced a crucial idea: the program could improve its evaluation function by playing against itself, a precursor of self-play learning23.

The 1960s brought alpha-beta pruning, which skips branches of the search tree that cannot affect the result, making deeper search feasible. The theoretical basis was worked out by several researchers, and the technique became standard. By the late 1960s, chess programs could beat casual players but not strong ones.

The Deep Blue Era, 1970s–1990s

Through the 1970s and 1980s, progress came from two directions: faster hardware and better search. The dominant approach remained brute-force search with hand-tuned evaluation functions, and the field split into competing schools. Some researchers, notably at MIT and Carnegie Mellon, believed raw speed and deep search would win; others argued that chess required human-like knowledge and pattern recognition. The debate was real and shaped funding and research directions.

The decisive event was Deep Blue, IBM's chess machine, which defeated world champion Garry Kasparov in a six-game match in 1997 (after losing a match in 1996)24. Deep Blue combined specialized hardware with alpha-beta search and an evaluation function tuned by grandmasters. It searched roughly 200 million positions per second. The win was a landmark, but it also marked the end of an era: brute-force search had conquered chess, and the approach did not obviously generalize to harder games.

The Wall: Go and the Limits of Search

Go, the ancient Chinese board game, became the next target. It resisted search-based methods for decades. The branching factor—the number of legal moves at each turn—is around 250, compared with about 35 in chess, and positions are hard to evaluate because influence is diffuse and long-range. Hand-crafted evaluation functions performed poorly. Through the 1990s and 2000s, the best Go programs were weak amateurs.

The breakthrough came from a different direction: Monte Carlo tree search (MCTS)5. Instead of evaluating positions with a formula, MCTS plays out many random games from a position (called rollouts) and uses the win rate to estimate value. The method was developed in the mid-2000s, with influential work by Rémi Coulom (2006) and others, and it transformed computer Go. By 2010, MCTS programs could play at strong amateur level. The idea was powerful because it replaced hand-crafted knowledge with statistical sampling—a shift from "what do experts know?" to "what does the data say?"

General Game Playing and the Rules of the Game

A parallel thread asked a different question: could a program play any game, given only its rules? General game playing emerged in the early 2000s, with the Stanford project led by Michael Genesereth and a annual competition (the General Game Playing Competition, starting in 2005). Programs received a formal description of a game's rules in a language called GDL (Game Description Language) and had to play without game-specific tuning. The challenge forced researchers to build general search and learning methods rather than hand-crafted heuristics. It remains an active benchmark, though general players still lag behind specialized ones.

Learning to Play: Reinforcement Learning and Self-Play

The 2010s brought a convergence. Deep reinforcement learning—combining deep neural networks with reinforcement learning, where an agent learns by trial and error from rewards—began to be applied to games6. The key insight was that a neural network could learn an evaluation function and a policy (a strategy for choosing moves) directly from experience, without hand-coding.

The landmark was AlphaGo, developed by DeepMind, which defeated Lee Sedol in 201678. AlphaGo combined MCTS with deep neural networks trained on human games and then refined by self-play—playing against itself to improve7. In 2017, AlphaGo Zero removed human data entirely, learning from scratch through self-play and surpassing the earlier version. The same year, AlphaZero generalized the approach to chess and shogi, beating the best existing programs79. The method was not new in principle—self-play and search had been around for decades—but the combination with deep learning made it work at scale.

Counterfactual regret minimization (CFR), developed by Martin Zinkevich and colleagues in 2007, addressed a different class of games: those with imperfect information, where players cannot see each other's cards or private states. Poker is the canonical example. CFR is an iterative algorithm that computes strategies minimizing regret—the difference between the outcome achieved and the best possible outcome in hindsight. It converges to a Nash equilibrium, a strategy profile where no player can improve by changing strategy alone10. CFR and its variants (such as Monte Carlo CFR) powered programs like Libratus (2017) and Pluribus (2019), which beat top human poker players. The approach is fundamentally different from search-based methods because it reasons about hidden information and mixed strategies.

The Present: Hybrid Methods and Open Questions

Today, game playing AI is less a single field than a toolkit. MCTS remains central for games with large search spaces. Deep reinforcement learning and self-play are standard for training agents. CFR and its descendants dominate imperfect-information games. Classical search still matters where speed and precision are needed, and general game playing continues as a test of generality.

The open questions are substantial. How do you build agents that transfer skills across games rather than mastering one? How do you handle games with continuous action spaces or real-time constraints? How do you make learned strategies interpretable and safe? And the oldest question remains: what does game-playing skill tell us about intelligence itself? The field has answered Shannon's challenge many times over, but each answer has opened new questions about what it means for a machine to play—and to think1.

References

  1. Shannon number, Wikipedia
  2. Artificial intelligence in video games (History), Wikipedia
  3. Arthur Samuel (computer scientist) (Computer checkers (draughts) development), Wikipedia
  4. Deep Blue (chess computer), Wikipedia
  5. Monte Carlo tree search, Wikipedia
  6. Deep reinforcement learning, Wikipedia
  7. AlphaGo, Wikipedia
  8. Computer Go, Wikipedia
  9. AlphaZero, Wikipedia
  10. Nash equilibrium, Wikipedia

The frameworks of Game Playing Ai

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1959-Present

    Self-play

    Can a program improve by playing against itself?

    Fly to its tower in Nousfarer

  2. 1965-1997

    Knowledge-Based Chess AI

    How much expert knowledge does a chess program need?

    Fly to its tower in Nousfarer

  3. 1975-2005

    Brute-Force Search and Special-Purpose Hardware

    Can raw speed and deep search beat human expertise?

    Fly to its tower in Nousfarer

  4. 2005-Present

    General Game Playing

    Can one program play any game given only its rules?

    Fly to its tower in Nousfarer

  5. 2007-Present

    Counterfactual Regret Minimization

    How do you find optimal strategies when information is hidden?

    Fly to its tower in Nousfarer

  6. 2013-Present

    Deep reinforcement learning

    Can neural networks learn to play games from experience alone?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Classical Game Tree SearchSelf-playKnowledge-Based Chess AIBrute-Force Search and Special-Purpos…General Game PlayingMonte Carlo Tree SearchCounterfactual Regret MinimizationDeep reinforcement learning

Beyond the field

Where Game Playing Ai's ideas came from in other fields, and what they led to there: the strongest 12 of 31 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Game Playing Ai is a planet in Nousfarer

Its city has a tower for each of these 8 frameworks. Land, walk in, and climb one concept at a time.

Land on Game Playing Ai

Related fields

Fields its ideas are linked to

More in Game Studies & Design

See where Game Playing Ai sits on the Map of Human Knowledge