Probability Theory

Probability theory is the mathematics of uncertainty: the study of how likely events are, how random quantities behave, and how beliefs should change when evidence arrives.

12 frameworks, from 1654 to today · Probability Theory · Mathematics

still a working view ended · each leads to its entry below

Overview

Probability theory is the mathematics of uncertainty: the study of how likely events are, how random quantities behave, and how beliefs should change when evidence arrives. Its practitioners build models of chance—coins, dice, insurance risks, gas molecules, stock prices—and prove theorems about them. What makes the field distinctive is that it began as a practical craft for gamblers and insurers, was rebuilt in the twentieth century as a branch of measure theory, and now supplies the mathematical language for statistics, physics, engineering, finance, and machine learning. The history of probability is a history of making "chance" precise.

Games of Chance and the Birth of Combinatorial Probability

Before the seventeenth century, chance was mostly discussed by gamblers and lawyers, not mathematicians. The turning point came in 1654, when the French gambler Antoine Gombaud, the Chevalier de Méré, posed problems about dice and card games to Blaise Pascal. Pascal corresponded with Pierre de Fermat, and together they solved problems such as how to divide the stakes fairly when a game of chance is interrupted1234. Their method was combinatorial: count the equally likely outcomes and compare them5. This "classical" probability—probability as the ratio of favorable to total equally likely cases—was the field's first working definition53.

The framework spread quickly. Christiaan Huygens published a short book on games of chance in 1657, and Jacob Bernoulli's Ars Conjectandi, published posthumously in 1713, gave the first proof of a version of the law of large numbers: in repeated independent trials with the same probability of success, the observed frequency tends to approach the true probability as the number of trials grows51267. Abraham de Moivre's The Doctrine of Chances (1718, with later editions) developed the combinatorics of gambling and approximated binomial probabilities with the normal curve28. These results were the first limit theorems—statements about what happens to probabilities in the long run.

Insurance, Annuities, and the Rise of Frequency

Meanwhile, a parallel tradition grew out of commerce. Life annuities and marine insurance required pricing risk over time, and the Dutch statesman Johan de Witt and the English astronomer Edmond Halley both produced mortality-based calculations in the late seventeenth century. In 1762 the Equitable Life Assurance Society was founded in London on actuarial principles, and the mathematician Richard Price worked on its mortality tables. This practical tradition treated probability as an empirical frequency observed in data—an early form of what would later be called frequentist thinking.

The two traditions—combinatorial and empirical—coexisted uneasily. In the eighteenth century, Thomas Bayes's posthumously published 1763 essay proposed a different question: given observed evidence, what is the probability that a hypothesis is true? Bayes's approach, and Pierre-Simon Laplace's systematic development of it in the late eighteenth and early nineteenth centuries, treated probability as a degree of rational belief, updated by observation910. Laplace's Théorie analytique des probabilités (1812) unified much of the field, applied it to astronomy and error analysis, and popularized the "inverse probability" method that later became known as Bayesian inference.

The Classical Era and Its Foundations

The nineteenth century was the age of "classical probability": a rich body of results built on the assumption of equally likely outcomes or on frequency intuitions, without a rigorous foundation. Siméon Denis Poisson studied the law of large numbers and the Poisson distribution (published in 1837)6711. Pafnuty Chebyshev and his students Andrey Markov and Aleksandr Lyapunov developed the theory of limit theorems, proving versions of the central limit theorem—the result that sums of many small independent random influences tend toward a normal distribution. Francis Galton, in the 1880s, introduced regression and correlation, connecting probability to statistics.

But the foundations were shaky. What exactly did it mean to say an event has probability one-half? The classical definition was circular (equally likely outcomes are defined by equal probability), and the frequency definition was vague about infinite sequences. These questions became urgent as probability was applied to physics—James Clerk Maxwell and Ludwig Boltzmann used probabilistic reasoning in statistical mechanics in the 1860s and 1870s—and to biology and social science.

Measure-Theoretic Axiomatics: Kolmogorov's Revolution

The decisive turning point came in 1933, when Andrey Kolmogorov published Grundbegriffe der Wahrscheinlichkeitsrechnung (Foundations of the Theory of Probability)51213. Kolmogorov's idea was to define a probability space as a triple: a set of outcomes, a collection of events (a sigma-algebra, meaning a family of subsets closed under complement and countable union), and a probability measure—a function assigning numbers between 0 and 1 to events, with the measure of the whole space equal to 1 and the measure of a countable union of disjoint events equal to the sum of their measures513. This borrowed the language of measure theory, developed by Émile Borel and Henri Lebesgue around 1900.

The payoff was enormous. Probability became a branch of mathematics with the same rigor as analysis. Random variables became measurable functions; expectation became an integral; independence became a product condition on measures. The measure-theoretic framework made it possible to state and prove limit theorems precisely, to define conditional expectation rigorously, and to build the theory of stochastic processes. Kolmogorov's axioms did not settle the philosophical question of what probability means—frequentists, Bayesians, and others still disagree—but they gave all sides a common mathematical language.

Limit Theorems and the Classical Limit Theory

With rigorous foundations in place, the limit theorems of the 1920s and 1930s were reformulated and extended. The central limit theorem was proved under general conditions by Jarl Waldemar Lindeberg (1922) and later by Paul Lévy and others. The law of large numbers was given precise forms—the weak law (convergence in probability) and the strong law (almost sure convergence), the latter proved by Kolmogorov in 1933. These results are the backbone of statistics: they explain why sample averages estimate population means and why the normal distribution appears so often.

Stochastic Processes: Randomness in Time

The study of stochastic processes—families of random variables indexed by time—grew alongside the foundations. Louis Bachelier's 1900 thesis modeled stock prices with what we now call Brownian motion, five years before Albert Einstein's 1905 physical derivation of the same process14151617. Norbert Wiener gave Brownian motion a rigorous mathematical construction in 1923. Andrey Markov introduced Markov chains in 1906 to study sequences of dependent trials, where the future depends only on the present state18. Kolmogorov's 1931 work on Markov processes and his later analytic theory of transition probabilities (with the Chapman–Kolmogorov equations) made the subject systematic18.

The 1930s and 1940s brought martingale theory. A martingale is a stochastic process whose expected future value, given the past, equals its present value—the mathematical model of a fair game19. Joseph Doob developed the modern theory in the 1940s and 1950s, proving convergence theorems and optional stopping results that became central to both probability and mathematical finance. Paul Lévy studied processes with independent increments and contributed deeply to the theory of Brownian motion and what became known as Lévy processes.

Stochastic Calculus and the Itô Revolution

The mid-twentieth century produced the field's most powerful computational tool: stochastic calculus. Ordinary calculus assumes smooth paths, but Brownian motion is nowhere differentiable—its path is infinitely jagged. Kiyosi Itô, in a series of papers beginning in 1944, developed a calculus for such processes. The key object is the stochastic integral, and the key result is Itô's formula, a chain rule for random processes that includes an extra correction term reflecting the roughness of Brownian paths20. This made it possible to write and solve stochastic differential equations—equations describing systems driven by noise.

Itô's calculus transformed probability into a tool for physics, engineering, and finance. In 1973, Fischer Black, Myron Scholes, and (in a parallel formulation) Robert Merton used it to derive a formula for pricing options, work that won the 1997 Nobel Prize in economics. The same mathematics describes diffusion in physics, filtering and control in engineering, and population dynamics in biology.

Frequentist and Bayesian Traditions in the Twentieth Century

The philosophical split between frequentists and Bayesians shaped twentieth-century statistics and probability. Frequentists, following Ronald Fisher, Jerzy Neyman, and Egon Pearson, defined probability as long-run frequency and built methods—confidence intervals, hypothesis tests—whose guarantees hold over repeated sampling. Bayesians, following Harold Jeffreys and later Leonard Savage and Bruno de Finetti, treated probability as subjective degree of belief updated by Bayes's theorem921. De Finetti's 1937 representation theorem showed that exchangeable sequences—where the order of observations does not matter—can be represented as mixtures of independent trials with a random parameter, giving Bayesian reasoning a rigorous probabilistic basis. The debate continues, but both traditions now share the measure-theoretic framework and often use the same computational tools.

The Present: Computation, Complexity, and New Frontiers

Today probability theory is both a mature mathematical discipline and a rapidly expanding toolkit. The measure-theoretic framework remains standard, but the field's center of gravity has shifted toward processes, computation, and applications. Markov chain Monte Carlo, developed from the 1950s onward (with the Metropolis algorithm in 1953 and the Gibbs sampler popularized in the 1980s), made Bayesian inference computationally feasible and is now central to machine learning. Large deviations theory, developed by Harald Cramér in the 1930s and Varadhan and others later, quantifies the probability of rare events. Rough path theory, introduced by Terry Lyons in the 1990s, extends stochastic calculus to rougher signals than Brownian motion.

Open questions remain: the foundations of probability are still debated philosophically; the interface between probability and quantum mechanics is unsettled; and the theory of random structures—random graphs, random matrices, interacting particle systems—is an active frontier. What began as a gambler's tool is now the mathematics of uncertainty itself, and its history is a record of making that uncertainty precise.

References

  1. History of probability (Origins), Wikipedia
  2. Probability (History), Wikipedia
  3. Classical definition of probability (History, introduction), Wikipedia
  4. Expected value (History), Wikipedia
  5. Probability theory (History of probability, Discrete probability distributions, Measure-theoretic probability theory), Wikipedia
  6. Frequentist probability (History), Wikipedia
  7. Law of large numbers (History), Wikipedia
  8. Normal distribution (Development), Wikipedia
  9. Bayesian probability (introduction, History), Wikipedia
  10. Bayesian statistics, Wikipedia
  11. Poisson distribution (History), Wikipedia
  12. Probability axioms, Wikipedia
  13. Probability space, Wikipedia
  14. Stochastic process, Wikipedia
  15. Stochastic calculus, Wikipedia
  16. Brownian motion (introduction, History), Wikipedia
  17. Stochastic differential equation (Background), Wikipedia
  18. Markov chain (introduction, History), Wikipedia
  19. Martingale (probability theory), Wikipedia
  20. Itô's lemma, Wikipedia
  21. Probability interpretations, Wikipedia

The frameworks of Probability Theory

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1654-1900

    Classical definition of probability

    How likely is an outcome when all cases are equally possible?

    About Classical definition of probability
    Classical probability arose from a simple but powerful question: how can we assign numerical odds to events in gambling games, using only symmetry and counting? At its core, this framework treats probability as a ratio — the number of ways an event can happen divided by the total number of equally likely outcomes. This approach, which dominated probability from the mid-1600s through the 1800s, turned luck into a calculus. For dice, cards, and lotteries, it was enough; for more complex or continuous problems, it eventually needed a deeper foundation.
    To master classical probability, a student learns a tightly connected set of ideas, best taken in this order:
    Foundations: The classical definition of probability — number of favorable cases divided by total cases — only works when every outcome is equally likely. This principle of equally likely outcomes is the bedrock. Combinatorial probability provides the counting tools, from simple arrangements to Pascal's triangle, which organizes binomial coefficients for problems like the number of heads in repeated coin flips.
    Basic Operations: The probability of complementary events (something either happening or not) always sums to 1. For incompatible events (outcomes that cannot happen together), probabilities add. For compound events (outcomes that can happen together), the multiplication rule for independent events applies: if two events do not influence each other, their joint probability is the product of their separate probabilities.
    Canonical Problems: Gambling problems — drawing lots, splitting stakes in an interrupted game — were the framework's original motivation. The concept of expectation in games of chance extends probability to weighted averages of payoffs. A fair game is one where the expected value of playing is zero (neither player has an advantage). Bernoulli trials — identical, independent experiments with exactly two outcomes — became the workhorse of classical inference. De Moivre's normal approximation to the binomial (1733) showed that as the number of trials grows, the binomial distribution's shape approaches a smooth, bell-shaped curve — an early glimpse of the Central Limit Theorem, though that theorem's modern form came later.
    The intellectual history of classical probability begins with a famous correspondence in 1654 between Blaise Pascal and Pierre de Fermat. They were asked by a gambler, the Chevalier de Méré, how to divide stakes in a game that had to be stopped early. Their solution — using combinations to count favorable cases — is considered the birth of probability. Christiaan Huygens published the first probability textbook, De Ratiociniis in Ludo Aleae (1657), which introduced expectation. Jacob Bernoulli, in his Ars Conjectandi (1713, published posthumously), gave the first law of large numbers for repeated trials and systematized the method of treating moral certainty. Abraham de Moivre, in The Doctrine of Chances (1718, with later expanded editions), developed the normal approximation and refined the mathematics of annuities and gaming. The framework's landmark codification came from Pierre-Simon Laplace, whose Théorie analytique des probabilités (1812) unified all these results with advanced calculus and applied probability to astronomy, legal evidence, and errors of observation.
    But classical probability had a fatal weakness. It could not reliably handle infinite possibilities: for example, the Bertrand paradox (1889) showed that asking for the probability of a random chord in a circle produced different answers depending on how one defined "random." This exposed the need for a more rigorous foundation, which arrived with Measure-Theoretic Probability (Kolmogorov, 1933). Rather than being replaced, classical probability was absorbed and transformed: its combinatorial core now lives within measure theory as a special case for finite, uniform probability spaces. Stochastic Processes (from about 1900 onward) used the new measure-theoretic tools to study random change over time — coin flips, queues, Brownian motion — far beyond the scope of classical fair-games reasoning.
    The 15 concepts its tower teaches
    • Bernoulli trials
    • Classical definition of probability
    • Combinatorial probability
    • De Moivre's normal approximation to the binomial
    • Drawing lots
    • Equally likely outcomes
    • Expectation in games of chance
    • Fair games
    • Favorable cases / total cases
    • Gambling problems
    • Multiplication rule for independent events
    • Pascal's triangle
    • Probability of complementary events
    • Probability of compound events
    • Probability of incompatible events

    Fly to its tower in Nousfarer

  2. 1670-1900

    Actuarial and Frequency Tradition

    How often does an event occur in the long run of data?

    Fly to its tower in Nousfarer

  3. 1763-Present

    Bayesian probability

    How should belief change when new evidence arrives?

    Fly to its tower in Nousfarer

  4. 1867-Present

    Limit Theory

    What happens to averages and sums as trials grow without bound?

    Fly to its tower in Nousfarer

  5. 1900-Present

    Stochastic process

    How do random quantities evolve over time?

    About Stochastic process
    Stochastic processes provide the mathematical language for describing systems that evolve randomly over time. Where classical probability treated isolated experiments—a coin flip, a die roll—stochastic processes capture the dynamics of chance: the jitter of a stock price, the spread of an epidemic, the queue at a coffee shop. The central object is a stochastic process, a collection of random variables indexed by time (or space), each representing the state of the system at a given moment. This framework emerged in the early twentieth century as a natural extension of classical probability, and it became the workhorse of modern probability theory, bridging pure mathematics with applications in physics, finance, biology, and engineering.
    Core Curriculum
    A first course in stochastic processes builds from a few fundamental building blocks. The curriculum typically proceeds as follows:
    1. Stochastic Process, State Space, Sample Path: A stochastic process is a family of random variables {Xt:t∈T}\{X_t : t \in T\}, where TT is an index set (often time). The state space is the set of possible values each XtX_t can take, and a sample path is a single realization of the process—a record of the states visited over time.
    1. Random Walk, Markov Chain, Transition Probability: The simplest dynamic model is a random walk, where each step is independent of the past. A Markov chain generalizes this: the future depends only on the present state, not on the full history. The transition probability P(Xn+1=j∣Xn=i)P(X_{n+1} = j \mid X_n = i) encodes the one-step dynamics.
    1. Poisson Process, Renewal Process: The Poisson process models events occurring randomly in time at a constant average rate—the classic example being radioactive decay or customer arrivals. A renewal process generalizes this by allowing arbitrary distributions for the times between events.
    1. Brownian Motion, Wiener Process: Brownian motion (also called the Wiener process) is the continuous-time limit of a random walk, describing the erratic motion of a particle suspended in a fluid. It is the foundational model for diffusion and the building block of stochastic calculus.
    1. Martingale: A martingale is a process where the expected future value, given all past information, equals the present value. It formalizes the idea of a "fair game" and is a central tool for analyzing stopping times and convergence.
    1. Stationary Process, Gaussian Process: A stationary process has statistical properties (like mean and variance) that do not change over time, making it a natural model for signals and time series. A Gaussian process is any process whose finite-dimensional distributions are multivariate normal; it is widely used in spatial statistics and machine learning.
    1. Branching Process, Birth-Death Process: A branching process models population growth where each individual produces a random number of offspring. A birth-death process is a continuous-time Markov chain where the state changes by ±1, representing population size under births and deaths.
    Intellectual History
    The story of stochastic processes begins in the early twentieth century, as mathematicians sought to extend probability beyond the static, combinatorial framework of classical probability. The first major step was the random walk, studied by Karl Pearson in 1905 in the con\text{} of mosquito dispersal. But the true catalyst was the mathematical modeling of Brownian motion. In 1900, Louis Bachelier, in his doctoral thesis Théorie de la spéculation, introduced a continuous-time stochastic process to model stock prices—what we now call Brownian motion—five years before Einstein's famous 1905 paper on the same phenomenon. Bachelier's work was largely ignored by probabilists for decades, but it planted the seed.
    The modern theory of stochastic processes took shape in the 1920s and 1930s. Andrey Kolmogorov, building on his measure-theoretic foundations (the Kolmogorov axioms), published Grundbegriffe der Wahrscheinlichkeitsrechnung in 1933, which provided the rigorous framework for probability spaces, σ\sigma-algebras, and conditional expectation. This work made stochastic processes mathematically respectable. In 1931, Kolmogorov also published a paper on Markov chains, establishing the transition probability as a fundamental object and deriving the forward and backward equations that bear his name.
    Around the same time, Norbert Wiener (1923) gave a rigorous construction of Brownian motion as a limit of random walks, proving the existence of a continuous but nowhere-differentiable sample path. This process is now called the Wiener process. In the 1930s, William Feller and others developed the theory of birth-death processes and branching processes, applying them to population biology and physics.
    The concept of the martingale was introduced by Paul Lévy in the 1930s, but its systematic development came later, in the 1950s and 1960s, through the work of Joseph Doob. Doob's Stochastic Processes (1953) became the definitive textbook, codifying the theory of martingales, stopping times, and convergence theorems. The Itô calculus, developed by Kiyoshi Itô in the 1940s, provided the tools to integrate with respect to Brownian motion, enabling the rigorous treatment of stochastic differential equations—a cornerstone of modern finance.
    Stochastic processes coexisted with and were eventually absorbed into the broader measure-theoretic framework. They did not replace classical probability but rather extended it to dynamic settings. Today, the theory of stochastic processes is a mature branch of probability, with deep connections to analysis, differential equations, and statistical physics. Its core concepts—Markov chains, Poisson processes, Brownian motion, martingales—remain the essential toolkit for anyone modeling randomness in time.
    The 14 concepts its tower teaches
    • Birth-Death Process
    • Branching Process
    • Brownian Motion
    • Gaussian Process
    • Markov Chain
    • Martingale
    • Poisson Process
    • Random Walk
    • Renewal Process
    • Sample Path
    • State Space
    • Stationary Process
    • Stochastic Process
    • Transition Probability

    Fly to its tower in Nousfarer

  6. 1920-Present

    Frequentist Inference

    How should we estimate and test using observed data?

    Fly to its tower in Nousfarer

  7. 1933-Present

    Probability axioms

    What are the minimal rules any probability must obey?

    Fly to its tower in Nousfarer

  8. 1938-Present

    Large Deviations Theory

    How fast do the probabilities of rare events decay?

    Fly to its tower in Nousfarer

  9. 1940-Present

    Martingale Theory

    How do fair games behave when observed over time?

    Fly to its tower in Nousfarer

  10. 1944-Present

    Stochastic Calculus

    How do we integrate and differentiate along random paths?

    Fly to its tower in Nousfarer

  11. 1953-Present

    Monte Carlo Methods

    How can random sampling estimate a quantity we cannot compute?

    Fly to its tower in Nousfarer

  12. 1998-Present

    Rough Path Theory

    How do we solve equations driven by very irregular signals?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Classical definition of probabilityActuarial and Frequency TraditionBayesian probabilityLimit TheoryStochastic processFrequentist InferenceProbability axiomsLarge Deviations TheoryMartingale TheoryStochastic CalculusMonte Carlo MethodsRough Path Theory

Beyond the field

Where Probability Theory's ideas came from in other fields, and what they led to there: the strongest 12 of 169 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Probability Theory is a planet in Nousfarer

Its city has a tower for each of these 12 frameworks. Land, walk in, and climb one concept at a time.

Land on Probability Theory

Related fields

Fields its ideas are linked to

More in Probability Theory

See where Probability Theory sits on the Map of Human Knowledge