Stochastic Processes

Stochastic processes are the branch of probability theory that studies systems evolving randomly in time.

17 frameworks, from 1906 to today · Probability Theory · Mathematics

still a working view ended · each leads to its entry below

Overview

Stochastic processes are the branch of probability theory that studies systems evolving randomly in time. Where classical probability asks how likely a single event is—a coin landing heads, a die showing six—a stochastic process asks how a random quantity unfolds: the jitter of a stock price across a trading day, the queue outside a ticket office, the diffusion of a drop of ink through water, the number of customers in a call centre at each minute. What is at stake is nothing less than a mathematical language for change under uncertainty, and that language now underpins physics, engineering, finance, biology, and statistics. The history of the subfield is the story of how a loose collection of practical problems—gambling, insurance, molecular motion—was forged into one of the most powerful and abstract branches of modern mathematics, and then fed back into the practical world on an enormous scale.

Gambling Problems and the First Markov Chains

Probability began in the seventeenth century with correspondence between Blaise Pascal and Pierre de Fermat about games of chance, and the "problem of points"—how to divide the stakes when a game is interrupted—produced the idea of expected value. For two centuries the subject dealt with independent trials: each coin flip, each roll of dice, unaffected by what came before. The first decisive break came from Andrey Markov, who around 1906 studied sequences of dependent trials. He asked whether the law of large numbers—the principle that long-run averages settle down—still held when each outcome depends on the previous one12. His answer was yes, provided the dependence has a special form: the future depends on the present state but not on the path taken to reach it1. This is the Markov property, and processes obeying it are Markov chains (discrete-time) or Markov processes (continuous-time). Markov demonstrated the idea concretely by analysing the alternation of vowels and consonants in Pushkin's verse, showing that a simple dependent model could reproduce statistical features of real text. The Markov property turned out to be the key that unlocked an entire theory: if the present state is all that matters, the process can be described by transition probabilities, and long-run behaviour can be computed from them.

Brownian Motion: From Pollen Grains to a Mathematical Object

In 1827 the botanist Robert Brown observed pollen grains suspended in water jittering ceaselessly under his microscope3. The cause—collisions with invisible water molecules—was not settled until Einstein's 1905 paper, which derived the diffusion equation from molecular assumptions and predicted that the mean squared displacement of a particle grows linearly with time. Einstein's work gave physicists a tool for estimating molecular sizes. Independently, Louis Bachelier in his 1900 thesis on speculation had already used a similar continuous random walk to model stock prices, though his work was largely ignored for decades13.

The mathematical object itself—Brownian motion, also called the Wiener process—was constructed rigorously by Norbert Wiener in the 1920s14. Wiener's construction showed that a continuous function of time could be random at every scale, with independent increments over disjoint time intervals and Gaussian distributions1. The path is continuous but nowhere differentiable: it has infinite variation, so ordinary calculus cannot integrate against it. This was the obstacle that would define the next era.

Measure Theory Rebuilds Probability

Before the 1930s, probability was a somewhat informal subject, and the construction of continuous-time processes was ad hoc. Andrey Kolmogorov's 1933 Grundbegriffe der Wahrscheinlichkeitsrechnung changed everything1. Kolmogorov axiomatised probability as a branch of measure theory: a probability space is a set of outcomes, a collection of events (a sigma-algebra), and a countably additive measure assigning each event a probability. Crucially, he gave a theorem—the Kolmogorov extension theorem—showing that a whole family of finite-dimensional distributions, consistent with one another, determines a stochastic process. This made it possible to construct processes rigorously from their finite-dimensional laws1. The measure-theoretic framework also gave precise meaning to conditional expectation, which had previously been a vague notion. Conditional expectation became the foundation for martingale theory.

Martingales and the Theory of Fair Games

A martingale is a process whose expected future value, given everything observed so far, equals its present value—the mathematical idealisation of a fair game15. The concept appeared in gambling literature (the "martingale" betting system) but was transformed into a powerful tool by Joseph Doob in the 1940s and 1950s. Doob proved the martingale convergence theorem: under mild conditions, a martingale converges almost surely. He also developed optional stopping theorems, which say when a gambler's expected fortune at a stopping time equals the starting fortune. These results were startling because they applied to very general processes, not just sums of independent variables. Martingale theory became the engine of modern probability: it gave proofs of the strong law of large numbers, the Radon–Nikodym theorem in probabilistic form, and later the foundations of stochastic integration. Paul Lévy contributed deeply to the same circle of ideas, and the Lévy–Doob martingale theory of the mid-twentieth century is one of the great achievements of the subject.

Stochastic Calculus: Integrating Against Noise

The central problem of the 1940s was how to define an integral against Brownian motion. Ordinary Riemann–Stieltjes integration fails because Brownian paths have infinite variation6. Kiyoshi Itô, in a series of papers from 1944 onward, defined a new integral—the Itô integral—by taking limits in mean square, and in doing so discovered that the chain rule of ordinary calculus must be corrected by an extra term. This is Itô's lemma: for a smooth function f of a Brownian motion B, the differential of f(B) includes a second-derivative term proportional to dt7. The correction arises because Brownian increments squared accumulate to time. Itô's calculus made it possible to write down and solve stochastic differential equations (SDEs)—equations like dX = μ(X)dt + σ(X)dB—which describe diffusions: particles in fluids, interest rates, population sizes18. The theory was extended by Itô and others to general semimartingales, the largest class of processes for which a sensible stochastic integral exists. This general theory, developed through the 1960s and 1970s (with major contributions from the Strasbourg school around Paul-André Meyer), unified Brownian motion, jump processes, and martingales under one framework.

Diffusion Theory and the Connection to Partial Differential Equations

Itô's calculus revealed a deep link between stochastic processes and partial differential equations. The generator of a diffusion—an operator like the Laplacian plus a drift term—governs the expected values of functions of the process, and the Feynman–Kac formula expresses solutions of parabolic PDEs as expectations over diffusion paths. This connection, developed in the mid-twentieth century, meant that probabilistic methods could solve PDE problems and vice versa9. It also gave rise to the theory of Lévy processes—processes with stationary independent increments that may jump—generalising Brownian motion and providing models for insurance claims, asset prices with sudden crashes, and queuing systems.

Queues, Point Processes, and the Applied Tradition

Alongside the abstract theory, a parallel applied tradition grew from practical problems. Agner Krarup Erlang, working for the Copenhagen Telephone Company around 1910, modelled telephone call arrivals as a Poisson process—a point process in which events occur randomly and independently at a constant rate—and derived formulas for waiting times and blocking probabilities110. This founded queueing theory10. The Poisson process and its generalisations (renewal processes, Cox processes) became the standard models for arrivals of customers, radioactive decays, and neural spikes. The applied tradition and the abstract tradition met in the mid-twentieth century: queueing networks, Markov decision processes, and reliability theory all drew on Markov chain theory and martingale methods.

Random Walks and Random Environments

The simple random walk—a particle stepping left or right with equal probability—is the discrete ancestor of Brownian motion and one of the most studied objects in probability111. Its recurrence properties (in one and two dimensions a random walk returns to its starting point with probability one; in three dimensions it may escape) were settled in the 1920s1. In the 1970s and 1980s, attention turned to random walks in random environments: the step probabilities themselves are random, fixed once and for all. This models transport through disordered media—electrons in impure metals, fluid through porous rock. The field produced striking phenomena, such as Sinai's result that a one-dimensional random walk in a random environment moves anomalously slowly, and it remains an active area connecting probability, statistical physics, and ergodic theory.

Large Deviations: The Mathematics of Rare Events

How likely is a very unusual outcome? The classical limit theorems describe typical behaviour; large deviations theory describes the exponentially small probabilities of rare events12. The theory grew from the work of Harald Cramér in the 1930s on insurance risk and was developed into a general framework by S. R. S. Varadhan and others in the 1960s and 1970s. The key object is a rate function that measures the "cost" of a deviation13. Large deviations theory has become essential in statistical mechanics (where it connects to entropy), in information theory (Sanov's theorem), and in risk management, where rare catastrophic events matter most.

Gaussian Processes and Random Fields

A Gaussian process is a stochastic process whose finite-dimensional distributions are multivariate normal14. Brownian motion is the canonical example, but the class is far larger and includes processes indexed by space as well as time—random fields1415. Gaussian random fields model spatial phenomena: the height of a rough surface, the temperature across a region, the fluctuations of the cosmic microwave background1617. The theory was developed through the mid-twentieth century, with important contributions from Kolmogorov and later from geostatistics (the "kriging" method of Danie Krige and Georges Matheron in the 1950s–60s). In modern machine learning, Gaussian processes serve as priors over functions for Bayesian regression, a development that began with work by David MacKay and others in the 1990s.

Hidden Markov Models and State-Space Models

In many real systems the state is not directly observed—only noisy measurements of it. Hidden Markov models (HMMs) formalise this: an underlying Markov chain evolves unseen, and at each step an observation is generated from the current state18. The forward–backward algorithm, developed in the 1960s and 1970s (with roots in the work of Leonard Baum and colleagues at the Institute for Defense Analyses), computes the likelihood of observations and the posterior distribution of hidden states. The Viterbi algorithm finds the most likely state sequence19. These methods transformed speech recognition in the 1970s and 1980s, and later bioinformatics (gene finding), finance, and signal processing18. The broader class of state-space models, including the Kalman filter for linear Gaussian systems (Rudolf Kalman, 1960), became standard tools in engineering and econometrics.

Monte Carlo and Markov Chain Monte Carlo

The Monte Carlo method—using random sampling to estimate quantities—emerged at Los Alamos in the 1940s, associated with Stanislaw Ulam, John von Neumann, and Nicholas Metropolis. The idea is simple: to estimate an integral, sample random points and average. But sampling from complicated distributions is hard. Markov chain Monte Carlo (MCMC) solves this by constructing a Markov chain whose stationary distribution is the target20. The Metropolis algorithm (1953) was the first; the Metropolis–Hastings algorithm (W. K. Hastings, 1970) generalised it. In the 1980s and 1990s, MCMC revolutionised Bayesian statistics: the Gibbs sampler (popularised by Stuart Geman and Donald Geman in 1984 and by Alan Gelfand and Adrian Smith in 1990) made it possible to fit complex hierarchical models. Modern computational statistics is unimaginable without MCMC, and the method's descendants—Hamiltonian Monte Carlo, variational inference—continue to drive machine learning and data science.

Mathematical Finance and Risk-Neutral Pricing

In 1973 Fischer Black, Myron Scholes, and (independently) Robert Merton published option-pricing formulas that used Itô calculus to model stock prices as geometric Brownian motion. The Black–Scholes formula expresses the price of a European call option in terms of the stock price, strike price, volatility, and interest rate. The deeper insight, developed by Harrison, Kreps, and Pliska in the late 1970s and early 1980s, is that arbitrage-free pricing is equivalent to the existence of a probability measure—the risk-neutral measure—under which discounted asset prices are martingales. This martingale approach to finance unified the field and made it possible to price complex derivatives. The 1987 crash and later crises exposed the limitations of the Gaussian model, spurring the development of jump-diffusion models (Merton, 1976), stochastic volatility models, and Lévy-process-based finance.

Stochastic Partial Differential Equations

The final major frontier is the study of random fields that evolve according to partial differential equations with random forcing—stochastic PDEs21. The prototype is the stochastic heat equation, driven by space-time white noise. These equations arise in physics (fluctuations in fluids, phase transitions), biology (population genetics), and engineering. The mathematical difficulties are severe: solutions may be distributions rather than functions, and nonlinear terms require renormalisation. The theory was developed from the 1970s onward, with major advances by John Walsh (martingale measures), and in the 2000s and 2010s by Martin Hairer, whose theory of regularity structures (for which he received the Fields Medal in 2014) provided a general framework for making sense of singular stochastic PDEs. This remains one of the most active areas of probability today.

The Present: Unified Theory and Computational Reach

Stochastic processes today are a mature but still expanding field. The measure-theoretic and semimartingale framework provides a unified language; Itô calculus and its generalisations are standard tools; and the computational revolution—MCMC, particle filters, Gaussian process regression, stochastic gradient methods—has made stochastic processes central to machine learning and artificial intelligence. Open questions remain: the rigorous understanding of singular stochastic PDEs, the behaviour of random walks in high-dimensional random environments, the foundations of stochastic analysis on rough paths (Terry Lyons's theory, developed from the 1990s), and the interface between stochastic processes and quantum field theory. The field that began with gamblers' dice and pollen grains now models everything from neural activity to financial markets to the large-scale structure of the universe, and its history is a case study in how abstract mathematics and concrete problems can transform each other1.

References

  1. Stochastic process (Markov processes, Markov processes and chains, Wiener process, Birth of modern probability theory, Mathematical construction, Martingale, Stochastic processes after World War II, Poisson process, Random walk, Random walks, introduction), Wikipedia
  2. Markov chain (History), Wikipedia
  3. Brownian motion (introduction, History), Wikipedia
  4. Wiener process, Wikipedia
  5. Martingale (probability theory), Wikipedia
  6. Itô calculus, Wikipedia
  7. Stochastic calculus, Wikipedia
  8. Stochastic differential equation, Wikipedia
  9. Feynman–Kac formula, Wikipedia
  10. Queueing theory (introduction, History), Wikipedia
  11. Random walk, Wikipedia
  12. Large deviations theory, Wikipedia
  13. Sanov's theorem, Wikipedia
  14. Gaussian process, Wikipedia
  15. Random field, Wikipedia
  16. Large deviations of Gaussian random functions, Wikipedia
  17. Gaussian random field, Wikipedia
  18. Hidden Markov model (introduction, History), Wikipedia
  19. Viterbi algorithm, Wikipedia
  20. Markov chain Monte Carlo, Wikipedia
  21. Stochastic partial differential equation, Wikipedia

The frameworks of Stochastic Processes

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1906-Present

    Markov chain

    How do systems evolve when the future depends only on the present?

    Fly to its tower in Nousfarer

  2. 1909-Present

    Queueing Theory

    How long do customers wait in line?

    Fly to its tower in Nousfarer

  3. 1920-Present

    Random walk

    Where does a randomly moving particle go?

    Fly to its tower in Nousfarer

  4. 1923-Present

    Wiener process

    What is the mathematical model of continuous random motion?

    Fly to its tower in Nousfarer

  5. 1930-Present

    Lévy process

    What processes have independent and stationary increments?

    Fly to its tower in Nousfarer

  6. 1933-Present
  7. 1938-Present

    Large Deviations Theory

    How likely are extremely rare events?

    Fly to its tower in Nousfarer

  8. 1940-Present

    Gaussian process

    How do jointly Gaussian random functions behave?

    Fly to its tower in Nousfarer

  9. 1940-Present

    Martingale

    When is a game fair?

    About Martingale
    Martingale theory addresses a deceptively simple question: if a process is 'fair' on average at every step, what can we say about its long-run behavior? A martingale is a stochastic process where the expected future value, given all past information, equals the present value. This property—no drift, no bias—turns out to be surprisingly powerful. It provides a unifying framework for analyzing sequences of random variables that are neither independent nor Markovian, yet obey a conditional fairness constraint. In stochastic processes, martingale theory is the lens through which we study the interplay between information, expectation, and time, revealing deep connections between gambling, finance, and the very structure of randomness.
    Core Curriculum
    A first course in martingale theory for stochastic processes typically builds around these fundamental concepts in logical order:
    1. Conditional Expectation: The mathematical tool that formalizes 'best guess given available information.' It is the engine of martingale theory, defining how expectations update as new information arrives.
    1. Filtration: A growing sequence of sigma-algebras representing the flow of information over time. Filtrations encode what is known at each moment and are essential for defining conditional expectations and martingales.
    1. Martingale: A stochastic process {Xn}\{X_n\} adapted to a filtration {Fn}\{\mathcal{F}_n\} such that E[Xn+1∣Fn]=Xn\mathbb{E}[X_{n+1} \mid \mathcal{F}_n] = X_n almost surely. This is the defining property of a 'fair game.'
    1. Submartingale and Supermartingale: Processes where the conditional expectation is greater than or equal to (submartingale) or less than or equal to (supermartingale) the current value. These model favorable and unfavorable games, respectively.
    1. Stopping Time: A random time τ\tau such that the event {τ≤n}\{\tau \leq n\} is known by time nn. Stopping times capture when a gambler might decide to quit, and are central to analyzing martingale behavior.
    1. Optional Stopping Theorem: A cornerstone result stating that, under certain conditions, the expected value of a martingale at a stopping time equals its initial value. This formalizes the idea that no betting strategy can beat a fair game.
    1. Martingale Convergence Theorem: A martingale that is bounded in L1L^1 converges almost surely to a finite limit. This theorem reveals that even without independence, martingales cannot oscillate forever; they must settle down.
    1. Doob's Maximal Inequality: An inequality bounding the probability that the maximum of a martingale exceeds a threshold, in terms of its terminal expectation. It is a key tool for controlling extreme values.
    1. **Doob's LpL^p Inequality**: A refinement of the maximal inequality for p>1p > 1, relating the LpL^p norm of the maximum to the LpL^p norm of the terminal value.
    1. Martingale Transform: A discrete-time stochastic integral, where a predictable process is used to 'bet' on the increments of a martingale. Martingale transforms preserve the martingale property under integrability conditions.
    1. Quadratic Variation: For a martingale, the quadratic variation measures the accumulated squared increments. It is a fundamental object in stochastic calculus and central to the analysis of path properties.
    1. Martingale Representation Theorem: In certain settings (e.g., Brownian filtration), every martingale can be represented as a stochastic integral. This theorem underpins much of mathematical finance.
    Intellectual History
    Martingale theory emerged in the mid-20th century, building on earlier work in probability and stochastic processes. The term 'martingale' itself has a long history in gambling, referring to a betting strategy where one doubles the stake after a loss, aiming to recover all previous losses with a single win. The mathematical theory, however, was developed to understand the limits of such strategies and the nature of fair games.
    Precursors: The concept of a fair game appears implicitly in the work of Louis Bachelier (1900) on the theory of speculation, where he modeled stock prices as a process with independent increments—a precursor to both Brownian motion and martingales. Andrey Kolmogorov's axiomatization of probability (1933) provided the measure-theoretic foundation for conditional expectation and filtrations, which are essential for martingale theory. Paul Lévy studied processes with independent increments and introduced the concept of a 'martingale' in a 1934 paper, though his definition was not yet fully formalized.
    Key Figures and Seminal Works: The modern theory was largely created by Joseph L. Doob in the 1940s and 1950s. Doob's 1953 book Stochastic Processes is the foundational text, systematically developing martingale theory within the framework of measure-theoretic probability. He proved the fundamental convergence theorems, the optional stopping theorem, and the maximal inequalities that bear his name. Doob's work established martingales as a central tool in stochastic processes, showing how they unify and extend results for sums of independent random variables and Markov chains.
    J. L. Doob (1910–2004) was an American mathematician who spent most of his career at the University of Illinois. His 1953 monograph remains a landmark, and his 1984 book Classical Potential Theory and Its Probabilistic Counterpart revealed deep connections between martingales and harmonic functions.
    Later Codifiers and Transmitters: In the 1960s and 1970s, martingale theory was absorbed into the broader framework of semimartingale theory and stochastic calculus, particularly through the work of Kiyosi Itô and Paul-André Meyer. Meyer's 1966 book Probability and Potentials (with Claude Dellacherie) integrated martingales into the theory of stochastic processes, and his 'Strasbourg school' developed the general theory of processes, where martingales play a central role. David Williams' 1991 book Probability with Martingales is a widely used modern introduction.
    Successor and Parallel Frameworks: Martingale theory coexists with and feeds into stochastic calculus, where it provides the foundation for stochastic integrals and Itô's formula. Semimartingale theory generalizes martingales to processes that can be decomposed into a martingale and a finite-variation part, forming the natural domain for stochastic integration. Stationary processes and Markov chains are distinct frameworks, but martingale methods are often applied to analyze them (e.g., using martingale central limit theorems for Markov chains).
    Legacy: Martingale theory transformed stochastic processes by providing a unified framework for analyzing dependence without requiring independence or the Markov property. Its key insights—that fair games have no systematic drift, that stopping times cannot create value, and that bounded martingales must converge—have become essential tools in probability, statistics, finance, and beyond. The theory remains a core part of the stochastic processes curriculum, serving as a bridge between elementary probability and advanced stochastic calculus.
    The 12 concepts its tower teaches
    • Conditional Expectation
    • Doob's L^p Inequality
    • Doob's Maximal Inequality
    • Filtration
    • Martingale
    • Martingale Convergence Theorem
    • Martingale Representation Theorem
    • Martingale Transform
    • Optional Stopping Theorem
    • Stopping Time
    • Submartingale
    • Supermartingale

    Fly to its tower in Nousfarer

  10. 1944-Present

    Itô calculus

    How do we integrate against random noise?

    Fly to its tower in Nousfarer

  11. 1949-Present
  12. 1950-Present

    Diffusion Theory and PDE Connections

    How are random motions linked to differential equations?

    Fly to its tower in Nousfarer

  13. 1960-Present

    Hidden Markov model

    How can we infer hidden states from observations?

    Fly to its tower in Nousfarer

  14. 1970-Present

    Semimartingale and General Stochastic Integration Theory

    What is the most general theory of stochastic integration?

    Fly to its tower in Nousfarer

  15. 1970-Present

    Stochastic partial differential equation

    How do random fields evolve in space and time?

    Fly to its tower in Nousfarer

  16. 1973-Present
  17. 1998-Present

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Markov chainQueueing TheoryRandom walkWiener processLévy processMeasure-Theoretic ProbabilityLarge Deviations TheoryGaussian processMartingaleItô calculusMonte Carlo and Markov Chain Monte Ca…Diffusion Theory and PDE ConnectionsHidden Markov modelSemimartingale and General Stochastic…Stochastic partial differential equat…Mathematical Finance and Risk-Neutral…Rough path

Beyond the field

Where Stochastic Processes's ideas came from in other fields, and what they led to there: the strongest 12 of 113 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Stochastic Processes is a planet in Nousfarer

Its city has a tower for each of these 17 frameworks. Land, walk in, and climb one concept at a time.

Land on Stochastic Processes

Related fields

Fields its ideas are linked to

More in Probability Theory

See where Stochastic Processes sits on the Map of Human Knowledge