- Atlas
- Mathematics
- Stochastic Processes
Stochastic Processes
Stochastic processes are the branch of probability theory that studies systems evolving randomly in time.
17 frameworks, from 1906 to today · Probability Theory · Mathematics
- Markov chain 1906-Present
- Queueing Theory 1909-Present
- Random walk 1920-Present
- Wiener process 1923-Present
- Lévy process 1930-Present
- Measure-Theoretic Probability 1933-Present
- Large Deviations Theory 1938-Present
- Gaussian process 1940-Present
- Martingale 1940-Present
- Itô calculus 1944-Present
- Monte Carlo and Markov Chain Monte Carlo 1949-Present
- Diffusion Theory and PDE Connections 1950-Present
- Hidden Markov model 1960-Present
- Semimartingale and General Stochastic Integration Theory 1970-Present
- Stochastic partial differential equation 1970-Present
- Mathematical Finance and Risk-Neutral Pricing 1973-Present
- Rough path 1998-Present
Overview
Stochastic processes are the branch of probability theory that studies systems evolving randomly in time. Where classical probability asks how likely a single event is—a coin landing heads, a die showing six—a stochastic process asks how a random quantity unfolds: the jitter of a stock price across a trading day, the queue outside a ticket office, the diffusion of a drop of ink through water, the number of customers in a call centre at each minute. What is at stake is nothing less than a mathematical language for change under uncertainty, and that language now underpins physics, engineering, finance, biology, and statistics. The history of the subfield is the story of how a loose collection of practical problems—gambling, insurance, molecular motion—was forged into one of the most powerful and abstract branches of modern mathematics, and then fed back into the practical world on an enormous scale.
Gambling Problems and the First Markov Chains
Probability began in the seventeenth century with correspondence between Blaise Pascal and Pierre de Fermat about games of chance, and the "problem of points"—how to divide the stakes when a game is interrupted—produced the idea of expected value. For two centuries the subject dealt with independent trials: each coin flip, each roll of dice, unaffected by what came before. The first decisive break came from Andrey Markov, who around 1906 studied sequences of dependent trials. He asked whether the law of large numbers—the principle that long-run averages settle down—still held when each outcome depends on the previous one12. His answer was yes, provided the dependence has a special form: the future depends on the present state but not on the path taken to reach it1. This is the Markov property, and processes obeying it are Markov chains (discrete-time) or Markov processes (continuous-time). Markov demonstrated the idea concretely by analysing the alternation of vowels and consonants in Pushkin's verse, showing that a simple dependent model could reproduce statistical features of real text. The Markov property turned out to be the key that unlocked an entire theory: if the present state is all that matters, the process can be described by transition probabilities, and long-run behaviour can be computed from them.
Brownian Motion: From Pollen Grains to a Mathematical Object
In 1827 the botanist Robert Brown observed pollen grains suspended in water jittering ceaselessly under his microscope3. The cause—collisions with invisible water molecules—was not settled until Einstein's 1905 paper, which derived the diffusion equation from molecular assumptions and predicted that the mean squared displacement of a particle grows linearly with time. Einstein's work gave physicists a tool for estimating molecular sizes. Independently, Louis Bachelier in his 1900 thesis on speculation had already used a similar continuous random walk to model stock prices, though his work was largely ignored for decades13.
The mathematical object itself—Brownian motion, also called the Wiener process—was constructed rigorously by Norbert Wiener in the 1920s14. Wiener's construction showed that a continuous function of time could be random at every scale, with independent increments over disjoint time intervals and Gaussian distributions1. The path is continuous but nowhere differentiable: it has infinite variation, so ordinary calculus cannot integrate against it. This was the obstacle that would define the next era.
Measure Theory Rebuilds Probability
Before the 1930s, probability was a somewhat informal subject, and the construction of continuous-time processes was ad hoc. Andrey Kolmogorov's 1933 Grundbegriffe der Wahrscheinlichkeitsrechnung changed everything1. Kolmogorov axiomatised probability as a branch of measure theory: a probability space is a set of outcomes, a collection of events (a sigma-algebra), and a countably additive measure assigning each event a probability. Crucially, he gave a theorem—the Kolmogorov extension theorem—showing that a whole family of finite-dimensional distributions, consistent with one another, determines a stochastic process. This made it possible to construct processes rigorously from their finite-dimensional laws1. The measure-theoretic framework also gave precise meaning to conditional expectation, which had previously been a vague notion. Conditional expectation became the foundation for martingale theory.
Martingales and the Theory of Fair Games
A martingale is a process whose expected future value, given everything observed so far, equals its present value—the mathematical idealisation of a fair game15. The concept appeared in gambling literature (the "martingale" betting system) but was transformed into a powerful tool by Joseph Doob in the 1940s and 1950s. Doob proved the martingale convergence theorem: under mild conditions, a martingale converges almost surely. He also developed optional stopping theorems, which say when a gambler's expected fortune at a stopping time equals the starting fortune. These results were startling because they applied to very general processes, not just sums of independent variables. Martingale theory became the engine of modern probability: it gave proofs of the strong law of large numbers, the Radon–Nikodym theorem in probabilistic form, and later the foundations of stochastic integration. Paul Lévy contributed deeply to the same circle of ideas, and the Lévy–Doob martingale theory of the mid-twentieth century is one of the great achievements of the subject.
Stochastic Calculus: Integrating Against Noise
The central problem of the 1940s was how to define an integral against Brownian motion. Ordinary Riemann–Stieltjes integration fails because Brownian paths have infinite variation6. Kiyoshi Itô, in a series of papers from 1944 onward, defined a new integral—the Itô integral—by taking limits in mean square, and in doing so discovered that the chain rule of ordinary calculus must be corrected by an extra term. This is Itô's lemma: for a smooth function f of a Brownian motion B, the differential of f(B) includes a second-derivative term proportional to dt7. The correction arises because Brownian increments squared accumulate to time. Itô's calculus made it possible to write down and solve stochastic differential equations (SDEs)—equations like dX = μ(X)dt + σ(X)dB—which describe diffusions: particles in fluids, interest rates, population sizes18. The theory was extended by Itô and others to general semimartingales, the largest class of processes for which a sensible stochastic integral exists. This general theory, developed through the 1960s and 1970s (with major contributions from the Strasbourg school around Paul-André Meyer), unified Brownian motion, jump processes, and martingales under one framework.
Diffusion Theory and the Connection to Partial Differential Equations
Itô's calculus revealed a deep link between stochastic processes and partial differential equations. The generator of a diffusion—an operator like the Laplacian plus a drift term—governs the expected values of functions of the process, and the Feynman–Kac formula expresses solutions of parabolic PDEs as expectations over diffusion paths. This connection, developed in the mid-twentieth century, meant that probabilistic methods could solve PDE problems and vice versa9. It also gave rise to the theory of Lévy processes—processes with stationary independent increments that may jump—generalising Brownian motion and providing models for insurance claims, asset prices with sudden crashes, and queuing systems.
Queues, Point Processes, and the Applied Tradition
Alongside the abstract theory, a parallel applied tradition grew from practical problems. Agner Krarup Erlang, working for the Copenhagen Telephone Company around 1910, modelled telephone call arrivals as a Poisson process—a point process in which events occur randomly and independently at a constant rate—and derived formulas for waiting times and blocking probabilities110. This founded queueing theory10. The Poisson process and its generalisations (renewal processes, Cox processes) became the standard models for arrivals of customers, radioactive decays, and neural spikes. The applied tradition and the abstract tradition met in the mid-twentieth century: queueing networks, Markov decision processes, and reliability theory all drew on Markov chain theory and martingale methods.
Random Walks and Random Environments
The simple random walk—a particle stepping left or right with equal probability—is the discrete ancestor of Brownian motion and one of the most studied objects in probability111. Its recurrence properties (in one and two dimensions a random walk returns to its starting point with probability one; in three dimensions it may escape) were settled in the 1920s1. In the 1970s and 1980s, attention turned to random walks in random environments: the step probabilities themselves are random, fixed once and for all. This models transport through disordered media—electrons in impure metals, fluid through porous rock. The field produced striking phenomena, such as Sinai's result that a one-dimensional random walk in a random environment moves anomalously slowly, and it remains an active area connecting probability, statistical physics, and ergodic theory.
Large Deviations: The Mathematics of Rare Events
How likely is a very unusual outcome? The classical limit theorems describe typical behaviour; large deviations theory describes the exponentially small probabilities of rare events12. The theory grew from the work of Harald Cramér in the 1930s on insurance risk and was developed into a general framework by S. R. S. Varadhan and others in the 1960s and 1970s. The key object is a rate function that measures the "cost" of a deviation13. Large deviations theory has become essential in statistical mechanics (where it connects to entropy), in information theory (Sanov's theorem), and in risk management, where rare catastrophic events matter most.
Gaussian Processes and Random Fields
A Gaussian process is a stochastic process whose finite-dimensional distributions are multivariate normal14. Brownian motion is the canonical example, but the class is far larger and includes processes indexed by space as well as time—random fields1415. Gaussian random fields model spatial phenomena: the height of a rough surface, the temperature across a region, the fluctuations of the cosmic microwave background1617. The theory was developed through the mid-twentieth century, with important contributions from Kolmogorov and later from geostatistics (the "kriging" method of Danie Krige and Georges Matheron in the 1950s–60s). In modern machine learning, Gaussian processes serve as priors over functions for Bayesian regression, a development that began with work by David MacKay and others in the 1990s.
Hidden Markov Models and State-Space Models
In many real systems the state is not directly observed—only noisy measurements of it. Hidden Markov models (HMMs) formalise this: an underlying Markov chain evolves unseen, and at each step an observation is generated from the current state18. The forward–backward algorithm, developed in the 1960s and 1970s (with roots in the work of Leonard Baum and colleagues at the Institute for Defense Analyses), computes the likelihood of observations and the posterior distribution of hidden states. The Viterbi algorithm finds the most likely state sequence19. These methods transformed speech recognition in the 1970s and 1980s, and later bioinformatics (gene finding), finance, and signal processing18. The broader class of state-space models, including the Kalman filter for linear Gaussian systems (Rudolf Kalman, 1960), became standard tools in engineering and econometrics.
Monte Carlo and Markov Chain Monte Carlo
The Monte Carlo method—using random sampling to estimate quantities—emerged at Los Alamos in the 1940s, associated with Stanislaw Ulam, John von Neumann, and Nicholas Metropolis. The idea is simple: to estimate an integral, sample random points and average. But sampling from complicated distributions is hard. Markov chain Monte Carlo (MCMC) solves this by constructing a Markov chain whose stationary distribution is the target20. The Metropolis algorithm (1953) was the first; the Metropolis–Hastings algorithm (W. K. Hastings, 1970) generalised it. In the 1980s and 1990s, MCMC revolutionised Bayesian statistics: the Gibbs sampler (popularised by Stuart Geman and Donald Geman in 1984 and by Alan Gelfand and Adrian Smith in 1990) made it possible to fit complex hierarchical models. Modern computational statistics is unimaginable without MCMC, and the method's descendants—Hamiltonian Monte Carlo, variational inference—continue to drive machine learning and data science.
Mathematical Finance and Risk-Neutral Pricing
In 1973 Fischer Black, Myron Scholes, and (independently) Robert Merton published option-pricing formulas that used Itô calculus to model stock prices as geometric Brownian motion. The Black–Scholes formula expresses the price of a European call option in terms of the stock price, strike price, volatility, and interest rate. The deeper insight, developed by Harrison, Kreps, and Pliska in the late 1970s and early 1980s, is that arbitrage-free pricing is equivalent to the existence of a probability measure—the risk-neutral measure—under which discounted asset prices are martingales. This martingale approach to finance unified the field and made it possible to price complex derivatives. The 1987 crash and later crises exposed the limitations of the Gaussian model, spurring the development of jump-diffusion models (Merton, 1976), stochastic volatility models, and Lévy-process-based finance.
Stochastic Partial Differential Equations
The final major frontier is the study of random fields that evolve according to partial differential equations with random forcing—stochastic PDEs21. The prototype is the stochastic heat equation, driven by space-time white noise. These equations arise in physics (fluctuations in fluids, phase transitions), biology (population genetics), and engineering. The mathematical difficulties are severe: solutions may be distributions rather than functions, and nonlinear terms require renormalisation. The theory was developed from the 1970s onward, with major advances by John Walsh (martingale measures), and in the 2000s and 2010s by Martin Hairer, whose theory of regularity structures (for which he received the Fields Medal in 2014) provided a general framework for making sense of singular stochastic PDEs. This remains one of the most active areas of probability today.
The Present: Unified Theory and Computational Reach
Stochastic processes today are a mature but still expanding field. The measure-theoretic and semimartingale framework provides a unified language; Itô calculus and its generalisations are standard tools; and the computational revolution—MCMC, particle filters, Gaussian process regression, stochastic gradient methods—has made stochastic processes central to machine learning and artificial intelligence. Open questions remain: the rigorous understanding of singular stochastic PDEs, the behaviour of random walks in high-dimensional random environments, the foundations of stochastic analysis on rough paths (Terry Lyons's theory, developed from the 1990s), and the interface between stochastic processes and quantum field theory. The field that began with gamblers' dice and pollen grains now models everything from neural activity to financial markets to the large-scale structure of the universe, and its history is a case study in how abstract mathematics and concrete problems can transform each other1.
References
- Stochastic process (Markov processes, Markov processes and chains, Wiener process, Birth of modern probability theory, Mathematical construction, Martingale, Stochastic processes after World War II, Poisson process, Random walk, Random walks, introduction), Wikipedia
- Markov chain (History), Wikipedia
- Brownian motion (introduction, History), Wikipedia
- Wiener process, Wikipedia
- Martingale (probability theory), Wikipedia
- Itô calculus, Wikipedia
- Stochastic calculus, Wikipedia
- Stochastic differential equation, Wikipedia
- Feynman–Kac formula, Wikipedia
- Queueing theory (introduction, History), Wikipedia
- Random walk, Wikipedia
- Large deviations theory, Wikipedia
- Sanov's theorem, Wikipedia
- Gaussian process, Wikipedia
- Random field, Wikipedia
- Large deviations of Gaussian random functions, Wikipedia
- Gaussian random field, Wikipedia
- Hidden Markov model (introduction, History), Wikipedia
- Viterbi algorithm, Wikipedia
- Markov chain Monte Carlo, Wikipedia
- Stochastic partial differential equation, Wikipedia
The frameworks of Stochastic Processes
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1906-Present
Markov chain
How do systems evolve when the future depends only on the present?
- 1909-Present
- 1920-Present
- 1923-Present
Wiener process
What is the mathematical model of continuous random motion?
- 1930-Present
- 1933-Present
- 1938-Present
- 1940-Present
- 1940-Present
Martingale
When is a game fair?
About Martingale
Martingale theory addresses a deceptively simple question: if a process is 'fair' on average at every step, what can we say about its long-run behavior? A martingale is a stochastic process where the expected future value, given all past information, equals the present value. This property—no drift, no bias—turns out to be surprisingly powerful. It provides a unifying framework for analyzing sequences of random variables that are neither independent nor Markovian, yet obey a conditional fairness constraint. In stochastic processes, martingale theory is the lens through which we study the interplay between information, expectation, and time, revealing deep connections between gambling, finance, and the very structure of randomness.Core CurriculumA first course in martingale theory for stochastic processes typically builds around these fundamental concepts in logical order:- Conditional Expectation: The mathematical tool that formalizes 'best guess given available information.' It is the engine of martingale theory, defining how expectations update as new information arrives.
- Filtration: A growing sequence of sigma-algebras representing the flow of information over time. Filtrations encode what is known at each moment and are essential for defining conditional expectations and martingales.
- Martingale: A stochastic process adapted to a filtration such that almost surely. This is the defining property of a 'fair game.'
- Submartingale and Supermartingale: Processes where the conditional expectation is greater than or equal to (submartingale) or less than or equal to (supermartingale) the current value. These model favorable and unfavorable games, respectively.
- Stopping Time: A random time such that the event is known by time . Stopping times capture when a gambler might decide to quit, and are central to analyzing martingale behavior.
- Optional Stopping Theorem: A cornerstone result stating that, under certain conditions, the expected value of a martingale at a stopping time equals its initial value. This formalizes the idea that no betting strategy can beat a fair game.
- Martingale Convergence Theorem: A martingale that is bounded in converges almost surely to a finite limit. This theorem reveals that even without independence, martingales cannot oscillate forever; they must settle down.
- Doob's Maximal Inequality: An inequality bounding the probability that the maximum of a martingale exceeds a threshold, in terms of its terminal expectation. It is a key tool for controlling extreme values.
- **Doob's Inequality**: A refinement of the maximal inequality for , relating the norm of the maximum to the norm of the terminal value.
- Martingale Transform: A discrete-time stochastic integral, where a predictable process is used to 'bet' on the increments of a martingale. Martingale transforms preserve the martingale property under integrability conditions.
- Quadratic Variation: For a martingale, the quadratic variation measures the accumulated squared increments. It is a fundamental object in stochastic calculus and central to the analysis of path properties.
- Martingale Representation Theorem: In certain settings (e.g., Brownian filtration), every martingale can be represented as a stochastic integral. This theorem underpins much of mathematical finance.
Intellectual HistoryMartingale theory emerged in the mid-20th century, building on earlier work in probability and stochastic processes. The term 'martingale' itself has a long history in gambling, referring to a betting strategy where one doubles the stake after a loss, aiming to recover all previous losses with a single win. The mathematical theory, however, was developed to understand the limits of such strategies and the nature of fair games.Precursors: The concept of a fair game appears implicitly in the work of Louis Bachelier (1900) on the theory of speculation, where he modeled stock prices as a process with independent increments—a precursor to both Brownian motion and martingales. Andrey Kolmogorov's axiomatization of probability (1933) provided the measure-theoretic foundation for conditional expectation and filtrations, which are essential for martingale theory. Paul Lévy studied processes with independent increments and introduced the concept of a 'martingale' in a 1934 paper, though his definition was not yet fully formalized.Key Figures and Seminal Works: The modern theory was largely created by Joseph L. Doob in the 1940s and 1950s. Doob's 1953 book Stochastic Processes is the foundational text, systematically developing martingale theory within the framework of measure-theoretic probability. He proved the fundamental convergence theorems, the optional stopping theorem, and the maximal inequalities that bear his name. Doob's work established martingales as a central tool in stochastic processes, showing how they unify and extend results for sums of independent random variables and Markov chains.J. L. Doob (1910–2004) was an American mathematician who spent most of his career at the University of Illinois. His 1953 monograph remains a landmark, and his 1984 book Classical Potential Theory and Its Probabilistic Counterpart revealed deep connections between martingales and harmonic functions.Later Codifiers and Transmitters: In the 1960s and 1970s, martingale theory was absorbed into the broader framework of semimartingale theory and stochastic calculus, particularly through the work of Kiyosi Itô and Paul-André Meyer. Meyer's 1966 book Probability and Potentials (with Claude Dellacherie) integrated martingales into the theory of stochastic processes, and his 'Strasbourg school' developed the general theory of processes, where martingales play a central role. David Williams' 1991 book Probability with Martingales is a widely used modern introduction.Successor and Parallel Frameworks: Martingale theory coexists with and feeds into stochastic calculus, where it provides the foundation for stochastic integrals and Itô's formula. Semimartingale theory generalizes martingales to processes that can be decomposed into a martingale and a finite-variation part, forming the natural domain for stochastic integration. Stationary processes and Markov chains are distinct frameworks, but martingale methods are often applied to analyze them (e.g., using martingale central limit theorems for Markov chains).Legacy: Martingale theory transformed stochastic processes by providing a unified framework for analyzing dependence without requiring independence or the Markov property. Its key insights—that fair games have no systematic drift, that stopping times cannot create value, and that bounded martingales must converge—have become essential tools in probability, statistics, finance, and beyond. The theory remains a core part of the stochastic processes curriculum, serving as a bridge between elementary probability and advanced stochastic calculus.The 12 concepts its tower teaches
- Conditional Expectation
- Doob's L^p Inequality
- Doob's Maximal Inequality
- Filtration
- Martingale
- Martingale Convergence Theorem
- Martingale Representation Theorem
- Martingale Transform
- Optional Stopping Theorem
- Stopping Time
- Submartingale
- Supermartingale
- 1944-Present
- 1949-Present
Monte Carlo and Markov Chain Monte Carlo
How can we sample from complex distributions?
- 1950-Present
Diffusion Theory and PDE Connections
How are random motions linked to differential equations?
- 1970-Present
Semimartingale and General Stochastic Integration Theory
What is the most general theory of stochastic integration?
- 1970-Present
Stochastic partial differential equation
How do random fields evolve in space and time?
- 1973-Present
Mathematical Finance and Risk-Neutral Pricing
How should we price risky assets?
- 1998-Present
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Hidden Markov model grew out of Markov chain
Hidden Markov models are Markov chains observed through a noisy channel; they were developed as an extension of Markov chain theory for inference.
Wiener process grew out of Random walk
Brownian motion (the Wiener process) is the continuous-time limit of the simple random walk; Wiener's construction formalised this scaling limit.
Carried by Norbert Wiener.
Lévy process grew out of Wiener process
Lévy processes generalise Brownian motion by allowing jumps while keeping stationary independent increments; Paul Lévy developed the theory building on Wiener's work.
Carried by Paul Lévy.
Gaussian process grew out of Wiener process
Gaussian processes generalise Brownian motion, which is a Gaussian process with a specific covariance; the theory of Gaussian processes was developed from the 1940s onward.
Itô calculus influenced by Wiener process
Itô calculus was created to define integrals against Brownian motion, whose paths have infinite variation; Itô's integral and lemma are built directly on the Wiener process.
Carried by Kiyoshi Itô.
Martingale grew out of Measure-Theoretic Probability
Martingale theory was built on Kolmogorov's measure-theoretic conditional expectation; Doob developed the theory rigorously in the 1940s and 1950s.
Carried by Joseph Doob; Andrey Kolmogorov.
Diffusion Theory and PDE Connections grew out of Itô calculus
The connection between stochastic differential equations and partial differential equations (Feynman–Kac, generator theory) was developed using Itô calculus.
Carried by Kiyoshi Itô; Richard Feynman; Mark Kac.
Semimartingale and General Stochastic Integration Theory grew out of Itô calculus
General stochastic integration theory for semimartingales extended Itô's calculus beyond Brownian motion; the Strasbourg school around Meyer developed this general theory.
Carried by Paul-André Meyer; Kiyoshi Itô.
Stochastic partial differential equation grew out of Itô calculus
Stochastic partial differential equations extend SDEs to infinite-dimensional settings; early SPDEs were driven by space-time white noise using Itô calculus.
Mathematical Finance and Risk-Neutral Pricing influenced by Itô calculus
The Black–Scholes–Merton option pricing formula uses Itô calculus to model stock prices as geometric Brownian motion and derive the Black–Scholes PDE.
Carried by Fischer Black; Myron Scholes; Robert Merton.
Monte Carlo and Markov Chain Monte Carlo influenced by Markov chain
Markov chain Monte Carlo methods use Markov chains to sample from probability distributions; the Metropolis algorithm (1953) and later MCMC methods rely on Markov chain theory.
Carried by Nicholas Metropolis.
Martingale influenced by Wiener process
Brownian motion is the canonical continuous martingale; Lévy's analysis of Brownian motion contributed to the development of martingale theory.
Carried by Paul Lévy; Joseph Doob.
Diffusion Theory and PDE Connections influenced by Wiener process
Diffusion theory links Brownian motion to the heat equation; the probabilistic representation of PDE solutions builds on the Wiener process.
Large Deviations Theory grew out of Measure-Theoretic Probability
Large deviations theory arose within measure-theoretic probability, using its tools to study rare events; early work by Cramér was later formalised in the measure-theoretic setting.
Carried by Harald Cramér.
Itô calculus influenced by Martingale
Itô's stochastic integral was later extended to semimartingales, a class defined via martingale theory; martingale techniques are central to stochastic integration.
Carried by Kiyoshi Itô.
Semimartingale and General Stochastic Integration Theory influenced by Martingale
Semimartingale theory relies on martingale decomposition theorems; the classification of semimartingales uses martingale theory.
Carried by Paul-André Meyer.
Mathematical Finance and Risk-Neutral Pricing influenced by Martingale
Risk-neutral pricing relies on martingale theory; the fundamental theorem of asset pricing characterises absence of arbitrage in terms of martingales.
Rough path influenced by Itô calculus
Rough path theory was developed to give a robust theory of controlled differential equations driven by rough signals, extending Itô integration to paths rougher than Brownian motion.
Carried by Terry Lyons.
Rough path influenced by Semimartingale and General Stochastic Integration Theory
Rough path theory addresses limitations of classical stochastic integration for paths with infinite variation, providing a deterministic alternative to semimartingale integration.
Carried by Terry Lyons.
Beyond the field
Where Stochastic Processes's ideas came from in other fields, and what they led to there: the strongest 12 of 113 links.
Markov chain influenced Queueing Theory in Stochastic Modeling
Erlang's queueing analysis and later queueing theory rest on Markov chains: birth-death models of arrivals and service are continuous-time Markov chains, and the memoryless property is the modelling core.
Carried by A. K. Erlang; Andrey Markov.
Markov chain led to Markov decision process in Stochastic Modeling
The Markov decision process is a direct extension of the Markov chain: Bellman added actions and rewards to Markov's transition model, giving a controlled chain optimised by dynamic programming.
Carried by Richard Bellman; Andrey Markov.
Markov chain influenced Markov decision process in Planning And Search
The Markov chain, formalised by Markov from 1906, is the stochastic substrate of the MDP. Bellman's 1957 formulation added decisions and rewards to the Markov transition structure.
Carried by Andrey Markov; Richard Bellman.
Markov chain influenced Markov chain Monte Carlo in Randomized Algorithms
The same Markov chain theory, stated under the Stochastic Processes label, underpins the Metropolis algorithm's construction and convergence arguments.
Carried by Andrey Markov; Nicholas Metropolis.
Markov chain influenced Markov Chain Monte Carlo in Markov Processes
MCMC simulates a Markov chain whose stationary distribution is the target; the Metropolis algorithm and later Hastings and Gibbs work rest directly on Markov chain theory.
Carried by Nicholas Metropolis; W. Keith Hastings.
Markov chain influenced Markov chain Monte Carlo in Statistical Inference
The stochastic-process formulation of Markov chains, including ergodicity and stationary distributions, is the theoretical basis on which MCMC convergence arguments rest.
Carried by Andrey Markov.
Markov chain influenced Markov Chain Monte Carlo in Bayesian Econometrics
Markov chains in stochastic processes supply the transition-kernel framework that MCMC relies on. The Metropolis-Hastings algorithm is defined as a Markov chain with a specified stationary distribution.
Carried by Andrey Markov.
Markov chain influenced Queueing Theory in Service Operations
Queueing theory is built on Markov chains: Erlang's waiting-line analysis and later queueing models use Markov processes to describe arrivals and service. The Markov property is the central probabilistic assumption of the field.
Carried by Andrey Markov; Agner Krarup Erlang.
Markov chain led to Markov decision process in Markov Processes
MDPs extend Markov chains by adding controls and rewards, and the theory of Markov chains supplies their transition probabilities and ergodic properties.
Carried by Richard Bellman.
Markov chain led to Ergodic Theory and Mixing Times in Markov Processes
Mixing times are a property of Markov chains: the field studies how fast a chain's distribution approaches its stationary distribution, building on Markov's and later ergodic work on chains.
Carried by Andrey Markov.
Markov chain led to Markov Chain Monte Carlo in Bayesian Statistics
The stochastic-processes entry for Markov chains is the same underlying theory that MCMC exploits: a chain whose long-run distribution is the target. MCMC grew out of that theory via the Metropolis algorithm.
Carried by Andrey Markov.
Markov chain led to Hidden Markov model in Markov Processes
HMMs extend Markov chain theory by adding an observation layer, and their inference procedures use the Markov property and transition matrices directly.
Carried by Leonard E. Baum.

Stochastic Processes is a planet in Nousfarer
Its city has a tower for each of these 17 frameworks. Land, walk in, and climb one concept at a time.
Land on Stochastic ProcessesRelated fields
Fields its ideas are linked to
- Markov Processes 18 links
- Probability Theory 15 links
- Stochastic Analysis 13 links
- Derivatives Pricing 10 links
- Measure Theoretic Probability 10 links
- Stochastic Modeling 6 links
- Derivatives And Risk Management 3 links
- Fixed Income 3 links
- Asset Pricing Theory 3 links
- Planning And Search 2 links
- Bayesian Econometrics 2 links
- Time Series Econometrics 2 links
More in Probability Theory
See where Stochastic Processes sits on the Map of Human Knowledge