Time Series Analysis

A time series is a sequence of measurements of the same thing, taken in order over time: a country's quarterly output, a city's daily temperature, the price of a stock at each minute of trading, the number of passengers on an airline each month.

11 frameworks, from 1900 to today · Statistics · Mathematics

still a working view ended · each leads to its entry below

Overview

A time series is a sequence of measurements of the same thing, taken in order over time: a country's quarterly output, a city's daily temperature, the price of a stock at each minute of trading, the number of passengers on an airline each month1. What makes such data a distinct statistical problem is that the observations are not independent1. Today's value usually resembles yesterday's, and the order in which the numbers arrive carries information1. Time series analysis is the branch of statistics that builds models and methods for data of this kind—to describe how a series behaves, to forecast where it is going, to separate a signal from noise, and to test whether apparent relationships between series are real.

The field's history is a long argument about how much structure to impose. One camp treats a series as a realization of a random process with a fixed probabilistic law and tries to estimate that law; another decomposes it into interpretable parts—trend, season, cycle, irregular; a third lets the data itself, through flexible algorithms, decide the shape1. These approaches were born in different problems—astronomy, economics, engineering, finance—and they have repeatedly borrowed from and competed with one another.

Astronomy, Economics, and the First Decompositions

Systematic study of periodic variation began in the eighteenth and nineteenth centuries with astronomical and meteorological data. The key mathematical tool was the idea that a complicated periodic curve could be built from simple sine and cosine waves. Joseph Fourier's work on heat conduction, published in the early nineteenth century, supplied the mathematics; Arthur Schuster, working on sunspot and terrestrial-magnetism data, gave the practical instrument in the 1890s when he introduced the periodogram—a plot of how much variance in a series is attributable to each frequency. The periodogram's weakness was soon apparent: with a finite sample, its values fluctuate wildly even when no true periodicity exists, so distinguishing a real cycle from noise required careful distribution theory, developed over the following decades.

A parallel tradition grew in economics, where business-cycle researchers wanted to split a series into components. The standard device was the moving average: replacing each value by the average of a window of nearby values smooths away short-term noise and leaves the trend2. Warren Persons and others at Harvard in the early twentieth century built elaborate "business barometers" on such decompositions, and the classical scheme—trend, seasonal, cyclical, and irregular components, combined either additively or multiplicatively—became standard teaching. Its limitation was that the components were defined by the smoothing procedure rather than by any probability model, so there was no principled way to say how uncertain a decomposition was.

Stationarity and the Stochastic Process View

The decisive shift came when statisticians began treating a series as a random process rather than a deterministic curve plus error. The central concept is stationarity: a series is (weakly) stationary if its mean and variance are constant over time and the covariance between two observations depends only on the distance between them, not on when they occur34. Stationarity is what makes estimation possible—without it, there is no stable quantity to estimate from a finite stretch of data.

The Russian school made the foundational contributions. Andrei Kolmogorov's work in the 1930s and 1940s put the theory of stationary stochastic processes on axiomatic footing. Norbert Wiener, working on anti-aircraft fire control and signal processing during and after the Second World War, developed the parallel engineering theory of prediction and filtering. The synthesis appeared in 1949 as Extrapolation, Interpolation, and Smoothing of Stationary Time Series, Wiener's famously difficult book. Its practical message: if you know the correlation structure of a stationary process, you can compute the best linear forecast of future values. The problem was that the correlation structure had to be known, and in real data it had to be estimated.

Spectral Analysis and the Frequency Domain

The periodogram matured into a full spectral analysis. The spectrum of a stationary process describes how its variance is distributed across frequencies, and the Wiener–Khinchin theorem connects it to the autocovariance function as a Fourier transform pair. In the 1950s and 1960s, statisticians—notably Maurice Bartlett, who proposed smoothing the periodogram to reduce its variance, and later John Tukey, whose Fast Fourier Transform work with James Cooley in the mid-1960s made computation cheap—turned spectral estimation into a practical toolkit. The frequency-domain view proved indispensable in engineering (vibration analysis, communications) and geophysics (tides, seismic signals), and it remains one of the two great languages of the field, alongside the time-domain language of autoregression.

Autoregression, Moving Averages, and the Box–Jenkins Revolution

The time-domain counterpart is the idea that a series can be predicted from its own past. The autoregressive (AR) model expresses each value as a weighted sum of previous values plus a random shock; the moving-average (MA) model expresses it as a weighted sum of past shocks5. The Russian statistician and economist Eugen Slutsky had shown in the 1920s that moving averages of random numbers could mimic business cycles, a result that unsettled the idea that cycles proved anything about causes. The English statistician Udny Yule introduced the autoregressive scheme for sunspot and other data around the same time.

The synthesis that made these models usable came from George Box and Gwilym Jenkins, whose 1970 book Time Series Analysis: Forecasting and Control gave the field its most influential methodology. Their ARIMA framework—autoregressive integrated moving average—handled non-stationary series by differencing (subtracting each value from the previous one) until stationarity was achieved, then fitting AR and MA terms, and checking the fit by examining whether the residuals looked like white noise. The accompanying iterative cycle—identify, estimate, diagnose—was as important as the mathematics: it told practitioners how to work. The method's appeal was that it required only the series itself, no external explanatory variables, and it produced forecasts with defensible uncertainty intervals. It became the default approach in forecasting practice for a generation.

State Space and the Kalman Filter

A different strand came from control engineering. In 1960 Rudolf Kalman published a recursive solution to the problem of estimating the hidden state of a dynamic system from noisy measurements—the Kalman filter6. The idea is to maintain a probability distribution over the unknown state, update it as each new observation arrives, and predict forward6. It is exact for linear systems with Gaussian noise and astonishingly efficient, requiring only matrix arithmetic at each step.

Statisticians recognized that many classical models, including ARIMA, could be written in state-space form, with an unobserved state vector evolving over time and observations generated from it. This unification, pushed by engineers and by statisticians such as Andrew Harvey, allowed missing data, irregular sampling, and time-varying parameters to be handled within one framework. Harvey's structural time series models—explicitly modeling level, trend, seasonal, and irregular components as unobserved stochastic processes—gave the old classical decomposition a probabilistic foundation. The Kalman filter then estimated the components, and the decomposition acquired standard errors.

Multivariate Systems and Vector Autoregression

Real questions are usually about several series at once: does money growth cause inflation, does one market move another? The univariate toolkit could not answer them. Christopher Sims's 1980 paper introduced the vector autoregression (VAR), which treats a set of series symmetrically, each regressed on lags of itself and of all the others. The VAR was deliberately theory-light: rather than imposing economic structure, it let the data reveal dynamic relationships, and its impulse-response functions traced how a shock to one variable propagated through the system. It became a workhorse of empirical macroeconomics, and its descendants—structural VARs with identifying restrictions, Bayesian VARs with shrinkage priors for large systems—remain central.

Volatility: ARCH and GARCH

Financial returns posed a puzzle the standard models could not handle. Returns themselves are nearly unpredictable, but their volatility clusters: turbulent days follow turbulent days7. Robert Engle's 1982 ARCH model (autoregressive conditional heteroskedasticity) let the conditional variance depend on past squared shocks; Tim Bollerslev's 1986 GARCH generalization added lagged variances, capturing volatility persistence with few parameters87. These models made it possible to forecast risk, price options with time-varying volatility, and test whether volatility responds asymmetrically to good and bad news. The 2003 Nobel Prize to Engle and Clive Granger marked the field's arrival as a mature empirical science.

Cointegration and Error Correction

Granger's contribution was the discovery, with Robert Engle in 1987, of cointegration: two non-stationary series can wander individually yet stay tied together, so that some linear combination of them is stationary9. Regressing one on the other in the usual way produces spurious results—the famous "spurious regression" problem—unless the series are cointegrated. The error-correction model captures the adjustment: a deviation from the long-run relationship pulls the series back toward it10. This gave econometrics a rigorous way to model long-run equilibrium alongside short-run dynamics, and it reshaped empirical work on prices, exchange rates, and consumption.

Bayesian and Machine-Learning Turns

Bayesian methods entered time series early—Kalman filtering has a natural Bayesian reading—but became a mainstream alternative with the spread of Markov chain Monte Carlo in the 1990s. Bayesian structural models, dynamic linear models, and Bayesian VARs allowed prior information and hierarchical structure to be incorporated, and they handled parameter uncertainty more directly than plug-in estimates. The approach is now standard in forecasting competitions and central-bank modeling.

The most recent turn is algorithmic. Neural networks and other machine-learning methods—recurrent networks, gradient boosting, and later transformer-based models—can learn complex nonlinear and long-range dependencies without a specified model form. Their record is mixed: on many classical benchmarks, simple statistical methods remain hard to beat, and the field has learned to be skeptical of claims that flexible models automatically forecast better. But for very large datasets, high-frequency data, and problems with many related series, learned models have become serious competitors, and hybrid approaches that combine statistical structure with learned components are an active frontier.

The Present

Today the field is plural. ARIMA and exponential smoothing remain the reliable baseline; state-space models and the Kalman filter underpin everything from macroeconomic nowcasting to satellite tracking; spectral methods serve engineering and the physical sciences; GARCH-family models dominate financial risk; VARs and cointegration anchor empirical macroeconomics; and Bayesian and machine-learning methods expand the range of what can be modeled. The open questions are familiar in new dress: how to forecast when the underlying process shifts, how to combine many models and many data sources, how to quantify uncertainty honestly when models are misspecified, and how to tell whether a flexible algorithm has learned structure or merely memorized noise. The argument between structure and flexibility that has run through the field since the nineteenth century is not settled—it has simply acquired more powerful tools on both sides.

References

  1. Time series (introduction, Methods for analysis, Estimation, filtering, and smoothing), Wikipedia
  2. Moving average, Wikipedia
  3. Stationary ergodic process, Wikipedia
  4. Stationary process, Wikipedia
  5. Moving-average model, Wikipedia
  6. Kalman filter (introduction, History), Wikipedia
  7. Volatility clustering, Wikipedia
  8. Autoregressive conditional heteroskedasticity, Wikipedia
  9. Cointegration (introduction, History), Wikipedia
  10. Error correction model, Wikipedia

The frameworks of Time Series Analysis

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1900-1970

    Decomposition of time series

    How can a series be split into trend, season and noise?

    Fly to its tower in Nousfarer

  2. 1930-Present

    Stationary process

    When does a random process have a stable law to estimate?

    Fly to its tower in Nousfarer

  3. 1950-Present

    Spectral Analysis

    How is variance distributed across frequencies?

    Fly to its tower in Nousfarer

  4. 1960-Present

    State Space Models and Kalman Filtering

    How can a hidden state be tracked from noisy readings?

    Fly to its tower in Nousfarer

  5. 1960-Present

    Structural Time Series Models

    How can trend and season be modeled as random processes?

    Fly to its tower in Nousfarer

  6. 1970-Present

    Box–Jenkins method

    How can a series forecast itself from its own past?

    Fly to its tower in Nousfarer

  7. 1980-Present

    Vector Autoregression

    How do several series move and influence one another over time?

    Fly to its tower in Nousfarer

  8. 1982-Present
  9. 1987-Present

    Cointegration

    When do nonstationary series move together in the long run?

    Fly to its tower in Nousfarer

  10. 1990-Present

    Bayesian Time Series Analysis

    How should prior belief be updated with each new observation?

    Fly to its tower in Nousfarer

  11. 2010-Present

    Machine Learning for Time Series

    Can flexible algorithms learn patterns better than fixed models?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Decomposition of time seriesStationary processSpectral AnalysisState Space Models and Kalman Filteri…Structural Time Series ModelsBox–Jenkins methodVector AutoregressionAutoregressive conditional heterosked…CointegrationBayesian Time Series AnalysisMachine Learning for Time Series

Beyond the field

Where Time Series Analysis's ideas came from in other fields, and what they led to there: the strongest 12 of 53 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Time Series Analysis is a planet in Nousfarer

Its city has a tower for each of these 11 frameworks. Land, walk in, and climb one concept at a time.

Land on Time Series Analysis

Related fields

Fields its ideas are linked to

More in Statistics

See where Time Series Analysis sits on the Map of Human Knowledge