Signal Processing

Signal processing is the engineering of extracting, shaping, and interpreting information carried in signals—quantities that vary over time or space.

13 frameworks, from 1920 to today · Electrical Engineering · Engineering

still a working view ended · each leads to its entry below

Overview

Signal processing is the engineering of extracting, shaping, and interpreting information carried in signals—quantities that vary over time or space1. A signal might be a voltage from a microphone, a radio wave, a seismic reading, a medical image, or a stream of bits from a sensor1. The field's practitioners build mathematical tools and physical systems that filter out noise, compress data, detect patterns, reconstruct images, and estimate what cannot be measured directly. What is at stake is fundamental: almost every technology that senses, communicates, or controls the physical world depends on the ability to separate signal from noise and to represent information efficiently. The history of signal processing is a story of increasingly powerful abstractions—from circuits to algorithms to statistical models to learned representations—each solving problems the previous approach could not.

Analog Filter Networks and the Birth of a Discipline

Signal processing began as a branch of circuit theory. In the early twentieth century, electrical engineers faced a practical problem: telephone and radio systems needed to separate frequency bands, suppress interference, and shape signals. The tools available were inductors, capacitors, and resistors arranged in networks. The question was how to design a network with a desired frequency response.

The key insight came from the recognition that filter behavior could be described mathematically before a circuit was built. In the 1920s and 1930s, engineers at Bell Telephone Laboratories and elsewhere developed the theory of image-parameter filters, which treated networks as cascades of symmetric sections and predicted passbands and stopbands from the sections' characteristic impedances. George Campbell in the United States and Otto Zobel in Germany contributed to this framework. A parallel development, the insertion-loss method, designed filters by specifying the attenuation they should provide between source and load; Sidney Darlington's work in the late 1930s showed that any physically realizable filter could be represented as a lossless network terminated in a resistor.

These were not abstract exercises. The telephone network needed channel filters to separate voice channels in carrier systems; radio receivers needed bandpass filters to select stations. The analog filter network became the first standardized signal-processing component, and its design theory—approximation of a desired response by rational functions, synthesis of a circuit realizing that function—established a pattern the field would follow repeatedly: specify what you want mathematically, then find a structure that delivers it.

Sampling, Reconstruction, and the Digital Turn

The decisive break came with the realization that signals could be represented as sequences of numbers rather than continuous voltages. The sampling theorem—often associated with Claude Shannon's 1949 paper but developed earlier by Harry Nyquist at Bell Labs in the 1920s and by Vladimir Kotelnikov in the Soviet Union in the 1930s—states that a signal bandlimited to a maximum frequency f can be perfectly reconstructed from samples taken at more than twice that frequency. This was a profound claim: continuous reality could be captured exactly by discrete numbers, provided you sampled fast enough.

The theorem mattered because it made digital processing possible in principle. If a signal is just a sequence of numbers, then any operation on it—filtering, modulation, detection—can be performed by arithmetic rather than by circuits. The practical obstacle was speed and cost: early digital computers were far too slow for real-time signal processing. The change came gradually through the 1950s and 1960s as digital hardware improved, and then rapidly with the advent of inexpensive microprocessors in the 1970s.

The first major digital signal-processing application was in telecommunications and in geophysics, where seismic data were processed on mainframe computers to locate oil deposits. By the late 1970s, specialized digital signal processor chips—notably Texas Instruments' TMS320 family, introduced in the early 1980s—made real-time digital filtering affordable. The sampling theorem had moved from a theoretical curiosity to the foundation of a trillion-dollar industry.

The Fast Fourier Transform and Spectral Analysis

The Fourier transform—which decomposes a signal into its constituent frequencies—had been known since the early nineteenth century, and its discrete counterpart, the discrete Fourier transform (DFT), was computable in principle. But direct computation of an N-point DFT requires on the order of N² operations, which for large N was prohibitively slow2.

In 1965, James Cooley of IBM and John Tukey of Bell Labs published an algorithm that computed the DFT in roughly N log N operations by recursively splitting the transform into smaller transforms. The fast Fourier transform (FFT) was not entirely new—Carl Friedrich Gauss had used a similar idea in the early nineteenth century, and others had rediscovered it—but Cooley and Tukey's paper arrived at the moment when digital computers made it practical, and it spread rapidly. The FFT transformed spectral analysis from a theoretical tool into a routine computation. Radar, sonar, speech analysis, vibration monitoring, and medical imaging all came to depend on it. The FFT also enabled a new approach to filtering: instead of convolving a signal with a filter's impulse response directly, one could transform both to the frequency domain, multiply, and transform back—often far faster for long signals.

Digital Filter Design and the State-Space Revolution

With digital processing came the need to design digital filters—algorithms that shape a sampled signal's frequency content. Two broad classes emerged. Finite impulse response (FIR) filters compute each output from a finite window of past inputs; they can be designed to have exactly linear phase, which preserves waveform shape, and they are always stable. Infinite impulse response (IIR) filters use feedback, achieving sharp frequency selectivity with fewer computations but risking instability and nonlinear phase. The design problem—choosing coefficients to approximate a desired frequency response—was solved through approximation theory: the Remez exchange algorithm for FIR filters, analog prototype transformations for IIR filters. By the 1970s, digital filter design was a mature engineering discipline with standard textbooks and computer-aided design tools.

A parallel development addressed a different problem: estimating the state of a dynamic system from noisy measurements. In 1960, Rudolf Kalman published a recursive solution to the linear quadratic estimation problem3. The Kalman filter maintains an estimate of a system's state and its uncertainty, updating both as new measurements arrive3. It is optimal for linear systems with Gaussian noise, and it is remarkably efficient—each update requires only matrix arithmetic, not the inversion of growing covariance matrices. The Kalman filter became essential in aerospace (Apollo's navigation computer used it), in control systems, and eventually in GPS receivers, robotics, and econometrics. Its later extensions—the extended Kalman filter for nonlinear systems, the unscented transform, particle filters—addressed cases where linearity or Gaussianity fails.

Statistical Signal Processing: From Deterministic to Probabilistic

Through the mid-twentieth century, signal processing was largely deterministic: given a signal and a filter, predict the output. But real signals are random—speech varies unpredictably, radar returns are corrupted by noise, communication channels introduce random distortions. The statistical approach, developed from the 1940s onward, treated signals as random processes and asked how to make optimal decisions under uncertainty.

Norbert Wiener's work during and after World War II on optimal filtering—finding the filter that minimizes mean-squared error between a desired signal and its estimate—established the framework4. Andrey Kolmogorov independently developed related theory in the Soviet Union. The Wiener filter assumes stationary signals and known statistics; later work extended to nonstationary cases and to adaptive implementations.

Statistical detection theory, developed by engineers and statisticians in the 1950s and 1960s, asked a complementary question: given a noisy observation, what is the best decision about what signal is present? The Neyman–Pearson lemma, the likelihood ratio test, and the matched filter (which maximizes signal-to-noise ratio for a known waveform) became standard tools. These ideas were driven by radar and sonar during the war and by digital communications afterward. Statistical signal processing also gave rise to parameter estimation theory—how to estimate unknown quantities from noisy data with minimum variance—and to spectral estimation methods that go beyond the FFT's limitations for short or noisy records.

Adaptive Filtering and Learning from Data

A fixed filter assumes the signal's statistics are known and constant. But in many applications—echo cancellation in telephone lines, channel equalization in modems, noise cancellation in headphones—the statistics change. Adaptive filters adjust their coefficients continuously based on the incoming data.

The least mean squares (LMS) algorithm, developed by Bernard Widrow and Marcian Hoff in 1960, is strikingly simple: at each step, adjust the filter coefficients in the direction that reduces the current error5. It converges slowly but is robust and cheap. The recursive least squares (RLS) algorithm converges faster but costs more computation6. These methods, and later variants, made adaptive filtering a practical reality. Echo cancellers in the telephone network, adaptive equalizers in high-speed modems, and active noise cancellation in consumer products all descend from this line of work. Adaptive filtering also connected signal processing to machine learning: the LMS algorithm is a form of stochastic gradient descent, and the distinction between the two fields has blurred over time.

Multirate Systems, Filter Banks, and Wavelets

A multirate system processes a signal at more than one sampling rate—decimating (downsampling) to reduce data volume, interpolating (upsampling) to increase it. Filter banks split a signal into subbands, process each separately, and recombine7. These ideas emerged in the 1970s and 1980s, driven by speech coding and subband coding of audio.

The theoretical breakthrough came with the recognition that filter banks could be designed to achieve perfect reconstruction—the original signal recovered exactly from its subbands—and that this required specific algebraic relationships among the filters. The quadrature mirror filter (QMF) structure, developed in the late 1970s, was an early solution; later work by P. P. Vaidyanathan, Martin Vetterli, and others generalized the theory.

This work converged with a parallel development in mathematics: wavelets. In the 1980s, Jean Morlet and Alex Grossmann in France, and later Ingrid Daubechies, Stéphane Mallat, and others, developed a theory of multiresolution analysis—representing a signal at multiple scales simultaneously. Wavelets are functions that are localized in both time and frequency, unlike sinusoids, which are localized in frequency but spread across all time. This makes them well suited to signals with sharp transients or nonstationary behavior—images with edges, seismic signals with sudden arrivals, audio with percussive sounds. The discrete wavelet transform, computable by a filter bank, became a standard tool in image compression (the JPEG 2000 standard uses wavelets), denoising, and fault detection. The connection between filter banks and wavelets was a rare moment of convergence between engineering and pure mathematics.

Sparse Recovery and Compressed Sensing

The sampling theorem says you must sample at twice the highest frequency. But what if the signal is sparse—meaning it can be represented with few nonzero coefficients in some basis? In 2006, Emmanuel Candès, Justin Romberg, and Terence Tao, and independently David Donoho, showed that a sparse signal can be recovered from far fewer measurements than the sampling theorem requires, provided the measurements are sufficiently incoherent with the sparsity basis and recovery is performed by an optimization that minimizes the number of nonzero coefficients (convexly relaxed to an ℓ₁ norm minimization).

Compressed sensing, as it became known, was a shock: it suggested that the standard pipeline of sample-then-compress was wasteful, and that one could acquire compressed measurements directly. The idea had practical impact in medical imaging (reducing MRI scan times), in radar, and in single-pixel cameras. It also stimulated a broader interest in sparsity, convex optimization, and inverse problems that reshaped signal processing's relationship to applied mathematics and statistics.

Graph Signal Processing and the Network Turn

Classical signal processing assumes signals live on regular grids—time samples, image pixels. But many modern datasets live on irregular structures: social networks, sensor networks, brain connectivity graphs, transportation networks. Graph signal processing, developed from the 2010s onward, extends filtering, Fourier analysis, and sampling to signals defined on graph vertices. The graph Laplacian plays the role of frequency: its eigenvectors define a graph Fourier transform, and filters are functions of the Laplacian8. This framework has been applied to denoising sensor network data, analyzing brain signals, and learning on structured data. It represents a broadening of what counts as a signal and what counts as a domain.

Data-Driven and Learned Signal Processing

The most recent turn is the integration of machine learning into signal processing. Neural networks—particularly convolutional networks and, more recently, transformers—can learn to perform tasks that classical methods handle only with hand-designed models: denoising, super-resolution, speech recognition, channel estimation. The relationship is complex. Learned methods often outperform classical ones when training data are abundant, but they can fail unpredictably outside their training distribution and offer less interpretability. Classical model-based methods remain preferred where data are scarce, where guarantees matter, or where the physics is well understood. The frontier is the combination: using known structure (sparsity, physics, symmetries) to constrain learned models, and using learned models to accelerate classical algorithms. The field's history suggests that each new abstraction—circuit, algorithm, statistical model, sparse representation, graph, neural network—has not replaced its predecessors but added a layer, and that the most powerful systems draw on several at once.

References

  1. Signal processing, Wikipedia
  2. Fast Fourier transform, Wikipedia
  3. Kalman filter (introduction, History), Wikipedia
  4. Wiener filter (introduction, History), Wikipedia
  5. Least mean squares filter, Wikipedia
  6. Recursive least squares filter, Wikipedia
  7. Filter bank, Wikipedia
  8. Graph Fourier transform, Wikipedia

The frameworks of Signal Processing

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1920-1970

    Network synthesis filters

    How can a circuit be designed to have a desired frequency response?

    Fly to its tower in Nousfarer

  2. 1942-Present

    Statistical signal processing

    How can we make optimal decisions from noisy observations?

    About Statistical signal processing
    Statistical signal processing is built on a deceptively simple shift in perspective: treat the signal you are observing not as a fixed, determinable function, but as one realization of a random process. Where classical Fourier analysis asks, "What frequencies are present in this waveform?", statistical signal processing asks, "What is the most likely waveform, given that my measurements are corrupted by noise?" This probabilistic viewpoint is what separates it from earlier and adjacent frameworks)Skip punctuation and returns the field's defining problem: inference under uncertainty.

    The Core Organizing Idea

    The motivating problem is that we almost never observe a signal directly. Every physical measurement is a mixture of the desired signal and an unwanted random disturbance, or noise. Thermal noise in a receiver, atmospheric turbulence in a radar return, or quantization error in a digital system all corrupt the signal unpredictably. If this corruption is modeled as a random process, then the observed data is a random variable, and our job becomes a problem of statistical inference: given this noisy observation, what can we say about the underlying signal?
    The distinctive move of statistical signal processing is to treat the signal itself, the noise, or both as stochastic processes—collections of random variables indexed by time. This stands in contrast to the deterministic viewpoint of Fourier analysis, which treats a signal as a fixed function to be decomposed into sinusoids. The statistical framework provides a rigorous foundation for questions that deterministic analysis cannot answer: What is the best estimate of a signal in the presence of noise? How much uncertainty remains after processing? How should we design a receiver to make the fewest errors?

    Core Concepts in a Learning Order

    A solid grasp of statistical signal processing builds on several interlocking ideasache.

    1. Random Processes and Stationarity

    A signal is modeled as a random process X(t)X(t), a collection of random variables indexed by time. The most practically important class is the wide-sense stationary process, whose mean is constant over time and whose autocorrelation function depends only on the time difference between two samples. This assumption is not merely mathematical convenience—it allows us to characterize the process by just two functions: the mean and the autocorrelation, and it makes possible the powerful frequency-domain tools described below.

    2. Correlation and Power Spectral Density

    The autocorrelation function measures how strongly a signal at one instant is related to a signal at a later instant. Its Fourier transform is the power spectral density (PSD), which describes how the signal's power is distributed across frequency. The PSD is the statistical counterpart to the deterministic spectrum of Fourier analysis: rather than describing the frequency content of one particular waveform, it describes the average frequency content of a random process. An entire class of estimators—the periodogram, the correlogram, and their smoothed variants—exists to estimate the PSD from a finite set of observed data.

    3. Optimal Filtering: The Wiener Filter

    Note: The numbering in this draft continues from the prior sections. See full article for the complete sequence.
    The foundational result of statistical signal processing is the Wiener filter, developed by Norbert Wiener during World War II. The problem is direct: given a noisy observation y(t)=s(t)+n(t)y(t) = s(t) + n(t), find the linear filter that minimizes the mean squared error between the filter's output and the true signal s(t)s(t). The solution involves computing the spectrum of the signal and the noise and forming a frequency-dependent weighting that attenuates frequencies where noise dominates and passes frequencies where the signal dominates. The Wiener filter set the template for all later optimal filtering: define a cost function (mean squared error), model the signal and noise statistically, then solve for the filter that minimizes that cost.

    Essential Concepts in Learning Order

    1. Random Processes and Stationarity

    A signal is a realization of a random process if, at each time instant, the value is a random variable. The full description of a random process is a joint probability distribution over all time instants, which is usually intractable. Statistical signal processing therefore relies on wide-sense stationarity: the assumption (or approximation) that the mean is constant and the autocorrelation depends only on the time difference, not on absolute time. This assumption is what makes a theory of filtering and estimation possible, because it gives the process stable statistical structure to exploit.

    2. Correlation and Power Spectral Density

    The central description of a stationary random signal is its autocorrelation function, which measures how strongly the signal at one time is related to the signal at another time. The Fourier transform of the autocorrelation is the power spectral density (PSD), which describes how the signal's average power is distributed across frequency. The PSD is the statistical counterpart to the deterministic Fourier spectrum: instead of showing the exact frequencies of a fixed waveform, it shows the frequencies where the random process tends to concentrate its energy. This is a crucial bridge between the older frequency-domain framework and the newer probabilistic one. The five sharp peaks visible in the spectral density estimate of a noisy signal, for example, are precisely what statistical signal processing aims to reveal.

    3. Optimal Filtering in the Wiener Tradition

    The central design problem is not simply to filter a signal, but to filter it optimally under a statistical criterion. The classic solution is the Wiener filter, developed by Norbert Wiener during World War II and published in full in 1949. The Wiener filter minimizes the mean-square error between the true signal and its estimate, given known statistics of both signal and noise. Its key assumption is that both are wide-sense stationary and that we only need their second-order statistics (means and autocorrelations). This is a landmark moment for the framework: it transformed filtering from an intuitive craft into a well-posed optimization problem, and it established mean-square error as the default performance criterion for signal estimation.

    4. Estimation and Detection: Two Sides of Inference

    The framework organizes itself around two families of problems, each with a rich toolkit of decision rules.
    • Estimation asks for a continuous quantity: What is the value of this parameter? The classic results are the maximum likelihood estimator, introduced by R. A. Fisher in the 1920s (though this is an older statistical concept that predates the framework), and the Bayesian estimator, which incorporates prior knowledge. In the 1940s, this framework crystallized around the work of Norbert Wiener (on filtering and prediction) and later, in the 1960s, Rudolf Kalman's state-space filtering (though Kalman filtering is a later codification of the core Wiener problem). The essential ideas are a criterion—usually mean-square error—and a systematic method for choosing the estimator that satisfies it.
    • Detection asks a yes/no question: Is a signal present, or is it just noise? The standard tool is the likelihood ratio test, which compares the probability of the observed data under the hypothesis that the signal is present against the probability under the null hypothesis. A receiver chooses the hypothesis with higher likelihood, often weighted by prior probabilities. This framework was formalized in the late 1940s and 1950s by researchers such as W. Petersen, T. Birdsall, and W. Fox, who introduced the receiver operating characteristic (ROC) curve to describe the trade-off between detecting a true signal and raising a false alarm snippet.

    5. Spectral Estimation

    When the signal's statistics are unknown—as they almost always are in practice—the PSD must be estimated from data. Classical nonparametric methods include the periodogram (the squared magnitude of the Fourier transform of the data) and the correlogram (the Fourier transform of the estimated autocorrelation). These methods are simple but suffer from high variance: a periodogram of noise looks almost nothing like the true PSDholierIt. The variance problem motivated a major line of research in the 1960s and 1970s, producing windowed and averaged periodograms (the Welch method), and later parametric methods like autoregressive (AR) modeling, which assume the data is generated by a linear filter driven by white noise. This estimation problem—recovering a smooth power spectrum from a finite, noisy sample—remains one of the field's most visible practical tasks.

    5. From Stationary to Adaptive

    A stationary random process has statistics that do not change over time generation. Real signals rarely obey this. The least mean squares (LMS) algorithm, introduced by Bernard Widrow and Marcian Hoff in 1960, extends Wiener's ideas to non-stationary environments by updating filter coefficients sample-by-sample. The LMS algorithm is simple, robust, and cheap to compute, which makes it the workhorse of adaptive filtering: echo cancellation, channel equalization, and noise suppression all use it in some form. It also marks the point where statistical signal processing begins to bleed into its successor frameworks of adaptive signal processing and machine learning.

    5. Spectral Estimation and the Periodogram Puzzle

    Estimating the PSD from finite data is a statistical problem in its own right. The naive approach—compute the Fourier transform of the data and take its magnitude squared (the periodogram)—is notoriously inconsistent: the variance of the estimate does not decrease as more data is added. The remedies developed here are still standard practice: Bartlett's method and Welch's method reduce variance by averaging periodograms over overlapping segments of the signal, at the cost of frequency resolution. This trade-off between variance and resolution is a recurring theme in the frameworkcars the framework's core diagnostic tool.

    The Organizing Problem: Inference Under Uncertainty

    The unifying aim of statistical signal processing is to answer, "What can we know about a signal when we only have noisy, incomplete observations?" The framework answers this by postulating a probabilistic model for the signal and noise, then applying statistical decision theory to find the optimal estimator or detector. Every major technique below is an answer to a different version of this question.

    Intellectual History and Major Figures

    The framework took shape during World War II, driven by concrete military needs: radar and sonar operators needed to detect weak echoes in noiseaged, and anti-aircraft gunners needed to predict the future position of a target. Norbert Wiener's work on interpolation, extrapolation, and smoothing of stationary time series (published in book form in 1949 as Extrapolation, Interpolation, and Smoothing of Stationary Time Series) provided the first systematic solution to the filtering problem and effectively founded the field. Independently, a similar theory was developed in the Soviet Union by Andrey Kolmogorov, whose 1941 work on stationary sequences also laid the foundations of linear least-squares prediction. These two lines of work formed the "Wiener–Kolmogorov" filter, the bedrock result of the field.
    A second foundation stone came from a different corner. In 1944, the statistician Abraham Wald published Sequential Analysis, which formalized the likelihood ratio test. The theory of hypothesis testing—deciding between two or more possible signal states—is the detection half of the field, and Wald's work, together with earlier results of Neyman and Pearson, gave it a rigorous probabilistic footing.
    The postwar decades saw the framework mature into its modern form. In 1960, Rudolf E. Kálmán introduced the Kalman filter for recursive estimation of linear dynamical systems, extending Wiener's ideas from stationary processes to non-stationary ones and from batch processing to a causal, real-time recursion. The matched filter, which maximizes the signal-to-noise ratio for a known signal in white noise, became the foundational result for radar and digital communications. The framework consolidated in the 1960s and 1970s with the development of the fast Fourier transform (1965), which made spectral estimation computationally feasible and drove the explosion of practical applications. By the 1980s, the framework's methods were encoded in standard textbooks and embedded in every digital communication receiver.

    Relations to Neighboring Frameworks

    • Fourier and Frequency-Domain Analysis is the deterministic predecessor. Statistical signal processing extends it by replacing the signal with a random process and by dealing with the fact that, with finite data, the Fourier transform of a noisy signal is itself a random quantity. The PSD is the point of contact between the two frameworks.
    • Time-Frequency Analysis, which emerged in parallel, addresses signals whose statistical and spectral properties change over time. Statistical signal processing typically assumes stationarity, at least locally; time-frequency methods add a joint description of time and frequency for non-stationary signals.
    • Adaptive Signal Processing can be seen as a practical, recursive implementation of statistical ideas: instead of assuming known statistics, an adaptive filter estimates the Wiener solution sequentially from the data itself (as in the LMS algorithm). The Wiener filter is the fixed, optimal solution; adaptive filters are its online, data-driven cousins.
    • Machine Learning in signal processing extends the same inference logic—estimating an unknown signal from noisy data—but replaces explicit statistical models with learned mappings from data. The framework's decision-theoretic foundations (likelihood, priors, risk) are the same concepts underlying much of modern machine learning.

    A Concrete Order of Learning

    A sensible path through the framework begins with a review of probability, random variables, and expectation. The next step is the theory of random processes: autocorrelation, cross-correlation, and stationarity. With that foundation, the Wiener filter gives the canonical example of optimal linear estimation. From there, one moves to detection theory and the Neyman-Pearson lemma, which provides the optimal test for a binary hypothesis under fixed error probabilities. The final topic is spectral estimation—using finite data to estimate the PSD—which is both practically essential and a rich source of insight into the fundamental limits of resolution and variance.
    No prior background in advanced probability is required, but comfort with random variables, expectation, and Fourier transforms is essential. A familiarity with linear systems (impulse response, transfer functions, convolution) is also necessary, since the filters studied are overwhelmingly linear.

    Boundaries with Neighboring Frameworks

    Fourier and Frequency-Domain Analysis is the deterministic predecessor. It describes exactly what is in the signal, assuming no noise and no randomness. Statistical signal processing subsumes it: the power spectral density of a stationary process is defined through the Fourier transform of the autocorrelation. The older framework remains a prerequisite and a computational engine, but it cannot express noise or the idea of an optimal estimate.
    Adaptive Signal Processing is a direct extension: where the Wiener filter assumes known, stationary statistics, adaptive filters (like the LMS algorithm) estimate those statistics on the fly and update the filter coefficients in real time. Statistical signal processing provides the theoretical ideal; adaptive processing provides the practical approximation.
    Machine Learning for Signal Processing is the modern downstream of the same inference problems. Deep networks and classical classifiers are now used for detection and estimation tasks, but they inherit the probabilistic framing, the cost functions (mean-square error, likelihood), and the evaluation methodology of statistical signal processing.
    Time-Frequency Analysis addresses the same noisy data but focuses on signals whose spectral content changes over time—something the stationary assumption explicitly forbids. It does not replace statistical signal processing; it relaxes its most restrictive assumption.

    Significance and Continuing Role

    Statistical signal processing is not merely an historical precursor; it is the theoretical backbone of much of modern engineering. Radar and sonar detection, global navigation satellite systems, speech recognition front ends, medical imaging reconstruction, digital communications receivers, and financial time-series modeling all rely on its core results. The Wiener filter, the matched filter, and the maximum-likelihood estimator are as fundamental to the practicing engineer as the Fourier transform.
    The framework's intellectual legacy is this: it turned signal processing from a craft of intuition into a science of decisions. By modeling signals as random processes and casting processing problems as problems of statistical inference, it supplied a single, rigorous criterion—mean-squared error, likelihood, or probability of error—against which any processing scheme can be judged. That normative stance remains its deepest contribution, and it is the reason subsequent frameworks in adaptive, sparse, and learned signal processing continue to measure themselves against it.
    The 14 concepts its tower teaches
    • Random Processes
    • Wide-Sense Stationarity
    • Autocorrelation Function
    • Power Spectral Density
    • Wiener Filter
    • Neyman-Pearson Lemma
    • Matched Filter
    • Maximum Likelihood Estimation
    • Minimum Mean-Square Error Estimation
    • Periodogram
    • Welch Method
    • Autoregressive Modeling
    • LMS Algorithm
    • Receiver Operating Characteristic

    Fly to its tower in Nousfarer

  3. 1948-Present

    Information theory and source coding

    How much can a message be compressed without losing information?

    Fly to its tower in Nousfarer

  4. 1949-Present

    Sampling and multirate theory

    When can a continuous signal be perfectly reconstructed from its samples?

    Fly to its tower in Nousfarer

  5. 1960-Present

    Adaptive filter

    How can a filter adjust itself to changing signal statistics?

    Fly to its tower in Nousfarer

  6. 1960-Present

    Kalman filter

    How can the state of a dynamic system be estimated from noisy measurements?

    Fly to its tower in Nousfarer

  7. 1965-Present

    FFT-based spectral analysis

    How can the frequency content of a signal be computed efficiently?

    Fly to its tower in Nousfarer

  8. 1970-Present

    Digital filter design

    How can a digital filter be designed to approximate a desired response?

    Fly to its tower in Nousfarer

  9. 1976-Present

    Filter banks and wavelets

    How can a signal be split into frequency bands and perfectly recombined?

    Fly to its tower in Nousfarer

  10. 1984-Present

    Wavelet and multiresolution analysis

    How can a signal be analyzed at multiple resolutions?

    Fly to its tower in Nousfarer

  11. 2006-Present

    Compressed sensing

    How can a sparse signal be recovered from few measurements?

    Fly to its tower in Nousfarer

  12. 2012-Present

    Data-driven and learned signal processing

    How can signal processing be learned from data rather than designed by hand?

    Fly to its tower in Nousfarer

  13. 2013-Present

    Graph signal processing

    How can signals defined on irregular structures be processed?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Network synthesis filtersStatistical signal processingInformation theory and source codingSampling and multirate theoryAdaptive filterKalman filterFFT-based spectral analysisDigital filter designFilter banks and waveletsWavelet and multiresolution analysisCompressed sensingData-driven and learned signal proces…Graph signal processing

Beyond the field

Where Signal Processing's ideas came from in other fields, and what they led to there: the strongest 12 of 74 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Signal Processing is a planet in Nousfarer

Its city has a tower for each of these 13 frameworks. Land, walk in, and climb one concept at a time.

Land on Signal Processing

Related fields

Fields its ideas are linked to

More in Electrical Engineering

See where Signal Processing sits on the Map of Human Knowledge