Signals And Systems

Signals and systems is the part of electrical engineering that asks how to describe a signal mathematically and predict what a system will do to it.

12 frameworks, from 1822 to today · Electrical Engineering · Engineering

still a working view ended · each leads to its entry below

Overview

Signals and systems is the part of electrical engineering that asks how to describe a signal mathematically and predict what a system will do to it1. A signal is any quantity that varies with time, space, or another independent variable and carries information—a microphone voltage, a radio wave, a digital image1. A system is anything that turns an input signal into an output signal: an amplifier, a filter circuit, a motor, a control loop, a communication channel. The stakes are practical. If engineers can write down a model of the signal and of the system, they can predict distortion, design filters, stabilize a feedback loop, compress data, and diagnose why a circuit misbehaves—all before building anything. The history of the subfield is largely the history of finding better representations: transforms, transfer functions, state variables, and, most recently, representations learned from data.

Fourier's heat problem and the birth of frequency analysis

In 1807, Joseph Fourier presented to the French Academy a claim that any function, however irregular, could be written as a sum of sines and cosines. This was the founding act of Fourier analysis—decomposing a signal into its frequency components, so that what looks like a messy waveform becomes a list of amplitudes at different frequencies2. The Academy was skeptical (Lagrange objected), and publication was delayed, but the method eventually appeared in Fourier's 1822 Théorie analytique de la chaleur. The idea mattered for engineering because it converts the hard problem of what a linear circuit does to a complex waveform into an easy problem: find what it does to each sine wave, then add the results.

Two tools grew from this. The Fourier series handles periodic signals; the Fourier transform extends the idea to aperiodic ones. For much of the nineteenth century these remained mathematical instruments. Their engineering use arrived with electrical communication: analyzing telegraph and telephone signals, designing filters, and understanding the spectrum occupied by a transmission. A frequency-domain view became the engineer's standard lens for interference, bandwidth, and distortion.

Linear systems and the transfer-function viewpoint

The crucial class of systems is the linear time-invariant (LTI) system: linear (output to a sum of inputs equals the sum of the outputs) and time-invariant (the behavior does not change over time)3. For these, the response to any input can be computed from the response to a single impulse—the impulse response—by a mathematical operation called convolution34. This is the central insight of classical signals and systems: a system is fully characterized by one function, and its effect on a sinusoid is simply a change in amplitude and phase, captured by the frequency response5.

The transfer function packaged this into algebraic form6. Working in the Laplace domain (for continuous time), a circuit or mechanical system becomes a ratio of polynomials, and stability, gain, and transient behavior can be read from the locations of its poles and zeros—the roots of those polynomials7. Oliver Heaviside's operational calculus in the 1880s–1890s and the rigorous Laplace-transform theory built by others turned this into a working method. By the mid-twentieth century, LTI theory—convolution, Fourier methods, the Laplace transform—was the backbone of electrical engineering education, applied to filters, amplifiers, servomechanisms, and communications.

A parallel frequency-domain device, the phasor, had already simplified sinusoidal steady-state circuit analysis: a sinusoid is represented by a complex number encoding its amplitude and phase, so differential equations become algebra. Later, the z-transform played the same role for discrete-time signals that the Laplace transform plays for continuous ones8.

Feedback control: making systems behave

Separate from signal analysis, a tradition of classical feedback control developed around the problem of keeping a system on target despite disturbances and imprecise components. The centrifugal governor, used on steam engines from the late eighteenth century, was the emblematic device; James Clerk Maxwell's 1868 paper "On Governors" gave the first mathematical stability analysis of such feedback loops. The central discovery was that feedback can stabilize or destabilize, and that stability depends on loop gain and phase.

Classical control was systematized in the 1930s–1940s, driven by servomechanisms for aircraft, ships, and industrial processes. The key tools were the Bode plot (a log-log graph of gain and phase versus frequency, developed by Hendrik Bode at Bell Labs), the Nyquist criterion (stability judged from a plot of the open-loop response around the critical point, from Harry Nyquist's 1932 work), and the root-locus method for tracking pole locations as gain changes9710. These let engineers design a compensator—typically a lead, lag, or PID (proportional–integral–derivative) controller—by shaping the frequency response. The approach was graphical, practical, and tolerant of partial models, which is why it dominated industrial control for decades.

A striking practical case is the negative-feedback amplifier: Harold Black's 1927 patent at Bell Labs showed that feeding back a fraction of the output could trade enormous gain for stability of gain, linearity, and bandwidth. That idea, formalized in feedback amplifier theory (Bode and others in the 1930s–40s), made long-distance telephony and later precision electronics possible.

State space and the modern control synthesis

Classical methods handled single-input, single-output systems well but struggled with multivariable systems, nonlinearities, and optimal design. The response was the state-space representation: describe a system by a set of first-order differential equations in internal variables (the "state"), written compactly with matrices11. This let engineers handle many inputs and outputs at once and analyze internal, not just input–output, behavior.

The state-space approach matured into modern control theory in the 1950s–60s, associated with Rudolf Kalman (controllability, observability, the Kalman filter for estimation in noise) and with optimal control: finding the control that minimizes a cost, as in the linear-quadratic regulator. The Kalman filter, published in 1960, recursively estimates a system's state from noisy measurements and became foundational for navigation, tracking, and later spacecraft guidance—most famously in the Apollo program12. These methods shifted design from graphical loop-shaping to solving equations: check controllability and observability, compute a gain matrix, estimate the unmeasured state.

Nonlinear systems theory asked what happens when linearity fails—when a system saturates, switches, or has multiple equilibria. Stability tools such as Lyapunov's "direct method" (from Aleksandr Lyapunov's 1892 work) were revived and extended; describing functions gave approximate frequency-domain answers; and later bifurcation and chaos theory described sudden qualitative changes in behavior. Nonlinear control became essential for robotics, power electronics, and aircraft, where linear approximations break down over large excursions.

Digital signal processing: the discrete revolution

The shift from analog to digital changed everything. Once signals are sampled—converted into a sequence of numbers—processing can be done by computation. The theoretical bridge was sampling theory: the Nyquist–Shannon sampling theorem states that a bandlimited signal can be reconstructed exactly from samples taken faster than twice its highest frequency13. Claude Shannon's 1948 paper and earlier work by Harry Nyquist and others made this precise, and it underpins every digital audio, video, and communication system.

Early digital processing was limited by hardware. A turning point was the fast Fourier transform (FFT), an efficient algorithm for computing the discrete Fourier transform, popularized by James Cooley and John Tukey in 1965 (with earlier roots including Gauss)14. The FFT reduced the computation from roughly N² to N log N operations, making spectral analysis practical on real machines14. Alongside it came the design of digital filters—finite-impulse-response (FIR) and infinite-impulse-response (IIR) types—whose coefficients are computed to meet a frequency specification, then run in software or dedicated chips.

Multirate and filter-bank processing extended these ideas: changing the sampling rate (decimation and interpolation), and splitting a signal into frequency bands with a bank of filters. This proved central to subband coding, speech compression, and eventually wavelets. A landmark application was the digital transmission of speech and music, and the compression standards that followed.

System identification: learning models from data

Not all systems can be modeled from first principles; sometimes the physics is too complex, and one must estimate the model from measured input–output data. System identification developed as a discipline in the 1960s and after, associated with control and econometrics researchers, providing methods to fit parametric models (difference equations, transfer functions, state-space forms) to experimental records, with attention to noise and to how well the resulting model will predict. It connected the deterministic theory of signals and systems to statistics, and it supplied the models that control and simulation needed.

Statistical signal processing: signals under uncertainty

Where classical analysis treated signals as deterministic, statistical signal processing treats them as random processes and asks how to extract information from noise. The foundations include Norbert Wiener's work on filtering and prediction (published in book form in 1949) and Andrey Kolmogorov's contemporaneous results; the Wiener filter gives the optimal linear estimate in a mean-square sense15. Later, adaptive filtering and estimation methods (including the Kalman filter, and recursive least squares) allowed filters to adjust to changing conditions1617. This tradition underlies radar, sonar, speech enhancement, communications receivers, and modern array processing—and it was the seedbed of the information-theoretic and detection-theoretic views of signals.

Sparsity and compressed sensing

A more recent turning point came from a surprising observation: many signals are sparse in some representation—most of their coefficients are zero or negligible—so they may be recoverable from far fewer measurements than classical sampling theory demands18. Compressed sensing, developed in the mid-2000s (notably by Emmanuel Candès, Justin Romberg, and Terence Tao, and independently by David Donoho), showed that under suitable conditions a sparse signal can be reconstructed exactly from a small number of random linear measurements by solving an optimization problem. This challenged the Nyquist-rate orthodoxy and influenced medical imaging, radar, and sensor design. It also linked signals and systems to convex optimization and statistics.

Data-driven modeling and learning

The newest phase treats the system itself as something to be learned from data rather than derived from governing equations. Data-driven modeling spans subspace identification (estimating state-space models directly from input–output records), and, more recently, machine-learning methods—neural networks and other flexible models—used to predict, classify, or control signals and systems19. The appeal is that such models can capture behavior too complex for hand-derived equations; the open questions are about guarantees: when can a learned model be trusted, how does it behave outside its training conditions, and how can it be combined with the physical insight that classical theory provides. The field today thus holds several strands in tension—transform-based analysis, state-space and optimal control, statistical estimation, and learned models—each descended from the turning points above. What remains open is how to unify them: how to certify a learned controller, how to identify nonlinear systems reliably, and how to design systems that reason about their own uncertainty.

References

  1. Signal (Signals and systems, introduction, Definitions), Wikipedia
  2. Fourier analysis, Wikipedia
  3. Linear time-invariant system, Wikipedia
  4. Impulse response, Wikipedia
  5. Linear filter, Wikipedia
  6. Transfer function, Wikipedia
  7. Nyquist stability criterion (Background, introduction), Wikipedia
  8. Z-transform, Wikipedia
  9. Bode plot, Wikipedia
  10. Root locus analysis, Wikipedia
  11. State-space representation, Wikipedia
  12. Kalman filter (History), Wikipedia
  13. Nyquist–Shannon sampling theorem, Wikipedia
  14. Fast Fourier transform (History, introduction), Wikipedia
  15. Wiener filter (introduction, History), Wikipedia
  16. Adaptive filter, Wikipedia
  17. Recursive least squares filter, Wikipedia
  18. Compressed sensing, Wikipedia
  19. Machine learning, Wikipedia

The frameworks of Signals And Systems

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1822-Present

    Fourier analysis

    How can any signal be broken into sine waves?

    Fly to its tower in Nousfarer

  2. 1880-Present

    Linear time-invariant system

    How does a linear system change any input?

    Fly to its tower in Nousfarer

  3. 1930-1990

    Classical feedback control

    How can feedback keep a system on target?

    Fly to its tower in Nousfarer

  4. 1948-Present

    Digital signal processing

    How can computers process sampled signals?

    Fly to its tower in Nousfarer

  5. 1949-Present

    Statistical signal processing

    How can signals be estimated despite noise?

    Fly to its tower in Nousfarer

  6. 1950-Present
  7. 1960-Present

    Adaptive Filtering

    How can filters adjust themselves to data?

    Fly to its tower in Nousfarer

  8. 1960-Present

    State-space representation

    How can many-variable systems be modeled and controlled?

    Fly to its tower in Nousfarer

  9. 1965-Present

    System identification

    How can models be built from measured data?

    Fly to its tower in Nousfarer

  10. 1980-Present
  11. 2004-Present

    Compressed sensing

    How few measurements are needed to reconstruct a signal?

    Fly to its tower in Nousfarer

  12. 2010-Present

    Data-driven modeling and learning

    How can data alone reveal how a system works?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Fourier analysisLinear time-invariant systemClassical feedback controlDigital signal processingStatistical signal processingNonlinear systemAdaptive FilteringState-space representationSystem identificationMultirate and Filter-Bank Signal Proc…Compressed sensingData-driven modeling and learning

Beyond the field

Where Signals And Systems's ideas came from in other fields, and what they led to there: the strongest 12 of 85 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Signals And Systems is a planet in Nousfarer

Its city has a tower for each of these 12 frameworks. Land, walk in, and climb one concept at a time.

Land on Signals And Systems

Related fields

Fields its ideas are linked to

More in Electrical Engineering

See where Signals And Systems sits on the Map of Human Knowledge