- Atlas
- Mathematics
- Approximation Theory
Approximation Theory
Approximation theory is the branch of analysis that studies how well complicated functions can be replaced by simpler ones, and how to find the best such replacement.
10 frameworks, from 1805 to today · Analysis · Mathematics
- Least Squares and Orthogonal Projection 1805-Present
- Minimax and Chebyshev Approximation 1850-Present
- Rational and Padé Approximation 1890-Present
- Constructive Approximation and Jackson–Bernstein Theory 1911-Present
- Functional-Analytic Approximation Theory 1932-Present
- Algorithmic and computational approximation 1934-Present
- Information-based complexity 1936-Present
- Spline and Piecewise-Polynomial Approximation 1940-Present
- Sparse approximation 1990-Present
- Adaptive and Neural-Network Approximation 2000-Present
Overview
Approximation theory is the branch of analysis that studies how well complicated functions can be replaced by simpler ones, and how to find the best such replacement1. Its central objects are families of "simple" functions—polynomials, rational functions, trigonometric polynomials, splines, wavelets—and its central questions are quantitative: given a function and a budget of complexity, how close can a simple function come, what does the best one look like, and how do you compute it? The subject matters because almost every numerical computation of a function, every compression scheme, every numerical solver for differential equations, and every model that replaces a hard function by an easy one is doing approximation theory whether it says so or not. The history of the field is a long negotiation between three demands that pull against each other: exactness of the best possible answer, constructibility of a practical recipe, and efficiency in high dimensions.
Least squares and orthogonal systems: the eighteenth-century start
The oldest strand is fitting data. Around 1805, Adrien-Marie Legendre published the method of least squares—choosing the curve that minimizes the sum of squared errors—and Carl Friedrich Gauss gave it a probabilistic justification through the normal distribution and used it in astronomical orbit determination. Least squares turned approximation into an optimization problem with a clean linear structure: if the approximating family is a linear space spanned by functions $\varphi1,\dots,\varphin$, the best fit is found by solving the normal equations, and the error is orthogonal to the span.
The decisive refinement came from choosing the spanning functions well. Legendre polynomials, studied by Legendre in 1782 and later by Laplace, are mutually orthogonal on $[-1,1]$ with respect to ordinary integration; Chebyshev polynomials, introduced by Pafnuty Chebyshev in the 1850s, are orthogonal with respect to a weight and have the striking property that among monic polynomials of a given degree, the scaled Chebyshev polynomial has the smallest maximum absolute value on the interval. Orthogonal systems—Legendre, Chebyshev, Hermite, Laguerre, and later the general theory of orthogonal polynomials—made least-squares approximation numerically stable and gave Fourier-type expansions their power. Joseph Fourier's trigonometric series (1807 onward) is the prototype: projecting onto sines and cosines, with coefficients computed by inner products, approximates a function by its partial sums. The idea that approximation is projection onto an orthogonal family, and that the error is what remains orthogonal to the family, is the seed of the functional-analytic view that would mature a century later.
Best approximation and the minimax turn
Least squares minimizes an average error. Chebyshev asked a different question: minimize the maximum error. This is best approximation in the uniform norm, and it is harder because the objective is nonlinear. Chebyshev's student Egor Zolotarev solved early cases in the 1870s, and in 1885 Karl Weierstrass proved the foundational existence theorem: every continuous function on a closed interval can be uniformly approximated arbitrarily well by polynomials. That theorem told the field that polynomials are universal, but said nothing about how good a given degree is or how to find the best one.
The structural answer came from Chebyshev's alternation idea, made rigorous by Boris Glebovich in 1899 and later by others: the best uniform polynomial approximation of degree $n$ to a continuous function is characterized by the error equioscillating at $n+2$ points with alternating signs. This "alternation theorem" converts an infinite-dimensional minimization into a finite, checkable condition. In 1934 Evgeny Remez turned it into an algorithm, and in the 1950s the exchange algorithms of Remez and of Charles Lawson became standard computational tools. The minimax viewpoint also produced the Chebyshev polynomials as the solution to a canonical extremal problem, and it connected approximation to the theory of moments and to duality—the best error equals a supremum over signed measures, a linear-programming duality in disguise.
Constructive approximation: Bernstein, Jackson, and the modulus of continuity
Weierstrass's theorem is existential. The constructive school asked for explicit approximants and for rates. Sergei Bernstein gave a famous constructive proof in 1912 using the Bernstein polynomials $Bn(f)(x)=\sumk f(k/n)\binom{n}{k}x^k(1-x)^{n-k}$, which converge uniformly to any continuous $f$ on $[0,1]$; these polynomials later became central to computer-aided design as Bézier curves. Dunham Jackson in 1911–1912 proved the quantitative converse to Weierstrass: the uniform error of the best degree-$n$ polynomial approximation is bounded by a constant times the modulus of continuity of $f$ evaluated at $1/n$. The modulus of continuity, $\omega(f,\delta)=\sup_{|x-y|\le\delta}|f(x)-f(y)|$, measures how much $f$ can change over short distances, and Jackson's theorem says smoothness controls approximation rate.
The matching lower bound came from Bernstein and others: if the approximation error decays like $n^{-k}$, then $f$ must have $k$ derivatives (in a suitable sense). Together these form the Jackson–Bernstein theory, the backbone of constructive approximation: the rate of polynomial approximation is equivalent to the smoothness of the function, measured by moduli of continuity or, in the sharper modern form, by $K$-functionals and Besov spaces. The same circle of ideas extends to trigonometric approximation, where the Jackson and Bernstein theorems and the work of Sergei Stechkin and others in the mid-twentieth century gave a complete dictionary between smoothness and approximation rate. This is the field's deepest structural result: approximation is a lens that reads off smoothness.
Rational approximation and Padé theory
Polynomials are not always the right simple functions. Rational functions—ratios of polynomials—can approximate functions with poles or sharp transitions far better per degree of freedom. The Padé approximant, developed by Henri Padé around 1890, matches the Taylor series of a function to the highest possible order using a rational function of prescribed numerator and denominator degrees; it is the rational analogue of a Taylor polynomial and often converges where the Taylor series diverges2. Padé theory connects to continued fractions, to the analytic theory of orthogonal polynomials (Stieltjes, Markov), and to the moment problem.
The twentieth century added a striking negative result and a striking positive one. In 1966 Donald Newman proved that rational functions approximate $|x|$ on $[-1,1]$ exponentially better than polynomials—a function with a corner, which polynomials handle poorly, is handled superbly by rationals. This launched a large theory of rational approximation rates, tied to the location of singularities of the function in the complex plane. Meanwhile the exact minimax theory of rational approximation is much harder than the polynomial case: the alternation theorem survives, but the error curve can have degenerate structure, and the problem is genuinely nonlinear. Rational approximation remains an active area, especially for functions with singularities and in model reduction.
Functional analysis recasts the problem
In the early twentieth century, approximation moved into the language of normed and Hilbert spaces. The Riesz representation theorem, the geometry of Hilbert spaces, and the projection theorem gave least-squares approximation a clean abstract form: the best approximation from a closed subspace is the orthogonal projection, and the error is orthogonal to the subspace. Stefan Banach's 1932 Théorie des opérations linéaires framed approximation in general normed spaces, and the question of when a best approximation exists and is unique became a structural question about the geometry of the space—strict convexity, smoothness, Chebyshev sets.
The Haar condition (a family whose every nontrivial linear combination has at most $n-1$ zeros) characterizes when best approximation is unique, and it underlies the alternation theory. The theory of Chebyshev systems, developed by Haar, Mairhuber, and others, unified polynomial, trigonometric, and spline approximation under one framework. Functional analysis also brought in the duality theory of approximation: the distance from a point to a subspace equals the norm of a functional vanishing on the subspace, a theorem of Hahn–Banach type that turns approximation into a problem about annihilating functionals. This abstract machinery is what makes modern approximation theory portable across norms and spaces.
Splines and piecewise polynomials
Polynomials are global and can oscillate badly. Piecewise polynomials—splines—trade global smoothness for local flexibility. The cubic spline, a piecewise cubic that is twice continuously differentiable at the knots, was studied systematically by Isaac Schoenberg in the 1940s, who coined the term and developed the theory; it had appeared earlier in actuarial and shipbuilding practice. Splines have a variational characterization: among interpolants with bounded second derivative, the cubic spline minimizes the integral of the squared second derivative—the "smoothest" interpolant in a precise sense.
The 1960s and 1970s brought B-splines, the basis functions with local support that make spline computation stable and efficient, developed by Schoenberg, Curry, and Carl de Boor. De Boor's 1978 A Practical Guide to Splines and his stable algorithms made splines the workhorse of numerical analysis and computer-aided geometric design. Splines connect to approximation rates through the same Jackson–Bernstein machinery: piecewise polynomials of degree $k$ with mesh size $h$ approximate smooth functions to order $h^{k+1}$, and the theory of spline approximation on nonuniform meshes became a major industry. The finite element method, developed in the 1950s and 1960s by engineers and mathematicians including Courant, Clough, and Zienkiewicz, is in effect a large-scale application of piecewise-polynomial approximation to partial differential equations.
N-widths and the information-based view
How well can any method approximate a given class of functions? This is the question of $n$-widths, introduced by Andrei Kolmogorov in 1936: the $n$-width of a set is the smallest possible error in approximating it by an $n$-dimensional linear space, minimized over all such spaces. Kolmogorov widths, and the related linear, Gelfand, and Bernstein widths, measure the intrinsic approximability of a function class independent of any particular method. They connect approximation to entropy, to the theory of compact operators (singular values are widths), and to the optimal recovery of functions from finite information.
The information-based theory, developed by Nikolai Bakhvalov, Joseph Traub, Henryk Woźniakowski, and others from the 1960s onward, asked a sharper question: given a budget of $n$ function evaluations or bits of information, what is the best possible approximation? This reframed approximation as a problem of optimal algorithms and information complexity, with sharp constants and lower bounds. The theory of optimal recovery, and the distinction between linear and nonlinear information, became central. These ideas feed directly into the modern study of sampling, compressed sensing, and the cost of approximation in high dimensions.
High dimensions, sparsity, and adaptivity
Classical approximation theory is largely one-dimensional or low-dimensional, and its rates degrade catastrophically with dimension: a polynomial of degree $n$ in $d$ variables has on the order of $n^d$ coefficients, so the "curse of dimensionality" makes naive approximation hopeless. The late twentieth and early twenty-first centuries produced responses.
Sparse approximation exploits the observation that many high-dimensional functions or signals are well described by a few significant coefficients in a suitable basis. The theory of nonlinear approximation, developed by Ronald DeVore, Björn Jawerth, and others in the 1990s, showed that keeping only the $n$ largest coefficients can achieve rates far better than linear approximation, governed by the function's membership in a Besov or Lorentz space. Compressed sensing, from work of Emmanuel Candès, Justin Romberg, Terence Tao, and David Donoho around 2006, showed that sparse signals can be recovered from far fewer measurements than classical sampling theory requires, using $\ell_1$ minimization—a result that reshaped signal processing and gave approximation theory a central role in data science.
Adaptive approximation chooses the approximating family based on the function: adaptive meshes refine where the function is rough, adaptive wavelet expansions keep coefficients where they matter, and nonlinear methods such as neural networks learn the basis itself. The theory of nonlinear widths and of adaptive approximation rates tries to say when adaptivity genuinely beats fixed linear methods. Deep neural networks, from the 2010s onward, raised the question of whether their approximation rates in high dimension escape the curse—results by several groups show that certain function classes are approximated efficiently by networks, but the full picture, especially for the functions that arise in practice, remains open.
The present
Approximation theory today is a federation of methods sharing a common question. Classical constructive approximation—Jackson–Bernstein theory, moduli of continuity, splines—remains the backbone and the source of sharp rates. Functional-analytic approximation supplies the abstract framework and the duality tools. Minimax and rational approximation remain active, especially for functions with singularities and in reduced-order modeling. The information-based and $n$-width tradition has merged with the study of sampling and compressed sensing. High-dimensional and nonlinear approximation—sparse representations, adaptive methods, and neural networks—is the fastest-moving frontier, driven by applications in signal processing, scientific computing, and machine learning.
The open questions are recognizably the old ones in new dress: what are the sharp rates for nonlinear and adaptive methods, and when do they beat linear ones? How do approximation rates in high dimension depend on the structure of the function class, and can neural networks exploit that structure? What is the right notion of complexity when the approximant must be computed, not just exist? The field's history is a record of answering such questions one norm at a time, and the questions keep returning because each new class of simple functions—polynomials, rationals, splines, wavelets, networks—reopens the same negotiation between exactness, constructibility, and efficiency.
References
- Approximation theory, Wikipedia
- Padé approximant, Wikipedia
The frameworks of Approximation Theory
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1805-Present
- 1850-Present
- 1890-Present
- 1911-Present
- 1932-Present
- 1934-Present
- 1936-Present
- 1940-Present
- 1990-Present
- 2000-Present

Approximation Theory is a planet in Nousfarer
Its city has a tower for each of these 10 frameworks. Land, walk in, and climb one concept at a time.
Land on Approximation TheoryRelated fields
More in Analysis
- Calculus Of Variations
- Complex Analysis
- Dynamical Systems
- Functional Analysis
- Geometric Measure Theory
- Harmonic Analysis
- Mathematical Analysis
- Measure Theory
- Operator Theory
- Ordinary Differential Equations
- Partial Differential Equations
- Real Analysis
See where Approximation Theory sits on the Map of Human Knowledge