- Atlas
- Mathematics
- Nonparametric Statistics
Nonparametric Statistics
Nonparametric statistics is the part of statistics that tries to learn from data while assuming as little as possible about the shape of the population that produced it.
11 frameworks, from 1933 to today · Statistics · Mathematics
- Empirical distribution function 1933-Present
- Order-statistics and quantile inference 1940-Present
- Distribution-free rank and sign inference 1945-Present
- Asymptotic relative efficiency theory 1948-1980
- Kernel and spline smoothing 1956-Present
- Nonparametric survival analysis 1958-Present
- Robust statistics 1964-Present
- Nonparametric Bayesian inference 1973-Present
- Resampling and computer-intensive inference 1979-Present
- Minimax rate theory 1982-Present
- Semiparametric model 1993-Present
Overview
Nonparametric statistics is the part of statistics that tries to learn from data while assuming as little as possible about the shape of the population that produced it1. A parametric method commits in advance to a formula with a few unknown numbers—a normal curve with a mean and a variance, say—and then estimates those numbers1. A nonparametric method refuses that commitment. It might estimate a whole curve from the data, compare groups using only the order of the observations, or build an uncertainty interval by resampling the data itself. The stakes are practical: real data are often skewed, clumped, contaminated by outliers, or simply unlike any textbook distribution, and a method that assumes the wrong shape can give confidently wrong answers. The history of the field is a long argument about how much structure a statistician is entitled to assume, and a series of ingenious devices for getting by with less.
Order, Ranks, and the First Distribution-Free Tests
The oldest strand is inference from ranks—the positions of observations when sorted from smallest to largest. In 1710 the Scottish physician John Arbuthnot used a sign argument to show that the observed run of years in which male births exceeded female births in London was too consistent to be chance. The sign test, which counts how often one of a paired difference is positive, needs no assumption about the size of the differences, only their direction12. It is the ancestor of a whole family.
The decisive early twentieth-century figure was Frank Wilcoxon, a chemist at the American Cyanamid company, who in 1945 published two tests for comparing two groups: one using the ranks of the pooled observations (now the Wilcoxon rank-sum test, also called the Mann–Whitney test after Henry Mann and Donald Whitney's 1947 reformulation), and one for paired data using the ranks of the absolute differences (the signed-rank test). The appeal was immediate: the tests were simple to compute by hand, and their validity did not depend on normality. Around the same time, the Swedish statistician Cramér and others were formalizing the idea of a distribution-free procedure—one whose probability of falsely rejecting a true hypothesis is the same whatever the underlying distribution, so long as it is continuous.
The 1950s and 1960s turned rank methods into a theory. Erich Lehmann's 1959 book Testing Statistical Hypotheses and the work of Jaroslav Hájek, Zbyněk Šidák, and later Pranab Sen and Malay Ghosh built the asymptotic machinery: under mild conditions, rank statistics are asymptotically normal, and their efficiency relative to the classical t-test can be computed exactly. This gave the field its central currency, asymptotic relative efficiency—the ratio of sample sizes the two methods need to achieve the same power in large samples3. The striking result, due to E. J. G. Pitman and others, was that for the normal distribution the rank tests lose only a small fraction of efficiency (about 95 percent for the sign test, more for the rank-sum test), while for heavy-tailed or contaminated distributions they can be far more efficient than the t-test. The rank methods were not a poor relation after all.
Estimating the Distribution Itself
A parallel line of work asked not how to test but how to estimate. The empirical distribution function—the step function that puts mass 1/n at each observation—is the natural estimate of the true distribution, and Andrey Kolmogorov (1933) and Nikolai Smirnov (1939) worked out its sampling behavior, giving the Kolmogorov–Smirnov test for whether a sample could have come from a specified distribution. The empirical distribution function is the foundation on which much of the field rests: the bootstrap, quantile estimates, and many confidence bands are built from it.
Quantiles—the values below which a given fraction of the data lies—are the natural nonparametric summary of a distribution, and their estimation and inference developed alongside the empirical distribution function. The sample median, the most robust of location estimates, has a long history; its asymptotic normality was established in the early twentieth century, and the theory of order statistics (the sorted observations) gave exact distributional results for any quantile. These results matter because quantiles describe skewed and heavy-tailed data—incomes, survival times, pollutant concentrations—where means mislead.
Robustness: Statistics That Resist Bad Data
A distinct but overlapping movement emerged in the 1960s around robust statistics: methods that perform well not only when the model is exactly right but when it is slightly wrong. The motivating problem was that a single wild outlier can drag a sample mean arbitrarily far, while the median barely moves4. Peter Huber's 1964 paper introduced M-estimators, which generalize maximum likelihood by replacing the squared-error loss with a loss that grows more slowly for large errors, bounding the influence of any one observation. Frank Hampel, in the late 1960s and 1970s, developed the influence function and the breakdown point—the fraction of the data that can be arbitrarily corrupted before the estimate becomes meaningless—as the field's diagnostic tools. John Tukey's work on exploratory data analysis and his resistant summaries (the median, the trimmed mean, the box plot) spread the robust outlook into everyday practice. Robustness and nonparametrics are not identical—a robust method may still assume a parametric family, just a flexible one—but they share the conviction that rigid assumptions are dangerous.
Smoothing: Density and Regression Curves
If the empirical distribution function estimates the distribution, what estimates its density—the smooth curve from which the histogram is a noisy sample? The histogram itself is the oldest density estimate, but it depends on arbitrary bin choices. The kernel density estimator, which places a small smooth bump (the kernel) at each observation and averages them, was proposed in various forms by Murray Rosenblatt (1956) and Emanuel Parzen (1962); their analysis showed how the smoothing width (the bandwidth) trades bias against variance, and how to choose it to minimize mean squared error. The estimator is now a standard tool, and the bandwidth-selection problem—too narrow and the curve is spiky, too wide and it is featureless—remains an active research area.
The same idea applied to regression: instead of fitting a line, fit a curve that follows the local average of the data. Kernel regression, local polynomial fitting (developed systematically by Jianqing Fan and Irene Gijbels in the 1990s), and smoothing splines—curves chosen to balance fit against roughness, with a penalty term controlling the trade-off—gave statisticians a flexible alternative to parametric regression. Splines, piecewise polynomials joined smoothly at knots, had been used in numerical analysis since the 1940s; their statistical theory, including the equivalence between smoothing splines and certain Bayesian and kernel methods, was worked out in the 1970s and 1980s by Grace Wahba and others. These methods made it possible to model a dose–response curve, a growth trajectory, or a spatial trend without guessing its functional form.
The Minimax Revolution: How Fast Can You Learn a Curve?
The 1980s brought a deeper theoretical question. If you estimate a curve nonparametrically, how accurate can any method possibly be? The answer depends on how smooth the curve is, and the framework that made this precise was minimax theory: for a class of possible curves (say, those with bounded second derivative), find the estimator whose worst-case error is smallest, and find the best possible worst-case error over all estimators15. Charles Stone's 1982 paper on optimal global rates of convergence established the rates for kernel and nearest-neighbor methods, and the field developed a vocabulary of convergence rates—n to the power −4/5 for twice-differentiable curves in one dimension, for instance—that told practitioners exactly what accuracy to expect and when a method was leaving performance on the table. This theory, associated with Stone, Peter Bickel, Alexandre Tsybakov, and others, is the backbone of modern nonparametric analysis: it explains why the curse of dimensionality makes high-dimensional smoothing hard, and it motivates the adaptive methods that try to learn the smoothness from the data.
Survival Analysis Without a Model
A major applied strand is nonparametric survival analysis, the analysis of time-to-event data when some subjects are still event-free at the end of study (censored)6. The Kaplan–Meier estimator, published by Edward Kaplan and Paul Meier in 1958, estimates the survival curve directly from censored data by multiplying together the conditional probabilities of surviving each successive event time61. It requires no assumption about the shape of the survival distribution and became the standard descriptive tool in medicine and engineering. The log-rank test, developed in the 1960s and 1970s, compares survival curves between groups using only the ordering of event times, and the proportional hazards model of David Cox (1972) is semiparametric: it leaves the baseline hazard unspecified while estimating the effect of covariates—a hybrid that shows how the nonparametric and parametric worlds can be combined.
Resampling: Let the Data Speak
The most influential late-twentieth-century development was the bootstrap, introduced by Bradley Efron in 19797. The idea is disarmingly simple: to estimate the variability of a statistic, resample the observed data with replacement many times, recompute the statistic on each resample, and look at the spread7. The empirical distribution function stands in for the unknown population, and the computer does the rest7. The bootstrap gave nonparametric confidence intervals and standard errors for statistics whose sampling distribution was unknown or intractable—correlation coefficients, ratios, regression coefficients—and it spread rapidly once computing became cheap. Related resampling ideas—the jackknife (developed by Maurice Quenouille in the 1940s and 1950s and named by Tukey), permutation tests (whose roots go back to R. A. Fisher and E. J. G. Pitman in the 1930s), and cross-validation for choosing smoothing parameters—form a family of computer-intensive methods that now underpin much of applied statistics.
Semiparametric Models and the Bayesian Turn
The boundary between parametric and nonparametric has proved productive rather than rigid. Semiparametric models combine a parametric part of interest (a treatment effect, a regression coefficient) with a nonparametric nuisance part (an unspecified baseline or error distribution)8. Cox's proportional hazards model is the classic example; another is the partially linear model, and the general theory of semiparametric efficiency, developed by Bickel, Chris Klaassen, Ya'acov Ritov, and Jon Wellner in their 1993 book, tells you the best possible precision for estimating the parametric part when the nuisance part is infinite-dimensional. This theory connects directly to modern causal inference, where doubly robust estimators combine a parametric outcome model with a nonparametric propensity model.
Meanwhile, Bayesian statisticians developed nonparametric Bayesian methods, which place a probability distribution over an infinite-dimensional space of functions or distributions rather than over a few parameters1. The Dirichlet process, introduced by Thomas Ferguson in 1973, is a distribution over distributions; it and its descendants (the Dirichlet process mixture, the Chinese restaurant process, Gaussian process priors) allow Bayesian inference with flexible, growing models. These methods became computationally practical with Markov chain Monte Carlo in the 1990s and remain a lively frontier.
The Present
Nonparametric statistics today is less a separate specialty than a set of tools and habits woven through all of statistics. Rank tests and the bootstrap are standard in applied work; kernel and spline smoothing underlie modern regression and machine learning; the minimax framework shapes how theorists think about any flexible model; and the semiparametric efficiency theory is the language of causal inference and high-dimensional statistics. The open questions are the old ones in new dress: how to choose smoothing parameters automatically and honestly, how to do nonparametric inference when the number of variables is large relative to the sample (where the curse of dimensionality bites hardest), how to give valid confidence statements after data-driven model selection, and how to reconcile the flexibility of nonparametric methods with the interpretability and efficiency that practitioners still want. The field's founding conviction—that you can learn a great deal from data without first assuming its shape—has never been more widely shared1.
References
- Nonparametric statistics (introduction, Definitions, Nonparametric tests, Minimax optimal rates of convergence), Wikipedia
- Sign test, Wikipedia
- Efficiency (statistics), Wikipedia
- Robust statistics (introduction, Breakdown point), Wikipedia
- Minimax estimator, Wikipedia
- Kaplan–Meier estimator, Wikipedia
- Bootstrapping (statistics) (History, introduction), Wikipedia
- Semiparametric model, Wikipedia
The frameworks of Nonparametric Statistics
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1933-Present
Empirical distribution function
How well does the sample step function estimate the true distribution?
- 1940-Present
Order-statistics and quantile inference
How can we estimate and test quantiles without a distributional model?
- 1945-Present
Distribution-free rank and sign inference
How can order alone support valid tests without distributional assumptions?
- 1948-1980
Asymptotic relative efficiency theory
How much efficiency is lost or gained by using ranks instead of values?
- 1956-Present
Kernel and spline smoothing
How do we estimate a smooth curve from noisy data?
- 1958-Present
Nonparametric survival analysis
How can survival be estimated when some subjects are censored?
- 1964-Present
Robust statistics
How can estimates resist outliers and slight model error?
- 1973-Present
Nonparametric Bayesian inference
How can we put a prior over whole distributions rather than parameters?
- 1979-Present
- 1982-Present
Minimax rate theory
What is the best possible worst-case accuracy for estimating a curve?
- 1993-Present
Semiparametric model
How can we combine a parametric part with an unspecified part?
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Order-statistics and quantile inference grew out of Empirical distribution function
Quantile inference treats the sample quantile as a functional of the empirical distribution function, and early asymptotic results for quantiles (standard error of the median, confidence intervals for quantiles) were obtained by inverting the EDF's binomial sampling behavior, a line consolidated in the 1940s and in Wilks's order-statistics results of 1948.
Carried by Samuel S. Wilks.
Kernel and spline smoothing grew out of Empirical distribution function
Kernel density estimation was formulated as the smoothed derivative of the empirical distribution function; Rosenblatt (1956) and Parzen (1962) analyzed the kernel estimator precisely by treating it as a smoothed EDF, so the EDF is the object whose smoothing the method studies.
Carried by Murray Rosenblatt; Emanuel Parzen.
Resampling and computer-intensive inference grew out of Empirical distribution function
The bootstrap replaces the unknown distribution by the empirical distribution function and resamples from it; Efron's 1979 paper presents the bootstrap explicitly as plugging the EDF into the functional of interest. Without the EDF as a consistent estimate of the population, the bootstrap has no object to resample from.
Carried by Bradley Efron.
Asymptotic relative efficiency theory grew out of Distribution-free rank and sign inference
Pitman (1948) computed the efficiency of rank and sign tests relative to the t-test, and the subsequent asymptotic normality theory for rank statistics (Hájek, Lehmann, 1950s–60s) made asymptotic relative efficiency the standard currency for comparing distribution-free and parametric procedures.
Carried by E. J. G. Pitman; Erich Lehmann; Jaroslav Hájek.
Minimax rate theory grew out of Kernel and spline smoothing
Stone's 1982 paper on optimal global rates of convergence derived minimax rates for kernel and nearest-neighbor estimators, taking the smoothing literature's bias–variance analyses and recasting them as worst-case optimality questions. Minimax rate theory grew directly out of the kernel/spline smoothing program.
Carried by Charles J. Stone.
Nonparametric Bayesian inference influenced by Empirical distribution function
Nonparametric Bayesian methods such as the Dirichlet process prior place a distribution over distributions centered on an EDF-like random measure; Ferguson's 1973 paper and later Polya-tree and Bayesian bootstrap constructions use the empirical distribution as the nonparametric anchor for posterior inference.
Carried by Thomas Ferguson.
Distribution-free rank and sign inference grew out of Order-statistics and quantile inference
Rank tests are built directly on order statistics: the Wilcoxon rank-sum test ranks the pooled sample, and the signed-rank test ranks absolute paired differences. The exact null distributions used in these tests came from order-statistics theory, and quantile/median thinking shaped the location-shift alternatives the rank tests target.
Carried by Frank Wilcoxon.
Resampling and computer-intensive inference influenced by Order-statistics and quantile inference
The bootstrap distribution of a sample quantile was among Efron's motivating examples, and resampling confidence intervals for quantiles became a standard application. Order-statistics theory supplied the exact finite-sample distributions against which bootstrap approximations were checked.
Carried by Bradley Efron.
Robust statistics influenced by Distribution-free rank and sign inference
Robust statistics drew on rank and sign procedures as exemplars of resistance to outliers; the median and trimmed mean, whose distribution theory came from order-statistics and rank work, became the prototypes for Huber's M-estimators and Hampel's influence-function analysis.
Carried by Peter Huber; Frank Hampel.
Robust statistics influenced by Asymptotic relative efficiency theory
Huber's 1964 M-estimator paper and the subsequent robustness literature compared robust estimators to classical ones by asymptotic variance and efficiency, directly reusing the asymptotic-relative-efficiency apparatus built for rank tests. Robustness adopted A.R.E. as its measure of the price of resistance.
Carried by Peter Huber.
Nonparametric Bayesian inference influenced by Kernel and spline smoothing
Smoothing splines were shown to have an exact Bayesian interpretation as posterior means under an integrated Wiener-process prior (Kimeldorf and Wahba, 1970), a result that carried kernel/spline smoothing into the nonparametric Bayesian framework and motivated later Bayesian curve estimation.
Carried by George Kimeldorf; Grace Wahba.
Semiparametric model influenced by Kernel and spline smoothing
Semiparametric models combine a parametric component with a nonparametric curve estimated by kernel or spline smoothing; the partially linear model and profile-likelihood methods for semiparametric regression were built using kernel and spline estimators as their nonparametric building block.
Carried by Peter Bickel; Jianqing Fan.
Semiparametric model influenced by Nonparametric survival analysis
Semiparametric survival models, notably the proportional hazards model with an unspecified baseline hazard, combine a parametric regression component with a nonparametric baseline estimated by a Nelson–Aalen/Kaplan–Meier-type construction. The semiparametric model framework grew out of nonparametric survival estimation.
Carried by David Cox.
Beyond the field
Where Nonparametric Statistics's ideas came from in other fields, and what they led to there: the strongest 12 of 44 links.
Order-statistics and quantile inference influenced Quantile Regression in Regression Analysis
Koenker and Bassett's 1978 paper on regression quantiles built directly on the classical theory of sample quantiles and order statistics, generalising the quantile of a distribution to a conditional quantile of a regression. The objective function minimised is an asymmetric absolute-loss check function whose minimiser for the intercept-only case is an order statistic.
Carried by Roger Koenker; Gilbert Bassett; Regression Quantiles (1978).
Nonparametric Bayesian inference grew out of Bayesian statistics in Statistical Inference
Nonparametric Bayesian inference is a branch of Bayesian statistics that replaces finite-dimensional priors with priors over function spaces, growing directly out of the Bayesian framework.
Carried by Thomas Ferguson.
Minimax rate theory grew out of Nonparametric inference in Statistical Inference
Minimax rate theory grew directly out of nonparametric inference, supplying optimal rates of convergence for density, regression and function estimation over smoothness classes.
Carried by Charles J. Stone; Richard Olshen.
Semiparametric model grew out of Nonparametric inference in Statistical Inference
Semiparametric models grew directly out of nonparametric inference by retaining a finite-dimensional parameter of interest alongside a nonparametric nuisance component.
Carried by Peter Bickel; Chris Klaassen.
Order-statistics and quantile inference influenced Nonparametric Survival Estimation in Survival Analysis
Order-statistics and quantile inference gave the product-limit estimator its mathematical basis: the Kaplan-Meier estimate is built from the ordered failure times and their ranks.
Carried by Edward L. Kaplan; Paul Meier.
Order-statistics and quantile inference influenced Quantile and Order-Statistics Summaries in Streaming Algorithms
Quantile and order-statistics summaries in streams took the statistical concept of order statistics and quantile estimation and adapted it to single-pass, bounded-memory computation.
Order-statistics and quantile inference influenced Estimating Equations and Semiparametric Inference in Statistical Inference
Estimating-equation and semiparametric methods use order statistics and quantile arguments, notably in quantile regression and rank-based estimating equations. The quantile-inference tradition supplied techniques taken up in semiparametric estimation.
Order-statistics and quantile inference influenced by Asymptotic inference in Statistical Inference
Asymptotic inference theory, especially the study of sample quantiles and their limiting distributions, provided the mathematical machinery for quantile inference. The asymptotic normality of sample quantiles was established in this tradition.
Order-statistics and quantile inference influenced Multiple testing and false discovery rate control in Statistical Inference
False discovery rate procedures and stepwise multiple-testing rules are built on order statistics of p-values and on quantile arguments. Benjamini and Hochberg's 1995 procedure works directly with the ordered p-values, a technique taken from order-statistics inference.
Carried by Yoav Benjamini; Yosef Hochberg.
Distribution-free rank and sign inference influenced by Design-based and randomization inference in Statistical Inference
Design-based randomization inference, especially permutation arguments for randomized experiments, supplied the combinatorial reasoning behind rank and sign tests. Wilcoxon's and Mann-Whitney's permutation justifications drew on this tradition.
Carried by Frank Wilcoxon; Henry Mann; Donald Ransom Whitney.
Asymptotic relative efficiency theory grew out of Nonparametric inference in Statistical Inference
Asymptotic relative efficiency theory was developed as a branch of nonparametric inference to compare distribution-free tests against parametric ones. Pitman efficiency and related measures were created for exactly this purpose.
Carried by E. J. G. Pitman.
Asymptotic relative efficiency theory grew out of Asymptotic inference in Statistical Inference
Asymptotic relative efficiency theory grew directly out of asymptotic inference, applying limiting-distribution comparisons of estimators and tests to rank and sign procedures. Pitman's and Noether's work in this tradition formalized the comparison.
Carried by E. J. G. Pitman.

Nonparametric Statistics is a planet in Nousfarer
Its city has a tower for each of these 11 frameworks. Land, walk in, and climb one concept at a time.
Land on Nonparametric StatisticsRelated fields
Fields its ideas are linked to
- Statistical Inference 19 links
- Survival Analysis 7 links
- Regression Analysis 3 links
- Biostatistics 2 links
- Statistical Learning 2 links
- Streaming Algorithms 1 link
- Game Theory 1 link
- Inverse Problems 1 link
- Mathematical Optimization 1 link
- Geometric Modeling 1 link
- Data Mining 1 link
- Operations Research 1 link
More in Statistics
- Bayesian Statistics
- Causal Inference
- Experimental Design
- Multivariate Statistics
- Regression Analysis
- Spatial Statistics
- Statistical Computing
- Statistical Inference
- Statistical Learning
- Statistics
- Survey Sampling
- Survival Analysis
- Time Series Analysis
See where Nonparametric Statistics sits on the Map of Human Knowledge