- Atlas
- Computer Science
- Data Mining
Data Mining
Data mining is the practice of searching large datasets for patterns that are useful, non-obvious, and not simply the product of chance.
11 frameworks, from 1957 to today · Data Science · Computer Science
- Cluster Analysis 1957-Present
- Exploratory Data Analysis 1965-Present
- Instance-Based Learning 1967-Present
- Anomaly detection 1980-Present
- Decision tree learning 1986-Present
- Knowledge Discovery in Databases 1989-Present
- Statistical Learning and Validation 1990-Present
- Association rule learning 1993-Present
- Ensemble Learning 1996-Present
- Cross-industry standard process for data mining 1999-Present
- Data Stream Mining 2000-Present
Overview
Data mining is the practice of searching large datasets for patterns that are useful, non-obvious, and not simply the product of chance1. Its practitioners build and apply algorithms that sort, group, predict, and flag: they classify customers, cluster documents, predict equipment failure, surface unusual transactions, and discover rules such as "people who buy diapers also buy beer" (a famous but largely apocryphal example from retail folklore)1. What is at stake is the difference between data that merely exists and data that informs a decision. Data mining sits at the meeting point of statistics, machine learning, and database engineering, and its history is the story of how those three communities learned to talk to one another1.
Statistics and Exploratory Data Analysis Before Computing
Long before the term existed, statisticians were mining data by hand1. In the nineteenth and early twentieth centuries, regression, correlation, and the analysis of variance gave researchers tools for finding relationships in tables of numbers. But these methods were built for confirmatory work: state a hypothesis, then test it. The exploratory side of statistics was less formalized until John Tukey, working at Bell Labs and Princeton, pushed it forward in the 1960s and 1970s. Tukey's 1977 book Exploratory Data Analysis argued that analysts should first look at data—with stem-and-leaf plots, box plots, and resistant summaries that are not thrown off by a few extreme values—before fitting models. The box plot, which shows a distribution's median, quartiles, and outliers in one compact graphic, is Tukey's most visible legacy. His message, that visual inspection and robust summaries reveal structure that formal tests can miss, became a founding assumption of data mining2.
Databases Grow Too Large to Inspect
Through the 1970s and 1980s, the relational database, formalized by Edgar Codd at IBM in 1970, made it possible to store and query enormous tables. But SQL queries answer questions you already know how to ask. By the late 1980s, businesses held gigabytes of transaction records and had no systematic way to ask "what patterns are in here?" The first workshop explicitly devoted to knowledge discovery in databases was held in 1989, and the field's vocabulary crystallized around that time1. Gregory Piatetsky-Shapiro, who organized that workshop, and others drew a distinction that has stuck: knowledge discovery in databases (KDD) is the whole process—selecting data, cleaning it, transforming it, mining it, and interpreting the results—while data mining is the algorithmic step in the middle1. The KDD conference series began in 1995, and the journal Data Mining and Knowledge Discovery followed in 1997. The process model that many practitioners still follow, CRISP-DM (Cross-Industry Standard Process for Data Mining), was published in 1999 by a consortium of European companies and research groups; it lays out six phases from business understanding through deployment, and its emphasis on iteration and business framing reflects the field's applied origins1.
Decision Trees and the Rise of Inductive Learning
The most influential early mining algorithm was the decision tree. A decision tree asks a sequence of yes/no questions about a record's attributes and ends in a prediction3. The idea traces to work in the 1960s, but the version that spread through data mining was ID3, published by Ross Quinlan in 1986, and its successor C4.5, published in 1993. ID3 chooses each split by information gain—a measure from Claude Shannon's information theory of how much a question reduces uncertainty about the outcome4. A concrete case: given records of weather conditions and whether a game was played, ID3 builds a tree whose first question might be "is it sunny?" because that attribute separates the outcomes most cleanly. Quinlan's trees were fast, produced rules humans could read, and handled real datasets with noise and missing values. They became the default first algorithm in countless courses and commercial tools, and their readable output made them the field's answer to the complaint that mining produces black boxes.
Nearest Neighbors and the Instance-Based View
A rival philosophy held that you need not build an explicit model at all5. Instance-based learning, articulated by David Aha, Dennis Kibler, and Marc Albert in 1991, and the k-nearest-neighbor classifier, whose statistical foundations were laid by Evelyn Fix and Joseph Hodges in the early 1950s and extended by Thomas Cover and Peter Hart in 1967, simply store the training examples. To classify a new case, find the k most similar stored cases and take a vote6. The method needs a notion of distance—usually Euclidean distance across numeric attributes—and it makes no assumptions about the shape of the underlying pattern67. Its weakness is equally plain: with millions of records, comparing a new case to every stored one is slow, and irrelevant attributes distort the distance. These trade-offs, interpretability versus flexibility, explicit model versus memorized examples, recur throughout the field's history.
Clustering: Finding Groups Without Labels
Cluster analysis addresses a different question: given data with no outcome to predict, can we find natural groupings18? The k-means algorithm, which alternates between assigning points to the nearest of k centers and recomputing those centers, was independently devised in the 1950s and 1960s (Stuart Lloyd's version circulated at Bell Labs from 1957; James MacQueen named it in 1967)9. Hierarchical clustering, which builds a tree of nested groupings, dates to the 1960s work of Joe Ward and others. The difficulty is that "natural grouping" is not self-defining: k-means will happily partition any dataset into k clusters whether or not real structure exists9. The field's response was to develop internal validation measures—silhouette scores, the gap statistic—and to insist that clustering results be checked against domain knowledge. Density-based methods such as DBSCAN, published by Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu in 1996, addressed a specific failure: k-means assumes clusters are roughly spherical and of similar size, while DBSCAN finds arbitrarily shaped clusters and labels sparse points as noise.
Association Rules and the Basket Problem
The algorithm that made data mining a household term in industry came from retail1. In 1993, Rakesh Agrawal, Tomasz Imieliński, and Arun Swami published AIS, and in 1994 Agrawal and Ramakrishnan Srikant published Apriori, for finding association rules: statements of the form "if a basket contains X, it often contains Y." Apriori's insight was a pruning principle—if an itemset is rare, every larger itemset containing it is also rare—which let it avoid checking the exponential number of possible combinations. The measure of interest was support (how often the rule's items appear together) and confidence (how often Y appears given X)1011. The method worked, and it spawned a decade of variants for efficiency and for other pattern types: sequential patterns, episodes, and frequent subgraphs. But it also exposed a problem that shaped the next phase: with low thresholds, the number of "interesting" rules explodes into the millions, most of them trivial or redundant. The study of interestingness measures—lift, conviction, and dozens of others—grew up around this problem, and researchers such as Sergey Brin, Rajeev Motwani, and Craig Silverstein argued in the late 1990s that statistical dependence, not raw frequency, should define what counts as a pattern worth reporting.
Outliers, Ensembles, and the Problem of Trust
Anomaly and outlier detection began as a statistical concern: an observation far from the rest may be an error or may be the most important record in the file. Tukey's exploratory work treated outliers as signals to investigate, and the field developed methods suited to high dimensions—distance-based definitions, local density measures such as the Local Outlier Factor of Breunig and colleagues in 2000, and later isolation-based methods. The practical stakes are concrete: fraud detection, network intrusion, and equipment failure all depend on flagging the rare case.
Meanwhile, the single-model approach ran into a wall. A decision tree trained on one sample of data can be unstable; small changes in the data produce different trees. Ensemble learning answered this by combining many models12. Bagging, introduced by Leo Breiman in 1996, trains models on bootstrap resamples and averages their votes. Boosting, developed by Yoav Freund and Robert Schapire in the late 1990s (AdaBoost, 1997), trains models sequentially, each focusing on the cases the previous ones got wrong. Breiman's Random Forests (2001) combined bagging with random attribute selection at each split and became one of the most reliable off-the-shelf miners. The empirical finding that ensembles of mediocre models often beat a single carefully tuned one was a genuine surprise and reshaped practice13.
Validation: The Discipline That Keeps Mining Honest
The field's credibility depended on a methodological turn as much as an algorithmic one. Mining millions of patterns guarantees that some will look significant by chance1. Statistical learning and validation supplied the safeguards: cross-validation, in which data is repeatedly split into training and held-out portions; the distinction between training error and generalization error; and the bias-variance trade-off, formalized in the 1990s, which explains why a model that fits its training data too closely fails on new data. The 2001 book The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani, and Jerome Friedman became the standard synthesis, and Vladimir Vapnik's support vector machines, developed from the 1960s onward and popularized in the 1990s, offered a principled alternative to trees and neighbors. The lesson that a pattern is only as good as its out-of-sample performance is now the field's first principle1.
Streams, Scale, and the Present
By the 2000s, data arrived continuously—clicks, sensor readings, transactions—and could no longer be stored and mined in batches. Data stream mining developed algorithms that process each record once, in bounded memory, and adapt as the distribution shifts1415. The landmark system was MOA (Massive Online Analysis), released in 2010 by Albert Bifet, Geoff Holmes, Richard Kirkby, and Bernhard Pfahringer, which collected stream classifiers and change detectors. The problem it solved is concrete: a model trained on last year's customer behavior silently degrades as behavior changes, and stream methods detect that drift and update14.
The present field is continuous with this history. Decision trees survive inside gradient-boosted ensembles such as XGBoost (2016), which dominate tabular prediction. Nearest-neighbor and clustering methods remain standard for exploration68. Association mining has receded from the spotlight but persists in recommendation and market-basket analysis. Deep learning has taken over domains with images, text, and speech, where hand-engineered features proved inadequate, but on the structured tables that data mining was built for, tree ensembles still often win. The open questions are the old ones sharpened: how to make complex models interpretable, how to mine data without amplifying bias, how to distinguish real structure from the patterns that any large dataset will contain, and how to keep models honest as the world they describe keeps changing1.
References
- Data mining (introduction, Data mining, Background, Etymology, Process, Standards, Results validation), Wikipedia
- Exploratory data analysis (Development), Wikipedia
- Decision tree learning, Wikipedia
- ID3 algorithm, Wikipedia
- Instance-based learning, Wikipedia
- K-nearest neighbors algorithm, Wikipedia
- Nearest neighbor search, Wikipedia
- Cluster analysis, Wikipedia
- K-means clustering (History), Wikipedia
- Association rule learning (History), Wikipedia
- Affinity analysis, Wikipedia
- Ensemble learning, Wikipedia
- Ensemble averaging (machine learning), Wikipedia
- Data stream mining, Wikipedia
- Streaming algorithm, Wikipedia
The frameworks of Data Mining
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1957-Present
- 1965-Present
- 1967-Present
Instance-Based Learning
Can we classify by comparing to stored examples?
- 1980-Present
- 1986-Present
- 1989-Present
Knowledge Discovery in Databases
How do we turn stored data into knowledge?
- 1990-Present
Statistical Learning and Validation
How do we know a pattern is real and not chance?
- 1993-Present
Association rule learning
Which items occur together more than expected?
- 1996-Present
- 1999-Present
Cross-industry standard process for data mining
What steps turn a business problem into a deployed model?
- 2000-Present
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Cross-industry standard process for data mining grew out of Knowledge Discovery in Databases
CRISP-DM grew directly out of the KDD distinction between the whole discovery process and the mining step, formalizing the select-clean-transform-mine-interpret cycle into six iterative phases.
Carried by Cross-Industry Standard Process for Data Mining (1999).
Anomaly detection influenced by Exploratory Data Analysis
Tukey's Exploratory Data Analysis treated outliers as signals worth investigating rather than mere errors, and this framing carried into later anomaly detection work that formalized distance- and density-based outlier definitions.
Carried by John Tukey; Exploratory Data Analysis (1977).
Knowledge Discovery in Databases influenced by Exploratory Data Analysis
Tukey's insistence that analysts first explore data visually and robustly before modeling became a founding assumption of the knowledge discovery in databases process, which places data exploration and interpretation around the mining step.
Carried by John Tukey; Gregory Piatetsky-Shapiro.
Ensemble Learning influenced by Decision tree learning
Ensemble methods were built to correct the instability of single decision trees; bagging and random forests combine many trees trained on resamples or random attribute subsets.
Carried by Leo Breiman; Random Forests (2001).
Association rule learning grew out of Knowledge Discovery in Databases
Agrawal's association rule algorithms gave the KDD field its signature pattern-discovery technique and its famous retail example, shaping what the community meant by a mined pattern.
Carried by Rakesh Agrawal; Apriori (1994).
Data Stream Mining grew out of Knowledge Discovery in Databases
Data stream mining grew out of the KDD agenda as practitioners confronted data too large or fast to store and mine in batches, adapting discovery algorithms to one-pass processing.
Knowledge Discovery in Databases influenced by Statistical Learning and Validation
Statistical learning contributed resampling-based validation and the concern with overfitting and chance patterns, which the KDD community adopted as criteria for what counts as a genuine discovery.
Ensemble Learning influenced by Statistical Learning and Validation
Boosting was developed within statistical learning theory by Freund and Schapire, and its margin-based analysis and validation practices came from that statistical tradition.
Carried by Yoav Freund; Robert Schapire.
Anomaly detection influenced by Cluster Analysis
Density-based clustering notions of local neighborhood density were taken up by Breunig and colleagues in the Local Outlier Factor, which measures how isolated a point is relative to its neighbors.
Carried by Markus M. Breunig.
Knowledge Discovery in Databases influenced by Cluster Analysis
Cluster analysis supplied unsupervised grouping methods that became one of the standard mining tasks within the knowledge discovery process.
Data Stream Mining influenced by Cluster Analysis
Streaming clustering algorithms adapted k-means and related partitioning methods to data that arrives continuously and cannot be revisited.
Anomaly detection influenced by Instance-Based Learning
Distance-based anomaly definitions reuse the nearest-neighbor distance computations central to instance-based learning, flagging points whose distance to their nearest neighbors is unusually large.
Statistical Learning and Validation influenced by Instance-Based Learning
The nearest-neighbor classifier's statistical foundations, laid by Fix and Hodges and extended by Cover and Hart, fed into the broader statistical learning account of nonparametric classification and its error bounds.
Carried by Thomas Cover; Peter Hart.
Data Stream Mining influenced by Association rule learning
Frequent-itemset and association rule techniques were adapted to streaming settings, where counts must be maintained over sliding windows rather than complete stored datasets.
Data Stream Mining influenced by Ensemble Learning
Ensemble techniques were adapted to streaming classification, where models must update incrementally as new examples arrive rather than being retrained on stored data.
Beyond the field
Where Data Mining's ideas came from in other fields, and what they led to there: the strongest 12 of 29 links.
Exploratory Data Analysis influenced by Statistical Graphics in Data Visualization
Tukey's exploratory data analysis drew directly on the tradition of statistical graphics, from Playfair through the scatterplots, histograms and box-like displays of nineteenth- and twentieth-century statistics, and made graphical display the core of exploratory practice.
Carried by John W. Tukey; Exploratory Data Analysis (1977).
Anomaly detection influenced by Detection theory in Biomedical Signal Processing
Detection theory, developed in radar and signal processing, supplied anomaly detection with its core formulation of separating signal from noise under false-alarm and detection probabilities, which data-mining anomaly detection adapted to multivariate data.
Decision tree learning led to Tree-Based Ensembles in Statistical Learning
Tree-based ensembles such as bagging and random forests were built directly on decision tree learning, combining many trees grown by the same recursive partitioning algorithms to reduce variance.
Carried by Leo Breiman.
Knowledge Discovery in Databases led to Cross-industry standard process for data mining in Data Science
CRISP-DM was built directly on the KDD process model, adopting its stages of selection, preprocessing, transformation, mining and interpretation and adding business understanding and deployment around them.
Association rule learning grew out of Data mining in Data Science
Association rule learning grew directly out of the data-mining programme of the early 1990s; the Apriori algorithm and market-basket analysis were framed as data-mining techniques for discovering patterns in large transaction databases.
Carried by Agrawal, Imielinski and Swami (1993); Agrawal and Srikant Apriori (1994).
Cross-industry standard process for data mining grew out of Data mining in Data Science
CRISP-DM was created by a consortium of data-mining practitioners and vendors to standardise the data-mining process, growing directly out of the data-mining field it codifies.
Carried by CRISP-DM consortium (1999).
Data Stream Mining grew out of Data mining in Data Science
Data stream mining is a branch of data mining that arose when the field turned to continuous, non-storable data; it reuses the mining task definitions (classification, clustering, frequent patterns) on streams.
Carried by Domingos and Hulten VFDT (2000).
Cluster Analysis influenced by Factor analysis in Multivariate Statistics
Factor analysis gave cluster analysis its central problem of grouping variables and cases by shared covariance structure, and early clustering work in psychometrics and statistics grew out of factor-analytic practice.
Cluster Analysis influenced by Principal component analysis in Multivariate Statistics
Principal component analysis supplied the dimension-reduction and projection techniques that clustering algorithms use to represent and group high-dimensional data, and PCA-based approaches to clustering were developed in the multivariate statistics literature from the 1950s onward.
Carried by Harold Hotelling.
Cluster Analysis influenced by Optimization in Applied Mathematics
Clustering was formulated as an optimization problem, minimizing within-cluster variance or similar criteria, drawing on the linear and nonlinear programming methods developed in operations research after 1947.
Cluster Analysis influenced Data mining in Data Science
Clustering methods developed in statistics and pattern recognition from the 1950s onward were taken up as core data-mining techniques, and cluster analysis is routinely presented as one of the main data-mining tasks.
Exploratory Data Analysis influenced Data mining in Data Science
Tukey's exploratory data analysis supplied the attitude and many of the techniques (residual plots, robust summaries, graphical exploration) that data mining adopted for finding patterns without prior hypotheses.
Carried by John Tukey; Exploratory Data Analysis (1977).

Data Mining is a planet in Nousfarer
Its city has a tower for each of these 11 frameworks. Land, walk in, and climb one concept at a time.
Land on Data MiningRelated fields
Fields its ideas are linked to
- Data Science 9 links
- Multivariate Statistics 3 links
- Statistical Learning 3 links
- Streaming Algorithms 2 links
- Machine Learning 1 link
- Randomized Algorithms 1 link
- Biomedical Signal Processing 1 link
- Big Data Processing 1 link
- Data Visualization 1 link
- Learning Sciences 1 link
- Public Health Informatics 1 link
- Business Analytics 1 link