- Atlas
- Social Sciences
- Information Retrieval
Information Retrieval
Information retrieval (IR) is the field concerned with finding material—documents, passages, images, or answers—that satisfies a person's information need, usually from a large store the person cannot read through themselves.
11 frameworks, from 1876 to today · Information Science · Social Sciences
- Documentation and Classification 1876-1950
- Coordinate Indexing and Post-Coordination 1947-1980
- Uniterm 1950-1970
- Statistical Indexing and Term Weighting 1957-Present
- Cranfield experiments 1958-Present
- Vector Space Model 1968-2000
- Probabilistic relevance model 1976-Present
- Human–computer information retrieval 1980-Present
- Link Analysis 1997-2010
- Learning to Rank 2005-Present
- Neural Information Retrieval 2013-Present
Overview
Information retrieval (IR) is the field concerned with finding material—documents, passages, images, or answers—that satisfies a person's information need, usually from a large store the person cannot read through themselves1. Its central artifacts are indexes and ranking systems: structures that let a machine decide, for a given query, which items are worth showing and in what order1. What is at stake is both technical and human. A retrieval system that ranks well saves a researcher hours; one that buries the right document under irrelevant ones can be worse than no system at all. And because "relevant" is a judgment made by people, IR has always had to wrestle with how to measure success at all. The field's history is largely a history of successive answers to two questions: how should documents and queries be represented, and how should we know whether a system works?
Documentation, Classification, and the Pre-Computational Index
Before electronic computers, the problem of retrieving recorded knowledge belonged to librarians and documentalists. Library catalogues and classification schemes—Dewey's decimal system from the 1870s, the Universal Decimal Classification developed by Paul Otlet and Henri La Fontaine around the turn of the twentieth century—organized books by subject so that a reader could browse to a shelf. Otlet's documentation movement went further, imagining a universal index of recorded knowledge on cards, with links between related topics.
These systems were pre-coordinated: the indexer decided in advance how concepts would be combined, so a subject like "the effect of pesticides on bees" got its own fixed heading or class number2. That made browsing coherent but rigid—a searcher could not easily combine terms the indexer had not anticipated. The card catalogue also imposed a physical limit: one card per heading, one place per card. The intellectual organization of knowledge was sophisticated, but the retrieval mechanism was manual and slow.
Coordinate Indexing and the Birth of Post-Coordination
The decisive break came with coordinate indexing, developed independently in several places in the mid-twentieth century. The idea: instead of assigning a pre-built compound heading, index each document by its individual concepts (terms, or later "uniterms"), and let the searcher combine them at search time2. This is post-coordination—coordination happens after indexing, at the moment of query2.
The best-known early implementation was Mortimer Taube's Uniterm system, introduced in the early 1950s at the U.S. Armed Forces Technical Information Agency and described in his 1953 book Studies in Coordinate Indexing. Each uniterm got a card listing the documents indexed by it; to find documents about, say, "copper" and "corrosion," a searcher compared the two cards and took the intersection2. The method was cheap, flexible, and allowed combinations the indexer never foresaw. It also exposed a new problem: false coordination, where terms that co-occur in a document's index have nothing to do with each other in meaning. The fix—and the foundation of modern search—was to let the machine do the intersecting, which pointed straight toward computer-based retrieval2.
Statistical Indexing and the Vector Space Model
Once computers could store and compare term lists, a new question opened: rather than a binary "this document has this term," could we weight terms by how informative they are? The intuition, formalized in the 1950s and 1960s, is that a term appearing in almost every document (like "the") discriminates poorly, while a rare term discriminates well. This is the basis of term weighting, and its most famous expression is inverse document frequency (IDF), commonly credited to Karen Spärck Jones's 1972 paper "A Statistical Interpretation of Term Specificity and Its Application in Retrieval." IDF down-weights terms that occur in many documents; combined with term frequency (how often a term appears in a given document), it yields the TF-IDF weighting that dominated retrieval for decades.
The framework that made these weights computable was the vector space model, set out by Gerard Salton and colleagues at Cornell in the late 1960s and 1970s. Each document and each query becomes a vector in a high-dimensional space whose axes are terms; relevance is estimated by the angle between query and document vectors, usually via cosine similarity. The model's appeal was that it turned retrieval into geometry: partial matches, ranked output, and term weighting all fell out naturally. Salton's SMART system (from the early 1960s onward) was the laboratory where these ideas were implemented and tested, and it trained a generation of researchers.
The Cranfield Paradigm: Making Evaluation Possible
None of this mattered unless one could tell whether a change in weighting actually improved results. The framework for that came from Cyril Cleverdon's work at the College of Aeronautics at Cranfield, England, beginning in the late 1950s. Cranfield built test collections—a fixed set of documents, a set of queries, and, crucially, human judgments of which documents were relevant to each query. A system could then be scored by recall (what fraction of relevant documents it found) and precision (what fraction of what it returned was relevant)13.
This Cranfield evaluation paradigm—test collections plus relevance judgments plus recall/precision measures—became the field's experimental backbone. It made retrieval research cumulative: competing methods could be compared on the same data. Its limitations were equally formative. Relevance judgments are expensive and subjective, and a fixed collection rewards tuning to that collection rather than to real users. The tension between laboratory control and real-world use runs through the rest of the field's history.
Probabilistic Retrieval
A rival to the geometric view asked a different question: given a query, what is the probability that a document is relevant1? The probability ranking principle, articulated by Stephen Robertson in the late 1970s, states that a system should rank documents by their estimated probability of relevance, since that ordering maximizes expected effectiveness. The binary independence model treated terms as independent and derived a ranking formula from term presence and absence.
The practical fruit was BM25, a ranking function developed by Robertson and Karen Spärck Jones in the 1990s (building on earlier work with Stephen Walker and others). BM25 combines term frequency, inverse document frequency, and a correction for document length, and it proved remarkably hard to beat—for years it was the default baseline against which new methods were measured, and it remains in wide use. Probabilistic retrieval's strength was that it grounded ranking in an explicit model of relevance rather than in geometric intuition; its weakness was the independence assumption, which real text violates.
Interactive Information Retrieval
Through the 1980s and 1990s, a countercurrent insisted that retrieval is not a one-shot query against a static collection but a process in which a person refines, reformulates, and learns. This is interactive information retrieval4. Its concerns are query reformulation, relevance feedback (letting a user mark results as relevant so the system can re-rank), interface design, and the staged nature of search sessions. The classic statement of the user's side is Carol Kuhlthau's information search process, which described search as moving through stages of uncertainty and increasing focus—a view that pushed IR toward studying searchers, not just systems. The practical legacy includes relevance feedback mechanisms and, later, the whole apparatus of query suggestion and session-aware ranking.
Link Analysis and the Web
The arrival of the World Wide Web in the 1990s broke the Cranfield assumptions: collections were enormous, heterogeneous, and constantly changing, and queries were short and ambiguous. A new signal became available—the link structure between pages1. Link analysis treats a hyperlink as a kind of endorsement: a page linked to by many important pages is itself likely important1. Google's PageRank, developed by Larry Page and Sergey Brin in the late 1990s, computed a page's importance from the pattern of links pointing to it, and combined it with text matching to rank web results. The related HITS algorithm, by Jon Kleinberg, identified "hub" and "authority" pages5. Link analysis made web search usable at scale and established that retrieval signals could come from outside the document text itself.
Learning to Rank
As search engines accumulated data—queries, clicks, and human relevance ratings—a natural next step was to learn the ranking function from examples rather than hand-tune it. Learning to rank treats ranking as a machine-learning problem: given features of query-document pairs (term matches, link scores, freshness, and so on), train a model to produce an ordering that matches human judgments6. Work in the 2000s, including the RankNet, LambdaRank, and ListNet methods, established the main approaches—pointwise, pairwise, and listwise—and the field moved from hand-crafted scoring formulas toward learned combination of many signals. This was the bridge from classical IR to modern machine learning.
Neural Information Retrieval
The most recent turn replaces hand-designed features with learned representations. Neural information retrieval uses neural networks to embed queries and documents into vectors, so that relevance becomes a similarity in a learned space rather than a count of shared terms1. Early neural ranking models in the 2010s were followed by dense retrieval, in which documents are encoded into dense vectors and searched by approximate nearest-neighbor methods—an approach that can match on meaning rather than exact words. The transformer architecture, introduced in 2017, made large pretrained language models available for ranking and, later, for generating answers directly. This has blurred the line between retrieval and generation: systems can now retrieve passages and synthesize a response, raising fresh questions about attribution and trust.
Where the Field Stands
Today's systems descend from every strand above. BM25-style lexical matching still anchors many pipelines; dense neural retrieval supplies semantic matching; learning-to-rank combines the signals; link and behavioral data inform web ranking; and evaluation still leans on Cranfield-style test collections, now supplemented by large-scale benchmarks and click logs. The open questions are the old ones in new dress: how to evaluate systems whose answers are generated rather than listed, how to handle relevance that depends on the individual searcher, and how to keep ranking fair and transparent when the signals are learned from data that carry their own biases.
References
- Information retrieval (introduction, Overview, Performance and correctness measures, First dimension: mathematical basis, History, Third Dimension: representational approach-based classification), Wikipedia
- Uniterm (introduction, History), Wikipedia
- Precision and recall, Wikipedia
- Human–computer information retrieval, Wikipedia
- HITS algorithm, Wikipedia
- Learning to rank, Wikipedia
The frameworks of Information Retrieval
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1876-1950
Documentation and Classification
How should recorded knowledge be organized for retrieval?
- 1947-1980
Coordinate Indexing and Post-Coordination
How can concepts be combined at search time?
- 1950-1970
- 1957-Present
Statistical Indexing and Term Weighting
Which terms best discriminate relevant documents?
- 1958-Present
- 1968-2000
Vector Space Model
How can documents and queries be compared geometrically?
- 1976-Present
Probabilistic relevance model
What is the probability a document is relevant?
- 1980-Present
Human–computer information retrieval
How do people search and refine over time?
- 1997-2010
- 2005-Present
- 2013-Present
Neural Information Retrieval
How can neural networks represent queries and documents?
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Coordinate Indexing and Post-Coordination reacted against Documentation and Classification
Coordinate indexing was defined against the pre-coordinated subject headings and classification schemes of documentation, which fixed concept combinations in advance. Taube's Uniterm system let searchers combine single concepts at query time, rejecting the rigidity of library classification.
Carried by Mortimer Taube; Studies in Coordinate Indexing (1953).
Vector Space Model grew out of Statistical Indexing and Term Weighting
The vector space model gave term weighting a geometric framework: documents and queries as vectors, with TF-IDF weights as components and cosine similarity as the ranking measure. Salton's SMART system implemented and tested these weights.
Carried by Gerard Salton; SMART.
Statistical Indexing and Term Weighting influenced by Uniterm
Uniterm's false coordination problem—terms co-occurring in an index without being related—motivated statistical weighting of terms by their specificity. The move from binary term presence to weighted terms addressed the same discrimination problem that uniterms exposed.
Carried by Karen Spärck Jones.
Statistical Indexing and Term Weighting influenced by Cranfield experiments
The Cranfield experiments provided the test collections and recall/precision measures needed to evaluate whether term weighting actually improved retrieval. Statistical indexing methods were among the first things tested under the Cranfield paradigm.
Carried by Cyril Cleverdon.
Vector Space Model influenced by Cranfield experiments
The Cranfield test collections and recall/precision measures let Salton's group evaluate vector-space weighting schemes against a fixed standard. The vector space model was developed and validated within the evaluation framework Cranfield established.
Carried by Cyril Cleverdon; Gerard Salton.
Probabilistic relevance model influenced by Cranfield experiments
Probabilistic retrieval was formulated and tested against Cranfield-style test collections, using relevance judgments to estimate the probability of relevance. The evaluation paradigm made it possible to compare probabilistic ranking with earlier methods.
Carried by Stephen Robertson; Karen Spärck Jones.
Human–computer information retrieval reacted against Cranfield experiments
Interactive information retrieval arose partly as a reaction to the Cranfield paradigm's one-shot, system-centered evaluation, insisting that retrieval is a process involving a user over time. It defined itself against laboratory test collections that ignored the searcher.
Carried by Carol Kuhlthau.
Probabilistic relevance model rival of Vector Space Model
The probabilistic relevance model competed with the vector space model as an account of ranking, grounding it in an explicit probability of relevance rather than geometric similarity. Both were developed and tested on the same Cranfield-style collections through the 1970s and 1980s.
Carried by Stephen Robertson; Gerard Salton.
Learning to Rank influenced by Vector Space Model
Learning to rank treats ranking as a machine-learning problem over features of query-document pairs, many of which are vector-space similarity scores and term-weighting features. The feature-based representation of documents and queries descends from the vector space model.
Learning to Rank influenced by Probabilistic relevance model
Learning to rank methods were evaluated against and often built on probabilistic ranking functions such as BM25, which supplied both baseline scores and features. The probabilistic model's ranking formula remained the standard to beat.
Carried by Stephen Robertson; Karen Spärck Jones.
Neural Information Retrieval influenced by Learning to Rank
Neural information retrieval adopted learning-to-rank training objectives and evaluation practices, treating ranking as a learning problem over query-document representations. Early neural ranking models were trained and benchmarked with the same labeled data and metrics.
Learning to Rank influenced by Link Analysis
Link-analysis signals such as PageRank became features in learned ranking functions for web search, alongside text-matching features. Learning to rank absorbed link structure as one input among many.
Carried by Larry Page; Sergey Brin.
Beyond the field
Where Information Retrieval's ideas came from in other fields, and what they led to there: the strongest 12 of 57 links.
Coordinate Indexing and Post-Coordination influenced by Faceted Classification in Knowledge Organization
Ranganathan's faceted classification (Colon Classification, 1933) supplied the idea of analyzing subjects into separate facets that could be combined at search time. This directly shaped post-coordinate indexing, where terms are combined after the fact rather than enumerated in advance.
Carried by S. R. Ranganathan; Colon Classification (1933).
Uniterm led to Boolean retrieval in Information Science
Uniterm indexing, with its cards combined by matching term numbers, is the direct ancestor of Boolean retrieval: Taube's uniterm system combined single-term postings with AND, OR and NOT logic. Boolean retrieval generalized this post-coordinate combination into a query model.
Carried by Mortimer Taube.
Statistical Indexing and Term Weighting influenced Vector space model in Information Science
Statistical indexing and term weighting, including inverse document frequency and term frequency weighting, supplied the components that the vector space model combined into its similarity measure. Salton's model built directly on these weighting schemes.
Carried by Gerard Salton.
Statistical Indexing and Term Weighting influenced Probabilistic retrieval in Information Science
Probabilistic retrieval models such as Robertson and Sparck Jones's relevance weighting took term-frequency and inverse-document-frequency weighting from statistical indexing work of the 1950s and 1960s and recast it in probabilistic terms.
Carried by Stephen E. Robertson; Karen Sparck Jones.
Cranfield experiments led to Information Retrieval Evaluation in Knowledge Organization
The Cranfield experiments established the test-collection methodology, with precision and recall measures, on which information retrieval evaluation was built. Later evaluation programs such as TREC grew directly out of Cranfield's design.
Carried by Cyril Cleverdon.
Vector Space Model influenced by Vector space in Linear Algebra
Salton's vector space model represents documents and queries as vectors in a term space and measures similarity by vector operations such as the cosine. The mathematical notion of a vector space supplied the formal apparatus.
Carried by Gerard Salton.
Vector Space Model influenced Word embedding in Distant Reading And Cultural Analytics
Word embeddings descend from the vector space model of information retrieval: documents and terms represented as vectors in a high-dimensional space with similarity measured geometrically. Word2vec and similar methods keep that representation while learning the vectors from context.
Carried by Gerard Salton; Tomas Mikolov.
Probabilistic relevance model grew out of Information retrieval in Information Science
The probabilistic relevance model was built as a formalisation of the relevance-ranking problem within information retrieval, growing out of the Cranfield tradition of measuring retrieval by relevance judgements.
Carried by Stephen E. Robertson; Karen Spärck Jones.
Human–computer information retrieval grew out of Information retrieval in Information Science
Human-computer information retrieval grew directly out of information retrieval as a branch that added the user and the interactive search session to the system-centred model.
Link Analysis grew out of Information retrieval in Information Science
Link analysis in retrieval grew out of information retrieval as a branch that exploited hyperlink structure, rather than term content, to rank web documents.
Carried by Sergey Brin; Lawrence Page.
Learning to Rank grew out of Information retrieval in Information Science
Learning to rank grew directly out of information retrieval as a branch that replaced hand-tuned ranking functions with machine-learned ones trained on relevance judgements.
Neural Information Retrieval influenced by Deep learning in Artificial Intelligence
Deep learning supplied the neural architectures, word embeddings and gradient-trained models that neural retrieval adopted for representing queries and documents.

Information Retrieval is a planet in Nousfarer
Its city has a tower for each of these 11 frameworks. Land, walk in, and climb one concept at a time.
Land on Information RetrievalRelated fields
Fields its ideas are linked to
- Information Science 23 links
- Knowledge Organization 14 links
- Artificial Intelligence 2 links
- Linear Algebra 2 links
- ML Systems 2 links
- Human Computer Interaction 2 links
- Machine Learning 1 link
- Graph Algorithms 1 link
- Digital Archives And Collections 1 link
- Distant Reading And Cultural Analytics 1 link
- ML Econometrics 1 link
- Cognitive Hci 1 link
More in Information Science
- Archival Science
- Bibliometrics
- Digital Curation
- Digital Libraries
- Information Behavior
- Information Ethics
- Information Science
- Knowledge Organization
- Records And Information Management
- Scholarly Communication
See where Information Retrieval sits on the Map of Human Knowledge