Psychometrics

Psychometrics is the science of measuring psychological attributes—intelligence, personality, attitudes, abilities, and more—by assigning numbers to them according to explicit rules.

13 frameworks, from 1880 to today · Psychology · Social Sciences

still a working view ended · each leads to its entry below

Overview

Psychometrics is the science of measuring psychological attributes—intelligence, personality, attitudes, abilities, and more—by assigning numbers to them according to explicit rules1. Its central problem is deceptively simple: how do you know whether a test measures what it claims to measure, and how do you know whether the numbers it produces mean the same thing across people, occasions, and groups? The answers to those questions have shaped education, employment, clinical diagnosis, and social policy for over a century. The history of psychometrics is the history of refining those answers—from early efforts to rank people by mental ability, through statistical theories of error and latent traits, to contemporary models that test whether a measurement works the same way across cultures and contexts.

Mental Testing and the First Measurement Models

Psychometrics grew out of nineteenth-century attempts to measure the mind scientifically1. Francis Galton, studying individual differences in sensory discrimination and reaction time in the 1880s, pioneered the use of statistical methods—including correlation—to describe how traits vary across people. His anthropometric laboratory in London collected data on thousands of visitors, treating mental characteristics as measurable quantities distributed continuously in a population.

James McKeen Cattell brought these methods to the United States, coining the term "mental test" in 1890 for a battery of sensory and motor tasks. But it was Alfred Binet, working with Théodore Simon in Paris, who made the decisive break. Commissioned in 1904 to identify schoolchildren who needed special education, Binet abandoned the sensory approach and built tasks that directly sampled complex mental functions—memory, comprehension, judgment, reasoning. The 1905 Binet–Simon scale arranged items by age level and introduced the idea of mental level: a child's performance could be summarized as the age at which typical children performed similarly. Lewis Terman's 1916 Stanford revision introduced the intelligence quotient (IQ)—originally the ratio of mental age to chronological age multiplied by 100—and the scale became the template for individual intelligence testing.

The First World War accelerated the enterprise. The U.S. Army needed to classify millions of recruits quickly, and the Army Alpha (verbal) and Army Beta (nonverbal) tests, developed by Robert Yerkes and colleagues, demonstrated that group testing was feasible. The war also exposed problems: scores depended heavily on literacy and cultural background, and the tests were used to make claims about "racial" differences in intelligence that later scholarship discredited. The episode taught psychometricians that a test's numbers are only as meaningful as the inferences drawn from them.

Classical Test Theory and the Problem of Error

As testing spread through schools, industry, and the military in the 1920s and 1930s, a theoretical framework emerged to make sense of scores. Classical test theory (CTT)—also called true-score theory—rests on a simple model: any observed score is the sum of a person's true score plus random error2. The true score is defined as the average score a person would obtain over infinitely many independent testings. From this model follow the concepts that organized the field for decades: reliability (the consistency of scores across occasions or item sets), standard error of measurement (the expected variability of observed scores around the true score), and the attenuation of correlations by measurement error.

The practical payoff was enormous. CTT gave test developers tools to estimate reliability—split-half correlations, test–retest correlations, internal consistency coefficients such as Cronbach's alpha (introduced by Lee Cronbach in 1951)—and to correct observed correlations for unreliability. It also clarified that a test's reliability is not a fixed property of the instrument but depends on the population tested and the conditions of administration. The framework's limitation was equally clear: its statistics were test-level, not item-level3. Two tests could have the same reliability but differ radically in which items were hard or easy, and CTT could not predict how a person would perform on a new set of items drawn from the same domain.

Factor Analysis and the Structure of Traits

A parallel development concerned what tests measure, not just how reliably. Charles Spearman, studying correlations among school subjects in 1904, observed that children who did well in one subject tended to do well in others4. He proposed that a single general factor—which he called g—underlay performance on all cognitive tasks, alongside task-specific factors. This was the origin of factor analysis: a statistical method for explaining correlations among many observed variables in terms of a smaller number of unobserved (latent) factors4.

The method was extended by Louis Thurstone in the 1930s and 1940s. Thurstone's multiple-factor analysis, applied to a battery of fifty-six tests, yielded several distinct primary mental abilities—verbal comprehension, numerical facility, spatial visualization, perceptual speed, and others—rather than a single g. The dispute between Spearman's general factor and Thurstone's multiple abilities was partly resolved by the recognition that the two models could coexist: a hierarchical structure with g at the top and specific abilities below. Raymond Cattell and John Horn later proposed a distinction between fluid intelligence (reasoning ability relatively independent of acquired knowledge) and crystallized intelligence (knowledge and skills acquired through experience), a distinction that remains influential.

Factor analysis became the workhorse of personality psychology as well4. Hans Eysenck's dimensional models of personality in the 1940s and 1950s, and the lexical studies that eventually produced the Big Five model of personality traits, all relied on factor-analytic methods to reduce large sets of questionnaire items to a few underlying dimensions. The method's appeal was that it made the structure of psychological attributes an empirical question rather than a matter of theoretical preference.

Attitude Scaling and the Measurement of Opinion

Not all psychometrics concerned abilities. In the 1920s and 1930s, researchers developed methods for measuring attitudes—evaluations of objects, people, or ideas—that were as rigorous as intelligence testing. Louis Thurstone's 1928 paper "Attitudes Can Be Measured" introduced the method of paired comparisons and the law of comparative judgment, which allowed stimuli (including statements of opinion) to be placed on an interval scale based on how often they were judged more favorable or less favorable than others. Thurstone and E. J. Chave applied the method to attitudes toward the church and toward war.

Rensis Likert's 1932 approach was simpler and more practical: respondents rated their agreement with statements on a five-point scale, and item scores were summed to produce a total attitude score. Likert scaling, with its emphasis on internal consistency and item-total correlations, became the dominant method for survey research5. The two approaches represented a lasting tension in psychometrics: Thurstone's method was theoretically elegant but laborious; Likert's was efficient but assumed that the intervals between response categories were equal. Both contributed to the broader enterprise of measuring subjective states—opinions, values, satisfaction—that had no external criterion against which to validate them.

Criterion-Keyed Tests and the Empirical Turn

A different tradition emerged from applied settings where the goal was prediction rather than theory. In the 1930s and 1940s, psychologists building selection tests for the military and industry developed criterion-keyed construction: items were retained not because they fit a theoretical model but because they empirically distinguished groups known to differ on a criterion. The Minnesota Multiphasic Personality Inventory (MMPI), published by Starke Hathaway and J. C. McKinley in 1943, was built this way—items were selected because they differentiated psychiatric patients from normal controls. The Strong Vocational Interest Blank, developed by Edward Strong in the 1920s and revised over decades, used a similar logic to predict occupational membership.

Criterion-keying was powerful but theoretically opaque: a test could predict well without anyone knowing why. It also made the test vulnerable to criterion contamination—if the criterion itself was biased, the test would inherit that bias. The approach nonetheless dominated applied testing for decades and remains the basis of many clinical and personnel selection instruments.

Construct Validity and the Meaning of Scores

The question of what a test measures—as opposed to whether it predicts a criterion—came to a head in the 1950s. The American Psychological Association's 1954 committee on test standards, chaired by Lee Cronbach, distinguished several types of validity: predictive, concurrent, content, and construct. Construct validity was the most ambitious: it asked whether a test actually measures the theoretical attribute it claims to measure, and it required an ongoing process of theory testing—showing that the test correlates with measures it should correlate with (convergent evidence) and fails to correlate with measures it should not (discriminant evidence).

Cronbach and Paul Meehl's 1955 paper "Construct Validity in Psychological Tests" made the case that validating a test is not a single study but a research program. The idea was that a test's meaning is embedded in a nomological network—a web of theoretical relationships among constructs—and that evidence for validity accumulates as the network is tested. This reframed validity from a property of the test to a property of the inferences drawn from scores6. The 1966 and 1974 APA standards progressively elevated construct validity, and by the 1980s the field had largely converged on the view that all validity is construct validity—that content and criterion evidence are contributions to the same inferential argument.

Item Response Theory and the Latent-Trait Revolution

Classical test theory's limitations became acute in the 1950s and 1960s as testing expanded and computers became available. The key limitation was that CTT statistics depended on the particular sample tested: an item's difficulty was defined relative to the group, and a person's ability was defined relative to the test. Item response theory (IRT) offered a solution13. Building on earlier work by Louis Guttman and others, researchers including Frederic Lord, Allan Birnbaum, and Georg Rasch developed models that specified the probability of a correct response as a function of a person's latent ability and item parameters (difficulty, discrimination, and sometimes guessing).

The Rasch model, introduced by Danish mathematician Georg Rasch in 1960, was the simplest: it assumed that the probability of a correct response depended only on the difference between a person's ability and an item's difficulty. The two-parameter logistic model, developed by Birnbaum in the late 1950s and published in 1968, added item discrimination; the three-parameter model added a guessing parameter. IRT's great advantage was invariance: item parameters estimated from one sample could, in principle, be used to predict performance in another, and person abilities could be estimated independently of the particular items administered1. This made possible computerized adaptive testing (CAT), in which each item is selected based on the examinee's estimated ability, shortening tests while maintaining precision. Lord's 1970s work on tailored testing and the later development of large-scale CAT programs—such as the Graduate Record Examination and the National Council Licensure Examination—demonstrated the practical value.

IRT also provided a framework for detecting differential item functioning (DIF): items that behave differently for members of different groups after controlling for ability7. This became a central tool for fairness analysis in educational and employment testing.

Structural Equation Modeling and the Integration of Measurement with Theory

By the 1970s, psychometricians had two powerful toolkits—factor analysis for latent structure and path analysis for causal relationships among observed variables—but they were separate. Structural equation modeling (SEM) unified them. Building on work by Karl Jöreskog, who developed the LISREL software in the early 1970s, and earlier contributions by Sewall Wright (path analysis) and others, SEM allowed researchers to specify models in which latent variables (measured by multiple indicators, as in factor analysis) were related to one another and to observed variables through a system of simultaneous equations.

The confirmatory factor analysis (CFA) model, a special case of SEM, became the standard method for testing whether a set of items measures the hypothesized latent constructs. Instead of exploratory factor analysis, which let the data determine the structure, CFA allowed researchers to specify a structure in advance and test how well it fit89. Fit indices—chi-square, comparative fit index (CFI), root mean square error of approximation (RMSEA)—became standard reporting practices. SEM made it possible to test complex theoretical models—mediational pathways, longitudinal change, latent growth—within a single coherent framework, and it became one of the most widely used statistical methods in psychology and the social sciences.

Measurement Invariance and the Problem of Comparison

As testing spread across cultures and as group comparisons became politically and scientifically salient, a new question arose: does a test measure the same construct in the same way across groups10? Measurement invariance (also called measurement equivalence) is the condition under which scores from different groups can be meaningfully compared10. If an item is interpreted differently by members of different cultures, or if the factor structure differs, then observed mean differences may reflect measurement artifacts rather than true differences in the attribute10.

The framework for testing invariance developed through the 1980s and 1990s, drawing on confirmatory factor analysis. Researchers test progressively stricter models: configural invariance (the same items load on the same factors), metric invariance (factor loadings are equal across groups), scalar invariance (item intercepts are equal), and strict invariance (residual variances are equal). Each level permits different kinds of comparisons. The framework became essential in cross-cultural psychology, international educational assessments (such as PISA and TIMSS), and fairness reviews of employment tests. It also exposed how often tests fail invariance—a finding with profound implications for the interpretation of group differences.

Multilevel Modeling and the Structure of Nested Data

Psychological data are rarely independent. Students are nested in classrooms, employees in organizations, repeated observations in persons11. Traditional statistical methods assumed independence and treated all observations as exchangeable, which could distort standard errors and lead to incorrect inferences. Multilevel modeling (also called hierarchical linear modeling), developed in the 1980s and 1990s by researchers including Harvey Goldstein, Stephen Raudenbush, and Anthony Bryk, provided methods for analyzing data with such nested structures.

In psychometrics, multilevel models addressed questions that single-level models could not: How much of the variance in a test score is due to differences between individuals versus differences between the contexts they inhabit11? How do item properties vary across schools or countries? Multilevel confirmatory factor analysis allowed researchers to separate within-group and between-group factor structures—a critical issue for understanding whether a construct has the same meaning at different levels of aggregation. The approach also connected to generalizability theory, which decomposes measurement error into multiple sources (occasions, raters, items) rather than treating it as a single undifferentiated quantity.

Bayesian Psychometrics

The most recent major development is the rise of Bayesian methods. Bayesian statistics treats parameters as having probability distributions rather than fixed unknown values, and it combines prior information with observed data to produce posterior distributions12. In psychometrics, Bayesian approaches offer solutions to problems that frequentist methods handle awkwardly: small samples, complex hierarchical models, and the incorporation of substantive prior knowledge.

Bayesian item response theory, developed from the 1990s onward by researchers including Albert, Patz, and Junker, allows the estimation of IRT models with small samples or sparse data by borrowing strength across items or persons through hierarchical priors. Bayesian estimation has become standard in IRT software such as BILOG-MG and in more recent packages. Bayesian methods have also been applied to factor analysis, SEM, and multilevel models, where they provide a natural framework for handling uncertainty in complex models. The approach remains a minority tradition in psychometrics but is growing, particularly in settings where data are scarce or where prior knowledge is substantial.

The Present Landscape

Contemporary psychometrics is a mature quantitative discipline that sits at the intersection of psychology, statistics, and educational measurement1. The classical concerns—reliability, validity, fairness—remain central, but the methods have become far more sophisticated1. IRT and its extensions (multidimensional IRT, cognitive diagnostic models) are standard in large-scale testing. SEM and multilevel modeling are routine in psychological research113. Measurement invariance testing is expected in any cross-group comparison. Bayesian methods are increasingly available and used14.

New challenges have emerged. The replication crisis in psychology has raised questions about the psychometric properties of widely used measures—many of which were validated on small samples and never rigorously tested for invariance or construct validity. The open-science movement has pushed for greater transparency in test development and validation. Computerized adaptive testing and online assessment have changed how tests are administered and scored15. And the rise of machine learning has introduced new methods for predicting outcomes from test data, raising questions about interpretability and fairness that the field is still working through.

The questions that remain open are recognizably the same ones that animated the field's founders: What does a score mean? For whom does it mean that? And how do we know? The difference is that psychometricians now have a century of theory and method—and a century of mistakes—to draw on in answering them.

References

  1. Psychometrics (introduction, Definition of measurement in the social sciences, Historical foundation, Theoretical approaches, Key concepts, Standards of quality), Wikipedia
  2. Classical test theory, Wikipedia
  3. Item response theory, Wikipedia
  4. Factor analysis (History, introduction), Wikipedia
  5. Likert scale, Wikipedia
  6. Construct validity, Wikipedia
  7. Differential item functioning, Wikipedia
  8. Exploratory factor analysis, Wikipedia
  9. Confirmatory factor analysis, Wikipedia
  10. Measurement invariance, Wikipedia
  11. Multilevel model, Wikipedia
  12. Bayesian statistics, Wikipedia
  13. Structural equation modeling, Wikipedia
  14. Bayesian inference (History), Wikipedia
  15. Computerized adaptive testing, Wikipedia

The frameworks of Psychometrics

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1880-1960
  2. 1904-Present

    Factor Analysis

    What latent factors explain test correlations?

    Fly to its tower in Nousfarer

  3. 1905-Present
  4. 1920-Present
  5. 1927-1980
  6. 1928-1970

    Attitude Scaling

    How can attitudes be measured?

    Fly to its tower in Nousfarer

  7. 1955-Present

    Construct Validity

    Does a test measure what it claims?

    Fly to its tower in Nousfarer

  8. 1960-Present

    Item Response Theory

    How do item responses relate to ability?

    Fly to its tower in Nousfarer

  9. 1970-Present
  10. 1970-Present

    Structural Equation Modeling

    How do latent variables relate causally?

    Fly to its tower in Nousfarer

  11. 1985-Present

    Measurement Invariance

    Does a test measure the same across groups?

    Fly to its tower in Nousfarer

  12. 1985-Present

    Multilevel model

    How do nested contexts affect measurements?

    Fly to its tower in Nousfarer

  13. 1995-Present

    Bayesian Psychometrics

    How can prior knowledge improve measurement?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Mental Testing and the IQ TraditionFactor AnalysisIntelligence Scale ConstructionClassical Test TheoryCriterion-Keyed Test ConstructionAttitude ScalingConstruct ValidityItem Response TheoryComputerized Adaptive TestingStructural Equation ModelingMeasurement InvarianceMultilevel modelBayesian Psychometrics

Beyond the field

Where Psychometrics's ideas came from in other fields, and what they led to there: the strongest 12 of 53 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Psychometrics is a planet in Nousfarer

Its city has a tower for each of these 13 frameworks. Land, walk in, and climb one concept at a time.

Land on Psychometrics

Related fields

Fields its ideas are linked to

More in Psychology

See where Psychometrics sits on the Map of Human Knowledge