- Atlas
- Medicine & Health
- Comparative Effectiveness Research
Comparative Effectiveness Research
Comparative effectiveness research (CER) asks a deceptively simple question: among the treatments that already exist for a given condition, which one actually works better, for whom, and at what cost?
10 frameworks, from 1970 to today · Health Services Research · Medicine & Health
- Outcomes Research 1970-1995
- Cost-Effectiveness Analysis 1975-Present
- Health Technology Assessment 1985-Present
- Observational Comparative Analysis 1985-Present
- Systematic review 1985-Present
- Heterogeneity of Treatment Effect 1989-Present
- Pragmatic clinical trial 2000-Present
- Learning Health Systems 2005-Present
- Patient-Centered Outcomes Research 2005-Present
- Target Trial Emulation 2010-Present
Overview
Comparative effectiveness research (CER) asks a deceptively simple question: among the treatments that already exist for a given condition, which one actually works better, for whom, and at what cost? Unlike the earlier stages of clinical research, which mainly test whether a new drug or device beats a placebo under tightly controlled conditions, CER compares real alternatives as they are used in ordinary practice. Its stakes are practical. Clinicians face competing options with little head-to-head evidence; payers must decide what to cover; patients want to know whether the newer, costlier choice is worth it. The history of CER is the story of how medicine learned to answer those questions with methods rigorous enough to trust — and of the political fights that erupted once the answers began to matter to money and markets.
Precursors: Outcomes Research and the Variation Puzzle
The intellectual roots lie in the 1970s and 1980s, when health services researchers noticed something puzzling. John Wennberg and colleagues documented that rates of common surgeries — tonsillectomy, hysterectomy, prostatectomy — varied several-fold between neighbouring communities with similar populations. If the evidence were clear, such variation should not exist. The finding implied that much medical practice rested on custom rather than proof, and it gave rise to what became known as outcomes research: the study of what actually happens to patients, using routine data rather than controlled experiments.
A landmark expression was the Medical Outcomes Study, begun in the 1980s by John Ware and colleagues, which developed ways to measure patients' functional status and well-being rather than just laboratory values. Its short-form questionnaire, the SF-36, became a standard instrument for capturing health status in populations. Around the same time, the RAND Health Insurance Experiment (1970s–1980s) showed that cost-sharing changed how people used care and, for some, their health — an early demonstration that system arrangements, not just treatments, are proper objects of comparison.
These strands — variation, outcomes measurement, and a suspicion that much care lacked an evidence base — set the stage. What was missing was a disciplined way to synthesize the scattered evidence and a way to compare alternatives directly.
Systematic Review and Meta-Analysis: Making the Evidence Add Up
By the late 1980s, the problem was not a shortage of trials but a flood of them, scattered across journals in conflicting directions. Individual studies were often too small to detect modest but real differences. The solution was the systematic review — a structured, reproducible search and appraisal of all relevant studies on a question — combined with meta-analysis, a statistical technique for pooling their results into a single estimate.
The approach was formalized through the 1980s and 1990s, notably by researchers associated with the Cochrane Collaboration, founded in 1993 and named for the British epidemiologist Archie Cochrane, who had argued that medicine should be built on systematic summaries of evidence. The Oxford Centre for Evidence-Based Medicine, established in the 1990s, popularized hierarchies ranking study designs. Meta-analysis could, for example, pool many small trials of a treatment and reveal a benefit that no single trial could confirm. But it also exposed a hazard: combining biased studies produces a confident but wrong answer, which pushed the field toward tools for assessing publication bias and heterogeneity.
Cost-Effectiveness and Decision Modeling: Comparing Value, Not Just Effect
Comparing treatments raised a second question: even if one option is more effective, is it worth the extra cost? Decision modeling and cost-effectiveness analysis gave CER a quantitative vocabulary. The central measure is the quality-adjusted life year (QALY), which combines length of life with its quality, so that a year in poor health counts for less than a year in full health1. A treatment's cost per QALY gained then allows unlike interventions to be compared on a common scale.
The approach was developed and codified from the 1970s onward, gaining institutional form through bodies such as the U.S. Panel on Cost-Effectiveness in Health and Medicine, convened in the 1990s, which issued methodological recommendations. Its most influential home became the UK's National Institute for Health and Care Excellence (NICE), established in 1999, whose appraisals weigh cost per QALY when deciding what the National Health Service will fund. The method made explicit trade-offs that had previously been hidden — and made it politically combustible, because a threshold above which a treatment is judged too expensive to fund is a rationing rule by another name.
Health Technology Assessment: Institutionalizing the Comparison
Health technology assessment (HTA) grew out of the same impulse: a formal process of evaluating the clinical, economic, and social consequences of health technologies to inform policy2. Early national agencies appeared in the 1980s — Sweden's SBU, established in 1987, is often cited as an early example — and the model spread across Europe and beyond. HTA brought CER into government: agencies synthesize evidence, commission economic models, and issue guidance that payers and clinicians may follow.
The United States took a more contested path. The Agency for Health Care Policy and Research, created in 1989, was attacked by surgical and industry groups after it published a guideline on back pain, and it was nearly eliminated; it survived reorganized as the Agency for Healthcare Research and Quality (AHRQ). The episode illustrated a recurring theme: CER's findings can threaten established practice and commercial interests, and its institutions are therefore politically fragile.
Observational Comparative Analysis and the Challenge of Confounding
Randomized trials are the gold standard for causal inference, but they are slow, expensive, and often unrepresentative of ordinary patients. Much CER has therefore relied on observational data — insurance claims, registries, electronic health records — comparing patients who happened to receive different treatments. The central danger is confounding: patients who receive one treatment may differ systematically from those who receive another, so an apparent effect may reflect who they were rather than what was done to them.
The field developed a toolkit to address this, including propensity scores — a method, advanced by Paul Rosenbaum and Donald Rubin in the 1980s, for balancing comparison groups on measured characteristics — and a discipline of sensitivity analysis for unmeasured differences. These methods made large observational comparisons possible, and debates over their trustworthiness have run through the field ever since.
Heterogeneity of Treatment Effect: One Size Does Not Fit All
A persistent finding of CER is that average effects conceal variation. A treatment that helps most patients may harm a subgroup, and the average can mask both. The study of heterogeneity of treatment effect asks which patient characteristics — age, comorbidity, genotype, preference — modify how well an intervention works.
One influential demonstration became a cornerstone of the field: the 1995 report of the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial's precursor analyses, and more famously the finding in the Cardiac Arrhythmia Suppression Trial, published in 1989, that antiarrhythmic drugs which reduced a surrogate marker (irregular heartbeats) actually increased mortality. Surrogate endpoints could mislead, and subgroup analysis could reveal that a therapy's value depended on who received it. Statisticians warned that searching many subgroups for significance produces false positives, so the field developed pre-specified subgroup analysis and interaction testing to distinguish real effect modification from noise. Recognizing heterogeneity is what makes the "for whom" in CER's central question operational.
Pragmatic Trials: Testing Treatments in the Real World
If observational studies risk confounding and explanatory trials risk irrelevance to ordinary practice, a middle path is the pragmatic trial: a randomized comparison conducted in routine settings, with broad eligibility, usual-care comparators, and outcomes that matter to patients and payers. The distinction between explanatory (idealized) and pragmatic (real-world) trials was articulated by A. Donabedian's successors and given a widely used framework in 2008 by the Pragmatic–Explanatory Continuum Indicator Summary (PRECIS) tool, developed by a group led by Doug Zwarenstein and colleagues, later revised as PRECIS-2.
Pragmatic trials allowed CER to answer questions like whether a new care model works when embedded in ordinary clinics rather than run by research staff. Their growth, including large simple trials embedded in health systems, blurred the line between research and routine operations.
Patient-Centered Outcomes Research: Bringing the Patient In
Through the 2000s a vocal patient and consumer movement argued that CER had been answering the wrong questions — efficacy defined by clinicians rather than outcomes that matter to patients, such as function, symptom burden, and quality of life. This fed into patient-centred outcomes research, which insists that patients help define the questions and the measures.
The idea gained institutional force in the United States with the Patient-Centered Outcomes Research Institute (PCORI), created by the Affordable Care Act in 2010, which funds CER explicitly framed around patient priorities and requires patient engagement in research. The same legislation created the Patient-Centered Outcomes Research Trust Fund, financed by a fee on health plans, to support this work. The turn toward patient-centredness changed what counts as a valid comparative question and who gets to ask it.
Target Trial Emulation: Making Observational Studies Answer Causal Questions
The most recent methodological turn addresses the credibility gap in observational CER. Target trial emulation, articulated influentially by Miguel Hernán and James Robins in the 2010s, asks the analyst to specify the randomized trial one would have run — its eligibility criteria, treatment strategies, follow-up, and outcome — and then design the observational analysis to mimic that trial as closely as the data allow. The discipline forces attention to pitfalls such as immortal time bias (in which follow-up time is misattributed to a treatment before it could have been received) and time-varying confounding3.
The approach gave observational comparisons a clearer causal framing and has been applied to questions from the effects of statins to the comparative safety of diabetes drugs. Its rise reflected a broader maturation: the field stopped treating observational and randomized evidence as rival camps and began asking how each could be made to answer the same well-posed question.
Learning Health Systems and the Present
The newest organizing idea is the learning health system: an arrangement in which routine care generates data that feed continuous comparison and improvement, closing the loop between practice and evidence. The Institute of Medicine (now the National Academy of Medicine) promoted the concept in the 2000s, and it draws together the strands above — pragmatic trials embedded in care, observational analysis of electronic records, systematic synthesis, and patient engagement — into a single cycle4.
Today CER's methods are plural and complementary. Systematic reviews and network meta-analysis synthesize what is known; pragmatic trials test in real settings; target trial emulation extracts causal answers from routine data; cost-effectiveness and HTA weigh value; and patient-centred measures judge success by what patients experience. Its institutions are spread unevenly — strong in the UK, Canada, Germany, and Australia through HTA bodies, more fragmented and politically contested in the United States. The open questions are the ones its history kept producing: how to make observational comparisons trustworthy, how to detect real heterogeneity without chasing noise, and how to keep evidence about comparative value from being overruled by the interests it inconveniences.
References
- Quality-adjusted life year, Wikipedia
- Health technology assessment, Wikipedia
- Survival epidemiology, Wikipedia
- Learning health systems, Wikipedia
The frameworks of Comparative Effectiveness Research
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1970-1995
Outcomes Research
What actually happens to patients in ordinary practice?
- 1975-Present
- 1985-Present
Health Technology Assessment
Should a health technology be adopted and paid for?
- 1985-Present
Observational Comparative Analysis
Which treatment works better in routine data?
- 1985-Present
- 1989-Present
Heterogeneity of Treatment Effect
For which patients does a treatment work?
- 2000-Present
Pragmatic clinical trial
Does a treatment work in ordinary care settings?
- 2005-Present
Learning Health Systems
How can care continuously learn from its own results?
- 2005-Present
Patient-Centered Outcomes Research
Which treatment matters most to patients?
- 2010-Present
Target Trial Emulation
What would a trial have shown, and can we emulate it?
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Observational Comparative Analysis grew out of Outcomes Research
Observational comparative analysis grew directly out of outcomes research's use of routine data to study what happens to patients. Wennberg's small-area variation work and the outcomes movement established the practice of comparing patient groups from claims and registries, which observational CER then formalized with causal-inference methods.
Carried by John Wennberg.
Health Technology Assessment influenced by Cost-Effectiveness Analysis
Health technology assessment incorporated cost-effectiveness analysis as a core component of its appraisals. HTA agencies commission economic models and weigh cost per QALY, a practice that made cost-effectiveness the quantitative backbone of technology assessment.
Target Trial Emulation grew out of Observational Comparative Analysis
Target trial emulation grew directly out of observational comparative analysis as a way to make its causal claims explicit. Hernán and Robins formalized the approach by specifying the randomized trial that would have answered the question and then emulating it with observational data, addressing the confounding problem that had dogged observational CER.
Carried by Miguel Hernán; James Robins.
Health Technology Assessment influenced by Outcomes Research
Health technology assessment took from outcomes research the insistence on measuring what happens to patients in routine practice rather than only efficacy in trials. Early HTA agencies such as Sweden's SBU (1987) built their assessments around outcomes and effectiveness evidence.
Pragmatic clinical trial influenced by Heterogeneity of Treatment Effect
The recognition that treatment effects vary across patients shaped the design of pragmatic trials, which seek broad eligibility to capture that variation. Pre-specified subgroup and interaction analyses became standard features of pragmatic trial protocols.
Learning Health Systems influenced by Pragmatic clinical trial
Learning health systems took pragmatic trials as a model for generating evidence within routine care. The vision of care and research as a continuous cycle drew on the demonstration that randomized comparisons could be embedded in ordinary clinical settings.
Carried by Institute of Medicine.
Patient-Centered Outcomes Research influenced by Pragmatic clinical trial
Patient-centered outcomes research adopted pragmatic trial designs as a principal method for comparing real-world alternatives. PCORI funded pragmatic trials that randomized patients in ordinary care settings and measured outcomes patients cared about.
Carried by Patient-Centered Outcomes Research Institute.
Heterogeneity of Treatment Effect influenced by Outcomes Research
Outcomes research's attention to patient-level variation in results fed the study of heterogeneity of treatment effect. The finding that outcomes differed across patient subgroups, central to outcomes measurement, became the explicit object of subgroup and interaction analysis.
Patient-Centered Outcomes Research influenced by Cost-Effectiveness Analysis
Patient-centered outcomes research adopted cost-effectiveness methods while reframing them around patient priorities. The PCORI methodology standards explicitly address cost-effectiveness analysis as one of the comparative methods to be applied to patient-centered questions.
Carried by Patient-Centered Outcomes Research Institute.
Beyond the field
Where Comparative Effectiveness Research's ideas came from in other fields, and what they led to there: the strongest 12 of 55 links.
Outcomes Research led to Core outcome set in Rehabilitation Outcomes
Core outcome sets grew directly out of outcomes research and its comparative-effectiveness agenda, which insisted that outcomes be measured in standard, comparable ways across studies. The COMET initiative (2010) formalised the practice of agreeing a minimum set of outcomes for a condition, extending the outcomes-research programme into trial methodology.
Carried by COMET Initiative.
Outcomes Research influenced Patient-reported outcome in Rehabilitation Outcomes
Outcomes research of the 1970s–80s, especially Wennberg's work on clinical variation and the call to measure results important to patients, shifted measurement to the patient's own health status; this directly fed into the development of patient-reported outcome measures in the late 1980s and 1990s.
Carried by John Wennberg.
Outcomes Research influenced Outcome registries and quality databases in Rehabilitation Outcomes
Outcomes research of the 1970s–80s established the measurement and comparison of results across providers and populations, the framework that rehabilitation outcome registries directly adopted to compare rehabilitation services.
Carried by John Wennberg.
Outcomes Research led to Comparative Effectiveness Research in Health Services Research
Comparative effectiveness research grew directly out of earlier outcomes research, keeping its aim of measuring the results of alternative treatments while shifting emphasis to head-to-head comparison. The later programme is a refinement of the earlier one.
Outcomes Research led to Health IT Evaluation and Outcomes Research in Digital Health
Health IT evaluation and outcomes research grew out of outcomes research, adopting its methods for measuring the effects of care on patient outcomes and applying them to information technology interventions. The outcomes framework was extended to evaluate health IT.
Outcomes Research led to Patient-Centred Outcomes Research in Health Services Research
Patient-centred outcomes research grew directly out of earlier outcomes research, keeping its focus on measuring results of care while shifting the outcome of interest to the patient's own experience. The later programme is a refinement of the earlier one.
Outcomes Research led to Clinical Registries and Outcome Benchmarking in Quality Improvement
Clinical registries and outcome benchmarking grew directly out of outcomes research, which established the measurement of patient outcomes and their comparison across providers. The registry movement extended that programme into ongoing, condition-specific data collection and benchmarking.
Cost-Effectiveness Analysis led to Cost-Utility Analysis in Health Technology Assessment
Cost-utility analysis is a direct refinement of cost-effectiveness analysis, substituting utility-weighted health states for single natural effect measures.
Observational Comparative Analysis influenced by Randomized controlled trial in Evidence Based Medicine
Evidence-based medicine's randomized trial ideal shaped observational comparative analysis, which developed methods to approximate trial comparisons from routine data. The trial remains the benchmark against which observational estimates are judged.
Observational Comparative Analysis influenced by Randomized Controlled Trial in Clinical Epidemiology
Clinical epidemiology's randomized trial supplied the comparative logic and bias concerns that observational comparative analysis adapted to non-randomized data. The later approach borrows its notions of confounding and treatment comparison.
Observational Comparative Analysis influenced by Randomized Controlled Trial in Epidemiological Methods
Observational comparative analysis defines itself against the randomized controlled trial, using observational data where randomization is impossible or unethical. The trial's logic of comparison and bias control set the standard the observational approach seeks to match.
Systematic review influenced by Meta-analysis in Biostatistics
Meta-analysis, developed in biostatistics and social science synthesis, supplied systematic review with its statistical method for pooling study results. Reviewers adopted meta-analysis as the quantitative arm of the review.

Comparative Effectiveness Research is a planet in Nousfarer
Its city has a tower for each of these 10 frameworks. Land, walk in, and climb one concept at a time.
Land on Comparative Effectiveness ResearchRelated fields
Fields its ideas are linked to
- Health Services Research 7 links
- Clinical Epidemiology 5 links
- Epidemiological Methods 4 links
- Health Policy Evaluation 3 links
- Evidence Based Medicine 3 links
- Rehabilitation Outcomes 3 links
- Impact Evaluation 2 links
- Health Technology Assessment 2 links
- Clinical Pharmacology 2 links
- Translational Pharmacology 2 links
- Health Economics 1 link
- Digital Health 1 link
More in Health Services Research
- Health Policy Evaluation
- Health Services Research
- Healthcare Delivery Systems
- Implementation And Dissemination Research
- Payment And Incentives
See where Comparative Effectiveness Research sits on the Map of Human Knowledge