Comparative Effectiveness Research

Comparative effectiveness research (CER) asks a deceptively simple question: among the treatments that already exist for a given condition, which one actually works better, for whom, and at what cost?

10 frameworks, from 1970 to today · Health Services Research · Medicine & Health

still a working view ended · each leads to its entry below

Overview

Comparative effectiveness research (CER) asks a deceptively simple question: among the treatments that already exist for a given condition, which one actually works better, for whom, and at what cost? Unlike the earlier stages of clinical research, which mainly test whether a new drug or device beats a placebo under tightly controlled conditions, CER compares real alternatives as they are used in ordinary practice. Its stakes are practical. Clinicians face competing options with little head-to-head evidence; payers must decide what to cover; patients want to know whether the newer, costlier choice is worth it. The history of CER is the story of how medicine learned to answer those questions with methods rigorous enough to trust — and of the political fights that erupted once the answers began to matter to money and markets.

Precursors: Outcomes Research and the Variation Puzzle

The intellectual roots lie in the 1970s and 1980s, when health services researchers noticed something puzzling. John Wennberg and colleagues documented that rates of common surgeries — tonsillectomy, hysterectomy, prostatectomy — varied several-fold between neighbouring communities with similar populations. If the evidence were clear, such variation should not exist. The finding implied that much medical practice rested on custom rather than proof, and it gave rise to what became known as outcomes research: the study of what actually happens to patients, using routine data rather than controlled experiments.

A landmark expression was the Medical Outcomes Study, begun in the 1980s by John Ware and colleagues, which developed ways to measure patients' functional status and well-being rather than just laboratory values. Its short-form questionnaire, the SF-36, became a standard instrument for capturing health status in populations. Around the same time, the RAND Health Insurance Experiment (1970s–1980s) showed that cost-sharing changed how people used care and, for some, their health — an early demonstration that system arrangements, not just treatments, are proper objects of comparison.

These strands — variation, outcomes measurement, and a suspicion that much care lacked an evidence base — set the stage. What was missing was a disciplined way to synthesize the scattered evidence and a way to compare alternatives directly.

Systematic Review and Meta-Analysis: Making the Evidence Add Up

By the late 1980s, the problem was not a shortage of trials but a flood of them, scattered across journals in conflicting directions. Individual studies were often too small to detect modest but real differences. The solution was the systematic review — a structured, reproducible search and appraisal of all relevant studies on a question — combined with meta-analysis, a statistical technique for pooling their results into a single estimate.

The approach was formalized through the 1980s and 1990s, notably by researchers associated with the Cochrane Collaboration, founded in 1993 and named for the British epidemiologist Archie Cochrane, who had argued that medicine should be built on systematic summaries of evidence. The Oxford Centre for Evidence-Based Medicine, established in the 1990s, popularized hierarchies ranking study designs. Meta-analysis could, for example, pool many small trials of a treatment and reveal a benefit that no single trial could confirm. But it also exposed a hazard: combining biased studies produces a confident but wrong answer, which pushed the field toward tools for assessing publication bias and heterogeneity.

Cost-Effectiveness and Decision Modeling: Comparing Value, Not Just Effect

Comparing treatments raised a second question: even if one option is more effective, is it worth the extra cost? Decision modeling and cost-effectiveness analysis gave CER a quantitative vocabulary. The central measure is the quality-adjusted life year (QALY), which combines length of life with its quality, so that a year in poor health counts for less than a year in full health1. A treatment's cost per QALY gained then allows unlike interventions to be compared on a common scale.

The approach was developed and codified from the 1970s onward, gaining institutional form through bodies such as the U.S. Panel on Cost-Effectiveness in Health and Medicine, convened in the 1990s, which issued methodological recommendations. Its most influential home became the UK's National Institute for Health and Care Excellence (NICE), established in 1999, whose appraisals weigh cost per QALY when deciding what the National Health Service will fund. The method made explicit trade-offs that had previously been hidden — and made it politically combustible, because a threshold above which a treatment is judged too expensive to fund is a rationing rule by another name.

Health Technology Assessment: Institutionalizing the Comparison

Health technology assessment (HTA) grew out of the same impulse: a formal process of evaluating the clinical, economic, and social consequences of health technologies to inform policy2. Early national agencies appeared in the 1980s — Sweden's SBU, established in 1987, is often cited as an early example — and the model spread across Europe and beyond. HTA brought CER into government: agencies synthesize evidence, commission economic models, and issue guidance that payers and clinicians may follow.

The United States took a more contested path. The Agency for Health Care Policy and Research, created in 1989, was attacked by surgical and industry groups after it published a guideline on back pain, and it was nearly eliminated; it survived reorganized as the Agency for Healthcare Research and Quality (AHRQ). The episode illustrated a recurring theme: CER's findings can threaten established practice and commercial interests, and its institutions are therefore politically fragile.

Observational Comparative Analysis and the Challenge of Confounding

Randomized trials are the gold standard for causal inference, but they are slow, expensive, and often unrepresentative of ordinary patients. Much CER has therefore relied on observational data — insurance claims, registries, electronic health records — comparing patients who happened to receive different treatments. The central danger is confounding: patients who receive one treatment may differ systematically from those who receive another, so an apparent effect may reflect who they were rather than what was done to them.

The field developed a toolkit to address this, including propensity scores — a method, advanced by Paul Rosenbaum and Donald Rubin in the 1980s, for balancing comparison groups on measured characteristics — and a discipline of sensitivity analysis for unmeasured differences. These methods made large observational comparisons possible, and debates over their trustworthiness have run through the field ever since.

Heterogeneity of Treatment Effect: One Size Does Not Fit All

A persistent finding of CER is that average effects conceal variation. A treatment that helps most patients may harm a subgroup, and the average can mask both. The study of heterogeneity of treatment effect asks which patient characteristics — age, comorbidity, genotype, preference — modify how well an intervention works.

One influential demonstration became a cornerstone of the field: the 1995 report of the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial's precursor analyses, and more famously the finding in the Cardiac Arrhythmia Suppression Trial, published in 1989, that antiarrhythmic drugs which reduced a surrogate marker (irregular heartbeats) actually increased mortality. Surrogate endpoints could mislead, and subgroup analysis could reveal that a therapy's value depended on who received it. Statisticians warned that searching many subgroups for significance produces false positives, so the field developed pre-specified subgroup analysis and interaction testing to distinguish real effect modification from noise. Recognizing heterogeneity is what makes the "for whom" in CER's central question operational.

Pragmatic Trials: Testing Treatments in the Real World

If observational studies risk confounding and explanatory trials risk irrelevance to ordinary practice, a middle path is the pragmatic trial: a randomized comparison conducted in routine settings, with broad eligibility, usual-care comparators, and outcomes that matter to patients and payers. The distinction between explanatory (idealized) and pragmatic (real-world) trials was articulated by A. Donabedian's successors and given a widely used framework in 2008 by the Pragmatic–Explanatory Continuum Indicator Summary (PRECIS) tool, developed by a group led by Doug Zwarenstein and colleagues, later revised as PRECIS-2.

Pragmatic trials allowed CER to answer questions like whether a new care model works when embedded in ordinary clinics rather than run by research staff. Their growth, including large simple trials embedded in health systems, blurred the line between research and routine operations.

Patient-Centered Outcomes Research: Bringing the Patient In

Through the 2000s a vocal patient and consumer movement argued that CER had been answering the wrong questions — efficacy defined by clinicians rather than outcomes that matter to patients, such as function, symptom burden, and quality of life. This fed into patient-centred outcomes research, which insists that patients help define the questions and the measures.

The idea gained institutional force in the United States with the Patient-Centered Outcomes Research Institute (PCORI), created by the Affordable Care Act in 2010, which funds CER explicitly framed around patient priorities and requires patient engagement in research. The same legislation created the Patient-Centered Outcomes Research Trust Fund, financed by a fee on health plans, to support this work. The turn toward patient-centredness changed what counts as a valid comparative question and who gets to ask it.

Target Trial Emulation: Making Observational Studies Answer Causal Questions

The most recent methodological turn addresses the credibility gap in observational CER. Target trial emulation, articulated influentially by Miguel Hernán and James Robins in the 2010s, asks the analyst to specify the randomized trial one would have run — its eligibility criteria, treatment strategies, follow-up, and outcome — and then design the observational analysis to mimic that trial as closely as the data allow. The discipline forces attention to pitfalls such as immortal time bias (in which follow-up time is misattributed to a treatment before it could have been received) and time-varying confounding3.

The approach gave observational comparisons a clearer causal framing and has been applied to questions from the effects of statins to the comparative safety of diabetes drugs. Its rise reflected a broader maturation: the field stopped treating observational and randomized evidence as rival camps and began asking how each could be made to answer the same well-posed question.

Learning Health Systems and the Present

The newest organizing idea is the learning health system: an arrangement in which routine care generates data that feed continuous comparison and improvement, closing the loop between practice and evidence. The Institute of Medicine (now the National Academy of Medicine) promoted the concept in the 2000s, and it draws together the strands above — pragmatic trials embedded in care, observational analysis of electronic records, systematic synthesis, and patient engagement — into a single cycle4.

Today CER's methods are plural and complementary. Systematic reviews and network meta-analysis synthesize what is known; pragmatic trials test in real settings; target trial emulation extracts causal answers from routine data; cost-effectiveness and HTA weigh value; and patient-centred measures judge success by what patients experience. Its institutions are spread unevenly — strong in the UK, Canada, Germany, and Australia through HTA bodies, more fragmented and politically contested in the United States. The open questions are the ones its history kept producing: how to make observational comparisons trustworthy, how to detect real heterogeneity without chasing noise, and how to keep evidence about comparative value from being overruled by the interests it inconveniences.

References

  1. Quality-adjusted life year, Wikipedia
  2. Health technology assessment, Wikipedia
  3. Survival epidemiology, Wikipedia
  4. Learning health systems, Wikipedia

The frameworks of Comparative Effectiveness Research

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1970-1995

    Outcomes Research

    What actually happens to patients in ordinary practice?

    Fly to its tower in Nousfarer

  2. 1975-Present

    Cost-Effectiveness Analysis

    Is the extra benefit worth the extra cost?

    Fly to its tower in Nousfarer

  3. 1985-Present

    Health Technology Assessment

    Should a health technology be adopted and paid for?

    Fly to its tower in Nousfarer

  4. 1985-Present

    Observational Comparative Analysis

    Which treatment works better in routine data?

    Fly to its tower in Nousfarer

  5. 1985-Present

    Systematic review

    What does the whole body of evidence show?

    Fly to its tower in Nousfarer

  6. 1989-Present
  7. 2000-Present

    Pragmatic clinical trial

    Does a treatment work in ordinary care settings?

    Fly to its tower in Nousfarer

  8. 2005-Present

    Learning Health Systems

    How can care continuously learn from its own results?

    Fly to its tower in Nousfarer

  9. 2005-Present
  10. 2010-Present

    Target Trial Emulation

    What would a trial have shown, and can we emulate it?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Outcomes ResearchCost-Effectiveness AnalysisHealth Technology AssessmentObservational Comparative AnalysisSystematic reviewHeterogeneity of Treatment EffectPragmatic clinical trialLearning Health SystemsPatient-Centered Outcomes ResearchTarget Trial Emulation

Beyond the field

Where Comparative Effectiveness Research's ideas came from in other fields, and what they led to there: the strongest 12 of 55 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Comparative Effectiveness Research is a planet in Nousfarer

Its city has a tower for each of these 10 frameworks. Land, walk in, and climb one concept at a time.

Land on Comparative Effectiveness Research

Related fields

Fields its ideas are linked to

More in Health Services Research

See where Comparative Effectiveness Research sits on the Map of Human Knowledge