- Atlas
- Medicine & Health
- Health Policy Evaluation
Health Policy Evaluation
Health policy evaluation is the systematic study of what happens after governments, insurers, and health systems change the rules of health care.
11 frameworks, from 1930 to today · Health Services Research · Medicine & Health
- Program Evaluation 1930-1980
- Randomized Policy Experimentation 1965-Present
- Donabedian model 1966-Present
- Economic Evaluation and Cost-Effectiveness Analysis 1970-Present
- Quasi-experiment 1975-Present
- Credibility Revolution 1990-Present
- Mixed Methods Evaluation 1990-Present
- Systematic review 1990-Present
- Implementation research 1995-Present
- Heterogeneity and Equity Analysis 2000-Present
- Natural Experiments 2000-Present
Overview
Health policy evaluation is the systematic study of what happens after governments, insurers, and health systems change the rules of health care1. When a country expands insurance, a state caps hospital payments, a health system pays doctors for quality, or a regulator approves a new drug, someone eventually asks: did it work, for whom, at what cost, and why? The subfield exists to answer those questions with evidence rather than assertion. Its history is a story of borrowed methods sharpened for political stakes — statistics from agriculture and psychology, economics from welfare theory, sociology from organizational studies — and of a slow reckoning with the fact that health policy is rarely a clean experiment.
The Birth of Program Evaluation in the Mid-Twentieth Century
Systematic evaluation of social programs emerged in the United States during the 1960s, when the War on Poverty and Great Society created large federal programs whose effectiveness Congress wanted measured. Evaluation borrowed its core logic from the randomized controlled trial (RCT), in which participants are assigned by chance to a treatment or comparison group so that differences in outcomes can be attributed to the program rather than to pre-existing differences. The 1962 amendments to the U.S. Food, Drug, and Cosmetic Act had already made controlled trials the standard for drug approval, and evaluators of social programs carried that template into education, housing, and health.
The period produced both landmark studies and a sobering lesson. The Negative Income Tax experiments of the late 1960s and 1970s tested guaranteed income in several U.S. cities and in Manitoba, Canada, and showed that randomized social experiments were feasible — but also expensive, ethically fraught, and politically vulnerable. In health, the RAND Health Insurance Experiment (1971–1982) remains the canonical case: families were randomly assigned to insurance plans with different levels of cost-sharing, and the study found that greater cost-sharing reduced health care use, with modest effects on health for most people but adverse effects for the poor and sick. The experiment answered a policy question — does free care improve health? — that observational data could not settle cleanly, and it established the template of the large, federally funded randomized policy trial.
Donabedian and the Quality Framework
Parallel to program evaluation, a clinical and organizational strand took shape. In 1966, Avedis Donabedian, a physician and researcher at the University of Michigan, published an article proposing that quality of care be assessed through three linked categories: structure (the resources and arrangements of care — staffing, facilities, organization), process (what is actually done to and for patients), and outcome (the resulting health status)2. The framework's power was that it made quality measurable and gave evaluators a causal chain to test: good structure should enable good process, which should produce good outcomes. It became the organizing grammar of health care quality research and, later, of the quality-measurement industry. Donabedian himself cautioned that the links between the three were empirical questions, not assumptions — a caution that later evaluators would take seriously when they found that structural investments often failed to change outcomes.
Cost-Benefit and Cost-Effectiveness Analysis
As health spending rose through the 1960s and 1970s, a second question pressed: not only whether a policy worked, but whether it was worth the money. Economic evaluation formalized this. Cost-benefit analysis values both costs and outcomes in monetary terms; cost-effectiveness analysis compares costs per unit of a health outcome, most influentially the quality-adjusted life year (QALY), a metric that combines length and quality of life into a single number34. The QALY was developed in the late 1960s and 1970s by researchers including Milton Weinstein and William Stason at Harvard, building on earlier work in decision analysis. Weinstein and Stason's 1977 paper on the cost-effectiveness of strategies to control hypertension is often cited as a founding application.
The approach spread through health technology assessment bodies — Britain's National Institute for Health and Care Excellence (NICE), established in 1999, made cost-per-QALY thresholds central to coverage decisions — and through the U.S. Panel on Cost-Effectiveness in Health and Medicine, whose 1996 report set methodological standards. The method's critics raised persistent objections: whose values go into a QALY, whether cost-effectiveness analysis systematically disadvantages disabled people, and whether a threshold is a technical parameter or a political choice. These debates shaped later work on equity and heterogeneity.
The Quasi-Experimental Turn
Randomization is often impossible in health policy: governments do not randomly assign laws, and insurers do not randomly set premiums. From the 1970s onward, evaluators developed quasi-experimental designs — methods that approximate a controlled comparison without random assignment. The most influential is difference-in-differences, which compares the change in outcomes in a group affected by a policy to the change in an unaffected comparison group, on the assumption that the two would have followed parallel trends absent the policy56. Interrupted time series examines whether an outcome shifts at the moment a policy takes effect7. Instrumental variables use an external source of variation — such as distance to a clinic or a policy rule that affects some people but not others — to isolate causal effects8.
The 1980s and 1990s brought a wave of influential applications. The Oregon Health Insurance Experiment, launched in 2008 after the state used a lottery to allocate limited Medicaid slots, used the lottery as a natural randomization and found that Medicaid coverage increased health care use and reduced financial strain and depression. The Medicaid expansions of the Affordable Care Act after 2014 were studied extensively with difference-in-differences designs comparing expansion and non-expansion states. These studies made clear both the power and the fragility of quasi-experimental claims: results depend on the credibility of the comparison group, and the parallel-trends assumption is often untestable in principle.
Natural Experiments and the Credibility Revolution
The quasi-experimental tradition matured into what economists call the credibility revolution: a shift in standards toward designs where the source of variation is transparent and the identifying assumption is explicit. Natural experiments — situations where policy or circumstance creates something close to random assignment — became prized9. Examples include the RAND experiment, the Oregon lottery, and the classic case of John Snow's 1854 investigation of cholera in London, where the coincidence of water supply boundaries with cholera deaths provided a natural comparison long before formal methods existed. In health policy, natural experiments include changes in reimbursement rules, eligibility thresholds, and geographic boundaries that create sharp contrasts between otherwise similar populations.
The credibility revolution also brought regression discontinuity, which exploits sharp cutoffs — an income threshold for subsidy eligibility, an age cutoff for program access — to compare people just above and just below the line10. It brought synthetic control methods, which construct a weighted combination of untreated units to serve as a counterfactual for a single treated unit such as a state or country. These methods made it possible to evaluate policies that affect whole jurisdictions, where no clean comparison group exists.
Systematic Review and Evidence Synthesis
As individual studies multiplied, the problem became aggregation: which findings should a policymaker trust? Systematic review — a structured, protocol-driven search and appraisal of all relevant studies on a question — emerged as the answer11. The Cochrane Collaboration, founded in 1993 and named for the British epidemiologist Archie Cochrane, built an international infrastructure for systematic reviews of health interventions. The Campbell Collaboration, founded in 2000, extended the model to social policy, including health-related programs. Meta-analysis, the statistical pooling of study results, gave reviews a quantitative backbone, though it also raised concerns about publication bias and the mixing of heterogeneous studies.
Evidence synthesis became institutionalized in bodies such as the U.S. Agency for Healthcare Research and Quality's Evidence-based Practice Centers and the international GRADE working group, which developed a system for rating the certainty of evidence. For health policy evaluation, the synthesis movement changed the unit of evidence from the single study to the body of studies — and forced evaluators to confront that the same policy can produce different results in different contexts.
Implementation Science and the Gap Between Evidence and Practice
By the 1990s, a troubling pattern was clear: many policies and interventions with strong evidence of efficacy were not adopted, or were adopted in ways that stripped them of effect. Implementation science emerged to study this gap — the systematic study of how to move evidence into routine practice12. Its roots include the diffusion-of-innovations research of Everett Rogers in the 1960s and the quality-improvement movement in health care. The field developed frameworks for understanding barriers and facilitators — the Consolidated Framework for Implementation Research (CFIR), published in 2009, and the RE-AIM framework for reach, effectiveness, adoption, implementation, and maintenance — and methods for studying implementation fidelity and adaptation.
Implementation science shifted evaluation's question from "does it work?" to "does it work when delivered in the real world, and what does it take to deliver it?" This made evaluation more realistic and more useful to policymakers, but also more complex: an intervention's effect now depended on context, and the same program could succeed in one setting and fail in another.
Mixed Methods and the Turn to Explanation
Quantitative designs answer whether and how much; they are weaker at answering why and how. Mixed methods evaluation combines quantitative and qualitative approaches — interviews, observation, document analysis — to explain mechanisms and capture stakeholder perspectives13. The approach gained ground in the 1990s and 2000s, drawing on traditions in sociology and anthropology and on the recognition that policy effects are mediated by how people understand and respond to rules. A typical mixed-methods evaluation might pair a difference-in-differences estimate of a payment reform's effect with interviews of hospital managers explaining why some responded and others did not. The method's claim is that explanation improves both interpretation and transferability: knowing why a policy worked in one place helps predict where else it will.
Heterogeneity and Equity
The most recent strand asks not whether a policy worked on average but for whom. Heterogeneity analysis examines whether effects differ across subgroups — by income, race, age, health status, or geography14. The RAND experiment's finding that cost-sharing harmed the poor more than the affluent was an early signal. Later work on Medicaid expansion, payment reform, and drug coverage repeatedly found that average effects masked divergent subgroup effects. Equity analysis goes further, asking whether a policy narrowed or widened disparities and whether benefits reached the populations most in need15. Methods include subgroup analysis, interaction terms in regression models, and distributional cost-effectiveness analysis, which weights health gains by their distribution across groups316. The field's growing attention to structural racism and social determinants has pushed equity analysis from a secondary concern to a central one.
The Present
Health policy evaluation today is a plural field. Randomized trials remain the gold standard where feasible, and large policy experiments — from insurance lotteries to conditional cash transfer programs — continue to be mounted. Quasi-experimental methods, especially difference-in-differences and regression discontinuity, dominate the evaluation of policies that cannot be randomized, and the credibility revolution has raised the bar for what counts as convincing evidence. Systematic reviews and evidence-rating systems organize the accumulated literature11. Economic evaluation, anchored in cost-effectiveness analysis, informs coverage and pricing decisions in many countries. Implementation science and mixed methods address the gap between efficacy and real-world effect. Equity analysis asks who benefits.
The open questions are substantive and methodological at once. How should evaluators handle policies that affect entire systems, where no clean comparison exists? How can evidence synthesized across contexts be made useful for a specific jurisdiction? How should cost-effectiveness account for equity? And how can evaluation keep pace with policies that change faster than studies can be completed? The field's history suggests that its methods will keep borrowing from neighboring disciplines and keep being reshaped by the political questions it is asked to answer.
References
- Program evaluation, Wikipedia
- Donabedian model (introduction, History), Wikipedia
- Cost-effectiveness analysis, Wikipedia
- Cost–benefit analysis, Wikipedia
- Difference in differences, Wikipedia
- Quasi-experiment, Wikipedia
- Interrupted time series, Wikipedia
- Instrumental variables, Wikipedia
- Natural experiment, Wikipedia
- Regression discontinuity design, Wikipedia
- Systematic review, Wikipedia
- Implementation research, Wikipedia
- Multimethodology, Wikipedia
- Subgroup analysis, Wikipedia
- Health equity, Wikipedia
- Interaction (statistics), Wikipedia
The frameworks of Health Policy Evaluation
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1930-1980
- 1965-Present
Randomized Policy Experimentation
What is the causal effect of a policy?
- 1966-Present
- 1970-Present
Economic Evaluation and Cost-Effectiveness Analysis
Is a policy worth the money?
- 1975-Present
Quasi-experiment
How can causal effects be estimated without randomization?
- 1990-Present
- 1990-Present
- 1990-Present
- 1995-Present
Implementation research
Why do evidence-based practices fail to be adopted?
- 2000-Present
Heterogeneity and Equity Analysis
Who gains and who loses from a policy?
- 2000-Present
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Randomized Policy Experimentation grew out of Program Evaluation
Program evaluation's core logic of comparing treated and comparison groups was carried into randomized policy experiments, which applied the RCT template to social programs. The Negative Income Tax experiments and the RAND Health Insurance Experiment (1971–1982) were direct outgrowths of the evaluation mandate created by 1960s social programs.
Carried by RAND Health Insurance Experiment (1971–1982).
Quasi-experiment grew out of Program Evaluation
Quasi-experimental designs were developed within program evaluation as methods to approximate controlled comparisons when randomization was impossible. Difference-in-differences, interrupted time series, and instrumental variables became standard tools for evaluating policies that could not be randomly assigned.
Credibility Revolution grew out of Quasi-experiment
The credibility revolution matured the quasi-experimental tradition by raising standards for transparent identification and explicit assumptions. It refined difference-in-differences, instrumental variables, and related designs into a coherent methodological program.
Economic Evaluation and Cost-Effectiveness Analysis grew out of Program Evaluation
Economic evaluation grew out of program evaluation's question of whether a policy worked, adding the question of whether it was worth the money. Cost-benefit and cost-effectiveness analysis formalized the comparison of costs and outcomes within the evaluation of social programs.
Mixed Methods Evaluation grew out of Program Evaluation
Mixed methods evaluation grew out of program evaluation's need to explain not only whether programs worked but why and how. It combined quantitative outcome measurement with qualitative process tracing within the evaluation tradition.
Systematic review grew out of Program Evaluation
Systematic review emerged from the proliferation of evaluation studies and the need to aggregate their findings for policymakers. It applied a structured, protocol-driven approach to synthesizing the evidence that program evaluation produced.
Carried by Cochrane Collaboration.
Quasi-experiment influenced by Randomized Policy Experimentation
Quasi-experimental methods arose partly as a response to the practical and ethical limits of randomized policy experiments, offering designs for situations where randomization was infeasible. They defined themselves by providing credible causal inference without random assignment.
Credibility Revolution grew out of Randomized Policy Experimentation
The credibility revolution extended the logic of randomized experiments by demanding transparent sources of variation and explicit identifying assumptions in observational designs. It grew out of the experimental tradition's emphasis on credible causal comparison.
Systematic review grew out of Randomized Policy Experimentation
Systematic reviews and meta-analysis were built to pool the results of randomized trials, including policy experiments, into quantitative summaries. The Cochrane Collaboration's methods were designed primarily for synthesizing controlled trials.
Carried by Cochrane Collaboration.
Implementation research grew out of Donabedian model
Implementation research grew out of the quality-of-care framework's concern with process—what is actually done to and for patients—by studying why evidence-based practices fail to be adopted. It extended the structure-process-outcome chain to the problem of translating evidence into routine practice.
Heterogeneity and Equity Analysis influenced by Economic Evaluation and Cost-Effectiveness Analysis
Heterogeneity and equity analysis arose partly in response to cost-effectiveness analysis's use of aggregate metrics like the QALY, which critics argued systematically disadvantaged disabled people and ignored distributional concerns. It defined itself by asking who gains and who loses from a policy, not just whether it is cost-effective overall.
Natural Experiments grew out of Quasi-experiment
Natural experiments are a branch of the quasi-experimental tradition that exploits situations where policy or circumstance creates something close to random assignment. They grew directly out of the search for credible comparison groups without randomization.
Beyond the field
Where Health Policy Evaluation's ideas came from in other fields, and what they led to there: the strongest 12 of 83 links.
Program Evaluation led to Utilization-Focused Evaluation in Public Health Program Evaluation
Patton's utilization-focused evaluation is a school within program evaluation, built on the field's accumulated practice and defined by its attention to intended use by intended users. It grew directly out of the program evaluation tradition.
Carried by Michael Quinn Patton.
Program Evaluation led to CIPP evaluation model in Public Health Program Evaluation
Stufflebeam's CIPP model was developed within the field of program evaluation as a decision-oriented alternative to objectives-based evaluation, and it is a branch of that broader tradition. It grew directly out of the evaluation practice and debates of the 1960s and 1970s.
Carried by Daniel Stufflebeam.
Randomized Policy Experimentation influenced by Randomized Controlled Trial in Clinical Epidemiology
Clinical epidemiology's RCT methodology, with randomization, control groups and blinded assessment, was carried over into the evaluation of social and health policies.
Randomized Policy Experimentation influenced by Randomized Controlled Trial in Epidemiological Methods
The randomized controlled trial developed in clinical and epidemiological research, notably the 1948 streptomycin trial, supplied the design that policy experimentation transferred from patients to programs.
Carried by Austin Bradford Hill; Medical Research Council streptomycin trial (1948).
Economic Evaluation and Cost-Effectiveness Analysis led to Distributional cost-effectiveness analysis in Health Policy Evaluation
Distributional cost-effectiveness analysis is a branch of economic evaluation in health policy, extending standard economic evaluation methods to incorporate equity alongside efficiency. It grew out of the economic evaluation tradition that compares costs and health outcomes of interventions.
Carried by Richard Cookson; Miqdad Asaria; Susan Griffin.
Economic Evaluation and Cost-Effectiveness Analysis influenced Burden of Disease and Cost-Effectiveness Priority Setting in Health Policy
Economic evaluation and cost-effectiveness analysis provided the analytic machinery for the burden of disease and priority-setting framework, which ranked interventions by cost per DALY averted.
Carried by World Development Report 1993.
Economic Evaluation and Cost-Effectiveness Analysis influenced by Cost-benefit analysis in Health Policy Evaluation
Cost-benefit analysis, developed for water and defense projects, supplied the discounting, valuation and incremental comparison techniques that health cost-effectiveness analysis adapted to health outcomes.
Economic Evaluation and Cost-Effectiveness Analysis influenced by Health Economics in Health Policy
Health economics, especially the work of Arrow on medical care and later cost-benefit applications, provided the conceptual basis for valuing health outcomes that economic evaluation depends on.
Carried by Kenneth Arrow.
Economic Evaluation and Cost-Effectiveness Analysis influenced Pharmacoeconomics in Social And Administrative Pharmacy
Pharmacoeconomics adopted economic evaluation and cost-effectiveness analysis as its core methods, transferring the techniques developed in health policy evaluation during the 1970s to the assessment of medicines.
Quasi-experiment influenced by Experimental and Quasi-Experimental Design in Public Health Program Evaluation
Campbell and Stanley's experimental and quasi-experimental design codified the quasi-experiment as a named design, which the health policy quasi-experiment framework then adopted.
Carried by Donald T. Campbell; Julian C. Stanley; Experimental and Quasi-Experimental Designs for Research (1963).
Systematic review influenced by Meta-analysis in Biostatistics
Statistical meta-analysis, developed by Glass, Hedges and others, provided the quantitative synthesis technique that systematic reviews in health policy use to combine study estimates.
Carried by Gene V. Glass; Larry V. Hedges.
Systematic review influenced by Systematic Review and Meta-analysis in Evidence Based Medicine
The systematic review and meta-analysis tradition of evidence-based medicine supplied the core methods, search protocols and pooling techniques that health policy systematic reviews adopted. Cochrane-style review methods were carried directly into policy review practice.
Carried by Archie Cochrane; Cochrane Collaboration.

Health Policy Evaluation is a planet in Nousfarer
Its city has a tower for each of these 11 frameworks. Land, walk in, and climb one concept at a time.
Land on Health Policy EvaluationRelated fields
Fields its ideas are linked to
- Public Health Program Evaluation 13 links
- Health Policy Evaluation 9 links
- Health Services Research 5 links
- Implementation And Dissemination Research 5 links
- Health Policy 5 links
- Applied Theater Theory 3 links
- Health Economics 3 links
- Health Informatics 3 links
- Epidemiological Methods 3 links
- Health Promotion 3 links
- Social Insurance And Redistribution 3 links
- Public Art Theory 2 links
More in Health Services Research
- Comparative Effectiveness Research
- Health Services Research
- Healthcare Delivery Systems
- Implementation And Dissemination Research
- Payment And Incentives
See where Health Policy Evaluation sits on the Map of Human Knowledge