Public Health Program Evaluation

Public health program evaluation is the systematic work of finding out whether an organized effort to improve population health actually did what it promised, and why.

14 frameworks, from 1840 to today · Public Health · Medicine & Health

still a working view ended · each leads to its entry below

Overview

Public health program evaluation is the systematic work of finding out whether an organized effort to improve population health actually did what it promised, and why1. Its practitioners ask three linked questions: Did the program reach the people it meant to reach1? Did it change anything1? And was the change worth what it cost1? The field matters because public health runs on collective action—vaccination campaigns, tobacco taxes, school meal standards, sanitation drives—and money spent on a program that does not work is money not spent on one that does. Evaluation is how the field learns from its own interventions rather than assuming good intentions produce good outcomes.

Counting the Population: Vital Statistics and the First Evaluations

The oldest strand is measurement. Before anyone could evaluate a program, someone had to know how many people were born, who died, and of what. In England, the Bills of Mortality were published weekly from the early seventeenth century, and John Graunt's Natural and Political Observations (1662) used them to estimate death rates and describe patterns of mortality. In the nineteenth century, William Farr, as compiler of abstracts in the General Register Office from 1839, built England's system of vital statistics into a tool for comparing death rates across districts and occupations. This was evaluation in embryo: a program or a condition could be judged by whether the numbers moved.

The sanitary movement made the link explicit. When John Snow investigated the Broad Street pump in 1854, he was not only explaining a cholera outbreak but testing a claim about how cholera spread—and the removal of the pump handle was an intervention whose effect could be read in the falling case counts. The idea that a public health action could be assessed by its consequences, counted before and after, is the ancestor of everything that follows.

The Randomized Controlled Trial and the Problem of Causation

The central difficulty of evaluation is that populations change for many reasons at once. A vaccination campaign that coincides with a mild season will look effective whether or not it was. The randomized controlled trial (RCT)—in which participants are assigned by chance to receive an intervention or not—solves this by making the groups comparable on everything except the intervention. The first widely recognized RCT in medicine is usually dated to the 1948 streptomycin trial for tuberculosis, run by the British Medical Research Council under Austin Bradford Hill. In public health, the classic early example is the field trial of the Salk polio vaccine in 1954, which randomized schoolchildren across the United States and showed the vaccine protected against paralytic polio.

The RCT gave evaluation a gold standard, but it also created a tension that has shaped the field ever since. Many public health programs cannot be randomized: a city cannot be randomly assigned to fluoridate its water without political and ethical complications, and a national tobacco policy cannot be withheld from a control group. This pushed evaluators toward quasi-experimental designs—studies that compare groups or periods without random assignment, using techniques such as before-and-after comparisons, matched comparison areas, interrupted time series, and regression discontinuity. Donald Campbell and Julian Stanley's Experimental and Quasi-Experimental Designs for Research (1963) became the standard reference, teaching a generation that credible causal claims could be built from carefully chosen comparison groups even when randomization was impossible. The community trial, in which whole communities rather than individuals are the unit of assignment, extended the logic to population-level interventions; the Stanford Five-City Project and the Minnesota Heart Health Program in the 1970s and 1980s were ambitious attempts to test community-wide cardiovascular prevention, and their mixed results taught the field how hard it is to detect the effect of a diffuse program.

Evaluation Comes of Age: Models and Debates

Through the 1960s and 1970s, evaluation grew into a profession with its own theories. In the United States, the Great Society programs of the 1960s—Head Start, Model Cities, community health centers—created a demand for evaluation and a federal expectation that programs would be judged. Out of this ferment came competing models.

Ralph Tyler's objectives-based approach, developed earlier in education, held that evaluation should measure whether stated objectives were met. Robert Stake's Countenance Model (1967) complicated this by distinguishing what was intended from what actually happened, and by looking at antecedents, transactions, and outcomes rather than outcomes alone. Daniel Stufflebeam's CIPP model (context, input, process, product) proposed that evaluation should examine the setting, the resources, the implementation, and the results—not just the final outcome12. Michael Scriven drew the distinction between formative evaluation, which feeds back during a program's development to improve it, and summative evaluation, which judges a finished program's worth3. Scriven also argued for goal-free evaluation: an evaluator who does not know the program's stated goals is less likely to miss unintended effects, whether good or bad.

These models were not merely academic. They reflected a real dispute about what evaluation was for. Was it a service to program managers, helping them improve? Or a service to funders and the public, helping them decide? Carol Weiss's work on how evaluation is actually used—her 1970s research showed that findings often influence decisions indirectly, by changing how people think about a problem, rather than by directly triggering a decision—reframed the question. Michael Quinn Patton's utilization-focused evaluation (1978 onward) built on this: evaluation should be designed from the start around the specific decisions its intended users will face, with those users involved in shaping the questions. The approach claimed that evaluations failed less often because of bad methods than because nobody used them.

Realist Evaluation and the Question of Mechanism

By the 1990s, a further dissatisfaction had set in. RCTs and quasi-experiments could show whether a program worked on average, but not why, for whom, or under what conditions. A program that succeeded in one city might fail in another, and the trial design offered no account of the difference. Realist evaluation, associated with Ray Pawson and Nick Tilley's Realistic Evaluation (1997), proposed that evaluation should identify the underlying mechanism—the process by which the program produces its effect—and the contexts in which that mechanism operates. The formula "$\text{mechanism} + \text{context} = \text{outcome}$" became its shorthand. A realist evaluator asks not "does this work?" but "how does this work, for whom, in what circumstances?" This was a direct challenge to the dominance of the experimental model and drew heavily on public health and social program experience.

Implementation Science and the Fidelity Problem

A parallel development asked a different question: even when a program is known to work, why does it so often fail when taken to scale? Implementation science emerged in the 1990s and 2000s to study the gap between evidence and practice. Its central concept is implementation fidelity—the degree to which a program is delivered as designed—and its central finding is that many programs fail not because the idea was wrong but because the delivery was incomplete, underfunded, or adapted beyond recognition. The field produced frameworks for understanding implementation (such as the Consolidated Framework for Implementation Research, published in 2009) and measures of fidelity, and it pushed evaluators to study process as carefully as outcomes. The logic model, a diagram tracing a program's inputs through activities and outputs to outcomes, became a standard tool for making a program's theory of change explicit and testable; the results chain is a close relative used in international development.

Evidence-Based Public Health and the Systematic Review

The final major strand is the movement toward synthesizing evidence. The Cochrane Collaboration, founded in 1993, and the Campbell Collaboration, founded in 2000, built systems for systematic reviews—structured, transparent summaries of all the studies on a question, often combined statistically in meta-analysis. The GRADE approach, developed in the 2000s, gave a common language for rating the certainty of evidence. Evidence-based public health, articulated in the 1990s and 2000s, argued that program decisions should rest on the best available research evidence, combined with local data and community values. This changed what evaluators were expected to produce: not just a verdict on one program, but a contribution to a cumulative evidence base that could guide future programs.

The Present

Today the field is plural. Experimental and quasi-experimental designs remain central, but they sit alongside realist, theory-based, and utilization-focused approaches, and mixed-methods designs that combine them are common. Implementation science has become a field in its own right, with its own journals and frameworks. Systematic reviews and evidence grading are routine. The open questions are real: how to evaluate complex, multi-component interventions whose effects are diffuse and long-delayed; how to weigh local context against generalizable evidence; how to involve communities in defining what counts as success; and how to evaluate programs in low- and middle-income settings where the experimental infrastructure is thin. The field's history is a record of successive attempts to answer a single stubborn question—how do we know whether we made a difference?—and each generation's answer has been more demanding than the last.

References

  1. Program evaluation (introduction, Assessing implementation, Assessing the impact (effectiveness), Conducting an evaluation, Assessing efficiency, History of the CIPP model), Wikipedia
  2. CIPP evaluation model, Wikipedia
  3. Formative assessment (Origin of the term), Wikipedia

The frameworks of Public Health Program Evaluation

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1840-1960
  2. 1930-1980

    Objectives-Based Evaluation

    Did the program meet its stated objectives?

    Fly to its tower in Nousfarer

  3. 1948-Present
  4. 1948-Present
  5. 1967-1990

    Countenance Model

    How do intentions compare with what actually happened?

    Fly to its tower in Nousfarer

  6. 1967-Present

    Formative and Summative Evaluation

    Is evaluation for improving or for judging?

    Fly to its tower in Nousfarer

  7. 1970-Present

    Logic model

    How does a program's theory of change run?

    Fly to its tower in Nousfarer

  8. 1971-Present

    CIPP evaluation model

    What should be examined: context, inputs, process, results?

    Fly to its tower in Nousfarer

  9. 1972-2000

    Goal-Free Evaluation

    What happens if we ignore the stated goals?

    Fly to its tower in Nousfarer

  10. 1978-Present

    Community Trial

    Does a program work across whole communities?

    Fly to its tower in Nousfarer

  11. 1978-Present

    Utilization-Focused Evaluation

    Will the intended users actually use the findings?

    Fly to its tower in Nousfarer

  12. 1993-Present
  13. 1995-Present

    Implementation research

    Why does a proven program fail at scale?

    Fly to its tower in Nousfarer

  14. 1997-Present

    Realist Evaluation

    How does a program work, for whom, in what circumstances?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Program Statistics and Vital Statisti…Objectives-Based EvaluationExperimental and Quasi-Experimental D…Randomized Controlled Trial EvaluationCountenance ModelFormative and Summative EvaluationLogic modelCIPP evaluation modelGoal-Free EvaluationCommunity TrialUtilization-Focused EvaluationEvidence-Based Public Health and Syst…Implementation researchRealist Evaluation

Beyond the field

Where Public Health Program Evaluation's ideas came from in other fields, and what they led to there: the strongest 12 of 95 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Public Health Program Evaluation is a planet in Nousfarer

Its city has a tower for each of these 14 frameworks. Land, walk in, and climb one concept at a time.

Land on Public Health Program Evaluation

Related fields

Fields its ideas are linked to

More in Public Health

See where Public Health Program Evaluation sits on the Map of Human Knowledge