Loading field map
Public health program evaluation is the systematic collection and analysis of information about the design, implementation, and effects of programs intended to improve population health. It is a practice-oriented discipline within public health that asks a deceptively simple question: does this program work, for whom, under what conditions, and at what cost? The answer requires far more than measuring outcomes. Evaluation is a structured way of thinking about what a program is supposed to do, how it actually operates in the real world, and what its consequences—intended and unintended—turn out to be.
The field exists because public health programs are investments of public resources made under uncertainty. A vaccination campaign, a smoking cessation hotline, a school-based nutrition curriculum, or a community needle-exchange service all embody assumptions about how change happens. Evaluation tests those assumptions. It serves multiple masters: funders who want accountability, program managers who want improvement, policymakers who want evidence for decisions, and communities who want to know whether an intervention respects and serves them. These different purposes have shaped the field's methods, its ethical codes, and its recurring debates.
At its heart, program evaluation in public health addresses three clusters of questions. The first concerns merit and worth: Is the program achieving its intended outcomes? Are those outcomes worth what the program costs? The second concerns process and implementation: Was the program delivered as planned? Did it reach the intended population? What barriers and facilitators shaped its operation? The third concerns explanation and generalization: Why did the program produce the results it did? What can be learned that applies beyond this specific program?
These questions carry real stakes. Evaluations can determine whether a program is scaled up, defunded, redesigned, or replicated. They can reveal that a well-intentioned intervention widened health inequities even while improving average outcomes. They can expose the difference between a program that works in a controlled trial and one that fails in routine practice. Because public health programs often target vulnerable populations and address sensitive issues—addiction, infectious disease, reproductive health, mental illness—evaluations also carry ethical weight. Poorly designed evaluations can burden participants, breach confidentiality, or extract data from communities without returning benefit.
A distinctive feature of public health program evaluation is its orientation toward population-level effects rather than individual patient outcomes. The evaluator's unit of analysis may be a clinic, a school district, a city, or a nation. This creates methodological challenges: populations are heterogeneous, contexts vary, and randomized assignment is often impossible or unethical. An evaluation of a citywide lead-poisoning prevention campaign cannot randomly assign some children to be exposed to lead. It must work with the natural variation that exists, using designs that can support causal inference without full experimental control.
The roots of program evaluation lie in several converging traditions. Nineteenth-century public health reformers conducted what would now be called evaluations when they documented the mortality reductions following sanitation improvements, though they lacked the formal methods and the label. Early twentieth-century social research contributed survey methods and statistical techniques. The term "evaluation research" emerged in the mid-twentieth century, particularly in education and social welfare, where evaluators assessed programs like Head Start and various anti-poverty initiatives.
Public health adopted and adapted these methods during the latter half of the twentieth century. The growth of federal funding for health programs in the United States and comparable developments elsewhere created demand for accountability. Epidemiologists brought their causal inference toolkit—cohort studies, case-control designs, and later, more sophisticated quasi-experimental methods. Health economists contributed cost-effectiveness analysis. The field crystallized as a distinct specialty with its own textbooks, journals, professional organizations, and training programs.
A crucial development was the shift from evaluation as an external, after-the-fact judgment to evaluation as an ongoing, integrated practice. This shift reflected both practical lessons—programs are easier to improve when evaluated during implementation—and philosophical arguments that communities and program staff should participate in shaping the questions asked about them. The result is a field that contains multiple traditions, each with its own assumptions about what counts as evidence and who should control the evaluation process.
The field is organized less by a single paradigm than by several enduring approaches that answer different questions and rest on different assumptions. These approaches coexist, overlap, and sometimes conflict. Understanding their distinctions is essential for reading evaluation literature and for designing evaluations.
The experimental tradition treats evaluation as a form of causal inference. Its gold standard is the randomized controlled trial (RCT), in which units—individuals, clinics, schools, or communities—are randomly assigned to receive the program or to serve as controls. Randomization, when feasible, ensures that the program and control groups are comparable on average, so that observed differences in outcomes can be attributed to the program with high confidence.
In public health, true experiments are often difficult. It may be unethical to withhold a promising intervention from a control group. Programs may operate at the community level, making individual randomization impossible. Political and practical constraints may prevent random assignment. The quasi-experimental tradition addresses these constraints by using designs that approximate experimental logic without randomization. Common approaches include interrupted time series (measuring outcomes repeatedly before and after program introduction), regression discontinuity (comparing units just above and below a program eligibility threshold), and difference-in-differences (comparing changes in the program group to changes in a comparison group over the same period). Propensity score matching and instrumental variable analysis are statistical techniques used to strengthen quasi-experimental designs.
The experimental tradition's strength is its capacity to support strong causal claims. Its limitation is that it often treats the program as a black box: it can establish that a program worked but not why or how. It also struggles with questions of generalizability—a program that works in one setting may fail in another, and experiments alone cannot explain why. Moreover, the conditions of an experiment (careful implementation, dedicated staff, motivated participants) may differ substantially from routine practice, a problem known as the efficacy-effectiveness gap.
The theory-driven tradition emerged partly as a reaction to the black-box character of experimental evaluation. Its central premise is that programs embody theories about how change happens—the program's theory of change or logic model. An evaluation should make that theory explicit, test its components, and identify which links in the causal chain are weak or broken.
A logic model typically specifies the program's inputs (resources), activities (what the program does), outputs (direct products of activities), and outcomes (changes in participants or populations), along with the assumptions linking each stage. The evaluator's task is to determine not only whether final outcomes improved but whether the intermediate steps occurred as expected. If a smoking cessation program failed to reduce quit rates, theory-driven evaluation asks: Did participants attend sessions? Did they learn the skills taught? Did they attempt to quit? Did they relapse? The answer locates the failure in the causal chain and suggests where to intervene.
This approach has several strengths. It produces actionable findings for program improvement. It can explain why a program works or fails, which supports adaptation to new settings. It accommodates complexity by acknowledging that programs operate through multiple pathways. Its limitations include the risk of imposing an overly linear model on messy reality, the difficulty of measuring every link in the chain, and the possibility that the program's actual mechanism differs from the theory its designers articulated.
A third tradition shifts the focus from methods to use. Utilization-focused evaluation, developed primarily by Michael Quinn Patton, begins with the question: Who will use the findings, and for what decisions? The evaluator works with intended users to identify their information needs, design questions that serve those needs, and present findings in forms that will actually be used. The success of an evaluation is measured not by its methodological elegance but by its contribution to program decisions.
Participatory evaluation extends this logic by involving program staff, participants, or community members in the evaluation process itself. Community-based participatory research (CBPR) is a related tradition in public health that emphasizes equitable partnerships between researchers and communities, shared ownership of the research process, and action to address community-identified concerns. These approaches argue that external evaluators may miss important questions, that communities have knowledge essential to interpreting findings, and that evaluation should build capacity rather than merely extract information.
These traditions have been influential in public health because many programs serve marginalized communities with histories of exploitation by researchers. Participatory approaches can improve the relevance, validity, and ethical quality of evaluations. Their limitations include the time and resources required for genuine partnership, the potential for conflicts between community interests and funder requirements, and the risk that close involvement with the program compromises the evaluator's independence.
Economic evaluation asks whether a program's benefits justify its costs. The main forms are cost-effectiveness analysis (comparing the cost per unit of health outcome, such as cost per quality-adjusted life year gained), cost-benefit analysis (expressing both costs and benefits in monetary terms), and cost-utility analysis (a variant of cost-effectiveness using preference-weighted outcomes). These methods are used to inform resource allocation decisions, particularly when programs compete for limited funds.
Economic evaluation is not a rival to other approaches but a complementary lens. It requires outcome data from effectiveness evaluations and adds the dimension of efficiency. Its limitations include the difficulty of valuing health outcomes in monetary terms, the challenge of capturing long-term and spillover effects, and the ethical question of whether cost considerations should determine access to health programs. In public health, economic evaluations often face the additional challenge that many benefits—such as reduced health inequities or improved community well-being—are difficult to quantify.
Realist evaluation, rooted in the work of Ray Pawson and Nick Tilley, offers a distinctive philosophical stance. It asks not "does this program work?" but "what works, for whom, in what circumstances, and how?" The approach is grounded in scientific realism, which holds that programs work by triggering mechanisms—underlying processes of change—that operate only in certain contexts. An evaluation should therefore identify the mechanisms a program activates and the contextual conditions that support or suppress them.
For example, a peer-support program for new mothers might work by providing social validation (mechanism), but only when participants trust the peers (context). A realist evaluation would test this configuration across different settings. This approach is particularly suited to complex interventions that interact with their environments. Its limitations include the difficulty of identifying mechanisms with confidence, the complexity of analyzing context-mechanism-outcome configurations, and the risk of producing findings so conditional that they offer little guidance for action.
These traditions are not mutually exclusive, and most evaluations combine elements of several. A large-scale evaluation of a national diabetes prevention program might use a quasi-experimental design to estimate effects, a logic model to guide process measurement, cost-effectiveness analysis to inform funding decisions, and participatory methods to engage community health workers in interpreting findings. The choice of approach depends on the evaluation's purpose, the stage of the program, the resources available, and the preferences of stakeholders.
The field does contain genuine disagreements. The most persistent is between those who prioritize internal validity (confidence that observed effects are caused by the program) and those who prioritize external validity (confidence that findings apply to other settings) or practical utility. Experimentalists sometimes criticize participatory approaches for sacrificing rigor; participatory evaluators sometimes criticize experiments for imposing external priorities and ignoring local knowledge. These debates are productive when they force evaluators to justify their choices, but they can also become unproductive when they harden into methodological orthodoxy.
A more recent development is the emphasis on implementation science, which studies the processes by which evidence-based programs are adopted, implemented, and sustained in real-world settings. Implementation science overlaps heavily with program evaluation, particularly the process-focused and theory-driven traditions, but it has its own identity and priorities. It asks how to close the gap between what is known to work and what is actually done in practice.
Contemporary public health program evaluation is characterized by several enduring features. First, it is methodologically pluralistic. Evaluators are expected to be conversant with experimental and quasi-experimental designs, qualitative methods, economic analysis, and participatory approaches, and to select methods based on the questions asked rather than ideological allegiance. Second, it is increasingly attentive to equity. Evaluators are asked to examine not only whether programs improve average outcomes but whether they reduce or exacerbate disparities across racial, ethnic, socioeconomic, and geographic groups. This has led to the development of equity-focused evaluation frameworks that center questions of fairness and justice.
Third, the field has embraced the complexity of real-world programs. The recognition that programs operate within dynamic systems—influenced by policy, culture, economics, and other programs—has led to interest in systems thinking and complexity-informed evaluation. These approaches acknowledge that simple linear models may be inadequate for understanding how health programs interact with their environments.
Fourth, the field has become more professionalized and standardized. Government agencies, international organizations, and professional associations have published evaluation standards and competencies. The Centers for Disease Control and Prevention's Framework for Program Evaluation in Public Health, first issued in 1999 and updated since, is widely used in the United States. Similar frameworks exist internationally, including those from the World Health Organization and various national public health agencies. These frameworks typically emphasize utility, feasibility, propriety, and accuracy as the criteria for good evaluation.
Finally, the field faces persistent challenges. Funding for evaluation is often inadequate, particularly for small community-based programs. The demand for evidence sometimes outpaces the supply of trained evaluators. Political pressures can compromise evaluation independence. And the gap between producing findings and ensuring their use remains wide. These challenges are not new, but they are enduring features of the landscape that any evaluator must navigate.
Public health program evaluation is thus best understood as a practical discipline that integrates scientific methods, ethical commitments, and political awareness. It is neither a pure science nor a pure management tool. It is a way of asking hard questions about programs that aim to improve health, and of answering those questions with evidence that is rigorous enough to trust, relevant enough to use, and honest enough to acknowledge its own limits. The field's durability rests on this combination: a commitment to truth-telling about programs, tempered by an understanding that programs are human creations, embedded in human contexts, and accountable to human communities.