Nonprofit Program Design: Building Evidence Into the Work From the Start

Most nonprofit programs are designed based on some combination of personal experience, theory, available funding, and organizational expertise rather than systematic review of what has worked in similar contexts. The result is that many programs are invented rather than adapted, missing the opportunity to build on the accumulated learning of similar efforts and setting up evaluation processes that attempt to demonstrate effectiveness without adequate understanding of the mechanisms through which change is supposed to occur. Building evidence into program design begins before a single participant is enrolled, with a systematic review of what is already known about effective approaches to the problem being addressed. Evidence reviews for practitioners, including those available through What Works Clearinghouses for various domains, provide accessible summaries of research on specific interventions and their outcomes. Program designers who invest time in understanding the evidence base for their domain approach design with more realistic theories of change and more targeted models of what their program will do differently or better than existing approaches. Theory of change, the logical framework connecting program activities to intended outcomes through specified mechanisms, is foundational to evidence-based program design. A well-developed theory of change is not simply a diagram but a testable hypothesis about how change occurs. It specifies what the program does, who it serves, what changes are expected, through what mechanisms, and under what conditions. A good theory of change is developed collaboratively with participants, community members, and subject matter experts, not only by program staff. Needs assessment, which systematically identifies the needs of the target population and the conditions that contribute to the problem being addressed, grounds program design in an accurate understanding of the context rather than assumptions. Research methods for needs assessment range from community surveys and focus groups to secondary data analysis to ethnographic approaches that embed program designers in community contexts. Needs assessments that identify the specific barriers and facilitators relevant to the target population enable more precisely tailored program designs. Evidence-based programs, defined as programs with demonstrated effectiveness in research studies, provide a starting point for practitioners who want to build on proven approaches rather than starting from scratch. The process of adapting evidence-based programs to new contexts requires balancing fidelity to the tested model with the adaptations necessary for cultural and contextual relevance. Research on program adaptation finds that adaptations that change surface features while maintaining core components preserve effectiveness, while adaptations that change core components produce less reliable results. Pilot testing is an essential step between program design and full implementation that many organizations skip due to funding pressure or urgency. Piloting with a small group of participants before scaling allows designers to identify implementation problems, refine program components, train staff, and develop the data systems needed for evaluation. Organizations that skip the pilot phase often encounter during full implementation the same problems they would have identified and addressed during piloting. Data collection for learning and improvement should be built into program operations from the beginning rather than added on as an evaluation requirement. Staff who understand what data are being collected and why are more likely to collect it consistently. Data systems that are integrated into program workflow rather than added as administrative burden produce better quality data. Participatory approaches to data collection that involve program participants in defining what is measured and how results are used can improve both data quality and program accountability. Formative evaluation, which provides ongoing feedback during program implementation to support learning and improvement, differs from summative evaluation, which assesses program outcomes at the end of an implementation period. Most programs need both, but formative evaluation is particularly valuable during early implementation when programs are still being refined. Building formative evaluation capacity requires that programs have the data infrastructure and staff capacity to regularly review and act on data about implementation quality and participant progress. The learning culture of an organization is as important as any specific evaluation mechanism. Programs designed and operated by organizations that genuinely want to know what is working and why, rather than those that are primarily interested in demonstrating effectiveness to funders, produce more honest and more useful evaluation information. Creating the organizational conditions for honest assessment, including leadership willingness to acknowledge problems and make changes, is the foundation on which evidence-based program design rests.