Association for Public Policy Analysis & Management

Roundtable: Building and Using Evidence-Based Practices: Emerging Approaches across Policy Domains

(Methods and Tools of Analysis)

Innovative social programs involve complex processes that are embedded in larger systems and organizational contexts. While large-scale, summative impact evaluations are one important approach for providing evidence about ‘what works,’ supporting learning and improvement on an ongoing basis is also important, if we are to solving persistent problems facing children, families, and communities. For evidence-based programming to thrive, we need multiple paths to support evidence-building.

This roundtable highlights four current efforts to improve the evidence base for social programs across a range of policy domains. These approaches focus on implementation and operational aspects of programs, with an eye toward actionable findings for managers and policy-makers.  Each participant is leading an initiative in this arena and brings perspectives from a variety of institutional contexts, including social policy, education, nonprofit management and international development.  Each panelist will 1) describe the initiative 2) discuss the challenges and advances implementing these initiatives, 3) discuss the implications for increasing the use of implementation-oriented evidence strategies. These short presentations will be followed by interchange among the panelists on common themes and challenges and 30 minutes of moderated discussion with the audience.

Participants will reflect on the following initiatives.

• Project Evident’s approach to Strategic Evidence Planning puts nonprofit organizations in the driver’s seat of evidence building. Nonprofits access evidence-building tools, talent, and approaches that speed up program improvement and position programs to scale. The focus is on self-driven evidence-building strategies for nonprofit organizations with varied evaluation capacity.
• The Goldilocks Challenge: Right-Fit Evidence for the Social Sector seeks to moderate the pressure faced by NGOs and nonprofits to demonstrate impact and focus attention instead on a parsimonious framework that guides both nonprofits and funders to focus on “right-fit” evidence for learning and improvement.
• The Implementation Research Incubator supports rigorous implementation analyses in social program evaluations and aims to share insights and approaches across communities of research and practice
• Rapid-cycle evaluation (RCE) provides decision makers with timely and actionable evidence of whether operational changes improve program outcomes, often using randomized controlled trials of A/B tests for these operational changes. The Ed Tech RCE Coach is a free, online interactive research tool that allows school administrators to execute low-cost, quick-turnaround evaluations. The Coach also aims to build capacity of administrators to interpret and generate evidence.