Association for Public Policy Analysis & Management

Roundtable: Building Evaluation into Program Design: Lessons from the Department of Labor’s Workforce Innovation Fund

(Employment and Training Programs)

The Workforce Innovation Fund (WIF), like several other Federal initiatives using a “tiered evidence” approach, provides multiple grants to State and local agencies to demonstrate innovative strategies or replicate effective ones in order is to build evidence-based practices in the workforce development field. Each of the 43 projects includes an independent evaluation and evaluators and practitioners must work together to design and implement interventions and the evaluations. Lessons from WIF can provide insight on how to design and evaluate social programs building on an existing evidence base. The roundtable session will focus on innovative practices in building in evaluation early in the program planning and implementation stages. Given that practitioners of social programs are increasingly being required to contract for independent evaluations of their programs, this session will offer an opportunity to discuss lessons learned in the WIF evaluations for both evaluators and practitioners alike. To support WIF, DOL contracted with Abt Associates to serve as the National Evaluation Coordinator and Technical Assistance provider (NEC/TA) for the third-party evaluations for all three rounds of funded grants. The WIF NEC/TA project director at Abt, Eliza Kean, will begin the session with a 15 minute presentation that provides background on the purpose and scope of WIF, a brief overview of the substance of WIF interventions and evaluations, and introduces some considerations and overarching lessons learned around evaluation resources, standards, and practice. While Abt, as the WIF NEC, has a high-level understanding of each of the WIF third party evaluations, the WIF grantees and evaluators have valuable first-hand knowledge of the planning, design, and execution of the WIF innovations and evaluations. As such, the Roundtable will include presentations from at least three third party evaluators. Each evaluator panelist will deliver a 10-15 minute presentation focused on their experiences working as a third party evaluator for a WIF grant. The presentation will include an overview of the WIF innovation evaluated, a discussion of opportunities and challenges in designing the evaluation alongside the program planning and implementation, and discussion of evaluation findings and implications for generalizability and program sustainability. Some key issues to be addressed by panelists include benefits and challenges of obtaining data for local evaluations, maintaining independence and implications for sustainability for grantee-led evaluations. Following the presentations, Eliza Kean will moderate 35 minutes of discussion, including discussion among the Roundtable presenters and questions and answers from the audience.