Panel Paper: Estimating Variation In Program Impacts: Theory, Practice and Applications

Saturday, November 10, 2012 : 1:45 PM
Hanover B (Radisson Plaza Lord Baltimore Hotel)

*Names in bold indicate Presenter

Howard Bloom, MDRC


This paper focuses on defining, estimating and interpreting variation in the impacts of programs. It begins by considering the consequences of impact variation for public policy, professional practice and evaluation research and noting some current opportunities for rigorously studying this variation through multi-site randomized trials that are being planned currently. The paper then introduces a theoretical model of impact variation based on the potential outcomes framework for causal inference. The paper next explores some key issues in the estimation of impact variation and presents some statistical and graphical approaches for reporting this variation. Lastly, the paper applies these approaches to study impact variation in two policy areas: welfare-to-work programs and high school reform.