Association for Public Policy Analysis & Management

Panel: Methods for Understanding Impact Variation

(Tools of Analysis: Methods, Data, Informatics and Research Design)

Policy makers, administrators and program evaluations focus increasing attention on the variation in policy impacts. That is, the field has tackled the challenge of estimating average treatment effects and is now innovating around how to say more about what it is about programs that drive their impacts. The proposed panel brings together three papers that focus on distinct aspects of this challenge. The first one draws on a three-armed experimental design and presents new analytic methods for improving non-experimental estimates of program component impacts. The second introduces a new approach for decomposing a total treatment effect into its direct and indirect effect, in the presence of treatment-by-mediator interaction. The third uses competing profiles that align with variation in treatment compliance to explore the associated variation in treatment effects. The proposed panel chair is the government project officer on a major national evaluation that has funded these methodological innovations; and the two discussants are senior evaluation scholars—one each in academic and research settings—whose insights will be of certain interest to the APPAM community.