Association for Public Policy Analysis & Management

Panel: Innovations in Evaluation for Improved Internal and External Validity

(Tools of Analysis: Methods, Data, Informatics and Research Design)

The evaluation field has historically focused on isolating the intervention effects from other potentially confounding factors to ensure that the evaluation results have high internal validity. Random assignment yields evaluations with high internal validity when attrition is low. However, the designs with the strongest internal validity sometimes have limited external validity. Randomized trials conducted in a convenience sample of sites may produce results that do not generalize to the broader population of sites. Regression discontinuity designs produce estimates for values close to the cutoff—but the results may not generalize to other points in the distribution. This panel includes papers designed to advance the internal and external validity of impact evaluations. The first paper is focused on improving the internal validity of propensity score matching estimators through machine learning. The second paper is focused on improving the external validity of Randomized Controlled Trials when sites are selected purposively. The last two papers are focused on assessing and enhancing the external validity of Regression Discontinuity Designs for points away from the cutoff—using methods that require high internal validity.