Association for Public Policy Analysis & Management

Panel: Exploring Challenges to Regression Discontinuity Estimation

(Tools of Analysis: Methods, Data, Informatics and Research Design)

In this session we explore three ways to mitigate potential problems with Regression Discontinuity (RD). First, while RD has a better theoretical warrant for producing unbiased estimates than other quasi-experimental methods, empirical evidence has not yet been systematically compiled to describe the extent to which it actually produces unbiased estimates across a wide range of applications, given the difficulties of implementing RD well. The first paper addresses this issue by comparing 231 RD and RCT estimates from 14 studies that deliberately sought to test for bias in the RD and that employed the same treatment in the RCT and RD. A second problem with RD is that its estimates cannot be generalized beyond the cut-point that determines treatment status. The second and third papers explore ways to combine RD estimates with other information to enable researchers to produce impact estimates that might be unbiased away from the cut-point as well as at that point. In paper two, a non-equivalent comparison group provides the information needed, and in paper three it is an RCT implemented close to the cutoff but not otherwise along the running variable. A final problem with RD estimation is that manipulation of the running variable can produce biased results but testing for this potential source of bias is challenging when the running variable is discrete. The final paper presents a method to test for manipulation of discrete running variables. This session brings together researchers from six institutions (including the chair and the discussant), all of whom are doing research on regression discontinuity, but who cover a variety of fields (economics, political science, education methods, and evaluation) and types of institutions (academic and non-academic). Two papers are on voting issues, one is on summer learning, and the meta-analysis covers a variety of social science topics. The list of co-authors also includes one current graduate student and one person who will start graduate school this fall. These sources of diversity should help to promote a rich exchange of ideas and development of areas for future research collaborations.