Panel: Methodological Developments in the Analysis of Quasi-Experimental Approaches
(Tools of Analysis: Methods, Data, Informatics and Empirical Design)

Thursday, November 6, 2014: 10:15 AM-11:45 AM
Laguna (Convention Center)

*Names in bold indicate Presenter

Panel Organizers:  Vivian C. Wong, University of Virginia
Panel Chairs:  Mark Long, University of Washington
Discussants:  Jeffrey Smith, University of Michigan


Alternative Tests for Manipulation and Sorting in Regression Discontinuity Designs
Coady Wing, Indiana University and Vivian C. Wong, University of Virginia


In many field settings, randomized control trials (RCTs) are not feasible, ethical, or desirable for estimating program impacts, so quasi-experimental (QE) approaches are also needed. However, quasi-experimental approaches have more assumptions than RCTs, their assumptions are less transparent, and the assumptions are often not empirically testable. This panel examines methods for analyzing three common QE approaches – the regression-discontinuity, matching, and instrumental variable designs – and the contexts and conditions under which these analytic methods should be employed. The first paper, presented by Wing and Wong, looks at approaches for detecting sorting behaviors by units in the regression-discontinuity design. When such sorting occurs, the validity of the regression-discontinuity design is undermined because units manipulate their assignment scores to enter a desired treatment condition. The paper proposes alternatives tests for detecting unit’s manipulation of assignment scores near cutoff thresholds, and compares the performance of these methods to the most commonly used approach for detecting manipulation at the cutoff (McCrary, 2008). The second paper, by Ferraro, Rundhammer, and Tchernis, examines analytic methods for addressing possible confounders when treatment and comparison groups are non-equivalent. Using results from a Monte Carlo study, the paper examines the conditions under which using a combination of regression-adjustment and matching approaches for estimating treatment effects is preferable to using one method alone. Finally, Jung and Pirog examine bias that may occur in the LATE estimate when instrumental variable approaches are applied. They examine the case when the ratio of treatment and comparison units in the sample differs from the population parameters of the treatment to comparison ratio, introduce approaches for addressing this bias, and discuss contexts under which their proposed approaches are appropriate. Finally, Jeff Smith (University of Michigan) will offer commentary and discussion of papers presented in this session.