Association for Public Policy Analysis & Management

Calculating impacts in a study that utilizes a regression discontinuity design (RDD) can be a technically complex affair. Bandwidth estimation, calculating “fuzzy” impacts, accounting for clustering of observations by unique values of the assignment variable, and incorporating multiple assignment variables or cutoffs all add to the complexity of estimation. When confronted with technically complex estimation methods researchers often turn to non-parametric bootstrapping as a simple and robust way to calculate standard errors. In this paper we use Monte Carlo simulations to examine the performance of non-parametric bootstrapping for RDD impact estimation scenarios that incorporate all of the estimation complexities described above. We also propose and examine a residual bootstrapping algorithm that is conceptually motivated by the idea that the RDD is akin to random assignment of residuals conditional on the assignment variable. The data generating processes used for the simulations are taken from previous Monte Carlo examinations of RDD estimation techniques found in the literature, as well as data generating processes that mimic data from our own empirical work in education. We find that both non-parametric bootstrapping and the proposed residual bootstrapping algorithm successfully control the type 1 error rate at the desired level, but that the residual bootstrapping algorithm can provide greater statistical power (that is, non-parametric bootstrapping can yield standard error estimates that are much too conservative).