Association for Public Policy Analysis & Management

A multi-site randomized trial provides an opportunity to create multiple instruments in the form of site-by-treatment interactions to estimate the effect of one or more mediators on an outcome of interest. The multiple-site, multiple mediator instrumental variables (MSMM-IV) model leverages cross-site variation in the effect of treatment on the mediators (“compliances”) and the outcome (“intent-to-treat effect”) that enable, in principle, the estimation of the causal effects of multiple mediators on the outcome of interest. (Raudenbush, Reardon, and Nomi, 2012; Reardon & Raudenbush, 2013; Reardon, Unlu, Zhu, & Bloom, 2014). There are two  challenges in using the MSMM-IV estimator in practice. First, the standard exclusion restriction is often not defensible. Second, the MSMM-IV estimator is succceptible to finite sample bias; this bias is in many cases larger than in IV models with a single instrument. This paper aims to address these challenges.

The paper starts by briefly describing and reviewing the generic MSMM-IV model, focusing on the assumptions required for the identification of the causal parameters of the model, and their statistical properties, especially in finite samples. One such assumption is the exclusion restriction which states that that the observed mediator(s) are the only pathway between the treatment and the outcome. We study the properties of an alternative model (MSMM-INT) that allows the estimation of the effect of a single mediator (which constitutes the indirect effect of the treatment on the outcome) in the presence of unobserved mediators whose collective effects are captured through an alternative pathway between the treatment and outcome (direct effect). The paper derives expressions for the expected values of the mediator effect estimate and the direct pathway estimate and their respective sampling variances. We also show that this model can be adapted to estimate the effects of multiple observed mediators separately.

The paper then demonstrates that both the MSMM-IV and MSMM-INT models are subject to finite sample bias. We propose alternative estimators for the direct and indirect effects in the MSMM-INT model that aim to eliminate this finite sample bias. 

Finally, the paper examines the finite sample properties of the three estimators (MSMM-IV, MSMM-INT, and de-biased MSMM-IV) through a series of simulations. These simulation results illustrate how key factors affecting the estimation (the number of sites, the reliability of the first-stage estimates, the degree of OLS bias, and the magnitudes and variances of the true direct and indirect effects) influence the bias and precision of the estimates from the three models. These simulation results, in conjunction with the theoretical findings, also provide heuristic guidance about the preferred estimator(s) under different conditions.