Association for Public Policy Analysis & Management

Many empirical studies draw on sophisticated theories that involve constructs that are not directly observable or are otherwise subject to measurement error (e.g., in latent independent variables, mediators, or outcomes). An important consideration in these studies is the treatment of measurement error. Prior literature has shown that ignoring measurement error in these types of variables can lead to biased estimates, inaccurate inferences and confidence intervals, and erroneous estimates of power (e.g., Cheung & Lau, 2008; 2015; Li & Beretvas, 2013)

A common approach to operationalize theories involving latent variables and address measurement error in multilevel settings is multilevel structural equation modeling (MLSEM). MLSEM has shown to be an effective strategy in large sample settings. However, the heavily parameterized nature of the approach requires large sample sizes at each level (e.g., 100 at least clusters) to dependably deliver stable and admissible solutions that provide unbiased estimates of effects and standard errors (Li & Beretvas, 2013).

For instance, with samples of 50 clusters and 20 individuals per cluster, research on multilevel mediation with MLSEM has demonstrated that about 15% of the time MLSEM provides no or inadmissible solutions (e.g., no convergence due to a non-positive definite matrix) and in the remaining 85% of the time when it does provide admissible solutions the results are substantially biased in terms of both the estimated effect and its standard error. This research has established that an absolute minimum of 80 to 100 clusters are required to ensure model convergence and suggested even larger samples are needed for MLSEM to dependably deliver unbiased estimates of effects and standard errors across common conditions (Li & Beretvas, 2013; Hox & Maas, 2001; Gagne & Hancock, 2006). Yet, in many areas of policy and management research, samples of less than 80 clusters are typical and samples greater than 80 clusters may be considered prohibitively large.

We develop a new class of analytic methods that can be an attractive alternative to MLSEM in small to moderate sample settings (e.g., less than 100 clusters). The proposed method—bias-corrected apportioned MLSEM—allows researchers to operationalize and test sophisticated theories involving latent variables while reliably accounting for the detrimental effects of measurement error even in small and moderate sample sizes. In this study, we examine apportioned MLSEM in the context multilevel mediation. Our investigation provides an accessible introduction and targeted assessment of the absolute and relative performance of the proposed method through the case study of one common type of MLSEM. The results are designed to serve four complementary purposes. First, we derive the bias-corrected estimators needed to implement the apportioned MLSEM method in multilevel mediation studies. Second, we develop and test a semi-parametric bootstrap approach to track the variability of the corrected mediation effect estimate. Third, the results are intended to provide an initial assessment of the amenability and promise of apportioned MLSEM. Fourth, the results provide an illustrative example of the method along with its implementation in software.