Association for Public Policy Analysis & Management

Multilevel mediation analyses play an essential role in helping researchers probe theories of action and document how policies impact outcomes. In this study, we discuss the planning of hierarchical experiments intended to detect multilevel mediation effects in terms of statistical power and sufficient sample sizes. We develop a statistical framework to estimate the power to detect upper-level, lower-level, and overall mediation with individual- or cluster-level mediators. We consider the power associated with four different types of tests that can be employed in the study planning phase before data is collected: the Sobel test, the joint test, the Monte Carlo interval test, and the partial posterior predictive distribution test. We draw on a running policy example to probe the pathways through which policies impact key outcomes and consider several different types of mediating variables. The power formulas are implemented in the free PowerUp software (causalevaluation.org).