Association for Public Policy Analysis & Management

The present paper is intended to provide researchers with sample design guidance for multi-site randomized trials. To do so, the paper will examine how the number of sites, the number of sample members per site and the ratio of treatment group members to control group members influences the statistical precision of estimates of three key program-effect parameters: (1) the cross-site grand mean effect, (2) the cross-site standard deviation of effects, and (3) the difference between grand mean effects for two categories of sites (e.g. urban vs. rural). The paper will first present simple statistical models for estimating these parameters using data from a multi-site trial and briefly explain why and how these models produce their desired estimates. The paper will then provide simple and intuitive closed-form expressions for: (1) the minimum detectable cross-site grand mean effect, (2) the minimum detectable cross-site standard deviation of effects, and (3) the minimum detectable difference between mean effects for two categories of sites. These expressions will be provided both for designs that randomize individuals within sites and designs that randomize clusters of individuals within sites. Lastly, a series of tables will be presented to illustrate how the number of sites, the sample size per site and the treatment/control group ratio influence the preceding minimum detectable parameter values and how these results depend on other parameters of one’s data such as the explanatory power of baseline covariates and the intra-class correlation of clusters.