Experiments that assign intact groups to treatment conditions are increasingly common in social research. Many interventions operate at a group level and therefore it may be difficult if not impossible to assign individuals to receive different treatment. However, it might be possible to assign intact groups to either treatment or control. This design is called as a group-randomized or cluster-randomized trial. Reasons for adopting cluster randomization are diverse, but include administrative convenience, reduction of treatment-control contamination and avoid ethical issues that arise. Many of the challenges of cluster randomization arise because inferences are frequently intended to apply at the individual level (unit of observation), while the randomization is at the cluster level (unit of randomization). If the analyses are performed at the individual level, the lack of statistical independence among members in a cluster will invalidate standard approaches to both the estimation of sample size and the analysis of the trial data. Application of standard sample size formulas and standard analysis methods will lead to underpowered studies and will tend to bias p-values downwards, thus risking spurious claim of statistical significance by elevating type I errors. The number of subjects required per treatment using standard sample size has to take into account estimate of the intra class correlation coefficient (ICC). As the magnitude of ICC increases, the more individuals within clusters resemble one another and less they resemble those from other clusters. As a result, the sample size required to detect a significant impact increases.  As far as we know, the designs of cluster randomized experiments in the job training literature rarely use ICC to estimate the sample size.

This paper provides new estimates of key empirical quantities related to the statistical power of impact estimates for experimental evaluations of job training programs applying statistical insights from the education literature. We estimate values of ICC using the previously collected randomized trial public-use data from the Job Training Partnership Act study and National Evaluation of Welfare to Work Study. Focusing on earnings data, this paper discusses appropriate precision standards and, for the cluster randomized design, the minimum detectable impacts using empirical values of ICC, regression R² values and standard deviation of earning outcomes. Our estimates of ICC are close to 0.02 which indicates that there is little variation between the sites as compared to the variation at the individual level. These estimates are in sharp contrast to ICCs in the education literature of about 0.2 for achievement outcomes (Hedges and Hedberg, 2007; Bloom, Hayes and Black, 2005) in cluster randomized trials. Our findings suggest that cluster randomized trials of job training programs may not need samples as large as those implied in the education. However, the sample sizes required in clustered randomized trials is still larger than in the case of experiments in which individuals are randomized by a factor of (1+(average_cluster_size-1)*ICC), which can be substantial. We believe that this will help in designing and planning of better future experiments in estimating impacts of job training programs.