Results from the Head Start Impact Study (HSIS; Puma et al., 2010) suggest that Head Start had small but significant impacts on children’s cognitive development after one year. Recent analyses find significant variation by program in Head Start impacts (Bloom & Weiland, 2014). As high quality early education is important for children’s development (Burchinal, Vandergrift, Pianta, & Mashburn, 2010), we hypothesize that one source of variation in Head Start’s impacts is classroom quality: Impacts of random assignment to Head Start may be larger (more positive) for children attending higher quality programs. This study uses data from the HSIS (n=4,440) to examine if classroom quality moderates impacts of random assignment to Head Start on children’s cognitive skills.

A challenge to testing our hypothesis is that classroom quality experienced by a child is endogenous to random assignment: Prior work indicates that random assignment to Head Start significantly increases the likelihood that a child attends a high quality learning environment. We address this endogeneity problem first by examining the moderating role of the quality of the Head Start program to which a child applied. We used hierarchical linear models with children nested in the Head Start center to which they applied and included random slopes for treatment to model spring outcomes (receptive vocabulary - Peabody Picture Vocabulary Test and early math -  Woodcock Johnson Applied Problems) as a function of pretest scores, center fixed effects, child- and family-level covariates, random assignment status, and an interaction between random assignment status and observed classroom quality (measured by the Early Childhood Environment Rating Scale).

Preliminary results support our hypothesis, suggesting that when children applied to high-quality Head Start programs, Head Start impacts on receptive vocabulary (b=0.14, SE=0.05, p=0.007) and early math (b=0.11, SE=0.06, p=0.07) were significantly larger compared to when children applied to low-quality Head Start programs. Final results will include more refined measures of quality and propensity score matching to ensure that children in the Head Start and control groups are equivalent on observed baseline characteristics in order to mitigate the threat of selection bias.

A second set of models to be included in this presentation will use principal stratification to classify children into latent subgroups based on the classroom quality they would have experienced under random assignment to both treatment and control (Page, 2012). Using this method we estimate a child’s counterfactual classroom quality, allowing us to examine heterogeneity in causal impacts of Head Start on cognitive outcomes for children in different latent subgroups based on their experienced and counterfactual classroom quality.  An advantage of this approach is that we can compare Head Start impacts within latent subgroups based on a post-treatment variable, experienced quality, while remaining within the experimental framework of the HSIS. 

These analyses provide evidence of the importance of program quality as a necessary condition for generating larger, positive impacts of large-scale early education programs. As new initiatives begin these results highlight the need to focus on the quality of new programs, rather than only the quantity of children served.