Association for Public Policy Analysis & Management

The evidence base on preschool programs is large and growing, but we severely lack evaluations of preschool programs that use experimental designs. Most of the current research reporting long-run preschool effects has depended on quasi-experimental designs (see Phillips et al., 2017 bibliography), leaving questions as to whether the widespread scale-up of public preschool should be expected to produce long-lasting effects on children’s lives. The Tennessee Voluntary Prekindergarten Program (TNVPK) was recently evaluated with a lottery-based random assignment design (Lipsey, Farran, & Hofer, 2015). However, study evaluators opted to disregard the random assignment design of the study and instead estimated treatment effects using propensity score techniques to adjust for selection into the TNVPK program. Moreover, the sample used for that study was a subset of children enrolled in the experiment for whom Tennessee administrative data are now available. In the current study, we provide a re-analysis of TNVPK data in which all of our intent to treat (ITT) impact models rely on variation generated by random assignment (i.e., lottery design).

Relying on administrative records from TN, we found that assignment to the program had no effect on retention, placement in gifted and talented programs, absences or major disciplinary offenses measured between kindergarten and grade 3. Children assigned to the treatment group were 4% more likely to be placed in special education over this period, with most of this placement occurring during either preschool or kindergarten. Perhaps more surprising, we also found a negative, and marginally statistically significant, effect of placement in the TNVPK program on third grade state test scores of -0.082 SD’s (SE = 0.045).

To further explore how these long-term null and iatrogenic effects arose, we relied on a subsample of students (n = 1065) who participated in a more intensive version of the study that used researcher-collected measures of cognitive skills and behavioral adjustment beginning at the end of preschool. We found that assignment to the program had a positive and statistically significant effect on end-of-preschool cognitive skills (β = 0.158, SE= 0.047), but this impact faded to 0 in kindergarten and second grade. Confirming the results for the state achievement tests, we again found negative point estimates for second (β = -0.087, SE= 0.048) and third grade (β = -0.050, SE= 0.053) cognitive skill measures, but the third grade effect was not statistically significant. For measures of teacher-rated behavior, we found no effect of assignment to the program on kindergarten or first grade behavioral ratings, but negative and statistically significant effects of the were detected in second grade.

In the full paper, we examine the sensitivity of these results to a number of alternative specifications. Because random assignment happened within each study preschool center, we investigate site heterogeneity, and we also present results that adjust for attrition and non-compliance. In general, we found that estimates were fairly robust to modelling specification, and we found no indication that attrition affected impact estimates. Implications for educational policy will be discussed.