Association for Public Policy Analysis & Management

The Commonwealth of Virginia received a federal Preschool Development Grant (PDG) to increase access and improve quality of a state-funded preschool program for four-year olds and their families in high need communities in 11 of Virginia’s 132 school divisions. All Virginia Preschool Imitative Plus (VPI+) classrooms are expected to include components consistent with a high-quality preschool program (e.g., highly educated teachers with child development expertise, use of an evidence-based curriculum, ongoing training and technical assistance). Evidence from Boston (Weiland & Yoshikawa, 2013) and Tulsa (Gormley, 2005) indicate that the provision of high quality preschool across a large school district can yield positive impacts for children but it remains unclear whether statewide efforts such as VPI+ can replicate these results (Farran, 2016). This paper examines the impact of participation in VPI+ on children’s early academic skills at kindergarten entry by using a regression discontinuity (RD) design.

To be eligible to participate in VPI+, a child’s family income had to be less than 200% of the federal poverty level (FPL) and the child had to be between the ages of four and five-years-old by September 30th of the preschool year. We leverage this strict age cut-off to compare children at kindergarten entry who just completed the VPI+ preschool intervention during the 2016-17 school year (VPI+ treatment, n = 969) with children who just began VPI+ preschool during the 2017-18 school year (control, n = 1,140). The impact of VPI+ is then operationalized as the difference in the expected achievement levels for a child that was just old enough to be eligible for VPI+ in fall of 2016 and a child who was just too young to be eligible for VPI+ in fall of 2016 and subsequently enrolled in the VPI+ program in fall 2017. Child outcome measures included the Phonological Awareness Literacy Screening lowercase letters and letter sounds assessments (Phonological awareness); Woodcock Johnson® III Applied Problems subtest (Mathematics skills); Peabody Picture Vocabulary Test (Vocabulary), and the Head Toes Knees Shoulders task (Self-regulation). Covariates included race/ethnicity, maternal education, family income measured at program entry.

Preliminary analyses indicate that RD assumptions were satisfied (compliance with the age cut-off, no evidence of “pile up, and no discontinuities of baseline covariates at the age cut-off) (Lipsey, et al, 2015). All analyses were conducted using a global regression as well as empirically derived optimal bandwidth approaches and included covariates and propensity score weights to account for baseline differences. Preliminary results show large impacts on measures of phonological awareness (ES>1.0) and moderate effects on self-regulation (ES=0.4) across all analytic models. We observe small impacts (ES=0.2) on mathematics skills in some models and no impacts on vocabulary skills.

For policy makers, the findings suggest the importance of strengthening some aspects of instructional quality, including math and language supports. The presentation will discuss how these data are being used to inform program improvement (Derrick-Mills, 2015; Metz, Naoom, Halle, & Bartley, 2015) support state department of education program improvement efforts, and to gauge progress toward state implementation and child outcome goals.