One of the challenges of the growing emphasis on evidence-based programs is how to understand and evaluate the mechanisms through which successful programs work.  Implementation research attempts to get into the “black box” of the intervention by measuring the process specified by a program’s theory of change. This is particularly important when comparing multiple interventions that aim to change the same set of outcomes through changes in the same mediating variable, but rely on different theories of change.  Though theoretically many evidence-based strategies rely on changes in mediating factors to improve outcomes, changes in mediating variables are rarely measured in the context of a randomized trial.

We address questions about the effectiveness of divergent theories for supporting children’s social-emotional development in a large-scale demonstration sponsored by the Administration for Children and Families known as Head Start Classroom-based Approaches and Resources for Emotion and Social-skill Promotion (CARES). The Head Start CARES is a large, national group randomized evaluation of three classroom-based enhancement strategies designed to improve children’s social-emotional competencies (Incredible Years Teacher Training, Preschool PATHS, Tools of the Mind).  Within a grantee Head Start centers were randomly assigned to one of the program enhancements or control. The demonstration included 307 classrooms in 104 Head Start centers within 17 sites across the country. As part of scaling up a demonstration program nationally, Head Start CARES implemented a systematic framework for the delivery of professional development across multiple programs at a national scale that included training, consistent weekly coaching, and technical assistance.

This randomized trial is unique in that it simultaneously tests the effects of three evidence-based strategies that differ in their theory of change. However, each program enhancement aims to affect the same outcomes through changing teacher behavior and the classroom context. In this paper, we test the divergent theories of change of the three program enhancements by examining the impacts on teacher practice.  These “first order” effects are critical to understanding whether each of these levers of change indeed has the potential to change social and emotional outcomes for children.

Highly encouraging preliminary findings show that each program model demonstrated impacts on teacher practices, consistent with theory. Relative to control group classrooms, classrooms assigned to one of the three enhancements improved teacher practice in the direction anticipated.    

Findings will be discussed in the context of how rigorous implementation research designs can inform both impact research and our understanding of programs’ theoretical underpinnings. As one of the largest demonstration studies of multiple social-emotional enhancements in Head Start programs nationally, the methods and findings from this study will be critical to informing future effectiveness trials of preschool programs and other efforts at changing teacher quality.