Association for Public Policy Analysis & Management

Teacher coaching is considered a high-quality professional development opportunity that emphasizes job-embedded practice, intense and sustained durations, and active learning. At the same time, this research base is new, and little is known about the effectiveness of specific design features or practices of different coaching models. Understanding which program features are critical for success is an important line of inquiry for the continued improvement of teacher professional development.

In this paper, we estimate the impact of MATCH Teacher Coaching (MTC) on a range of teacher practices using a blocked randomized trial and explore how changes in the coaching model across two cohorts are related to program effects. In the 2011-12 and 2012-13 school years, coaches worked with treatment teachers in charter schools across the Recovery School District in New Orleans on improving practices common across grades and subjects, including behavior management, instructional delivery, and student engagement. We explore whether this program is effective at improving teaching practices by drawing on classroom observations, principal evaluations, and student surveys.

Importantly, several significant changes in the design and delivery of the coaching model in the second cohort provide a unique opportunity to explore potential mechanisms by which coaching may lead to improved teacher practice. By design, the scale of the MTC program increased between the two years, with 49 teachers offered coaching in cohort 2 compared to 30 teachers in cohort 1. To accommodate this change, MTC reduced the average amount of coaching it provided to teachers from four weeks to three weeks throughout the school year and increased teacher-to-coach ratios slightly. In addition, all of the coaches except for the program director changed across years. Finally, programmatic changes induced an increased focus on behavior management over other classroom practices. All of these were strategic changes made by the MTC staff rather than a targeted response to the specific set of teachers who participated in the second cohort. Therefore, between-cohort differences reflect plausibly exogenous variation in program characteristics, which we exploit in our analyses.

Results indicate no effect of coaching on any of our outcome measures when data are pooled across all teachers. However, this finding masks substantial variability in the effectiveness of coaching across cohorts. For cohort 1, we find that coached teachers scored 0.56 standard deviations higher on a summary index of effective teaching practices. In contrast, we find no effect of coaching among cohort 2 teachers with a negative but non-significant point estimate for our summary index. By ruling out explanations related to the research design (i.e., differences in the counterfactual and potential spillover effects, the sample of teachers included in each cohort, randomization block outliers), we attribute differential treatment effects to changes in the program model, which we describe in detail using a rich set of qualitative data including coaching logs and conversations with coaches. A set of exploratory analyses provide suggestive evidence of differences in treatment effects by coach and the focus of coaching.