*Names in bold indicate Presenter
Because so many factors together shape the choices of teachers in schools, it is often difficult to determine what causes any observed changes during a given period of time. In the absence of random assignment, the current study determined program-eligibility by a strict formula based on school-level FRPL percentages. This approach created the possibility of conducting two quasi-experimental analyses which compare schools within the district who were and were not assigned to implement the Program:
We are concerned that any observed trends in outcomes within the treatment schools might not be due to the Program, but instead trends may reflect changes in outcomes that were taking place throughout the district. The difference-in-differences (DiD) analysis therefore incorporates the trends in outcomes in a set of matched, non-treated Comparison schools, chosen specifically because they were comparable to the treatment schools in key ways. Because the DiD analysis compares treatment schools to Comparison schools both before and after the implementation of the Program, we can eliminate all unobserved time-invariant factors as potential explanations for the estimated impact of the program. However, the DiD analysis is susceptible to sources of bias from unobserved time-varying factors. We therefore also implement a second methodological approach that has the stronger causal warrant, but has a more limited population of causal inference.
The regression discontinuity (RD) approach relies on comparing the outcomes of schools that were just above and below the threshold for eligibility to participate in the intervention. Because whether a school has 49% or 50% FRPL is arguably random, we expect all factors other than program participation to be very similar. The RD has a strong causal warrant for identifying the changes in student- and teacher-outcomes that were caused by the Program. The RD analysis has a few short-comings however: The analysis has low statistical power, since there are relatively few schools very “near” the threshold for eligibility. In addition, the RD estimates pertain to schools that have about 50% FRPL students and may not apply to extremely high- or low- SES schools (limited external validity).