Screen Twice, Cut Once: Assessing the Predictive Validity of Teacher Selection Tools
*Names in bold indicate Presenter
In this paper we analyze the relationship between two teacher selection rubrics that are used in Spokane Public Schools and three teacher outcomes: value-added measures of effectiveness, teacher absence behavior, and the likelihood of attrition. Three aspects of our study are unique. First, we observe whether a prospective teacher has been offered a job (which he or she may reject), not just whether a teacher is employed in a position. Because teachers in 95 percent of job searches that include an offer from Spokane accept the offer, the difference between a teacher who is hired in Spokane and a teacher who is not is largely a decision on the district’s part, rather than due to employee preference.
Second, unlike previous studies of teacher hiring, we observe all applicants to SPS, and both the employment outcomes for applicants who are hired into SPS and applicants who are not hired by SPS but are employed in other public school districts in Washington State. We are not limited to analyzing the outcomes for teachers who perform well enough to progress through the SPS hiring process, which means that we can better assess the ability of the selection rubrics to perform their intended job in determining how desirable a potential employee is, without assuming that the screening rubric has a constant predictive validity at all points in its range. Finally, given that we observe all applicants, we are able to employ a Heckman selection correction to account for the potential that estimates are biased by only observing the outcomes of public school teachers.
We find that the ratings on the two selection instruments used by Spokane are significantly associated with teacher effectiveness, particularly in math, and teacher attrition. The magnitudes of these relationships are educationally meaningful: A one standard deviation increase in screening scores is associated with an increase of about 0.06 standard deviations of student math achievement and a decrease in teacher attrition of three percentage points. Correcting for selection into an SPS job does not significantly change the findings.
- CEDR WP 2014-9.1.pdf (2062.6KB)