Over the past two decades, both economists and education researchers have devoted considerable effort to trying to identify measures that would predict effective teachers at the time of hire. Yet, no single indicator (e.g., high school grades, scores on college entrance exams, subject area expertise, or the competitiveness of college attended) has consistently predicted teacher effectiveness.

One interpretation of this research literature is that it is not the level of cognitive and non-cognitive skills that will predict teacher effectiveness. Rather, it is the interactionbetween individuals’ skills and their ability to apply them to a classroom setting that will shed light on their potential to become effective instructors. If this were true, demonstration lessons could be a useful hiring tool for school districts.

To address this question, we randomly selected 60 college graduates who applied to an alternative pathway into teaching in Argentina in 2013 to prepare and deliver two 15-minute demonstration lessons for a grade and subject of their choice. Applicants were randomly assigned to one of five rater pairs that graded their lesson plans, demonstration lessons, and debrief on their lessons using an adaptation of a widely-used rubric. We then measured the effectiveness of these applicants after they entered the teaching program using student surveys, principal surveys, classroom observations, students’ grades, and teacher-reported measures of students’ non-cognitive skills. We collected data on these measures during a summer training institute, right after applicants had entered the program, and we are collecting further data on multiple occasions during the two years that the applicants will devote to the program. We are collecting these validation data for both study participants and all of their peers in the program.

Our study will answer two questions: (i) can demonstration lessons predict teacher effectiveness, as measured by multiple measures (both individually and combined)?; and (ii) how many raters, interview tasks, and occasions of measurement are necessary to obtain reliable scores for these demonstration lessons? We will answer the first question with a number of descriptive analyses using the measures of effective teaching from the summer training institute and the first year of teaching of study participants. We will answer the second question with a generalizability study, which employs a multilevel modeling framework to distinguish among different sources of variability in a score, and with a decision study, which shows how to improve reliability by adjusting the number of raters, interview tasks, and occasions of measurement.

Our study is the first to assess the usefulness of demonstration lessons at time of hire. It is also among the first to analyze the predictive power and reliability of demonstration lessons in a middle-income country. And it is among the few to validate hiring information with multiple measures of effective teaching beyond teachers’ value-added.