Panel Paper: Impact of Curricular Reform On Course-Taking, Classroom Composition, and Achievement: A Multi-Site Regression Discontinuity Design

Saturday, November 10, 2012 : 1:45 PM
Salon A (Radisson Plaza Lord Baltimore Hotel)

*Names in bold indicate Presenter

Takako Nomi and Stephen W. Raudenbush, University of Chicago

Inequality in access to rigorous high-school mathematics courses is closely linked to inequality in access to college admissions and labor market opportunities. In an attempt to overcome this inequality, the Chicago Public Schools launched a program called "double-dose algebra" for ninth graders. The idea is to engage all students in rigorous math coursework by ensuring that those with little prior preparation receive enhanced instructional opportunities. The district required students with incoming test scores below the national average to enroll in support coursework in addition to regular algebra. However, this policy had the unintended effect of increasing skill-based segregation in math classrooms.

An earlier study, using a regression discontinuity design, showed that both the intent-to-treat effect and the compliers’ average treatment effect were modest but highly statistically significant. Moreover, the effect of taking double-dose algebra varied significantly across schools. The current study extends this earlier work to understand heterogeneity in the treatment effect across schools as well as mediating mechanisms of the policy that may be related to treatment effect heterogeneity. We explore conditions under which schools produced the largest effects.    

We focus on two policy implementation features and their variation across schools. The first source of variation is the degree to which schools complied with the cutoff-based course assignment policy. The second source is the degree to which schools segregated algebra classes on the basis of students’ prior skills in response to the policy. We ask: 1) Is the total effect of taking double-dose algebra related to the degree of segregation?; 2) What is the effect of taking double-dose Algebra, holding constant classroom peer ability?; and 3) What is the effect of classroom peer ability, holding constant taking double-dose Algebra?

We use data from Chicago Public Schools, the third largest district in the nation, predominantly serving low-income and minority students. Our sample consists of 12,916 students in 60 high schools. To estimate the school-specific total effect of double-dose algebra and its between-school variability, we use school-by-cutoff score interaction terms as instruments in a random effects framework. We then use the same instruments to identify the independent effect of course taking and classroom peer ability as the two mediators of the policy.

The total effect of double-dose algebra (i.e., the combined effect of double-dose Algebra coursework and peer academic composition) was positive and significant with an effect size of 0.20, and the effect varied considerably across schools. Schools with the lowest level of skill-based segregation produced the largest effect, and it was more than twice as large as the average effect among schools with the largest skill-based segregation. The effect of double-dose algebra coursework net of classroom peer ability effect was 0.30, while the effect of classroom peer ability net of double-dose algebra effect was 0.40. In other words, the benefit of additional algebra instruction could be compromised if students are placed in classrooms with relatively lower-ability peers.