Policymakers have launched an array of policies to increase Advanced Placement (AP) course-taking.  President Obama’s Council of Advisors on Science and Technology reported that “student enrollment in [AP] programs is one of the bright spots of STEM education in recent years.”  Yet, there is a high degree of uncertainty about the effects of AP courses.

We are conducting a randomized control trial (RCT) evaluating the impacts of taking an AP biology or AP chemistry high school course on students’ success in high school and college success.  Funded by the National Science Foundation, this study is the first RCT to evaluate the efficacy of the AP curriculum.  Our ongoing multi-year study has thus far recruited more than 1,100 student participants in 29 high schools.  Eligible student participants are randomly assigned access to either the treatment group (those offered enrollment in the school’s new AP biology/chemistry class) or the control group (“business as usual”).

The AP curriculum developed by the College Board aims to prepare high school students for the rigor of college coursework. The College Board has recently revised AP science courses to better develop students’ ability to conduct scientific inquiry. The revised courses emphasize depth of inquiry and application as opposed to the acquisition of limited knowledge in many content areas.

The goals of our study are to conduct a formative and summative evaluation of the new curriculum and produce findings that can be used by educators to strengthen the teaching of advanced science courses in high school.  To evaluate and improve the fidelity of AP course implementation, we are conducting surveys, interviews, and observations of teachers and students in the AP courses.  Our summative evaluation will determine the effects of the curricula on students’ ability to conduct scientific inquiry and their overall educational performance and aspirations, including the number, quality, and type of colleges to which students apply and enroll. We will examine variation in the effects of the new courses according to students’ prior level of preparation and the level of fidelity with which the courses are implemented.

The results of this study will provide the first experimental evidence on the effects of taking an inquiry-based AP science course on students’ educational progress generally and their success in STEM fields in particular, and help inform future policy efforts to increase enrollment in AP courses.

The paper we will present at APPAM will describe our RCT study design and specifically focus on our methods and processes for developing a psychometrically authenticated assessment to measure students’ scientific inquiry skills.  The development of this assessment is timely given the absence of any validated tool that measures students’ inquiry skills and has significant relevance to emergent education policies (e.g., Common Core State Standards and the Next Generation Science Standards).  We will share lessons learned, discuss how the first cohort of students performed on our assessment, and show patterns in the data, including the extent to which observable student characteristics such as gender, race, and income are associated with their scientific inquiry skills.