Association for Public Policy Analysis & Management

Panel: New Measurements on the Effectiveness of Public Organizations and Employees: Examples from Education

(Tools of Analysis: Methods, Data, Informatics and Research Design)

Measuring the effectiveness of public organizations and employees has been a perennial and core policy and management issue to improve the quality of public services. The measurement, trade-off, and property of effectiveness metrics based on multiple socially desirable outcomes is key toward making wise decisions from restructuring organizations to personnel management reform. Education is such a case that in the past decade, estimating school and teacher effectiveness using test scores has greatly shaped education policies and practices. However, on one hand, although test scores have a number of measurement strengths, it only captures one narrow aspect of contributions schools and teachers make to student learning outcomes, ignoring non-cognitive abilities or long-run success. On the other hand, our understanding of the variation of effectiveness measures including value-added to achievement is still in its infancy. This panel presents new development of school and teacher effectiveness. The four papers cover a wide range of issues including new effectiveness measures based on non-test score outcomes, the relationship between different metrics, validity and reliability, and mechanisms that explain the variation in effectiveness. 

 

The first two papers explore the multidimensionality of school effectiveness. The first paper compares school contributions to average gains in student achievement, attendance and the extent of reducing gaps between ethno-racial student subgroups. The second paper estimates school value-added to students’ short-run test scores and attendance, and further examines how they predict long-run outcomes including dropout and high school graduation rates. Both papers use either survey data or measures of school practices generated from textual data to explain the variation of different school effectiveness metrics.

 

The other two papers focus on teachers. Specifically, the third paper uses highly detailed class-level attendance data to identify teacher effectiveness in reducing class absences in secondary schools. The last paper examines the portability of teacher effectiveness using experimental data from the Talent Transfer Initiative study conducted in 10 districts across seven states.

 

Collectively, this panel makes three important contributions: (a) they exemplify the use of new educational measurements that speak to both students’ cognitive gains and socio-emotional development, and both short-and long-run outcomes; (b) they unpack the “black box” of teaching and add new insights into education production “processes”; and (c) the new knowledge informs ways in which we can improve school and teacher effectiveness in order to promote both educational goals of excellence and equity. Panel participants include leading scholars and junior researchers, and practitioners from multiple institutions (e.g., University of Washington, Stanford University, Brown University, University of Virginia, and Seattle Public Schools).