Association for Public Policy Analysis & Management

Panel: Identifying and Misidentifying Effective Principals from Teacher and Student Outcomes

(Education)

States and school districts regard principals as major contributors to the success or failure of schools and their students. For this reason, nearly all states in recent years have established systems for evaluating principals' performance based on subjective measures (ratings from supervisors) and objective measures (teacher and student outcomes). However, most of these measures have been designed on the basis of very little evidence for their validity. Pursuing the conference's theme that measurement matters, the papers in this panel investigate whether it is possible to distinguish effective and ineffective principals using objective data on teacher and student outcomes from states' administrative records. The first paper (by Grissom and Bartanen) shows that objective patterns of teacher retention can distinguish principals who earn high subjective ratings. The second paper (by Chiang et al.) shows that certain performance measures based on student test scores provide a small amount of information for predicting principals' future impacts, whereas other measures provide no information at all. The third paper (by Cullen et al.) demonstrates the consequences of using poorly designed test-score measures: because of their use in accountability systems, poorly designed measures have a greater influence on principals' career outcomes than do measures that more accurately reflect the principals' impacts.

These papers raise several questions that could stimulate further discussion with the audience. To what extent should principal evaluation systems rely on objective versus subjective measures? Among the objective measures, how much weight should be placed on teacher versus student outcomes? Given the findings in these papers, to what extent should high-stakes personnel decisions be based on objective measures of principal performance?