Association for Public Policy Analysis & Management

We introduce a novel approach of evaluating performance of public universities in the US. We propose a series of dynamic performance metrics that evaluate 575 higher education institutions based on a 28 year history of performance data. Contrary to existing measurement approaches that rely on relative performance and rankings of higher education institutions at a specific time, our methodology considers the dynamic performance trajectories over the entire history of observed indicators.  Realizing that higher education institutions try to accomplish a multitude of goals that can be complementary or rival in their use of resources, we identify five distinct dimensions of performance: 1) mission effectiveness, 2) instruction and teaching, 3) equity and diversity, 4) cost and efficiency and 5) research effectiveness. These dimensions have routinely been highlighted in existing literatures in higher education.

For each dimension we estimate a dynamic performance index using indicators that reflect existing academic research on performance in higher education, along with criteria that have been salient in political and policy debates.  However, our selection of indicators is flexible and can be adapted based on feedback and further discussions with experts and policymakers in higher education.  Furthermore, the dimensions that we identify can be updated if new measures are created or if additional data sources are identified.

Estimation of the dynamic indices is conducted using a Dynamic Factor Model approach. This model uses the statistical methodology of factor analysis in a dynamic setting in order to summarize a series of observed indicators into five dimensions of performance. One of the benefits of the proposed methodology is that no prior assumptions need to be made with regards to the importance (i.e. weight) of each indicator. In our model, indicator weights are parameters estimated by the model. This alleviates the need to impose subjective weights in the determination of the performance indices. In addition, the resulting indices are smoothed in that, at every point in time, performance is evaluated based on information from the entire sample, not just from a specific year. That process conveys complete and detailed information about the performance trajectory of an institution.  This is particularly useful when we compare an institution’s performance not only with other institutions but also relative to its own past performance.

The results of our work are of particular importance to education officials, university administrators as well as parents and students. Our goal is to highlight the importance of estimating performance trajectories of higher education institutions rather than focusing on static rankings that are determined based on subjective weights. In addition, our methodology is based on the fact that universities try to achieve a series of different goals.

The set of performance metrics that we estimate are used as dependent variables in a mixed model approach to identify trends between groups of institutions. These groups are based both on the institutions missions as well as their geographic location. This analysis provides valuable insights to policy makers as it allows them to identify which institutions have been historically underperforming in specific areas.