Association for Public Policy Analysis & Management

The U.S. science and innovation system includes multiple actors and institutions that mutually support and enable the conduct of science and the production of key outcomes. Among these institutions is the university-based research system, which is a key producer of knowledge and discovery. The Evidence Act (P.L.115-435) calls for greater accountability of science and technology projects funded by Federal agencies. While institutions are ranked in terms of reputation, selectivity and production, those metrics do not adequately capture the complexity of research capacity and competitiveness. Based on a comprehensive literature review we develop the Academic Research Excellence and Competitiveness (AREC) framework for evaluation that distinguishes four levels that reflect different activities, production, and outcomes relevant to the research process: ecosystem, institution, project/team, and individual. Ecosystem captures the macro-dimension of the AREC framework, which corresponds to the jurisdiction level associated with states, regions, or countries. All other levels of the framework are embedded within the ecosystem level. Institution level recognizes the operational units where research occurs, including a specific type of institution, such as a university, center, department, or major laboratory. Projects and team dimension taps into a substantial literature on team science and demonstrates that effective management of projects and teams is an important determinant of their success. This work is valuable in informing decisions about Federal investments toward science and technology research, development, and innovation in academia.

Ultimately it is the ability of individual researchers to convert training, support, networks, and opportunities into advances that bring further gains and increase their relative advantage within the relevant community of researchers. Each of the four framework levels also includes the multiple dimensions or elements of AREC including resource acquisition, knowledge production, scientific and technical workforce, visibility and reputation, and economic development. Resource acquisition captures the ability to obtain resources – financial, human, infrastructure, administrative and political – at each level of analysis. Knowledge production represents the ability to leverage existing capacities and new investments to produce new knowledge and innovation. Workforce development recognizes the importance of developing a multi-dimensional, multi-skilled workforce that has the capacity and flexibility to respond and provides solutions to critical scientific and societal challenges. Visibility and reputation involves the relative prominence of an entity at any of the different levels. Reputation is a key currency in academia and is a critical factor for signaling capacity and potential contribution to science and innovation. It also has an indirect effect on the perceived capacity and potential contribution of other associated entities in the framework. Economic development considers competence and capacity for commercialization and potential contribution of research to industry, as well as other stakeholder impacts. It includes the production of new inventions, successful innovation of new products and processes, and successful commercialization of research outputs. Entities at different levels advance competitiveness through partnerships, management of technology transfer, commercialization policies, and researcher ability and motivation. The paper presents this framework and demonstrates how it can be operationalized into a comprehensive theory of change.