Panel Paper: New Data to Describe Science Investments: STAR Metrics

Saturday, November 10, 2012 : 2:05 PM
Chesapeake (Sheraton Baltimore City Center Hotel)

*Names in bold indicate Presenter

Julia Lane, New York University and Lou Schwarz, Factor21

In common with many countries, the substantial United States investment in R&D is characterized by limited documentation of the nature and results of those investments. Despite the increased calls for reporting by key stakeholders, current data systems cannot meet the new requirements; indeed, the conclusion of the Science of Science Policy interagency group’s Federal Research Roadmap  was that the science policy data infrastructure was inadequate for decision-making. In response to this need, a new data system is being built (STAR METRICS[1]) drawing from administrative records of both research institutions and federal agencies; this paper describes the initial results of that effort – focusing on expenditures during the 2011 Federal fiscal year from awards made by the National Science Foundation. 

The contribution of the paper is three fold. First it describes in a non-technical fashion how these new data can contribute to our understanding of the initial results of science investments. Second, it shows how new computational technologies can be used to go beyond the traditional methods of manual reporting and administrative program coding to capture information at the most granular units of analysis possible. Finally, it discusses the lessons learned for the collection and analysis of data. The most important is leveraging existing data, not relying on surveys and manual reporting; the deficiencies of each have been well documented.

[1] Science and Technology for America’s Reinvestment – Measuring the Effect of Research on Innovation, Competitiveness, and Science