Association for Public Policy Analysis & Management

Scientific community is often disappointed by the limited use of scientific research in policymaking, and likewise, policymakers also display frustration with the inability of scientific research to provide useful or actionable information in policymaking process. Majority would agree that public policy decisions should only be made when all the relevant information have been made available to and consider by policymakers. However, there are fewer consensuses on whether policymakers should put premium on scientific and technical information (STI) over other source of information that are of political, social, or economic nature.

While there are plenty researches looking into the relation between science and politics, such as “regulatory science”, or “post-normal” science,” describe a shift from traditional emphasis on scientific epistemic criteria (e.g. validity and reliability) towards a broader question of policy relevance and usefulness. Surprisingly, the study of the institutional design in knowledge-producing organizations and how it might affects the use and impact of the STI they produced, is largely unexplored. In this study, we are interested in finding out how STI are being use in public policymaking process.

Few studies have empirically examined the use and impact of STI on policy reports. In this study, we examine this relationship based on an analysis of the cited references in reports of the US National Research Council (NRC), which is the working arm of the National Academies. The main role of the NRC is to advise Congress on scientific and technical issues through synthesizing credible and relevant STI into a serviceable format for policymakers. Each year Congress and other federal agencies call upon NRC to provide scientific and technical knowledge and recommendations on important policy issues. NRC selects over 8,000 scientists with diverse scientific background and produced more than 200 scientific reports for US policymakers.

We’ve constructed a database of 589 reports produced by NRC between 2005-2012. The 589 reports provided the basis for the measure and analysis of STI content, we linked these reports to other datasets, including EBSCOhost, Engineering Village, Sage, Scopus, and Web of Science to analyze sources of information including STI. In terms of outcome measures, we went to NRC official website to obtain information on briefings given to Congress; accompanied by the direct references of these NRC reports in Congressional documents from Proquest and the University of North Texas Digital Library database.

Our results indicate that STI are widely used in the NRC report-writing process, but more than half of the reports failed to transmit its findings to Congress, we also found that NRC committees’ pattern of STI use will differ depending on the policy area. In addition, reports with more intensive use of STI are less likely to convey to Congress, and more importantly, reports produced by authors of diverse expertise are less likely to have policy impact compared to reports produced by homogeneous committees. These findings have important implications for policymakers and researchers that are trying to foster the evidence-based culture in federal government, and introduce a new area of study for future research.