Association for Public Policy Analysis & Management

Training future scientists is critical for all federal scientific agencies and for the future of biomedical research in the United States. The National Institutes of Health (NIH), the federal scientific agency tasked with enhancing health and reducing disability and illness through research advancements, dedicates funding to training the next generation of biological and medical researchers. Ultimately, NIH’s goal is to prepare budding scientists to eventually become NIH-funded investigators and scientific leaders. In a time of limited resources, assessing the impact of these training programs on career outcomes is important.

We are not the first to study career outcomes of federally funded science research training programs. A previous research study reported positive outcomes related to time to degree completion, receipt of tenure-track positions, tenure, and timely applications for federal research grants for Ruth L. Kirschstein National Research Service Award (NRSA) pre-doctoral recipients (Pion, 2001). Follow-up studies of the NRSA postdoctoral fellowship and trainee awards found differences related to the time to first independent researcher grant award for NRSA postdoctoral recipients compared to non-recipients (Mantovani, Look & Wueker, 2006).

These studies provide descriptive information to policymakers that enhance our understanding of the outcomes for individuals in training related programs. However, the methodologies used do not specifically disambiguate the impact of the NRSA program from the impact of other individual and environmental factors. Recent studies revealing racial disparities in NIH grant funding also suggested that training did not improve award probabilities for African-American, Asian or Hispanic applicants (Ginther et al, 2010). Ginther and colleagues recommended NIH conduct a closer investigation of the impact of training by race and ethnicity to provide additional insight into differences in R01 award probability and perhaps identify potential policy levers for diversifying the scientific workforce. With this recommendation and current workforce conditions for biomedical researchers in mind, it is important to update and increase the rigor associated with evaluations of NIH training programs.

We analyze the impact of NIH funded training on an individual researcher’s NIH-funded independent investigator research career. We use data from the NIH Information for Management, Planning and Coordination II (IMPACII) database, which contains administrative records and demographic data for all NIH administered grants and awardees. Using regression discontinuity, we calculate estimates of the causal effects of programs on anticipated outcomes when treatment is based on an assignment variable such as fellowship criterion score. NIH’s multi-step selection process for NRSA fellowships generates a score via a peer review system. We use this score to empirically examine the impact of the NRSA training for fellows who just scored into a fellowship to those who just scored outside of the fellowship on their longer term NIH-funded career outcomes. We apply a “fuzzy” regression discontinuity design using NIH fellowship application scores. Our study will increase NIH’s understanding of the impact of its training and provide the agency with additional evaluation of programs that are relevant to stakeholders.