Association for Public Policy Analysis & Management

This paper reports results from an evaluation of the Graduate Research Fellowship Program (GRFP), the National Science Foundation’s flagship fellowship program that provides funding for about 2,000 applicants per year in the STEM fields of graduate education, of whom 14% are underrepresented minority (URM) students.  The findings are drawn from a 2012 quasi-experimental evaluation of the impacts of GRFP participation on various graduate school and early career outcomes compared to those receiving Honorable Mention (HM) for the fellowship. Outcomes were measured using survey data collected from 7,014 current and former Fellows and 3,199 HM designees (including 926 URM Fellows and 351 URM HM designees) from Fellowship award years 1994 through 2011.

Though Fellows and HM designees ostensibly had very similar qualifications predicting success in graduate school and beyond, some differences may still exist and are controlled for in this study.  We used all available background information (that is, pre-GRPF award) to match applicants based on the predicted probability that they would receive the treatment (the Fellowship) instead of what we consider the control (the Honorable Mention designation). The estimated probabilities, or propensity scores, were estimated using a logistic regression with a set of individual and institutional background measures as covariates. The impact of the GRFP Fellowship on the outcomes was then calculated as the difference between the average outcomes of the GRFP Fellows and HM designees, with propensity score weighting applied. The propensity-weighted differences were subjected to a series of additional analyses designed to assess possible differences in the impacts between different demographic groups, graduate fields of study, and graduate institutions.

For graduate school experiences, we find that, overall, GRFP participation had a positive impact on the likelihood of completion of a Ph.D. within ten years, but a negative impact on working for pay and applying for grants or contracts during graduate school. In addition, Fellows reported fewer opportunities to receive training or instruction on research, teaching, industry, or policy and to engage in other research activities through training compared to HM designees. When we model program impacts for URM students, we observe no differences in graduate school experiences by treatment status.

For post-graduate careers and experiences, we find that the Fellowship had positive impacts, overall, on the number of papers presented at national or international meetings, the number of papers published (both in refereed journals and overall), and the number of grants and contracts awarded as a principal investigator after graduate school. The program also had positive impacts on the likelihood of serving on a committee or panel and providing review services, both activities related to successful STEM-related careers.  Additional regression analyses found no differential effects for URMs on these outcomes, with the exception of a negative impact of the Fellowship for URMs on the number of patents applied for after graduate school.

We are planning further analyses to measure program effects for URMs, including GRFP effects on the level of financial support and, after controlling for level of financial support, on graduate education experiences and early career outcomes.