Panel Paper: The Importance of Integrating Risk In Retrospective Evaluations of Research and Development

Friday, November 9, 2012 : 9:45 AM
Chesapeake (Sheraton Baltimore City Center Hotel)

*Names in bold indicate Presenter

Peter Linquiti, George Washington University


The typical retrospective evaluation of a research and development program characterizes the social benefits of the results and compares them to the social cost of the program.  A positive result indicates a socially beneficial program and a negative result is seen as a poor use of resources.  While this approach yields substantial insight into the value of R&D investments, it yields an incomplete understanding of the phenomenon of interest because it neglects the risks of R&D projects.  This paper focuses on retrospective evaluations of R&D and argues that irrespective of whether the analyst is trying to understand the operation of the studied program or trying to develop generalizable knowledge to inform the design and predict the results of other nascent programs, risks and returns should always be considered together.  More specifically, if a retrospective evaluation ignores risk, the validity of its results may be suspect.  A review of guidelines for practitioners and examples from the literature demonstrate that risk is rarely considered in retrospective evaluations of R&D investments.  In order to close this gap between a normative “to-be” condition and an empirical “as-is” condition, the feasibility of characterizing ex-ante risks in R&D projects is demonstrated and suggestions are made about how to conceptualize the construct of risk in relationship to returns.  Methodologies from the fields of finance and risk assessment are presented as possible points of departure for incorporating risk and return when evaluating  R&D initiatives.  Implications for policymakers are also characterized.