Association for Public Policy Analysis & Management

Rigorous evaluations of social programs are increasingly used to inform evidence-based policy in the US and abroad. Given the complexity of social programs, evaluation methods used to analyze them (such as randomized controlled trials) are also complex. To understand the nature, rigor, and applicability of evaluations of social programs, detailed reports are needed of the programs tested as well as the methods used to evaluate them. However, reports often omit important information, hindering proper critical appraisal and the effective transfer of this research evidence to policy and practice decision-making.

Reporting guidelines have been developed by health researchers over the last 20 years to address this problem. These guidelines involve evidence- and consensus-based recommendations about the information that needs to be reported in empirical studies. Use of reporting guidelines for randomized trials of medical interventions has led to more useful reports within biomedical research areas. Similar reporting guidelines for evaluations of social programs would help to ensure that this area of research meets its obligations to evidence-based policy-making.

This presentation will discuss the applicability of current reporting guidelines to evaluations of social programs. The presentation will begin with an overview of reporting guidelines and the development of the EQUATOR Network for enhancing the quality and transparency of research. The presenter will then describe a project to develop a new reporting guideline for randomized trials of social and psychological interventions (CONSORT-SPI). Development of this reporting guideline included an online Delphi process with 384 stakeholders (researchers, policy-makers, practitioners, funders, journal editors) from 32 countries, as well as a three-day consensus meeting of key researchers, journal editors, and funders. After reviewing the content of this checklist, the presenter will conclude with suggestions for future reporting guidelines that would be useful for evaluations of social programs.

Reporting guidelines are an important step toward improving reports of many designs for evaluating social programs for evidence-based policy. As demonstrated by existing guidelines, the development of reporting guidelines specific to social programs should improve the accuracy, comprehensiveness, and transparency of study reports. Improved reporting of evaluations promises to facilitate the critical appraisal of research and its use in policy and practice decision-making.