This paper will describe how the growing use of rigorous impact evaluations—particularly randomized control trials (RCTs)—as a routine management tool in the private sector might be carried over to administration in the public sector. The paper will consist of three parts:

1)      A description of the growing role of experimentation and rigorous testing of management changes (often incremental changes in products, service or service delivery) in the private sector. These experiments, often using existing data sources, are integrated into performance management systems and are reviewed regularly at senior management meetings. The paper will also discuss the value of these ongoing, typically interconnected trials for the firms’ effectiveness, creating an iterative system of continuous quality improvement.

2)      Based on conclusions from the private sector experiences, the paper will consider the feasibility and potential value of using RCTs and other rigorous impact evaluations in assessing the impact of incremental changes in services, delivery mechanisms, facilities, customer interfaces, and other means under the ordinary control of public managers and executives. This discussion will also delineate the conditions—e.g., types of programs or services, pre-conditions regarding data on services and outcomes—under which RCTs might be used as a management tool in the public sector.

3)      A third section in the paper will review and assess several options for promoting the use of experimentation and rigorous impact evaluations in public administration. Such options may include national awards or prizes, training/educational programs, and federal or state assistance grants supporting such work in specific program areas.

Throughout the paper, the authors will contrast this way of using impact evaluation to traditional rigorous program evaluations, which are typically multi-year projects—conducted by outside contractors—that assess programs with multiple attributes that involve costly data collection methods. These evaluations seek to test new or existing policy or program approaches whose adoption, continuation, and funding often depend on high-level decision-makers, such as governors or mayors and legislatures. By contrast, the approach we advocate uses RCTs to evaluate incremental changes generally within the control of managers and administrators. By focusing on small modifications in technique or services, the political risks of failure are reduced for public officials in comparison to evaluations of entire programs or major policies.  In addition, the paper will compare methodological and governance advantages and disadvantages of relying on impact evaluations rather than outcome or process measures as the basis for making management changes.

Data on evaluations of different types will come from a wide-ranging search of evaluation reports, authors’ experiences with evaluations, interviews with private and public managers, and secondary sources.