Association for Public Policy Analysis & Management

Roundtable: The Role of Analysis and Program Evaluation in Policy-Making

(The Impacts of Politics on the Policy Process)

What do we know about the factors that determine the incorporation of good analysis into public policy decisions? Many of us teach program evaluation, policy analysis, economics, and decision theory, and we do so in the hope that our students will use these techniques (and other forms of analysis) to influence policy decisions. But the interface between those doing the analysis and those making the decisions is poorly understood. This roundtable will explore this interface and talk about when analysis makes a difference in policy-making . . . and when it doesn’t. One can broadly categorize the factors that affect the role of policy analysis. Political factors, including the salience of the issue at hand and the interest group environment are well recognized to be important influences on policy decisions, and by extension on the role analysis can play in those decisions. Policy-making is a fundamentally political task but decision-makers still value analysis both for its ability to help them make better decisions, and to enhance the legitimacy of those decisions. This leaves room for analysis and program evaluation to make a difference but this room varies from issue to issue. The organizational structure in which analysts operate influences when they are brought into policy deliberations and their ability to raise questions about the preferred policy alternatives. Little thought has been given to whether a bureaucracy’s structure empowers analysts or disempowers them. Finally, the legal setting of a policy decision can restrict the use of analysis and program evaluation by eliminating policy options or forbidding the consideration of certain factors. It can also create incentives to rely upon or base decisions on data and analyses (i.e., evidence). These broad categories however, leave much room for more particular questions. How do politics, law, and bureaucracy play out in the real world when analysts inside and outside government are trying to be heard? The members of this roundtable have a wide variety of experiences in academia, foundations, and government. They will be able to speak the role of how politics, law, bureaucracy, and other factors enhance or impede analytical thinking and the use of “evidence” in government decisions. Among the questions that the roundtable will discuss: When and how does policy analysis and program evaluation make a difference in policy-making? What specific factors are most critical in determining the effectiveness of policy analysts? What reforms (if any) would enhance the role of analysis and program evaluation in policy-making? How has the role of analysis and “evidence” in policy-making changed over the past few decades?