Association for Public Policy Analysis & Management

Evaluation is an evidence-generating activity that informs decisions on public social programs and policies. Large-scale organizational change has been, will be, and is already happening across the U.S. Federal government regarding the value and use of evaluation. Including the rise of the information age, there have been multiple shifts throughout history, both internal and external to the government, that have influenced the evolution of evaluation use and capacity in the government. However, the full breadth of factors leading to large and small changes toward institutionalizing Federal evaluation remain unknown. It is beneficial to understand the barriers and facilitators (i.e. actors, events, policies, etc.) of evaluation use in complex systems to understand the (in)effective mechanisms that can be wielded or established. The overarching aim of this dissertation is to employ a new theoretical framework of valuing evaluation that was used in conjunction with a novel mapping method to capture influential factors on the evolution of Federal evaluation over the last 25 years.

One way in which government demonstrates value in evidence-generating activities is by investing resources toward greater evaluation capacity. The Office of Management and Budget (OMB) publishes the President’s Budget projections for each fiscal year. Although researchers have found an increase in Federal spending on evaluation over time, there is more to the story behind the numbers. Each Budget includes supplementary documents that append narratives, called Analytical Perspectives, which explicate the rationale and analyses that informed the changes to the proposed budget. Understanding the language, rhetoric, and persuasion of policy processes can be an important approach to tracking the patterns of discourse and observe how change has been codified through language. Findings from the linguistic and discursive analyses were complemented by interviews with 15 key stakeholders using an ethnographic mapping method to paint a comprehensive picture of how evaluation has become institutionalized in the Federal government over time.

Although analyses are currently underway, many factors have already been identified as having facilitated or impeded the institutionalization of evaluation. Political agendas, policies, organizational milieu, turnover, and national crises have all significantly influenced the typification of ideology and practices, albeit disproportionately, across the Federal government. Mapping social and cultural processes for organizational learning has helped to uncover trends and lessons (that should be) learned to inform future policymaking decisions. Most importantly, the ramifications from deinstitutionalizing evaluation continue to have an impact on current policy decisions. To date, there has not been a study of this kind or magnitude done to understand the (de)institutionalization of evaluation. As Federal agencies work to respond to the mandates of the Foundations for Evidence-Based Policymaking Act (P.L. 115-435), this technique can be adopted as an ethnographic process to investigate the micro- and meso-system factors that exist within each agency to be responsive and productive in the given organizational culture.