Association for Public Policy Analysis & Management

For over a decade, government officials in the U.S., Australia, Denmark, Korea, and elsewhere have regularly used large surveys of public sector employees to gauge employee perceptions and attitudes toward their jobs, working conditions, organizational policies, coworkers and managers, and performance. In 2002, the U.S. Office of Personnel Management (OPM) began using the Federal Human Capital Survey (FHCS) – now the Federal Employee Viewpoint Survey (FEVS) – to collect attitudinal and perceptual data from tens of thousands of federal employees (e.g., the 2015 FEVS had over 400,000 survey respondents).  

Scholars have used these survey data to produce dozens of peer-reviewed articles that examine a variety of issues central to public management and governance. Despite the proliferation of published work based on FEVS data, the field of public management to date lacks a critical assessment of the quality and value of how scholars have used the survey data in their research. Fernandez and colleagues (2015) provided a critical assessment of the FEVS instrument, finding weaknesses in its content, design, and implementation. The authors offered a set of recommendations for refining the survey and its implementation with the aim to improve the quality and value of the data.  Their efforts subsequently led to a proposed OPM regulation that revised many of the statutorily required questions in the survey “to stronger, relevant and unambiguous questions as well as questions that capture a single concept” (OPM, 2016). 

We extend this area of inquiry by turning the critical lens back to the public management research that has used the FEVS to this point. We provide a critical assessment of the quality and value of the scholarship employing this data for explanatory, inferential modeling, and analysis. We also explore the extent to which researchers have established reliable and valid measures of various constructs using the data; the extent of rigor used in tests of causality and/or association; the ways in which FEVS data have been merged with data from other sources; and how scholars have used this survey to influence public management research and theory (for good and bad).

To address these questions, we conduct a bibliometric analysis of all published, peer-reviewed journal articles that use FHCS/FEVS data as a central element of the quantitative inferential modeling used to answer a respective research question. We code and assess each of the articles according to the criteria of (1) item selection and scaling; (3) convergent/discriminant validity of constructs derived from FEVS data; (4) tests of scale reliability; (5) modeling approaches; (6) endogeneity issues; and (7) common source bias. Our findings provide a constructively critical assessment of public management scholarship employing the FEVS, so as to strengthen the partnership between OPM and public management scholars as they jointly pursue questions critical to effective governance.

References

Fernandez, Sergio, et al. "Assessing the Past and Promise of the Federal Employee Viewpoint Survey for Public Management Research: A Research Synthesis." Public Administration Review 75.3 (2015): 382-94.

OPM. "Personnel Management in Agencies." 5 CFR 250. Washington, DC: Office of the Federal Register, 2016. OPM-2016-0001