Association for Public Policy Analysis & Management

The Wisconsin Department of Children and Families (DCF) is responsible for overseeing the state’s county-administered child welfare system. Beginning in 2013, DCF adopted a Continuous Quality Improvement (CQI) approach to better understand the system’s performance and improve child welfare policy, practice, and outcomes through systematic data collection and analysis. An important source of child welfare performance data comes from case reviews, which are mandated by the federal government. Previously, DCF relied on a qualitative practice appraisal protocol to review caseworker practice on a small sample of child welfare cases in few counties. This approach was resource-intensive and resulted in limited information. In adopting CQI, DCF created a new case review process that collects quantifiable data from a statewide representative sample of cases large enough to detect statistically significant results.

Our team has worked closely and transparently with local agencies to develop and implement new review protocols, beginning with the design of standardized review tools to collect quantifiable data on practice/service delivery in different Child Protective Services (CPS) program areas. This involved operationalizing Wisconsin’s CPS Standards, Public Child Welfare Practice Model, and Child Welfare Safety Model into discrete data elements by CPS program area in order to create valid surveys with single-construct questions. After piloting the surveys (i.e., review tools) to ensure reliability, we created an electronic version in Microsoft Access for improved data collection and integrity. The electronic review tools contain embedded IF-THEN logic to guide reviewers and allow entries only in applicable fields. We have also created random sampling procedures for identifying a statewide representative sample of cases for review and developed standardized, online trainings for case reviewers.

At least one round of case reviews in all program areas has been completed using the new process, and to date more than 1,900 case record reviews have been completed. Part of our research design also involves a process for generating testable hypotheses with input from Child Protective Services experts from the Department and the University of Wisconsin-Madison School of Social work. After cases are reviewed and hypotheses formulated, we use statistical analysis software (SAS) to match case review and administrative data to examine relationships between case practice/service delivery and outcomes. 

These results have been compiled in published reports, communicated with county agencies and local child welfare staff, and presented at conferences throughout Wisconsin. Stakeholders widely support the new process, as it offers an objective view of the current status of child welfare practice. Most importantly, the new case review process is analytically robust and makes use of data and technology to yield actionable information for system improvement.