Association for Public Policy Analysis & Management

In 2014, JBS International, Inc. and its partner, the Center for the Support of Families, developed and delivered the Child Welfare Continuous Quality Improvement (CQI) Training Academy (Academy), a seven-unit training program for child welfare managers who are responsible for performing and strengthening CQI activities and processes in State or local public child welfare agencies. The purpose of the effort, funded by the Children's Bureau (CB), Administration for Children and Families, U.S. Department of Health and Human Services, was two-fold: 1) to support child welfare professionals across the 50 United States, the District of Columbia, and Puerto Rico through a distance learning program that would assist them in performing and improving CQI activities and processes; and 2) to develop and refine a replicable, virtual, group-based learning model for dissemination throughout the child welfare field.

To reach these aims, the Academy used training and coaching by CQI experts to increase knowledge and skills in core competencies needed to equip participating State child welfare managers to facilitate a systematic problem-solving cycle, foster cultures of continuous learning and improvement, and identify and implement sustainable changes in their agencies. The content of the training and coaching was grounded in a broad view of CQI as an essential component of the agencies’ policy-making processes with the role of identifying, implementing, monitoring and adjusting solutions to identified problems in the States’ child welfare programs. Participants represented middle managers with a variety of functions within their agencies, as opposed to a more traditional set of CQI monitoring and evaluation staff. Simultaneously, the Academy was rigorously evaluated, using a mixed method, pre-post test design with random assignment and a comparison group, to answer key process and outcome questions related to satisfaction, accessibility, usability, participation, reach, use, and acquisition of competencies and skills. In addition, the evaluation helped the Academy to clearly describe and specify key aspects of a replicable group-based learning model, monitor adherence to the model during program delivery, and assess the model’s ability to affect proximal learning and behavior-related outcomes.

We propose a discussion of what was learned through the Academy and the implications of those lessons for building CQI capacity and roles in child welfare, as well as other human service systems. Areas of focus include: 1) the relationship between prior participant knowledge and experience and learning outcomes and implications for designing a CQI training program that contributes strongly to the agencies’ broader policy-making process; 2) the efficacy of entirely self-directed learning as compared to blended learning—combining self-directed and group-based activities—in achieving participant learning outcomes; 3) the factors that keep participants motivated and engaged in virtual learning about CQI; 4) evidence from prior studies of self-directed, web-based and virtual group-based learning that was corroborated through the Academy’s evaluation; and 5) the content of CQI training that participants found most accessible, most challenging, and most applicable to their roles within the child welfare agency.