Association for Public Policy Analysis & Management

In partnership with the Broward Sheriff’s Office (BSO), Childnet and the Children’s Services Council of Broward County (CSC) we developed predictive and prescriptive models for cases entering the child welfare system. The overall goal was to determine whether the effectiveness of child welfare decision making, particularly at the child protective services level, could be enhanced by applying machine learning to administrative data in order to improve outcomes for children and families and for the system as a whole.

The data for this study consisted of administrative child welfare records from the three agencies listed above for the years beginning in 2010 and ending in 2015, totaling 57,358 cases. Machine learning algorithms were applied to develop accurate predictive models determining if cases would return to the system, as well as machine-guided propensity score matching models to rigorously build prescriptive models that could be deployed to provide tailored recommendations on a case-by-case basis.

Twenty-five percent of the cases reported to BSO during the study period returned to the system as a result of a new report of abuse or neglect. By withholding 30% of the cases from the modeling, and then “testing” the models on these unseen cases, the machine learning predictive and prescriptive models would have reduced that rate of return to the system by 30% if the recommended decisions were followed. This is a significant improvement in the current child welfare decision-making models, and if applied through technological deployment (i.e., a software “scoring and reporting engine” that could sit alongside the administrative dataset) they would benefit the system as follows:

• Improved capability for verifying and/or substantiating reports of abuse and neglect.

• More accuracy in differentiating and classifying low, moderate and high risk cases.

•  Recognizing that child protective service investigators appear to have differential investigative skills and assigning cases to them accordingly.

• Using prescriptive analytics to improve matching between the needs of families and children and available services.

• Being able to identify very low risk cases needing little or no services, estimated to be as high as 30% of all referrals to child protective services.

• Making more appropriate and impactful referrals to Childnet.

• Making more appropriate and impactful referrals to CSC-funded programs.

• Recognizing that low risk and moderate risk cases referred for services that are not needed increases the odds that these families will return to the system with a subsequent child protective service report. In other words, providing services to families where the need for such services is not required appears to have adverse consequences for many cases.

• Recognizing that a significant proportion (40%) of the child welfare cases currently being referred to the courts do not warrant such a referral.

The paper discusses these findings in greater detail. The important point to be made, however, is that the machine learning and predictive analytics models that were developed suggest that the rate of return to the child welfare system in Broward County could be reduced by as much as 28%.