Saving children, one algorithm at a time

The Oklahoma Department of Human Services (DHS) is the latest government entity to take advantage of predictive analytics from non-profit Eckerd Kids to prevent child welfare tragedies.

The pilot program, which uses Eckerd Rapid Safety Feedback (ERSF) software, analyzes child welfare data and flags high-risk cases. Eckerd partnered with Tampa, Fla.,-based Mindshare Technology, which provides the predictive analytics and machine learning to identify the cases most likely to lead to child fatalities.

By analyzing real-time data pulled from disparate sources on cases that had produced poor safety outcomes for children, Eckerd and Mindshare Technology found 15 data points -- such as a child under the age of three, a lover in the home, intergenerational abuse, young parents, mental health problems and a history of substance abuse -- that were highly correlated with poor outcomes.

Using those data points, Eckerd began electronically scanning all of its cases to flag those that matched a number of the risk factors and give them a more thorough review.

Often the review leads to an immediate meeting between quality management specialists and the case management team for the family. Front-line staff receive best practices coaching sessions where deficits were identified and actions needed are tracked for accountability.

Unlike traditional quality assurance programs, cases are chosen based on specific children at greatest risk of severe maltreatment.

“Instead of randomly pulling cases to look at, we’re pulling the cases we need to look at,” Eckerd COO Ron Zychowski told Alliance for Children and Families Magazine.

“Eckerd’s ERSF tool has shown some promising results in Florida preventing tragedies in open child welfare cases as well as improving the quality of casework,” DHS Director Ed Lake told the Daily Ardmoreite. “While the results from the tool are neither foolproof nor take the place of considered judgments of front-line DHS staff and other professionals, we believe this software will add an extra layer of protection for the children we serve by helping our workers focus special attention on those families with high risk factors.”

The software is currently being piloted in Oklahoma County with hopes to expand its use statewide. Tom Ward and White Fields Inc., a home for boys, together with Eckerd Kids, provided funding for the project.

Besides Oklahoma, Eckerd Rapid Safety Feedback is being deployed in Maine, Connecticut, Alaska and Illinois.

In May, Bryan Lindert, senior quality director at Eckerd Kids, testified before Congress on the link between substance and child abuse and how data can be used to minimize harm to children. 

Lindert cited results from the organization's work with Hillsborough County, Fla., which began in July 2012. “In Hillsborough, there were no maltreatment fatalities in the three-year period following implementation of the program in the population served by Eckerd. Critical case practices also improved an average of 22 percent,” he said.

According to Eckerd, using the software has led to improvements in information sharing, supervisory reviews, follow ups by case managers, safety planning, quality of case management contacts and discussions with families.

Lindert named several success factors for a jurisdiction looking to employ data analytics for improving child welfare that include daily access to the State Automated Child Welfare Information System (SACWIS) and access to quality assurance reviews assessing case practice.

Greg Povolny, founder and CEO of Mindshare, cited in the appendix of Lindert’s testimony, added a another: mining data daily. “Predictive analytics is not a one‐time job. The intention is to zero in on children for the long haul,” he said.

“States are sitting on valuable mountains of data about the families that come to their attention that could be used to predict the children who may experience poor outcomes beyond child fatality,” Lindert said. “We are not advocating decisions made by machines. We are advocating that data and coaching together provide a support for those men and women working with families to help them focus attention where it is needed most.”

According to Lindert, the startup cost for a jurisdiction to set up such a system is approximately $200,000, with about $90,000 in yearly fees to support the portal maintenance and ongoing activities.

Florida has also worked with data analytics provider SAS of Cary, N.C., to determine key risk factors for child fatalities. Florida’s Department of Children and Families released a report in 2014 analyzing child fatalities reported to the Florida Abuse Hotline from Jan. 1, 2007 to June 30, 2013. Similar to the Eckerd-Mindshare project, the goal was to improve child welfare practices by using predictive analytics tools and techniques to confirm general trends in child fatalities and to determine key risk factors.

Not everyone supports the use of analytics for improving child welfare, however.

“We should not make more of the math than it really is -- just a very good tool for providing information, if you know how to use it well,” Meta S. Brown, a data analytics specialist wrote in a column in Forbes. “Predictions are often wrong, even when they are made using excellent data and excellent mathematical technique,” she said.

“There is no real evidence that the improvements [in Hillsborough County] had anything to do with predictive analytics,” Richard Wexler, executive director of National Coalition for Child Protection Reform, said in a recent report. “Correlation is not causation.”

Wexler cited several reasons for his conclusion:

• The subjective nature of determining whether a child’s death is due to maltreatment.
• Florida’s changing state policy and guidance concerning what kinds of deaths to label as maltreatment, which make it hard to do an apples-to-apples comparison.
• The hiring of several additional caseworkers, which could have caused the improved numbers.

Wexler is particularly concerned about “false positives” -- child welfare agencies classifying and treating too many cases as high risk. False positives increase expenses with added investigations, divert staff from cases of real need and place children in foster homes unnecessarily, which may increase, rather than decrease, abuse in the home.

Instead, Wexler argues for random sampling -- currently done under Child and Family Services Reviews of state child welfare systems -- but with a larger sample size. A multidisciplinary team taking an in-depth look at these cases should examine both cases where the children were left at home and cases where the children were placed in foster care -- everything that went wrong and right in each case.