An algorithm that screens for child neglect raises concerns

Apr 28, 2022, 10:10 AM | Updated: Apr 29, 2022, 10:41 am
The Family Law Center in Pittsburgh is seen on Wednesday, March 16, 2022. Around the country, as ch...

The Family Law Center in Pittsburgh is seen on Wednesday, March 16, 2022. Around the country, as child welfare agencies use or consider algorithmic tools like in Allegheny County, an Associated Press review has identified a number of concerns about the technology, including questions about its reliability and its potential to harden racial disparities in the child welfare system. (AP Photo/Matt Rourke)

(AP Photo/Matt Rourke)

For family law attorney Robin Frank, defending parents at one of their lowest points — when they risk losing their children — has never been easy.

The job is never easy, but in the past she knew what she was up against when squaring off against child protective services in family court. Now, she worries she’s fighting something she can’t see: an opaque algorithm whose statistical calculations help social workers decide which families should be investigated in the first place.

“A lot of people don’t know that it’s even being used,” Frank said. “Families should have the right to have all of the information in their file.”

From Los Angeles to Colorado and throughout Oregon, as child welfare agencies use or consider tools similar to the one in Allegheny County, Pennsylvania, an Associated Press review has identified a number of concerns about the technology, including questions about its reliability and its potential to harden racial disparities in the child welfare system. Related issues have already torpedoed some jurisdictions’ plans to use predictive models, such as the tool notably dropped by the state of Illinois.

According to new research from a Carnegie Mellon University team obtained exclusively by AP, Allegheny’s algorithm in its first years of operation showed a pattern of flagging a disproportionate number of Black children for a “mandatory” neglect investigation, when compared with white children. The independent researchers, who received data from the county, also found that social workers disagreed with the risk scores the algorithm produced about one-third of the time.

County officials said that social workers can always override the tool, and called the research “hypothetical.”

Child welfare officials in Allegheny County, the cradle of Mister Rogers’ TV neighborhood and the icon’s child-centric innovations, say the cutting-edge tool – which is capturing attention around the country – uses data to support agency workers as they try to protect children from neglect. That nuanced term can include everything from inadequate housing to poor hygiene, but is a different category from physical or sexual abuse, which is investigated separately in Pennsylvania and is not subject to the algorithm.

“Workers, whoever they are, shouldn’t be asked to make, in a given year, 14, 15, 16,000 of these kinds of decisions with incredibly imperfect information,” said Erin Dalton, director of the county’s Department of Human Services and a pioneer in implementing the predictive child welfare algorithm.

____

This story, supported by the Pulitzer Center for Crisis Reporting, is part of an ongoing Associated Press series, “Tracked,” that investigates the power and consequences of decisions driven by algorithms on people’s everyday lives.

____

Critics say it gives a program powered by data mostly collected about poor people an outsized role in deciding families’ fates, and they warn against local officials’ growing reliance on artificial intelligence tools.

If the tool had acted on its own to screen in a comparable rate of calls, it would have recommended that two-thirds of Black children be investigated, compared with about half of all other children reported, according to another study published last month and co-authored by a researcher who has audited the county’s algorithm.

Advocates worry that if similar tools are used in other child welfare systems with minimal or no human intervention — akin to how algorithms have been used to make decisions in the criminal justice system — they could reinforce existing racial disparities in the child welfare system.

“It’s not decreasing the impact among Black families,” said Logan Stapleton, a researcher at Carnegie Mellon University. “On the point of accuracy and disparity, (the county is) making strong statements that I think are misleading.”

Because family court hearings are closed to the public and the records are sealed, AP wasn’t able to identify first-hand any families who the algorithm recommended be mandatorily investigated for child neglect, nor any cases that resulted in a child being sent to foster care. Families and their attorneys can never be sure of the algorithm’s role in their lives either because they aren’t allowed to know the scores.

Child welfare agencies in at least 26 states and Washington, D.C., have considered using algorithmic tools, and at least 11 have deployed them, according to American Civil Liberties Union.

Larimer County, Colorado, home to Fort Collins, is now testing a tool modeled on Allegheny’s and plans to share scores with families if it moves forward with the program.

“It’s their life and their history,” said Thad Paul, a manager with the county’s Children Youth & Family Services. “We want to minimize the power differential that comes with being involved in child welfare … we just really think it is unethical not to share the score with families.”

Oregon does not share risk score numbers from its statewide screening tool, which was first implemented in 2018 and was inspired by Allegheny’s algorithm. The Oregon Department of Human Services – currently preparing to hire its eighth new child welfare director in six years – explored at least four other algorithms while the agency was under scrutiny by a crisis oversight board ordered by the governor.

It recently paused a pilot algorithm built to help decide when foster care children can be reunified with their families. Oregon also explored three other tools – predictive models to assess a child’s risk for death and severe injury, whether children should be placed in foster care and if so, where.

For years, California explored data-driven approaches to the statewide child welfare system before abandoning a proposal to use a predictive risk modeling tool in 2019.

“During the project, the state also explored concerns about how the tool may impact racial equity. These findings resulted in the state ceasing exploration,” department spokesman Scott Murray said in an email.

Los Angeles County’s Department of Children and Family Services is being audited following high-profile child deaths, and is seeking a new director after its previous one stepped down late last year. It’s piloting a “complex-risk algorithm” that helps to isolate the highest-risk cases that are being investigated, the county said.

In the first few months that social workers in the Mojave Desert city of Lancaster started using the tool, however, county data shows that Black children were the subject of nearly half of all the investigations flagged for additional scrutiny, despite making up 22% of the city’s child population, according to the U.S. Census.

The county did not immediately say why, but said it will decide whether to expand the tool later this year.

___

Associated Press reporter Camille Fassett contributed to this report.

___

Follow Sally Ho and Garance Burke on Twitter at @_sallyho and @garanceburke.

___

Contact AP’s global investigative team at [email protected] or https://www.ap.org/tips/

Copyright © The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.

AP

(AP Photo/Gerry Broome, File)...
Associated Press

Fish and Wildlife shoots wrong wolf, more attacks confirmed

Wolves from two packs in northeast Washington state have attacked more cattle
12 hours ago
Snowbird William Wellema talks about his frustration at not being allowed on to the island to surve...
Associated Press

As search goes on, Floridians await OK to survey Ian’s wrath

FORT MYERS BEACH, Fla. (AP) — William Wellema has been living under a bridge for four days, waiting to get to Fort Myers Beach on Florida’s Estero Island to see if his vacation home survived Hurricane Ian. On Friday, he was beyond frustrated as he continued to await permission to drive across. “They’re saying it’s […]
1 day ago
FILE - Nobel Commitee chairman Thorbjorn Jagland sits next to an empty chair with the Nobel Peace P...
Associated Press

Bialiatski joins small group of jailed Nobel Peace laureates

STOCKHOLM (AP) — Belarussian pro-democracy campaigner Ales Bialiatski, who shared the Nobel Peace Prize with human rights groups in Russia and Ukraine, is the fourth person in the 121-year history of the Nobel Prizes to receive the award while in prison or detention. Bialiatski, 60, who founded the non-governmental organization Human Rights Center Viasna, was […]
1 day ago
Associated Press

Judge eyes summer trial in Kesha’s legal clash with Dr. Luke

NEW YORK (AP) — After almost a decade, the legal fight between pop singer Kesha and producer Dr. Luke is headed to trial in New York next summer, a judge said Friday. Manhattan Judge Jennifer Schecter said jury selection could start in either late June or early July, asking the two sides to choose and […]
1 day ago
Associated Press

Mexican drug lord claims attack was aimed at him

MEXICO CITY (AP) — In a video posted on social media, a man claiming to be a Mexican drug cartel leader says an attack that killed 20 people including a mayor was in fact aimed at him. Authorities in Mexico said Friday they are still investigating the authenticity of a video attributed to José Alfredo […]
1 day ago
FILE — Police walk near Robb Elementary School following a shooting, May 24, 2022, in Uvalde, Tex...
Associated Press

Uvalde schools suspend entire police force after outrage

AUSTIN, Texas (AP) — The Uvalde school district on Friday pulled its embattled campus police force off the job following a wave of new outrage over the hiring of a former Texas state trooper who was part of the hesitant law enforcement response during the May shooting at Robb Elementary School. School leaders also put […]
1 day ago

Sponsored Articles

Anacortes Christmas Tree...

Come one, come all! Food, Drink, and Coastal Christmas – Anacortes has it all!

Come celebrate Anacortes’ 11th annual Bier on the Pier! Bier on the Pier takes place on October 7th and 8th and features local ciders, food trucks and live music - not to mention the beautiful views of the Guemes Channel and backdrop of downtown Anacortes.
Swedish Cyberknife Treatment...

The revolutionary treatment of Swedish CyberKnife provides better quality of life for majority of patients

There are a wide variety of treatments options available for men with prostate cancer. One of the most technologically advanced treatment options in the Pacific Northwest is Stereotactic Body Radiation Therapy using the CyberKnife platform at Swedish Medical Center.
Work at Zum Services...

Seattle Public Schools announces three-year contract with Zum

Seattle Public Schools just announced a three-year contract with a brand-new company to the Pacific Northwest to assist with their student transportation: Zum.
Swedish Cyberknife 900x506...

June is Men’s Health Month: Here’s Why It’s Important To Speak About Your Health

According to the Centers for Disease Control and Prevention, men in the United States, on average, die five years earlier than women.
...

Anacortes – A Must Visit Summertime Destination

While Anacortes is certainly on the way to the San Juan Islands (SJI), it is not just a destination to get to the ferry… Anacortes is a destination in and of itself!
...

Ready for your 2022 Alaskan Adventure with Celebrity Cruises?

Celebrity Cruises SPONSORED — A round-trip Alaska cruise from Seattle is an amazing treat for you and a loved one. Not only are you able to see and explore some of the most incredible and visually appealing natural sights on the planet, but you’re also able to relax and re-energize while aboard a luxury cruise […]
An algorithm that screens for child neglect raises concerns