Report urges fixes to online child exploitation CyberTipline before AI makes it worse

A tipline set up 26 years ago to combat online child exploitation has not lived up to its potential and needs technological and other improvements to help law enforcement go after abusers and rescue victims, a new report from the Stanford Internet Observatory has found.

The fixes to what the researchers describe as an “enormously valuable” service must also come urgently as new artificial intelligence technology threatens to worsen its problems.

“The CyberTipline will just be flooded with highly realistic-looking AI content, which is going to make it even harder for law enforcement to identify real children who need to be rescued,” said researcher Shelby Grossman, an author of the report.

The service was established by Congress as the main line of defense for children who are exploited online. By law, tech companies must report any child sexual abuse material they find on their platforms to the system, which is operated by the National Center for Missing and Exploited Children. After it receives the reports, NCMEC attempts to find the people who sent or received the material — as well as the victims, if possible. These reports are then sent to law enforcement.

While the sheer amount of CyberTipline reports is overwhelming law enforcement, researchers say volume is just one of several problems core to the system. For instance, many of the reports sent by tech companies — like Google, Amazon, and Meta — lack important details, such as enough information about an offender’s identity, the report said. This makes it hard for law enforcement to know which reports to prioritize.

“There are significant issues with the entire system right now and those cracks are going to become chasms in a world in which AI is generating brand new CSAM,” said Alex Stamos, using the initials for child sexual abuse materials. Stamos is a Stanford lecturer and cybersecurity expert.

In addition, the system is behind technologically and plagued by a constant challenge among government and nonprofit tech platforms: the lack of highly skilled engineers, who can get paid far higher salaries in the tech industry. Sometimes those employees are even poached by the same companies that send in the reports.

Then there are the legal constraints. According to the report, court decisions have led the staff at NCMEC to stop vetting some files (for instance, if they are not publicly available) before sending them to law enforcement. Many law enforcement officials believe they need a search warrant to access such images, slowing down the entire process. At times, multiple warrants or subpoenas are needed to identify the same offender.

It’s also easy for the system to get distracted. The report reveals that NCMEC recently hit a milestone of a million reports in a single day due to a viral meme that was spreading on multiple platforms — which some people thought was funny and others were sharing out of outrage.

“That day actually led them to make some changes,” Stamos said. “It took them weeks to get through that backlog” by making it easier to cluster those images together in certain ways.

The CyberTipline received more than 36 million reports in 2023, nearly all of them from online platforms. Facebook, Instagram and Google were the companies that sent in the highest number of reports. The overall number of reports has been dramatically increasing, according to the study.

Nearly half of the tips sent last year were actionable, meaning NCMEC and law enforcement could follow up on it. The rest either didn’t have enough information or because the image or video reported had been reported many times before. Others can be classified as a meme that’s spread “without malicious intent.”

Hundreds of reports concerned the same offender, and many of the reports included multiple images or videos. Around 92% of the reports filed in 2023 involved countries outside the U.S., a large shift from 2008 when the majority involved victims or offenders inside the U.S.

Some are false alarms. “It drives law enforcement nuts when they get these reports that they perceive are definitely adults,” Grossman said. “But the system incentivizes platforms to be very conservative or to report potentially borderline content, because if it’s found to have been CSAM and they knew about it and they didn’t report it, they could receive fines.”

One relatively easy fix proposed in the report would improve how tech platforms label what they are reporting to distinguish between widely shared memes and something that deserves closer investigation.

The Stanford researchers interviewed 66 people involved with the CyberTipLine, ranging from law enforcement to NCMEC staff to online platform employees. Many said they’ve been raising concerns about the system for years.

The NCMEC said it looked forward to “exploring the recommendations internally and with key stakeholders.”

“Over the years, the complexity of reports and the severity of the crimes against children continue to evolve. Therefore, leveraging emerging technological solutions into the entire CyberTipline process leads to more children being safeguarded and offenders being held accountable,” it said in a statement.

Among the report’s other findings:

— The CyberTipline reporting form doesn’t have a dedicated field for submitting chat-related material, such as sextortion messaging. The FBI recently warned of a “huge increase” in sextortion cases targeting children — including financial sextortion, where someone threatens to release compromising images unless the victim pays. The victims are primarily boys between the ages of 14 to 17.

— Police detectives told Stanford researchers they are also having a hard time persuading their higher-ups to prioritize these crimes. Even when they write up detailed descriptions of the images to emphasize their gravity, “their higher-ups often won’t even want to read their description,” Grossman said. “They wince when they read it and they don’t really want to think about this.”

— Many law enforcement officials said they were not able to fully investigate all reports due to time and resource constraints. For instance, a single detective may be responsible for 2,000 reports a year.

— Outside the U.S., especially in poorer countries, the challenges around child exploitation reports are especially severe. For example, law enforcement agencies might not have reliable internet connections, “decent computers” or even gas for cars to execute search warrants.

— Pending legislation passed by the U.S. Senate in December would require online platforms to report child sex trafficking and online enticement to the CyberTipline and give law enforcement more time to investigate child sexual exploitation. Currently, the tipline doesn’t offer clear, straightforward ways to report suspected sex trafficking.

While some advocates have proposed more intrusive surveillance laws to catch abusers, Stamos, the former chief security officer at Facebook, said they should try simpler fixes first.

“There’s no need to violate the privacy of users if you want to put more pedophiles in jail. They’re sitting right there,” Stamos said. “The system does not work very well at taking the information that currently exists and then turning it into prosecutions.”