CSAM – Excited Pixels

While some emerging generative technologies may positively affect various domains (e.g., certain aspects of drug discovery and biological research, efficient translation between certain languages, speeding up certain administrative tasks, etc) they are, also, enabling new forms of harmful activities. Case in point, some individuals and groups are using generative technologies to generate child sexual abuse or exploitation materials:

Sexton says criminals are using older versions of AI models and fine-tuning them to create illegal material of children. This involves feeding a model existing abuse images or photos of people’s faces, allowing the AI to create images of specific individuals. “We’re seeing fine-tuned models which create new imagery of existing victims,” Sexton says. Perpetrators are “exchanging hundreds of new images of existing victims” and making requests about individuals, he says. Some threads on dark web forums share sets of faces of victims, the research says, and one thread was called: “Photo Resources for AI and Deepfaking Specific Girls.”

…

… realism also presents potential problems for investigators who spend hours trawling through abuse images to classify them and help identify victims. Analysts at the IWF, according to the organization’s new report, say the quality has improved quickly—although there are still some simple signs that images may not be real, such as extra fingers or incorrect lighting. “I am also concerned that future images may be of such good quality that we won’t even notice,” says one unnamed analyst quoted in the report.

The ability to produce generative child abuse content is becoming a wicked problem with few (if any) “good” solutions. It will be imperative for policy professionals to learn from past situations where technologies were found to sometimes facilitate child abuse related harms. In doing so, these professionals will need to draw lessons concerning what kinds of responses demonstrate necessity and proportionality with respect to the emergent harms of the day.

As just one example, we will have to carefully consider how generative AI-created child sexual abuse content is similar to, and distinctive from, past policy debates on the policing of online child sexual abuse content. Such care in developing policy responses will be needed to address these harms and to avoid undertaking performative actions that do little to address the underlying issues that drive this kind of behaviour.

Relatedly, we must also beware the promise that past (ineffective) solutions will somehow address the newest wicked problem. Novel solutions that are custom built to generative systems may be needed, and these solutions must simultaneously protect our privacy, Charter, and human rights while mitigating harms. Doing anything less will, at best, “merely” exchange one class of emergent harms for others.

In early 2021, John Buckley, Malia Andrus, and Chris Williams published an article entitled, “Understanding the intentions of Child Sexual Abuse Material (CSAM) sharers” on Meta’s research website. They relied on information that Facebook/Meta had submitted to NCMEC to better understand why individuals they reported had likely shared illegal content.

The issue of CSAM on Facebook’s networks has risen in prominence following a report in 2019 in the New York Times. That piece indicated that Facebook was responsible for reporting the vast majority of the 45 million online photos and videos of children being sexually abused online. Ever since, Facebook has sought to contextualize the information it discloses to NCMEC and explain the efforts it has put in place to prevent CSAM from appearing on its services.

So what was the key finding from the research?

We evaluated 150 accounts that we reported to NCMEC for uploading CSAM in July and August of 2020 and January 2021, and we estimate that more than 75% of these did not exhibit malicious intent (i.e. did not intend to harm a child), but appeared to share for other reasons, such as outrage or poor humor. While this study represents our best understanding, these findings should not be considered a precise measure of the child safety ecosystem.

This finding is significant, as it quickly becomes suggestive that the mass majority of the content reported by Facebook—while illegal!—is not deliberately being shared for malicious purposes. Even if we assume that the number sampled should be adjusted—perhaps only 50% of individuals were malicious—we are still left with a significant finding.

There are, of course, limitations to the research. First, it excludes all end-to-end encrypted messages. So there is some volume of content that cannot be detected using these methods. Second, it remains unclear how scientifically robust it was to choose the selected 150 accounts for analysis. Third, and related, there is a subsequent question of whether the selected accounts are necessarily representative of the broader pool of accounts that are associated with distributing CSAM.

Nevertheless, this seeming sleeper-research hit has significant implications insofar as it would compress the number of problematic accounts/individuals disclosing CSAM to other parties. Clearly more work along this line is required, ideally across Internet platforms, in order to add further context and details to the extent of the CSAM problem and subsequently define what policy solutions are necessary and proportionate.