Tag: Policy

Categories
Writing

Establishing Confidence Ratings in Policy Assessments

Few government policy analysts are trained in assessing evidence in a structured way and then assigning confidence ratings when providing formal advice to decision makers. This lack of training can be particularly problematic when certain language (“likely,” “probable,” “believe,” etc) is used in an unstructured way because these words can lead decision makers to conclude that recommended actions have a heft of evidence and assessment that, in fact, may not be present in the assessment.

Put simply: we don’t train people to clinically assess the evidence provided to them in a rigorous and structured way, and upon which they make analyses. This has the effect of potentially driving certain decisions that otherwise might not be made.

The government analysts who do have this training tend to come from the intelligence community, which has spend decades (if not centuries) attempting to divine how reliable or confident assessments are because the sources of their data are often partial or questionable.

I have to wonder just what can be done to address this kind of training gap. It doesn’t make sense to send all policy analysts to an intelligence training camp because the needs are not the same. But there should be some kind of training that’s widely and commonly available.

Robert Lee, who works in private practice these days but was formerly in intelligence, set out some high-level framings for how private threat intelligence companies might delineate between different confidence ratings in a blog he posted a few years ago. His categories (and descriptions) were:

Low Confidence: A hypothesis that is supported with available information. The information is likely single sourced and there are known collection/information gaps. However, this is a good assessment that is supported. It may not be finished intelligence though and may not be appropriate to be the only factor in making a decision.

Moderate Confidence: A hypothesis that is supported with multiple pieces of available information and collection gaps are significantly reduced. The information may still be single sourced but there’s multiple pieces of data or information supporting this hypothesis. We have accounted for the collection/information gaps even if we haven’t been able to address all of them.

High Confidence: A hypothesis is supported by a predominant amount of the available data and information, it is supported through multiple sources, and the risk of collection gaps are all but eliminated. High confidence assessments are almost never single sourced. There will likely always be a collection gap even if we do not know what it is but we have accounted for everything possible and reduced the risk of that collection gap; i.e. even if we cannot get collection/information in a certain area it’s all but certain to not change the outcome of the assessment.

While this kind of categorization helps to clarify intelligence products I’m less certain how effective it is when it comes to more general policy advice. In these situations assessments of likely behaviours may be predicated on ‘softer’ sources of data such as a policy actor’s past behaviours. The result is that predictions may sometimes be based less on specific and novel data points and, instead, on a broader psychographic or historical understanding of how an actor is likely to behave in certain situations and conditions.

Example from Kent’s Words of Estimative Probability

Lee, also, provided the estimation probability that was developed in the early 1980s for CIA assessments. And I think that I like the Kent Word approach more if only because it provides a broader kind of language around “why” a given assessment is more or less accurate.

While I understand and appreciate that threat intelligence companies are often working with specific datapoints and this is what can lead to analytic determinations, most policy work is much softer than this and consequently doesn’t (to me) clearly align with the more robust efforts to achieve confidence ratings that we see today. Nevertheless, some kind of more robust approach to providing recommendations to decision makers is needed so that executives have a strong sense of an analyst’s confidence in any recommendation, and especially when there may be competing policy options at play. While intuition drives a considerable amount of policy work at least a little more formalized structure and analysis would almost certainly benefit public policy decision making processes.

Categories
Writing

Quick Thoughts on Academics and Policy Impact

I regularly speak with scholars who complain policy makers don’t read their work. 95% of the time that work is either published in books costing hundreds of dollars (in excess of department budgets) or behind a journal paywall that departments lack access to.1

Bluntly, it’s hard to have impact if your work is behind paywalls.

Moreover, in an era of ‘evidence-based policymaking’ dedicated public servants will regularly want to assess some of the references or underlying data in the work in question. They perform due diligence when they read facts, arguments, or policy recommendations.

However, the very work that a scholar is using to develop their arguments or recommendations may, also, lay behind paywalls. Purchasing access to the underlying books and papers that go into writing a paper could run a public servant, or their department, even more hundreds or thousands of dollars. Frankly they’re not likely to spend that amount of money and it’d often be irresponsible for them to do so.

So what are the effect of all these paywalls? Even if the government policymaker can get access to the scholar’s paper they cannot fact-check or assess how it was built. It is thus hard for them to validate conclusions and policy recommendations. This, in turn, means that committed public servants may put important scholarly research into an ‘interesting but not sufficiently evidence-based’ bucket.

Does this mean that academics shouldn’t publish in paywalled journals or books? No, because they have lots of audiences, and publications are the coin of the academic realm. But it does mean that academics who want to have near- or middle-term impacts need to do the work and make their findings, conclusions, and recommendations publicly available.

What to do, then?

Broadly, it is helpful to prepare and publish summaries of research to open-source and public-available outlets. The targets for this are, often, think tanks or venues that let academics write long-form pieces (think maximum of 1,200-1,500 words). Alternately, scholars can just start and maintain a blog and host summaries of their ideas, there, along with an offer to share papers that folks in government might be interested in but to which they lack access.

I can say with some degree of authority from my time in academia that publishing publicly-available reports, or summarising paywalled work, can do a great deal to move the needle in how government policies are developed. But, at the same time, moving that needle requires spending the time and effort. You should not just expect busy government employees to randomly come across your paywalled article, buy it, read it, and take your policy recommendations seriously.

  1. Few government departments have extensive access to academic journals. Indeed, even working at one of the top universities at the world and having access to a wealth of journals, I regularly came across articles that I couldn’t access! ↩︎
Categories
Links Writing

Generative AI Technologies and Emerging Wicked Policy Problems

While some emerging generative technologies may positively affect various domains (e.g., certain aspects of drug discovery and biological research, efficient translation between certain languages, speeding up certain administrative tasks, etc) they are, also, enabling new forms of harmful activities. Case in point, some individuals and groups are using generative technologies to generate child sexual abuse or exploitation materials:

Sexton says criminals are using older versions of AI models and fine-tuning them to create illegal material of children. This involves feeding a model existing abuse images or photos of people’s faces, allowing the AI to create images of specific individuals. “We’re seeing fine-tuned models which create new imagery of existing victims,” Sexton says. Perpetrators are “exchanging hundreds of new images of existing victims” and making requests about individuals, he says. Some threads on dark web forums share sets of faces of victims, the research says, and one thread was called: “Photo Resources for AI and Deepfaking Specific Girls.”

…

… realism also presents potential problems for investigators who spend hours trawling through abuse images to classify them and help identify victims. Analysts at the IWF, according to the organization’s new report, say the quality has improved quickly—although there are still some simple signs that images may not be real, such as extra fingers or incorrect lighting. “I am also concerned that future images may be of such good quality that we won’t even notice,” says one unnamed analyst quoted in the report.

The ability to produce generative child abuse content is becoming a wicked problem with few (if any) “good” solutions. It will be imperative for policy professionals to learn from past situations where technologies were found to sometimes facilitate child abuse related harms. In doing so, these professionals will need to draw lessons concerning what kinds of responses demonstrate necessity and proportionality with respect to the emergent harms of the day.

As just one example, we will have to carefully consider how generative AI-created child sexual abuse content is similar to, and distinctive from, past policy debates on the policing of online child sexual abuse content. Such care in developing policy responses will be needed to address these harms and to avoid undertaking performative actions that do little to address the underlying issues that drive this kind of behaviour.

Relatedly, we must also beware the promise that past (ineffective) solutions will somehow address the newest wicked problem. Novel solutions that are custom built to generative systems may be needed, and these solutions must simultaneously protect our privacy, Charter, and human rights while mitigating harms. Doing anything less will, at best, “merely” exchange one class of emergent harms for others.

Categories
Aside Links

Highlights from TBS’ Guidance on Publicly Available Information

The Treasury Board Secretariat has released, “Privacy Implementation Notice 2023-03: Guidance pertaining to the collection, use, retention and disclosure of personal information that is publicly available online.”

This is an important document, insofar as it clarifies a legal grey space in Canadian federal government policies. Some of the Notice’s highlights include:

  1. Clarifies (some may assert expand) how government agencies can collect, use, retain, or disclose publicly available online information (PAOI). This includes from commercial data brokers or online social networking services
  2. PAOI can be collected for administrative or non-administrative purposes, including for communications and outreach, research purposes, or facilitating law enforcement or intelligence operations
  3. Overcollection is an acknowledged problem that organizations should address. Notably, “[a]s a general rule, [PAOI] disclosed online by inadvertence, leak, hack or theft should not be considered [PAOI] as the disclosure, by its very nature, would have occurred without the knowledge or consent of the individual to whom the personal information pertains; thereby intruding upon a reasonable expectation of privacy.”
  4. Notice of collection should be undertaken, though this may not occur due to some investigations or uses of PAOI
  5. Third-parties collecting PAOI on the behalf of organizations should be assessed. Organizations should ensure PAOI is being legitimately and legally obtained
  6. “[I]nstitutions can no longer, without the consent of the individual to whom the information relates, use the [PAOI] except for the purpose for which the information was originally obtained or for a use consistent with that purpose”
  7. Organizations are encouraged to assess their confidence in PAOI’s accuracy and potentially evaluate collected information against several data sources to confidence
  8. Combinations of PAOI can be used to create an expanded profile that may amplify the privacy equities associated with the PAOI or profile
  9. Retained PAOI should be denoted with “publicly available information” to assist individuals in determining whether it is useful for an initial, or continuing, use or disclosure
  10. Government legal officers should be consulted prior to organizations collecting PAOI from websites or services that explicitly bar either data scraping or governments obtaining information from them
  11. There are number pieces of advice concerning the privacy protections that should be applied to PAOI. These include: ensuring there is authorization to collect PAOI, assessing the privacy implications of the collection, adopting privacy preserving techniques (e.g., de-identification or data minimization), adopting internal policies, as well as advice around using attributable versus non-attributable accounts to obtain publicly available information
  12. Organizations should not use profile information from real persons. Doing otherwise runs the risk of an organization violating s. 366 (Forgery) or s.403 (Fraudulently impersonate another person) of the Criminal Code
Categories
Links

Deskilling and Human-in-the-Loop

Photo by Digital Buggu on Pexels.com

I found boyd’s “Deskilling on the Job” to be a useful framing for how to be broadly concerned, or at least thoughtful, about using emerging A.I. technologies in professional as well as training environments.

Most technologies serve to augment human activity. In sensitive situations we often already require a human-in-the-loop to respond to dangerous errors (see: dam operators, nuclear power staff, etc). However, should emerging A.I. systems’ risks be mitigated by also placing humans-in-the-loop then it behooves policymakers to ask: how well does this actually work when we thrust humans into correcting often highly complicated issues moments before a disaster?

Not to spoil things, but it often goes poorly, and we then blame the humans in the loop instead of the technical design of the system.1

AI technologies offer an amazing bevy of possibilities. But thinking more carefully on how to integrate them into society while, also, digging into history and scholarly writing in automation will almost certainly help us avoid obvious, if recurring, errors in how policy makers think about adding guardrails around AI systems.

  1. If this idea of humans-in-the-loop and the regularity of errors in automated systems interests you, I’d highly encourage you to get a copy of ‘Normal Accidents’ by Perrow. ↩︎
Categories
Links Writing

Doing A Policy-Oriented PhD

Steve Saideman has a good, short, thought on why doing a PhD is rarely a good idea for Canadians who want to get into policy work. Specifically, he writes:

In Canada, alas, there is not that much of a market for policy-oriented PhDs. We don’t have much in the way of think tanks, there are only a few govt jobs that either require PhDs or where the PhD gives one an advantage over an MA, and, the govt does not pay someone more if they have a PhD.

I concur that there are few places, including think tanks or civil society organizations, where you’re likely to find a job if you have a policy-related PhD. Moreover, when you do find one it can be challenging, if not impossible, to find promotion opportunities because the organizations tend to be so small.

That said, I do in fact think that doing a policy-related PhD can sometimes be helpful if you stay pretty applied in your outputs while pursuing your degree. In my case, I spent a lot of time during my PhD on many of the same topics that I still focus on, today, and can command a premium in consulting rates and seniority for other positions because I’ve been doing applied policy work for about 15 years now, inclusive of my time in my PhD. I, also, developed a lot of skills in my PhD—and in particular the ability to ask and assess good questions, know how questions or policy issues had been previously answered and to what effect, and a reflexive or historical thinking capacity I lacked previously—that are all helpful soft skills in actually doing policy work. Moreover, being able to study policy and politics, and basically act as an independent agent for the time of my PhD, meant I had a much better sense of what I thought about issues, why, and how to see them put into practice than I would have gained with just a master’s degree.

Does that mean I’d recommend doing a PhD? Well…no. There are huge opportunity costs you incur in doing them and, also, you can narrow you job market searches by appearing both over-educated and under-qualified. The benefits of holding a PhD tend to become more apparent after a few years in a job as opposed to being helpful in netting that first one out of school.

I don’t regret doing a PhD but, if someone is particularly committed to doing one, I think that they should hurl themselves into it with absolute abandon and treat it as a super-intensive 40-65 hour/week job, and be damn sure that you have a lot of non-academic outputs to prove to a future employer that you understand the world and not just academic journals. It’s hard work, which is sometimes rewarding, and there are arguably different (and less unpleasant) ways of getting to a relatively similar end point. But if someone is so motivated by a hard question that they’d be doing the research and thinking about it, regardless of whether they were in a PhD program? Then they might as well go and get the piece of paper while figuring out the answer.

Categories
Writing

Why Is(n’t) TikTok A National Security Risk?

Photo by Ron Lach on Pexels.com

There have been grumblings about TikTok being a national security risk for many years and they’re getting louder with each passing month. Indeed, in the United States a bill has been presented to ban TikTok (“The ANTI-SOCIAL CCP ACT“) and a separate bill (“No TikTok on Government Devices Act“) has passed the Senate and would bar the application from being used on government devices. In Canada, the Prime Minister noted that the country’s signals intelligence agency, the Communications Security Establishment, is “watching very carefully.”

I recently provided commentary where I outlined some of the potential risks associated with TikTok and where it likely should fit into Canada’s national security priorities (spoiler: probably pretty low). Here I just want to expand on my comments a bit to provide some deeper context and reflections.

As with all things security-related you need to think through what assets you are attempting to protect, the sensitivity of what you’re trying to protect, and what measures are more or less likely to protect those assets. Further, in developing a protection strategy you need to think through how many resources you’re willing to invest to achieve the sought-after protection. This applies as much to national security policy makers as it does to individuals trying to secure devices or networks.

What Is Being Protected

Most public figures who talk about TikTok and national security are presently focused on one or two assets.

First, they worry that a large volume of data may be collected and used by Chinese government agencies, after these agencies receive it either voluntarily from TikTok or after compelling its disclosure. Commentators argue that Chinese companies are bound to obey the national security laws of China and, as such, may be forced to disclose data without any notice to users or non-Chinese government agencies. This information could be used to obtain information about specific individuals or communities, inclusive of what people are searching on the platform (e.g., medical information, financial information, sexual preference information), what they are themselves posting and could be embarrassing, or metadata which could be used for subsequent targeting.

Second, commentators are adopting a somewhat odious language of ‘cognitive warfare’ in talking about TikTok.1 The argument is that the Chinese government might compel the company to modify its algorithms so as to influence what people are seeing on the platform. The intent of this modification would be to influence political preferences or social and cultural perceptions. Some worry this kind of influence could guide whom individuals are more likely to vote for (e.g., you see a number of videos that directly or indirectly encourage you to support particular political parties), cause generalised apathy (e.g., you see videos that suggest that all parties are bad and none worth voting for), or enhance societal tensions (e.g., work to inflame partisanship and impair the functioning of otherwise moderate democracies). Or, as likely, a combination of each of these kinds of influence operations. Moreover, the TikTok algorithm could be modified by government compulsion to prioritise videos that praise some countries or that suppress videos which negatively portray other countries.

What Is the Sensitivity of the Assets?

When we consider the sensitivity of the information and data which is collected by TikTok it can be potentially high but, in practice, possesses differing sensitivities based on the person(s) in question. Research conducted by the University of Toronto’s Citizen Lab found that while TikTok does collect a significant volume of information, that volume largely parallels what Facebook or other Western companies collect. To put this slightly differently, a lot of information is collected and the sensitivity is associated with whom it belongs to, who may have access to it, and what those parties do with it.

When we consider who is using TikTok and having their information uploaded to the company’s servers, then, the question becomes whether there is a particular national security risk linked with this activity. While some individuals may potentially be targets based on their political, business, or civil society bonafides this will not be the case with all (or most) users. However, in even assessing the national security risks linked to individuals (or associated groups) it’s helpful to do a little more thinking.

First, the amount of information that is collected by TikTok, when merged with other data which could theoretically be collected using other signals intelligence methods (e.g., extracting metadata and select content from middle-boxes, Internet platforms, open-source locations, etc) could be very revealing. Five Eyes countries (i.e., Australia, Canada, New Zealand, the United Kingdom, and the United States of America) collect large volumes of metadata on vast swathes of the world’s populations in order to develop patterns of life which, when added together, can be deeply revelatory. When and how those countries’ intelligence agencies actually use the collected information varies and is kept very secretive. Generally, however, only a small subset of individuals whose information is collected and retained for any period of time have actions taken towards them. Nonetheless, we know that there is a genuine concern about information from private companies being obtained by intelligence services in the Five Eyes and it’s reasonable to be concerned that similar activities might be undertaken by Chinese intelligence services.

Second, the kinds of content information which are retained by TikTok could be embarrassing at a future time, or used by state agencies in ways that users would not expect or prefer. Imagine a situation where a young person says or does something on TikTok which is deeply offensive. Fast forward 3-4 years and their parents are diplomats or significant members of the business community, and that offensive content is used by Chinese security services to embarrass or otherwise inconvenience the parents. Such influence operations might impede Canada’s ability to conduct its diplomacy abroad or undermine the a business’s ability to prosper.

Third, the TikTok algorithm is not well understood. There is a risk that the Chinese government might compel ByteDance, and through them the TikTok platform, to modify algorithms to amplify some content and not others. It is hard to assess how ‘sensitive’ a population’s general sense of the world is but, broadly, if a surreptitious foreign influence operation occurred it might potentially affect how a population behaves or sees the world. To be clear this kind of shift in behaviour would not follow from a single video but from a concerted effort over time that shifted social perceptions amongst at least some distinct social communities. The sensitivity of the information used to identify videos to play, then, could be quite high across a substantial swathe of the population using the platform.

It’s important to recognise that in the aforementioned examples there is no evidence that ByteDance, which owns TikTok, has been compelled by the Chinese government to perform these activities. But these are the kinds of sensitivities that are linked to using TikTok and are popularly discussed.

What Should Be Done To Protect Assets?

The threats which are posed by TikTok are, at the moment, specious: it could be used for any number of things. Why people are concerned are linked less to the algorithm or data that is collected but, instead, to ByteDance being a Chinese company that might be influenced by the Chinese government to share data or undertake activities which are deleterious to Western countries’ interests.

Bluntly: the issue raised by TikTok is not necessarily linked to the platform itself but to the geopolitical struggles between China and other advanced economies throughout the world. We don’t have a TikTok problem per se but, instead, have a Chinese national security and foreign policy problem. TikTok is just a very narrow lens through which concerns and fears are being channelled.

So in the absence of obvious and deliberate harmful activities being undertaken by ByteDance and TikTok at the behest of the Chinese government what should be done? At the outset it’s worth recognising that many of the concerns expressed by politicians–and especially those linked to surreptitious influence operations–would already run afoul of Canadian law. The CSIS Act bars clandestine foreign intelligence operations which are regarded as threatening the security of Canada. Specifically, threats to the security of Canada means:

(a) espionage or sabotage that is against Canada or is detrimental to the interests of Canada or activities directed toward or in support of such espionage or sabotage,

(b) foreign influenced activities within or relating to Canada that are detrimental to the interests of Canada and are clandestine or deceptive or involve a threat to any person,

(c) activities within or relating to Canada directed toward or in support of the threat or use of acts of serious violence against persons or property for the purpose of achieving a political, religious or ideological objective within Canada or a foreign state, and

(d) activities directed toward undermining by covert unlawful acts, or directed toward or intended ultimately to lead to the destruction or overthrow by violence of, the constitutionally established system of government in Canada,

CSIS is authorised to undertake measures which would reduce the threats to the security of Canada, perhaps in partnership with the Communications Security Establishment, should such a threat be identified and a warrant obtained from the federal court.

On the whole a general ban on TikTok is almost certainly disproportionate and unreasonable at this point in time. There is no evidence of harm. There is no evidence of influence by the Chinese government. Rather than banning the platform generally I think that more focused legislation or policy could make sense.

First, I think that legislation or (preferably) policies precluding at least some members of government and senior civil servants from using TikTok has some merit. In these cases a risk analysis should be conducted to determine if collected information would undermine the Government of Canada’s ability to secure confidential information or if the collected information could be used for intelligence operations against the government officials. Advice might, also, be issued by the Canadian Security Intelligence Service so that private organisations are aware of their risks. In exceptional situations some kind of security requirements might also be imposed on private organisations and individuals, such as those who are involved in especially sensitive roles managing critical infrastructure systems. Ultimately, I suspect the number of people who should fall under this ban would, and should, be pretty small.

Second, what makes sense is legislation that requires social media companies writ large–not just TikTok–to make their algorithms and data flows legible to regulators. Moreover, individual users should be able to learn, and understand, why certain content is being prioritised or shown to them. Should platforms decline to comply with such a the law then sanctions may be merited. Similarly, should algorithmic legibility showcase that platforms are being manipulated or developed in ways that deliberately undermine social cohesion then some sanctions might be merited, though with the caveat that “social cohesion” should be understood as referring to platforms being deliberately designed to incite rage or other strong emotions with the effect of continually, and artificially, weakening social cohesion and amplifying social cleavages. The term should not, however, be seen as a kind of code for creating exclusionary social environments where underprivileged groups continue to be treated in discriminatory ways.

So Is TikTok ‘Dangerous’ From A National Security Perspective?

Based on open source information2 there is no reason to think that TikTok is currently a national security threat. Are there any risks associated with the platform? Sure, but they need to be juxtaposed against equivalent or more serious threats and priorities. We only have so many resources to direct towards the growing legion of legitimate national security risks and issues; funnelling a limited set of resources towards TikTok may not be the best kind of prioritisation.

Consider that while the Chinese government could compel TikTok to disclose information about its users to intelligence and security services…the same government could also use business cutouts and purchase much of the same information from data brokers operating in the United States and other jurisdictions. There would be no need to secretly force a company to do something when, instead, it could just lawfully acquire equivalent (or more extensive!) information. This is a pressing and real national security (and privacy!) issue and is deserving of legislative scrutiny and attention.

Further, while there is a risk that TikTok could be used to manipulate social values…the same is true of other social networking services. Indeed, academic and journalistic research over the past 5-7 years has drawn attention to how popular social media services are designed to deliver dopamine hits and keep us on them. We know that various private companies and public organisations around the world work tirelessly to ‘hack’ those algorithms and manipulate social values. Of course this broader manipulation doesn’t mean that we shouldn’t care but, also, makes clear that TikTok isn’t the sole vector of these efforts. Moreover, there are real questions about the how well social influence campaigns work: do they influence behaviour–are they supplying change?–or is the efficaciousness of any campaign representative of an attentive and interested pre-existing audience–is demand for the content the problem?

The nice thing about banning, blocking, or censoring material, or undertaking some other kind of binary decision, is that you feel like you’ve done something. Bans, blocks, and censors are typically designed for a black and white world. We, however, live in a world that is actually shrouded in greys. We only have so much legislative time, so much policy capacity, so much enforcement ability: it should all be directed efficiently to understanding, appreciating, and addressing the fulness of the challenges facing states and society. This time and effort should not be spent on performative politics that is great for providing a dopamine hit but which fails to address the real underlying issues.

  1. I have previously talked about the broader risks of correlating national security and information security. ↩︎
  2. Open source information means information which you or I can find, and read, without requiring a security clearance. ↩︎
Categories
Links

Adding Context to Facebook’s CSAM Reporting

In early 2021, John Buckley, Malia Andrus, and Chris Williams published an article entitled, “Understanding the intentions of Child Sexual Abuse Material (CSAM) sharers” on Meta’s research website. They relied on information that Facebook/Meta had submitted to NCMEC to better understand why individuals they reported had likely shared illegal content.

The issue of CSAM on Facebook’s networks has risen in prominence following a report in 2019 in the New York Times. That piece indicated that Facebook was responsible for reporting the vast majority of the 45 million online photos and videos of children being sexually abused online. Ever since, Facebook has sought to contextualize the information it discloses to NCMEC and explain the efforts it has put in place to prevent CSAM from appearing on its services.

So what was the key finding from the research?

We evaluated 150 accounts that we reported to NCMEC for uploading CSAM in July and August of 2020 and January 2021, and we estimate that more than 75% of these did not exhibit malicious intent (i.e. did not intend to harm a child), but appeared to share for other reasons, such as outrage or poor humor. While this study represents our best understanding, these findings should not be considered a precise measure of the child safety ecosystem.

This finding is significant, as it quickly becomes suggestive that the mass majority of the content reported by Facebook—while illegal!—is not deliberately being shared for malicious purposes. Even if we assume that the number sampled should be adjusted—perhaps only 50% of individuals were malicious—we are still left with a significant finding.

There are, of course, limitations to the research. First, it excludes all end-to-end encrypted messages. So there is some volume of content that cannot be detected using these methods. Second, it remains unclear how scientifically robust it was to choose the selected 150 accounts for analysis. Third, and related, there is a subsequent question of whether the selected accounts are necessarily representative of the broader pool of accounts that are associated with distributing CSAM.

Nevertheless, this seeming sleeper-research hit has significant implications insofar as it would compress the number of problematic accounts/individuals disclosing CSAM to other parties. Clearly more work along this line is required, ideally across Internet platforms, in order to add further context and details to the extent of the CSAM problem and subsequently define what policy solutions are necessary and proportionate.

Categories
Links Writing

The So-Called Privacy Problems with WhatsApp

(Photo by Anton on Pexels.com)

ProPublica, which is typically known for its excellent journalism, published a particularly terrible piece earlier this week that fundamentally miscast how encryption works and how Facebook vis-a-vis WhatsApp works to keep communications secured. The article, “How Facebook Undermines Privacy Protections for Its 2 Billion WhatsApp Users,” focuses on two so-called problems.

The So-Called Privacy Problems with WhatsApp

First, the authors explain that WhatsApp has a system whereby recipients of messages can report content they have received to WhatsApp on the basis that it is abusive or otherwise violates WhatsApp’s Terms of Service. The article frames this reporting process as a way of undermining privacy on the basis that secured messages are not kept solely between the sender(s) and recipient(s) of the communications but can be sent to other parties, such as WhatsApp. In effect, the ability to voluntarily forward messages to WhatsApp that someone has received is cast as breaking the privacy promises that have been made by WhatsApp.

Second, the authors note that WhatsApp collects a large volume of metadata in the course of using the application. Using lawful processes, government agencies have compelled WhatsApp to disclose metadata on some of their users in order to pursue investigations and secure convictions against individuals. The case that is focused on involves a government employee who leaked confidential banking information to Buzzfeed, and which were subsequently reported out.

Assessing the Problems

In the case of forwarding messages for abuse reporting purposes, encryption is not broken and the feature is not new. These kinds of processes offer a mechanism that lets individuals self-identify and report on problematic content. Such content can include child grooming, the communications of illicit or inappropriate messages or audio-visual content, or other abusive information.

What we do learn, however, is that the ‘reactive’ and ‘proactive’ methods of detecting abuse need to be fixed. In the case of the former, only about 1,000 people are responsible for intaking and reviewing the reported content after it has first been filtered by an AI:

Seated at computers in pods organized by work assignments, these hourly workers use special Facebook software to sift through streams of private messages, images and videos that have been reported by WhatsApp users as improper and then screened by the company’s artificial intelligence systems. These contractors pass judgment on whatever flashes on their screen — claims of everything from fraud or spam to child porn and potential terrorist plotting — typically in less than a minute.


Further, the employees are often reliant on machine learning-based translations of content which makes it challenging to assess what is, in fact, being communicated in abusive messages. As reported,

… using Facebook’s language-translation tool, which reviewers said could be so inaccurate that it sometimes labeled messages in Arabic as being in Spanish. The tool also offered little guidance on local slang, political context or sexual innuendo. “In the three years I’ve been there,” one moderator said, “it’s always been horrible.”

There are also proactive modes of watching for abusive content using AI-based systems. As noted in the article,

Artificial intelligence initiates a second set of queues — so-called proactive ones — by scanning unencrypted data that WhatsApp collects about its users and comparing it against suspicious account information and messaging patterns (a new account rapidly sending out a high volume of chats is evidence of spam), as well as terms and images that have previously been deemed abusive. The unencrypted data available for scrutiny is extensive. It includes the names and profile images of a user’s WhatsApp groups as well as their phone number, profile photo, status message, phone battery level, language and time zone, unique mobile phone ID and IP address, wireless signal strength and phone operating system, as a list of their electronic devices, any related Facebook and Instagram accounts, the last time they used the app and any previous history of violations.

Unfortunately, the AI often makes mistakes. This led one interviewed content reviewer to state that, “[t]here were a lot of innocent photos on there that were not allowed to be on there … It might have been a photo of a child taking a bath, and there was nothing wrong with it.” Often, “the artificial intelligence is not that intelligent.”

The vast collection of metadata has been a long-reported concern and issue associated with WhatsApp and, in fact, was one of the many reasons why many individuals advocate for the use of Signal instead. The reporting in the ProPublica article helpfully summarizes the vast amount of metadata that is collected but that collection, in and of itself, does not present any evidence that Facebook or WhatsApp have transformed the application into one which inappropriately intrudes into persons’ privacy.

ProPublica Sets Back Reasonable Encryption Policy Debates

The ProPublica article harmfully sets back broader policy discussion around what is, and is not, a reasonable approach for platforms to take in moderating abuse when they have integrated strong end-to-end encryption. Such encryption prevents unauthorized third-parties–inclusive of the platform providers themselves–from reading or analyzing the content of the communications themselves. Enabling a reporting feature means that individuals who receive a communication are empowered to report it to a company, and the company can subsequently analyze what has been sent and take action if the content violates a terms of service or privacy policy clause.

In suggesting that what WhatsApp has implemented is somehow wrong, it becomes more challenging for other companies to deploy similar reporting features without fearing that their decision will be reported on as ‘undermining privacy’. While there may be a valid policy discussion to be had–is a reporting process the correct way of dealing with abusive content and messages?–the authors didn’t go there. Nor did they seriously investigate whether additional resources should be adopted to analyze reported content, or talk with artificial intelligence experts or machine-based translation experts on whether Facebook’s efforts to automate the reporting process are adequate, appropriate, or flawed from the start. All those would be very interesting, valid, and important contributions to the broader discussion about integrating trust and safety features into encrypted messaging applications. But…those are not things that the authors choose to delve into.

The authors could have, also, discussed the broader importance (and challenges) in building out messaging systems that can deliberately conceal metadata, and the benefits and drawbacks of such systems. While the authors do discuss how metadata can be used to crack down on individuals in government who leak data, as well as assist in criminal investigations and prosecutions, there is little said about what kinds of metadata are most important to conceal and the tradeoffs in doing so. Again, there are some who think that all or most metadata should be concealed, and others who hold opposite views: there is room for a reasonable policy debate to be had and reported on.

Unfortunately, instead of actually taking up and reporting on the very valid policy discussions that are at the edges of their article, the authors choose to just be bombastic and asserted that WhatsApp was undermining the privacy protections that individuals thought they have when using the application. It’s bad reporting, insofar as it distorts the facts, and is particularly disappointing given that ProPublica has shown it has the chops to do good investigative work that is well sourced and nuanced in its outputs. This article, however, absolutely failed to make the cut.

Categories
Links

Alarmist Takes On Chinese Influence Operations Must Be Set Aside

Lotus Ruan and Gabrielle Lim have a terrific piece in Just Security ‌which strongly makes the case that, “fears of Chinese disinformation are often exaggerated by overblown assessments of the effects of China’s propaganda campaigns and casually drawn attributions.”

The two make clear that there are serious issues with how some Western policy analysts and politicians are suggesting that their governments respond to foreign influence operations that are associated with Chinese public and private parties. To begin, the very efficacy of influence operations remains mired in questions. While this is an area that is seeing more research of late, academics and policy analysts alike cannot assert with significant accuracy whether foreign influence operations have any real impact on domestic opinions or feelings. This should call for conservatism in the policies which are advanced but, instead, we often see calls for Western nations to adopt the internet ‘sovereignty’ positions championed by Russia and China themselves. These analysts and politicians are, in other words, asserting that they only way to be safe from China (and Russia) is to adopt those countries’ own policies.

Even were such (bad) policies adopted, it’s unclear that they would resolve the worst challenges facing countries such as the United States today. Anti-vaxxers, pro-coup supporters, and Big Lie advocates have all been affected by domestic influence operations that were (and are) championed by legitimately elected politicians, celebrities, and major media personalities. Building a sovereign internet ecosystem will do nothing to protect from the threats that are inside the continental United States and which are clearly having a deleterious effect on American society.

What I think I most appreciated in the piece by Ruan and Lim is that they frankly and directly called out many of the so-called solutions to disinformation and influence operations as racist. As just one example, there are those who call for ‘clean’ technologies that juxtapose Western against non-Western technologies. These kinds of arguments often directly perpetuate racist policies; they will not only do nothing to mitigate the spread of misinformation but will simultaneously cast suspicion and violence towards non-Caucasian members of society. Such proposals must be resisted and the authors are to be congratulated for directly and forcefully calling out the policies for what they are instead of carefully critiquing the proposals without actually calling them as racist as they are.