This article is part 2 of the four-part series. Part 1: Setting the scene offered an introduction to the series by clarifying definitions, provided an estimated magnitude of the online child sexual exploitation (OCSE) in South and Southeast Asia and hypothesised that continued over-reliance on automated methods of OCSE detection are unlikely to yield strong long-term results and that investment into prevention programmes, digital literacy efforts and regional partnerships will continue to be of utmost importance.
The nature of the issue
It has long been recognised that child sexual exploitation is a complex and a nuanced problem with numerous contributing factors that are deeply embedded in social structures and norms. As shown on the figure below (adapted from Radford, Allnock & Hynes, 2015 for UNICEF) known risks to victims are spread across several interconnected levels. Individual as well as family, community and broader societal factors influence the likelihood of victimisation and perpetration.
Views on how perpetration in online spaces occurs and who may become a target tend to be influenced by moral panic pieces still prevalent in mass media. Yet, due to the sensitivity and an explicit nature of the crime, sometimes bordering on taboo, few people are willing to go on an independent fact-finding mission to learn the facts.
South and Southeast Asia are the regions where OCSE is wrapped up in a plethora of gender imbalances which exacerbate the problem further. Data shows that, worldwide, about half of all children and a third of women are subject to physical or sexual violence in their lifetime simply because of their gender. Furthermore, certain cultures have societal norms that view violence against women and children as normal and appropriate under certain circumstances. UNESCO research shows that 52% of women in South Asia and 30% in East Asia and the Pacific aged 15 to 49 believe that physical force against a wife or a life partner can be justified.
These gender imbalances extend into the online realm, where they are often amplified further. Globally, despite accounting for half of the population, 49.7% to be exact, with very little variance in the past 60 years, women in low- and middle-income countries are 19% less likely than men to use mobile internet, which translates into around 310 million fewer women than men. In the countries that are the focus of this overview, almost 114 million fever women are using the internet compared to men, according to ITU data.
South Asia faces persistent gender inequalities, limiting access to devices and safe online experiences for women and girls. GSMA reports that literacy and digital skills remain one of the top barriers to mobile internet adoption in those countries. Female mobile users in Bangladesh, India and Pakistan who were aware of mobile internet but did not use it reported that reading and writing difficulties was the main reason.
Technology-facilitated violence against women and girls (VAWG) exists on the same plane as offline VAWG and it is a part of the systemic discrimination and gender-based violence that many women and girls experience. A Plan International study across 22 countries, including India, the Philippines, and Thailand, found that 58% of young women aged 15 to 25 have experienced online harassment on social media. They also indicated that male harassers tend to be more aggressive or display inappropriate behaviour online after they felt the girl had rejected or turned them down in some way. The Internet Watch Foundation indicates that over 90% of child sexual abuse images analysed by them consistently show girls only. In the International Child Sexual Exploitation (ICSE) Database managed by INTERPOL 64.8% of unidentified victims are female and 92.7% of offenders are male. Despite having less presence online, females feature in the majority of child sexual exploitation cases and, more broadly, tend to be on the receiving end of more severe types of harassment including stalking and sexual harassment.
What does it have to do with AI?
These profound inequalities in access and safety are deeply embedded in the language, communication styles, and the way various groups represent themselves in online spaces that eventually serve as source data for training AI models. In addition to serving as a representation of the long-standing issues, language also reflects the shifting opinions regarding many contentious societal problems. In their paper on large language models (LLMs) Bender, Gebru, McMillan-Major & Shmitchell demonstrate this using the global #MeToo movement as an example, arguing that it
“...has spurred broad-reaching conversations about inappropriate sexual behavior from men in power, as well as men more generally [84]. These conversations challenge behaviors that have been historically considered appropriate or even the fault of women, shifting notions of sexually inappropriate behavior. Any product development that involves operationalizing definitions around such shifting topics into algorithms is necessarily political (whether or not developers choose the path of maintaining the status quo ante). For example, men and women make significantly different assessments of sexual harassment online [40]. An algorithmic definition of what constitutes inappropriately sexual communication will inherently be concordant with some views and discordant with others.” (p.615).
In short, the fluid nature of more contentious societal problems, the way they are (not) discussed and (not) represented on different platforms offers source data for algorithms that are expected to generalise multiple layers of nuance rooted in history, culture and politics.
In the context of OCSE, with utilisation of AI-driven tools for content detection still being in the early days and being limited to large companies who can afford to build or buy such tools, it is yet to be seen how well they can identify risky or illicit communications with a high degree of variance in gender, age, background, language and motivations of victims, traffickers or end customers.
Evidence from the Philippines offers one example that demonstrates the nuanced nature of OCSE perpetration. Globally there is an estimated 7.5% of female offenders in online child sexual exploitation cases. However, in the Philippines 87% of cases analysed by the International Justice Mission involved at least one female perpetrator, usually a mother or other female relative. Due to this pattern being uncommon in other regions, instances of grooming or trafficking may remain undetected until it’s assessed by a subject matter expert familiar with this regional pattern or, in the worst case, until the situation escalated to the point where violative media is produced and can be detected via hash matching or other form of image analysis.
It is important to remember that Southeast Asia has over 100 languages written in multiple scripts. There are instances when a language is written in a script that is different to a canonical one. For example, in casual communications like instant messaging Hindi may be written in Latin script instead of Devanagari.
It may be easy to argue that these are outlier cases which are not the ultimate targets of scaled abuse detection mechanisms. Yet OCSE consists of a multitude of such outliers that are unique to location, culture and language and, if overlooked, they lead to thousands of victims slipping through the cracks of AI-driven solutions that often come with a promise of a universal applicability.
Hidden human resource involved in AI development
Despite the convincing facade of self-sufficiency, AI relies heavily on human labour, often unnoticed in the automation experience. From hail apps’ drivers and content moderators to data clerks and software engineers, AI systems depend on these workers for making automated systems to operate without an apparent need for human interventions.
Data used to train models requires review and correction for validity, accuracy, completeness, consistency, and uniformity. While programs handle some data cleaning, human data annotators play a significant role.
The same applies to the process called reinforcement learning from human feedback which is a technique for optimising algorithms’ performance by providing input from humans by asking them, for example, to rank responses in terms of accuracy. Training of models with assistance from human feedback has been shown to outperform baselines on various tasks including summarisation and following instructions.
It’s been reported that increasingly agents tasked with data cleaning and data annotation - the processes designed to improve the performance of AI models - are based in the more economically disadvantaged countries of Africa and Asia due to the low costs. They have to follow a set of very specific, often lengthy and constantly changing instructions provided by the developing company. The entire process adds a whole new layer of interpretation to the source data - it is viewed through the eyes of annotators who have their own biases and lived experiences but they are added on top of the instructions often provided by someone employed by a large tech company and likely living a completely different reality.
As a result the performance of AI systems is affected by a multitude of human influences intrinsically linked to societies, cultures and languages but, by the virtue of established practices, they are less transparent and open to probing.
Performance reliability
In addition to the varied influence of human factors on AI performance, it’s been documented that the accuracy of models can degrade over time or that models can make mistakes. The former phenomenon known as “drift” refers to instances when Large Language Models (LLMs) behave in unexpected or unpredictable ways that stray away from the original parameters. This may happen due to attempts to improve certain parts, while inadvertently degrading others. The latter is called “hallucination” and refers to mistakes in the generated output that are semantically or syntactically plausible but are in fact incorrect.
The risks of both are reportedly lower with smaller, more tailored models, like the ones that may get developed specifically for detection of OCSE. However, it is reasonable to expect that a larger abuse fighting model, designed to detect various types of problematic content, would be deployed instead, in an effort to maximise return on investment.
One of the documented ways to combat hallucination on AI tools is reinforced learning from human feedback, the problematic aspects of which have been covered above.
Chapter conclusion
The complexities and limitations surrounding the interactions between humans and AI are being widely discussed today. They are known, they are no longer brand new and there are also ways to overcome them. OpenAI previously stated that “long-term AI safety research needs social scientists to ensure AI alignment algorithms succeed when actual humans are involved”.
Despite the declared commitment and pledges made in front of the policymakers and regulators, child safety seems to continuously fall to the bottom of the tech agenda. As a result, when commercially driven applications and solutions are being deployed, responsible development and ethical design of products continue to meet only the minimum compliance benchmark at best or to be an afterthought or even a ‘non-thought’ at worst.
See other chapters in the "Combatting OCSE in Asia Pacific" series:
Comments