An AI-generated fake political advert, a fake image of an explosion at the Pentagon that caused the spread of disinformation, a dating app that uses AI to start conversations with women without disclosing to them that they are talking to a bot, are just a sampling of countless reports this year.
We have seen of how generative AI could be used to generate convincing fake news articles and abusive content amplifying gender-based harms.
Such gender-based harms especially target women and girls, a phenomenon called technology-facilitated gender-based violence (TFGBV).
The emergence of generative AI, a class of AI technology that creates realistic text, images, audio, and video with a simple text input, has amplified existing methods and expanded the potential avenues for TFGBV faced by online communities.
Generative AI has opened up new avenues and exacerbated TFGBV against women and girls online, rights organisations have warned in a report.
According to the report published by Unesco, the proliferation of generative AI technology brought forth new harms, such as the creation of more convincing fake media, unintended biases in the outputs, automated harassment campaigns, and the ability to build realistic false narratives, known as “synthetic histories”.
As a result, existing harms such as hate speech, cyber harassment, misinformation, and impersonation, which are among the top vectors of TFGBV, now have a much wider reach and are more dangerous.
TFGBV is a pervasive problem across the globe, with a staggering 58 percent of young women and girls experiencing online harassment on social media platforms, according to a global study.
Even in regions with a digital divide, such as Sub-Saharan Africa, the 19 percent of women who used the internet in 2020 faced disproportionate gender-based violence, according to Unesco.
This report is aligned with Unesco’s work on raising awareness and producing solutions for gender-based violence on the internet.
Furthermore, generative AI tools have been misused by attackers to inflict TFGBV through various techniques. For instance, chatbots can be manipulated to provide harmful information by asking what users should not do, bypassing safeguards.
These tools are also utilised to create realistic templates for phishing campaigns, derogatory comments, and accessing social media APIs. Image generation AIs, particularly techniques like inpainting, lack prompt restrictions, thereby increasing the potential for AI porn and the spread of misinformation.
For example, text-to-image models can easily generate images of women in situations they did not consent to being in, thus creating a more realistic vector of image-based abuse.
Creating synthetic histories is a new vector of TFGBV harm. It allows attackers intending on spreading misinformation to use text-generative AI models to come up with convincing fake reports and histories that cast the target in a bad light, with the objective of casting doubt and defaming the individual - one of the top methods of inflicting TFGBV today.
“Generative AI technology enables an increase in the number of attackers, sustained and automated attacks, and the generation of content that convincingly mimics multiple perspectives,” says Unesco in its report.
This abuse is differentiated because women and girls are attacked for simply being online and for being girls/women. TFGBV also manifests similarly to real-world violence in that it tends to be enacted more on the most vulnerable and the disempowered.
Unesco defines TFGBV as use of digital tools to inflict physical, sexual, psychological, social, political or economic harm or other infringements of rights and freedoms. These are forms of violence that are directed against women because they are women and/or that affect women disproportionately,” says.
Specific communities amongst women are more likely to be targeted. Unesco’s research “The Chilling” identified that women journalists in prominent and visible positions tend to attract more virulent abuse. In their survey of 901 journalists, nearly three quarters (73 percent) said they had experienced online violence.
Campaigns against TFGBV against women have been up since the 2010s when Black feminists sounded the alarm following the fake movement to #EndFathersDay trended as part of a coordinated online campaign of fake accounts to disparage black feminism, perpetuate harmful stereotypes, and create conservative outrage about black feminist movements.
This campaign was intended to discredit and ridicule the movement for progressive thinking by creating bots to impersonate black women who fed into stereotypes of black feminists. Since, others such as Gamergate, a sustained coordinated online harassment campaign against women video game programmers.
Current generative AI technologies include ChatGPT, Claude, Bard, LlaMA, MidJourney, and Dall-E.
Unesco says combating TFGBV harms due to generative AI requires joint efforts from both generative AI developers and technology companies that platform them, regulators, policy makers, civil society organizations, independent researchers, and raising awareness at an individual level.
“Education on media and information literacy will give women knowledge to navigate the new world of generative AI,” adds the report.
Similar to the work Unesco has done on regulatory solutions and providing policy advice to address hate speech, this report intends to provide guidance to a variety of stakeholders for generative AI fueled technology-facilitated gender-based violence.
