Deepfakes and the manosphere: A dangerous combination

If you think revenge porn couldn’t happen to you because you’ve never allowed anyone to take photos or videos of yourself in intimate situations, think again. Search “remove clothes app” or similar terms online, and you’ll find a dizzying array of free and paid image editing tools promising to undress anyone in a perfectly innocuous photo quickly and realistically.

This is the era of deepfake porn, where generative AI is being used by many around the world to create pornographic images so real that the subjects – typically women – find it difficult for others to believe that it isn’t really them.

In November last year, students from the Singapore Sports School came under police investigation for creating and sharing deepfake nude photos of their female schoolmates. In the same month, five Cabinet ministers and over 100 public servants received e-mails demanding payment in return for not publishing faked images of them in compromising positions. And women make up a disproportionate share of these cases.

In 2019, a company called Deeptrace Labs identified 14,678 deepfake videos across several streaming platforms and porn sites, and Meanwhile, a 2023 report by Singapore-based non-profit SG Her Empowerment (SHE) found that female youth were twice as likely to experience sexual harassment compared to male youth. 

The rise of tech-facilitated sexual violence

Deepfake porn is one of many forms of techfacilitated sexual violence, which the Association of Women for Action and Research (Aware) defines as “sexual violence committed, amplified, or aided through digital technologies or online platforms”. This ranges from non-consensual creation or obtaining of intimate images, to distributing these images, to sending unwanted explicit images, to sending sexually harassing messages or comments.

According to SHE, its one-stop support centre for victims or survivors of online harms – Shecares@SCWO – has seen over 200 cases since it began operating in 2023, and image-based sexual abuse is among the top harms seen. In one case, a client had a deepfake image of her uploaded on a social media platform, which quickly went viral.

“Of course, she was very distressed when she came to us. We reached out to the platform and it was removed shortly after,” shares How Kay Lii, chief operating officer at SHE. 

In its report published last year, Aware cited a case where a woman was warned by an anonymous account on social media that intimate images of her were circulating online.

“She was not sure if the claims were authentic and wanted to get to the bottom of the issue. However, the police were unable to investigate because the anonymous account did not provide the client with any evidence,” said Aware in the report. “The uncertainty of the situation made her feel extremely uncomfortable and anxious.”

The impact of deepfake harms on victims can be severe, says Kay Lii. “They feel shame, anger, humiliation and anxiety. They have to deal with the stigma. They feel like their privacy has been badly violated, and it takes a significant toll on their mental health and quality of life.

“It also affects their personal relationships, professional opportunities, and their ability to return to normal online activity.”

Noting that women are disproportionately affected by sexualised deepfakes, Kay Lii adds: “If such online harms against women continue to proliferate, women will find it challenging, and even dangerous, to go online for their education, work or recreation. “They will find themselves disadvantaged, and unfairly so.”

Tighter laws, but can they keep up?

In Singapore, the authorities have been taking steps to act more swiftly against online harms, most recently with the establishment of a new agency for online safety, which will be able to act on behalf of victims to order perpetrators and online service providers – such as social media platforms – to take down harmful and offensive content. 

The government also plans to introduce new legislation to allow victims of online harms to seek civil remedies from their perpetrators. These moves allow victims to bypass the current court process, which can take too long. And they come on the back of other legislation introduced in recent years to combat online harms, such as the Online Safety Act and Online Criminal Harms Act, which give the government more teeth to take perpetrators to court.

But Chew Han Ei, an adjunct senior research fellow at the Institute of Policy Studies whose research focus includes technology adoption by societies, says that tech platforms also need to do more and become part of the solution. 

“Technology like generative AI has lowered the barrier for creating harmful content. And platforms, whether intentionally or not, sometimes contribute to the problem,” explains Han Ei, who also serves on the board of SG Her Empowerment.

Citing Telegram, which has fostered the rise of groups with tens of thousands of members sharing sexually offensive images – such as SG Nasi Lemak – Han Ei notes that “algorithms designed to maximise engagement can end up amplifying harmful content or reinforcing damaging norms”.  

And among the darker undercurrents being amplified by the Internet is the manosphere, a collective term for the loosely networked websites, blogs, and online forums that promote masculinity, misogyny, and opposition to feminism. 

A megaphone for misogyny

In the manosphere, men are portrayed as victims in a world that empowers women and minorities by putting men at a disadvantage, and gender equality is perceived as an existential threat to men.

Increasingly, these views have gone from online forums and subreddits into the mainstream, thanks to social media personalities like Andrew Tate and Myron Gaines, who make millions by trafficking in toxic masculinity and misogynistic views on their platforms, and exploiting fears among men about their role in society. 

In Singapore, you may be surprised to know that in a survey of about 500 Singaporeans published by market research firm Ipsos last year, 57 per cent of Singapore men agreed that promoting women’s equality has “gone so far” that “men are being discriminated against”. Han Ei says that technology has given the age-old problem of misogyny a “megaphone”.

“The manosphere exists largely because platforms are designed to push content that gets clicks and reactions, even if it’s toxic,” he explains.

“The more polarising the content, the more it spreads. Algorithms love that.”

For example, in a 2022 study in Australia that tracked Youtube recommendations to boys and men, researchers found that among the accounts being studied, Youtube was recommending videos with messages antagonistic towards women and feminism.

Following the recommendations and viewing the suggested content resulted in more overtly misogynist content being recommended, said the researchers.

Lawyer Simran Toor, who served as SHE’s first CEO, adds that the manosphere may be filling a gap for men who are seeking answers to their fears and sense of alienation from society.

“Over the past few decades, women’s roles in society have evolved and grown… while a lot of men have this question in the back of their minds: ‘If women have taken on all these realms, what’s my role?’ Yet, there aren’t many spaces where men can have that open and thorough discussion.”

A fear of “saying the wrong thing” may lead men, particularly young men and boys, to look online for guidance, says Simran. “Unfortunately, the most prolific discussions tend to be quite specific, with your Andrew Tates and the like. And when you’re feeling lost and insecure, you may well resonate with that rhetoric.”

And the manosphere is more than just rhetoric; Han Ei points to SG Nasi Lemak, as well as an online poll rating the looks of women Islamic religious teachers, as examples of harmful actions resulting from misogynistic attitudes.

“It normalised voyeuristic behaviour and dismissed the harm done to victims, and reduced respected leaders to objects of ridicule. These aren’t just isolated incidents – they reflect a culture that tolerates, even amplifies, harmful narratives about women,” he states.

“The manosphere isn’t just online noise – it shapes real attitudes and behaviours.” And globally, there are growing concerns over the link between misogyny and far-right violent extremism, noted researcher Yasmine Wong in a commentary published last year.

“Researchers hypothesise that misogyny has become a ‘gateway drug’ to far-right violent extremism,” wrote Yasmine, who is senior research fellow at the S Rajaratnam School of International Studies. 

“For instance, the perceived existential threat posed by feminism and gender equality on men has been likened to the perceived existential threat that migrants pose to the white male identity.”

Already in the UK, the authorities have announced that it is treating extreme misogyny as a form of extremism, as it believes misogyny to be fuelling extremism in the country, she wrote.

“Rather than normalising the gender divide and misogynistic attitudes, it is imperative to understand the risks associated with the global growth of misogyny, not just in its insidious role in violent extremism, but also its potential to damage the social fabric.”

A space for hard conversations

Not all is lost – Simran points out that the online advocacy against sexual violence on women has grown over the years, while enhanced legislation shows that the problem is being taken seriously. But the deeper issues behind our gender attitudes are a “societal problem” that cannot be fixed by laws.

“If people are still going to have these mindsets, you can’t pass a law against a mindset right?” says Simran. “So we need community efforts and outreach.”

With online discourse often polarised between extremes, Simran calls for more conversations to happen offline, in schools and other community and social settings.

“It would be helpful to have proper discourse, and as a society decide what are the qualities of healthy masculinity,” she suggests.

Han Ei echoes the need for more public discourse on the issue: “Public education is key to equipping people with the tools to challenge harmful norms, and build a culture that values equality and empathy.”

He adds: “This is about more than fixing the Internet. It’s about shaping the kind of society we want to live in.”

What to do if you’re a victim of a deepfake

●DOCUMENT THE EVIDENCE Take screenshots of posts, messages, or images, and capture any links and timestamps. These are crucial when reporting the incident to social media platforms to request takedowns.

Most social media platforms have policies against non-consensual intimate imagery and impersonation, so take note of what is required to make a report and document all evidence accordingly.

●SEEK SUPPORT Don’t keep it to yourself – talk to trusted friends or loved ones for support. And seek out organisations that specialise in providing emotional, legal and technical support for victims of online harms.

Aware’s Sexual Assault Care Centre provides support for victims, including assistance with gathering evidence, filing police reports or magistrate’s complaints, and applying for court orders against harassment.

Shecares@SCWO centre provides free legal advice through clinics with volunteer lawyers and free counselling support, and can help reach out to tech platforms to take down harmful content.

●LODGE A POLICE REPORT Although it can be distressing to share your experience even before you have had time to process it, lodging a police report allows the authorities to investigate and take action against chargeable offences. 

●SHOUTOUT The SG Her Empowerment (SHE) Annual Symposium is diving deep into pressing topics like gender norms and their intersections with Artificial Intelligence, online harms, and women in the workforce. Expect a dynamic line-up of panels, workshops, films, and a special presentation that will challenge perspectives and spark meaningful conversations.

The SHE Annual Symposium 2025 will bring together thought leaders, policymakers, and advocates to address the most urgent issues shaping the digital world for women and girls.

Date: Monday, 24 February 2025
Venue: YWCA Fort Canning
Time: 9:00 AM to 4:30 PM

Find out more and register here.