#386: New Nth Room: Middle Schoolers Deepfake Videos Of Mom, Sister, Classmates In "Humiliation Room"

In light of a pervasive crisis in South Korea, growing alarm is spreading over the mass production and sharing of non-consensual deepfake pornography, with a disconcerting number of cases involving women and minors.

Thousands of Telegram chat rooms across South Korea have been uncovering a massive network of perpetrators creating and sharing deepfake sexual content of victims, many of whom are minors.

The controversy intensified with the revelation that over 400 schools had related Telegram chat rooms. This network of perpetrators demanded users submit personal information and images of individuals they know—including classmates, friends, and family members—to be manipulated into pornographic deepfake videos.

The chat rooms require users to submit personal details and photos of acquaintances, friends, and family members, which are then used to generate deepfake pornographic videos.

To access these chat rooms, users had to provide extensive personal details and photos of their victims. Facts required could include name, date of birth, occupation, school, grade, age, Instagram handle, phone number, and even home addresses.

The scale of this epidemic is staggering, with over 400 schools, or 7% of all middle schools, high schools, and colleges in South Korea, identified as having associated Telegram chat rooms.

Affected institutions list includes dozens of middle schools, high schools, and even top universities. Over 70 individual chat rooms were identified within one Telegram channel, each corresponding to different universities or schools.

Perpetrators, often young students, view creating and sharing these deepfakes as a form of "entertainment" or "prank", with little awareness of the severe consequences.

Many perpetrators are students, some as young as middle schoolers, who aim at their own classmates and teachers. The casual nature of their conversations in the threads reveals a disturbing attitude toward creating and distributing these images.

Many perpetrators are students themselves, even as young as middle schoolers, who target their own classmates and teachers.

Reports have surfaced of middle school students deepfaking their peers and of a middle school boy who produced deepfake content of 12 female classmates and two teachers.

Some perpetrators go to extreme lengths, such as secretly recording victims or obtaining personal photos, in order to create more content to share in the chat rooms.

The lengths to which perpetrators will go to create this content are alarming; cases of secret recordings and obtaining personal photographs are common. Some have gone as far as committing physical assaults to garner material for their deepfakes.

The epidemic’s reach extends to a range of victims, from schoolchildren to professional adults such as lawyers and teachers. The deep fakes have been used to create videos that depict individuals in sexually compromising and abusive situations. For instance, Stephanie Soo brought up a case of a woman who became a lawyer, only to discover her images shared in a group chat from her law school days, implicating someone within her professional circle.

Thi ...

The rise of deepfake technology has brought about an undercurrent of psychological trauma and societal disruption, particularly for female victims. Their lived experiences depict an alarming picture of the repercussions deepfakes have on individuals' sense of security and their trust in the social framework.

Victims, upon discovering the deepfake content, experience intense trauma, panic, and a profound loss of trust in those around them.

Women in Korea express a feeling akin to 'social collapse,' which speaks volumes about the impact on their sense of security. The distribution of these deepfakes—often through numerous chat rooms—promotes this distrust, potentially affecting victims' employment opportunities and exposing them to physical harm.

Both students and teachers have found themselves embroiled in deepfake scandals. A high schooler was horrified by explicit deepfake photos of herself that appeared authentic, and a teacher discovered deepfake material of herself being disseminated by her students. This sense of betrayal, particularly when those involved were trusted individuals such as students or, in one horrific instance, a victim's brother, disrupts the foundational sense of safety one feels in personal relationships and community.

Victims report intense emotional disturbances, such as feeling the need to cut themselves out of their own bodies—a graphic illustration of the severity of the psychological impact of being deepfaked. This trauma reinforces paranoid thoughts, leaving victims in constant fear of who might have seen the manipulated images and videos.

Victims report feeling disgusted with themselves and their bodies, and become paranoid about every interaction, unsure of who may have seen the content.

Victims endure an overwhelming sense of paranoia, constantly questioning who among their acquaintances or passersby may have seen or distributed the harmful content. This paranoia extends beyond the digital realm, with implications on every personal interaction they have.

The case of Stephanie Soo, who struggled for a year after discovering her brother had taken compromising photos of her, underscores this burden. Unable to trust even her family, she illustrates the deeply personal nature of the violation and the struggle to reclaim ownership and comfort within one's own body.

The deepfake videos can have far-reaching consequences, including jeopardizing victims' future employment prospects and exposing them to potential physical harm if recognized.

Beyond personal trauma, the threat looms of potential blackmail and physical danger due to recognition from deepfake content. Officials warn that such images can ruin job prospects and lead to physical violence. Victims are left to grapple with both the immediate emotional toll and the long-term social and economic repercussions.

The impact of these violations extends beyond the individual victims, damaging the broader social ...

In South Korea, the systemic failures and dismissive attitudes towards deepfake-related crimes allow such activities to continue with minimal deterrence. Despite the severity of these crimes, the legal system often fails to provide significant consequences for the perpetrators.

Despite the severity of these crimes, the legal system in South Korea has largely failed to provide adequate deterrence or meaningful consequences for perpetrators.

Lenient sentences that frequently involve probation or suspended sentences send a clear message that these crimes are not taken seriously. An egregious example is a high schooler convicted of creating and distributing deepfake videos who received only six months in prison because he admitted his mistakes and had no prior criminal record. In reality, perpetrators often receive probation and suspension without a fine, even though making deepfakes for distribution can be punished with up to five years in prison or a 50 million won fine. Moreover, only five individuals were sentenced to prison for purely deepfake-related crimes in the previous year.

Even when perpetrators are identified and prosecuted, the penalties are viewed as a "slap on the wrist," doing little to discourage future offenders.

Officers note a lack of awareness about deepfaking as a crime, suggesting deficiencies in education and law enforcement regarding the issue. A middle schooler who produced deepfakes facilitated the narrative that there are negligible consequences for young offenders. Perpetrators share among themselves that unless they've left identifying information, they likely will not be caught, due to Telegram's lack of cooperation with the police.

The dismissive attitudes of some government officials and the public have further compounded the problem, minimizing the experiences of victims.

Parents have been noted to question the fairness of severely punishing their sons for what they perceive to be a mere "joke," trivializing the crime and downplaying its impacts. Even when victims seek help from authorities, they're often told there's little that can be done against crimes committed on secure platforms like Telegram. Victims' family reactions can also be dismissive, such as a mother who merely apologized during an argument, and a father who questioned the duration his daughter would talk about her experience.

Some officials have expressed concerns about "overregulation" of deepfakes, while others have suggested that women are partially res ...