Bot Generated Fake Nudes of Over 100,000 Women Without Their Knowledge, Says Report


Key Facts

  • These so-called “deepfake” images were created by an ecosystem of bots on the messaging app Telegram that could generate fake nudes on request, according to a report released by Sensity, an intelligence firm that specializes in deepfakes.
  • The report found that users interacting with these bots were mainly creating fake nudes of women they know from images taken from social media, which is then shared and traded on other Telegram channels.
  • The Telegram channels the researchers examined were made up of 101,080 members worldwide, with 70% coming from Russia and other eastern European countries.
  • A small number of individuals targeted by the bot appear to be underage.
  • According to the report, the bots received significant advertising on the Russian social media website VK.
  • However, the Russian social platform’s press team told Forbes that these communities or links were not promoted using VK’s advertising tools, adding “VK doesn’t tolerate such content or links… and blocks communities that distribute them.”

Big Number

70%. That’s the percentage of bot’s victims who were private individuals, not celebrities or influencers. The fake nudes of these women were generated using photos that were either taken from social media or private material. This is unlike deepfake non-consensual pornographic videos where celebrities are often the target.

Crucial Quote

Sensity’s CEO and Chief Scientist Giorgio Patrini told Forbes that unlike celebrity deepfake videos, which require multiple images and videos, “only a single image is needed to operate this technology, and simply by uploading to a chat room,” He notes that this completely changes completely who can be targeted, “if you have ever shared publicly one photo on social media, you may be under threat.”

Surprising Fact

While direct messages on Telegram feature unbreakable end-to-end encryption, which makes illegal content hard to trace, Sensity found that the Telegram channels used in this investigation were simply accessible by searching the right keywords among public groups. According to Patrini, these channels don’t even try to hide and they include tens of thousands of users, with no vetting or selection. “Actually, bot services like this one are made for business, they charge by usage, and they want to reach a big audience to monetize. There is little to no attempt to police and take them down,” he added.

Key Background

Deepfakes are a form of forged media generated by using a type of artificial intelligence called neural networks in which a person an image or video is replaced by someone else’s likeness. The Telegram bot discovered by Sensity’s researcher sounds similar to an app called DeepNude, that used AI to automatically generate non-consensual nudes of women in photos, by ‘stripping’ their clothing from the images. DeepNude was shut down by its developers in a single day after it received extensive critical media coverage. The app’s creators however sold DeepNude’s license on an online marketplace later for $30,000, following which the app was reverse-engineered. The researchers note that the Telegram bot is likely based on an open-source version of DeepNude, however, simpler and easier to use than the original desktop app.

Tangent

While most of the coverage on deepfakes has focused heavily on politics and elections many experts worry that the actual victims of this technology may be people who are socially vulnerable. Sam Gregory, a program director with human-rights video organization Witness, told CNET that, the “focus on deepfakes is in an electoral context,” overlooks the harm being caused to regular people, where even a poor-quality deepfake is still deeply harmful.


* This article was automatically syndicated and expanded from Forbes.


Be the first to comment

Leave a comment: