Facebook removed fake accounts that used AI to create fake profile pictures
Facebook said Friday that it shut down hundreds of fake accounts, pages and groups that misled users, including some that used artificial intelligence to generate fake profile pictures. Researchers who studied the group of accounts said it’s the first time they’ve seen AI-generated images used on such a large scale, raising concerns about how the social network will tackle this problem ahead of the .
The company also took down fake accounts from the photo-sharing service Instagram , which it owns. It pulled down 610 Facebook accounts, 89 pages, 156 groups and 72 Instagram accounts that were from the US and Vietnam. About 55 million accounts followed at least one of the pages and most of these followers were from outside of the US.
“We’re constantly working to detect and stop this type of activity,” Facebook said in a blog post. “We don’t want our services to be used to manipulate people.”
The company said it removed the accounts for engaging in foreign and government interference or “coordinated inauthentic behavior,” which means that whoever was running the accounts misled others about their identity and what they were doing.
Targeting US citizens as well as Vietnamese, Spanish and Chinese-speaking audiences, many of the fake accounts promoted content by a US media outlet called “The BL,” short for “The Beauty of Life.” The BL has been linked to a US media group that owns The Epoch Times and pushes pro-Trump content, according to fact-checking group Snopes . The fake accounts “typically posted memes and other content about US political news and issues including impeachment, conservative ideology, political candidates, elections, trade, family values and freedom of religion,” Facebook said.
Graphika, a social media analysis company, and the Atlantic Council’s Digital Forensics Lab analyzed the network of fake accounts and found that a lot of the English language content was pro-Trump or attacked the president’s rivals.
The accounts used fake profile pictures and pretended to be Americans to join groups such as “America Needs President Trump” and “Trump for America’s President.” Using AI, these fake accounts included profile pictures of people that aren’t real.
Researchers noticed subtle details in the profile pictures that signaled they were fake. For example, in some photos accessories such as glasses and earrings were not symmetrical. The dimensions of someone’s neck might be slightly distorted or the background of the photo looks off.
“The ease with which the operation managed to generate so many synthetic pictures, in order to give its fake accounts (mostly) convincing faces, is a concern,” according to the report by Graphika and the Atlantic Council.
In other cases, fake accounts used photos from stock images or glamor shots found online.
Another network of fake accounts originated from the country of Georgia. Facebook removed 39 Facebook accounts, 344 pages, 13 groups and 22 Instagram accounts, which collectively had more than 450,000 followers and members. The network pretended to be media, political parties and activists, posting about news, political issues and political criticism.
“These accounts represent the core portion of a larger network of more than 88,000 accounts engaged in spammy behavior across a wide range of topics,” Twitter said. “We have permanently suspended all of these accounts.”
Originally published Dec 20, 10:52 a.m. PT
Update 1:30 p.m. PT: Adds more background about AI-generated profile pictures.
Author Queenie Wong