Most people can’t tell a bot from a human online, and the bots most likely to fool them are the polite ones, according to a new Surfshark study.
The company analyzed 1,722 participants worldwide, testing their ability to separate human comments from AI-generated ones in a social media setting. Overall, people caught just 40% of the bots placed in front of them.
(Source: Surfshark)
The bots that slipped by most often weren’t loud or aggressive. They were positive, friendly, and logical-sounding, the exact traits that make a stranger’s comment feel safe to trust.
“On social media, everyone notices the angry trolls. That’s why the angry trolls are the ones who often get caught. In the simulated social media environment, participants flagged half (50.2%) of negative AI-generated bots,” said Luís Costa, Research and Insights Lead at Surfshark.
When a bot took on a positive, agreeable persona, detection dropped to 38%, a 12-point gap from the negative bots.
“Often, AI-powered accounts used in sophisticated manipulation campaigns are agreeable, logical, or simply unremarkable. They can support real users’ opinions and just inflate the number of comments in the discussion to make a minority view look like everyone feels the same way. Seldom do people report such accounts,” added Costa.
Serious topics make it worse
Participants spotted more bots on light topics, like pineapple on pizza, than on serious ones, like women’s rights. On the more serious topic, people weren’t only missing more bots. They were also wrongly accusing more real humans of being fake.
Neutral bots caused similar trouble, ranking near the bottom for detection across most topics. Immigration was the exception. Negative bots were still easiest to catch there, but positive bots were the hardest of all four topics to spot.
Bots that used emojis heavily were caught over 60% of the time, while bots that kept their language plain were detected only 35% of the time. An account that simply stops overusing emojis becomes twice as hard to identify.
Friendly bots do more damage than angry ones
Researchers argue that the agreeable, logical bot is a bigger privacy and security concern than the confrontational one, precisely because it doesn’t set off suspicion.
A bot that agrees with someone, sounds reasonable, and never picks a fight doesn’t register as a threat. People are more likely to move that conversation into private chat, hand over personal details, click a link, or accept a “logical” argument at face value, without knowing they’re talking to a machine.
Contact like that can shift someone’s political or social views before they ever suspect they’ve been talking to a bot.
Platform, age, and habits all play a role
Text-driven platforms performed better than visual ones. Threads users had the highest detection rate in the study, though Surfshark notes this is based on a small sample. X users followed close behind. TikTok and Facebook users trailed well behind both.
Detection peaked among the youngest players and declined steadily from there, bottoming out among people over 50, the lowest score of any group. Over-50s also had the weakest accuracy, meaning they were the most likely to wrongly flag a real person as a bot.
People on social media “almost all the time” caught close to half the bots they saw, while people who don’t use social media at all caught roughly a third.