“Don’t believe what AI told you I said”
Chatbots are making up quotes that were never actually said.
 

This is Atlantic Intelligence, a newsletter in which our writers help you wrap your mind around artificial intelligence and a new machine age.

Did someone forward you this newsletter? Sign up here.

Damon Beres

Senior editor, technology

My colleague Yair Rosenberg was recently quoted in a blog post published by the Times of Israel. Except, well, he wasn’t: The post, or at least that portion of it, seemed to be AI-generated, attributing statements to Yair that he had never actually made. Something similar recently happened to the science-fiction author John Scalzi and to the writer Gabriel Yoran, both of whom spoke with Yair about encountering AI-generated posts on social media that effectively put words into their mouths.

“The internet has long been awash with fake quotations attributed to prominent personalities,” Yair writes in a new article for The Atlantic. “As Abraham Lincoln once said, ‘You can’t trust every witticism superimposed over the image of a famous person on the internet.’ But the advent of AI interfaces churning out millions of replies to hundreds of millions of people—ChatGPT and Google’s Gemini have more than 1 billion active users combined—has turned what was once a manageable chronic condition into an acute infection that is metastasizing beyond all containment.”

(Illustration by The Atlantic. Source: Getty.)

 

John Scalzi is a voluble man. He is the author of several New York Times best sellers and has been nominated for nearly every major award that the science-fiction industry has to offer—some of which he’s won multiple times. Over the course of his career, he has written millions of words, filling dozens of books and 27 years’ worth of posts on his personal blog. All of this is to say that if one wants to cite Scalzi, there is no shortage of material. But this month, the author noticed something odd: He was being quoted as saying things he’d never said.

“The universe is a joke,” reads a meme featuring his face. “A bad one.” The lines are credited to Scalzi and were posted, atop different pictures of him, to two Facebook communities boasting almost 1 million collective members. But Scalzi never wrote or said those words. He also never posed for the pictures that appeared with them online. The quote and the images that accompanied them were all “pretty clearly” AI generated, Scalzi wrote on his blog. “The whole vibe was off,” Scalzi told me. Although the material bore a superficial similarity to something he might have said—“it’s talking about the universe, it’s vaguely philosophical, I’m a science-fiction writer”—it was not something he agreed with. “I know what I sound like; I live with me all the time,” he noted.

What to Read Next

Sign up for Work in Progress, a newsletter in which Rogé Karma, Annie Lowrey, and others explain today’s news and tomorrow’s trends in work, technology, and culture.

For full access to our journalism, subscribe to The Atlantic.

Summer reads, games, and more

 

Your Atlantic summer is here. Read, listen, play, and find fascinating stories to inspire this season’s adventures. Subscribe today for unlimited access to all of The Atlantic, all summer long.

Subscribe