Generative artificial intelligence has brought us the Pope in a coat and deepfake Drake, but also a potentially immense problem in separating fact from fiction. Today, anyone can create photorealistic images with Midjourney, churn out prose with ChatGPT or clone a voice in minutes.
These tools will inevitably be used – and are already being used, in some cases – to perpetuate financial scams, hoaxes and spread misinformation to sway public opinion and divide populaces. There are no easy fixes, and you could be on your own. So how good are you at spotting fakes?
Look at these examples and pick which one you think is made by AI:
We’ve already seen many examples of how generative AI can be misused. Deepfake pornography, in which a person’s likeness is used without consent, started proliferating a few years ago. In September, Microsoft said it found China-affiliated actors spreading AI-generated images on divisive topics, such as gun control. “We can expect China to continue to hone this technology over time, though it remains to be seen how and when it will deploy it at scale,” Microsoft Corp. researchers wrote.
Bill Browder, a frequent critic of Russia, found himself on a video call with former Ukrainian president Petro Poroshenko in May, only to belatedly realize it was a “clumsy but convincing” deepfake aimed at goading him into saying something embarrassing.
“I don’t think we’ve seen the power that AI can have in terms of spreading misinformation yet,” said Jack Brewster, enterprise editor with NewsGuard Technologies Inc., which provides tools to train AI models to be more accurate. “There’s more room for this technology to advance.”
Image generators have improved significantly in recent months. Wonky hands used to be a dead giveaway that a picture was AI-generated, but after an update earlier this year, Midjourney became more skilled at rendering human appendages.
An image generated by Midjourney text-to-image AI generator of a Shakespearian character to illustrate the poor human anatomy such generators are known to create.AI-GENERATED IMAGE VIA MIDJOURNEY
A number of startups have launched in recent months to identify AI-generated images. Andrey Doronichev was prompted to build a detection tool earlier this year through his San Francisco-based company, Optic, after coming across a deepfake video of Ukrainian President Volodymyr Zelensky.
Generative AI applications such as Dall-E and Midjourney leave artifacts in the pictures they create – patterns that we cannot see. An algorithm trained on a huge volume of these images can detect those hallmarks, even if we don’t know specifically what they are, Mr. Doronichev explained.
Detection algorithms are not entirely reliable, however, and require retraining whenever an update is made to an image generator or when a new model is released. “This is what makes this job hard, and potentially endless,” Mr. Doronichev said.
Determining whether the text you’re reading was generated by AI may be even more challenging than assessing images, given how writing styles can differ from one person to another. Still, NewsGuard has identified more than 450 so-called “news” websites that consist entirely or mostly of AI-written articles, in multiple languages, replete with false claims, fictitious news and old events presented as new.
Companies such as OpenAI, maker of ChatGPT, have put guardrails in place to prevent misuse, but some users have easily bypassed these restrictions, including to produce blatant falsehoods. NewsGuard found that GPT-3.5, the large language model that powered the first iteration of ChatGPT, would not generate false news articles about China in English, but it did so in traditional and simplified Chinese languages.
“We’ve seen this over and over, with content moderation on big platforms being much less efficient in other languages,” said Chine Labbe, vice-president of partnerships for Europe and Canada at NewsGuard.
Who wrote the following article about Justin Trudeau’s election in 2015: The Globe and Mail or ChatGPT?
One of the statements below was written by AI, the other was spoken by Telus chief executive Darren Entwistle. Which one is AI?
Did Bank of Canada governor Tiff Macklem provide the following explanation for raising interest rates? Or did ChatGPT?
Services to identify AI-generated text have proliferated over the past few months, with varying degrees of accuracy. OpenAI’s own tool correctly identified only 26 per cent of AI-written text, and falsely branded human prose as being machine-generated 9 per cent of the time, according to the company. In July, OpenAI shut down the tool because of its “low rate of accuracy.”
A study published in May by researchers at Stanford University found that AI text detectors were fairly reliable when analyzing essays written by eighth graders, but incorrectly labelled more than half of those penned by non-native English speakers as AI-generated. The researchers also found detectors could be easily fooled by prompting ChatGPT to rewrite text in a more literary style.
Jon Gillham launched a company called Originality.AI in Collingwood, Ont. last November for publishers and content marketers to assess whether copy was written by AI. Academic institutions have flocked to the service, too. “We don’t love that use case. There’s a lot more hair on it,” Mr. Gillham said, referring to the possibility of students being falsely accused.
The company uses an AI model trained on reams of copy to distinguish between human and machine-generated text, but Mr. Gillham can’t exactly say how it does so. “AI sees patterns that we don’t see,” he said.
The results are not always definitive, however. When The Globe inputted a public address by Bank of Canada Governor Tiff Macklem, Originality.AI determined that 40 per cent of it was machine-generated, denoting entire paragraphs as having been written by AI.
Mr. Gillham speculated that the writing style, replete with short sentences, was somewhat uncommon, which could account for the drop in accuracy. It’s also possible the Bank of Canada really did use ChatGPT, he said.
A spokesperson for the Bank of Canada said it does not, in fact, use ChatGPT or any other AI tool to author public statements.
Services that rewrite AI text in order to fool detectors have also popped up too, sparking a constant arms race between those trying to identify the hallmarks of AI, and those trying to thwart them.
AI models are now powerful enough to clone somebody’s voice based on only a snippet of audio. Microsoft researchers unveiled a model earlier this year that can mimic a voice with only three seconds worth of input.
Meta Platforms then trumped Microsoft in June when it announced a model that needed just two seconds worth of audio. While the company said it has built a “highly effective” classifier to distinguish between genuine and AI-created audio, Meta did not release the model or code to the public because of the “potential risks of misuse.”
Scammers have already adopted existing AI voice cloning tools to dupe the public. Some fraudsters have used AI to clone the voice of a victim’s family member, pretending to be in distress to convince the target to send money. A 75-year-old woman in Regina shelled out more than $7,000 recently after receiving a phone call from someone who sounded like her grandson, claiming to need money for bail.
Listen to these examples and pick the one you think is made by AI:
To root out deepfakes, some companies are trying to co-ordinate specifications to watermark digital media. “This has to be done at the source,” said Hany Farid, a professor at the University of California, Berkeley, who has expertise in digital forensics. “Ideally what would happen is your web browser would be enabled, and it would read the watermark for you.”
An initiative called the Coalition for Content Provenance and Authenticity, for example, counts Adobe Inc., Microsoft, Intel Corp. and camera manufacturers as members. But such efforts require widespread buy-in, and open-source AI models (as opposed to those owned and operated by corporations) won’t necessarily adopt watermarking standards.
Alberto Fittarelli, a senior researcher with Citizen Lab, suspects digital fingerprinting could be easily circumvented, too. “Models will likely be reverse-engineered and replicated by an array of actors in the future, and the malicious ones will certainly not adhere to such regulations.”
Detection services, meanwhile, will constantly be playing catch-up. “As detection tools and techniques will evolve and improve over time, so will AI generative models and their ability to become unrecognizable,” Mr. Fittarelli said. “Unfortunately, I suspect that the biggest investment will be in developing the latter.”
Videos made with AI tools are easy to spot today. They’re glitchy, awkward and vaguely psychedelic. But if the improvement in AI-generated images is any indication (along with the money pouring into the sector), generated video is only set to get better.
What does a cat eating pizza, or goldfish on the moon look like? See samples of what RunwayML's AI video generator can create from a text prompt or still image.
Despite the widespread concern, some experts argue that generative AI on its own may not lead to more misinformation and scams. Arvind Narayanan, a computer science professor at Princeton University, has said that while creating misinformation is easier and cheaper, spreading it widely is not.
“The cost of distributing misinformation by getting it into people’s social media feeds remains far higher than the cost of creating it,” he wrote recently along with Sayash Kapoor, a Princeton PhD candidate. “In other words, the bottleneck for successful disinformation operations is not the cost of creating it.”
That puts more pressure on social media platforms to invest in fact-checking and content moderation, but also suggests the potential fixes are not all technical in nature. We all have to educate ourselves on what generative AI is capable of, maintain skepticism, and rely on trusted sources for information.
All source images and the underlying animations for the cloned voice videos are from Getty.
How The Globe’s senior visuals editor made the AI content in this story
Making AI content with the intent to deceive was a challenge. I did photo research first to find source images that a generative AI would be able to mimic. That meant detailed close shots of faces or machinery were out in favour of more general scenes that would still be recognizable.
I’ve used Midjourney the most so it was my main tool to make the fake images for this story. My prompts were in plain language without elaborate commands. I iterated on the versions that Midjourney created to work toward a final image that could pass as realistic. Bing’s images came with a watermark that I removed using Photoshop.
A set of four initial images created by the Midjourney text-to-image AI generator.AI-GENERATED IMAGE VIA MIDJOURNEY
Voice cloning is extraordinarily capable and hard to notice without the cues to fakery that are in images or video. The llElevenLabs system we used did well with Joe Castaldo and Janice Dickson’s North American accents but wasn’t able to match my Australian accent. Be on guard: the next spam call you get could be using a voice clone to imitate a relative or celebrity.
AI video generation has improved significantly in just a few months. The RunwayML system I used can make a video from a still image, as well as a text prompt. The level of detail and motion is very good but not fully realistic yet. While videos made this way have that “uncanny valley” feeling, some creators are embracing the technique to make short films entirely from AI-generated assets.
– Patrick Dell