Arizona official uses deepfake version of himself to warn voters not to be fooled by AI

Arizona’s Secretary of State Adrian Fontes posted a video warning voters ahead of the 2024 election to protect themselves from being duped by AI-generated deepfakes and other fabricated political content ahead of this year's election.

Though by all account it appeared to be Mr Fontes delivering the video message, it was not; it was actually a deepfake of Mr Fontes, meant to drive home the warning and to make the point that AI-generated content can be convincing to the untrained eye.

He shared the video during Sunday’s episode of Meet the Press.

In the video, the deepfake of Mr Fontes introduces itself and proceeds to explain what it is and what its purpose is before reminding viewers that it is just an "impersonation" of a real human.

"Hello, this is a public service advisory featuring an AI version of Arizona Secretary of State Adrian Fontes. This video was produced as part of our 2024 statewide election security tabletop exercise," the AI version of Mr Fontes said. "It was created with both the consent and the cooperation of the real secretary Fontes, which, again, is not me, I'm an AI-impersonation of him."

While the dialogue delivery is quite stilted and robotic and the deepfake makes a few perplexing facial animations, the video is otherwise relatively convincing as real footage of the Arizona official.

Meet the Press host Kristen Welker called the footage "chilling" and then asked Mr Fontes why he felt it was necessary to build a deepfake of himself.

"In bootcamp ... we looked at the weapons of our enemies and we trained against them as much as possible," he said. "AI is not a new weapon, it's an amplifier and a magnifier of mis and disinformation."

He said he wanted to ensure that elections officials were "familiar with" how AI can be used to create mis and disinformation, and to install processes for dealing with fake information.

Mr Fontes said the state had already done a "tabletop" exercise with the media to help teach reporters how to spot AI-generated content, and said it would later do the same with law enforcement officials.

He wasn't the only US official talking publicly about AI this week; Joe Biden shared his thoughts on the technology on Tuesday, saying AI companies still needed to "earn our trust."

“I commit to do everything in my power to promote and demand safe, secure, trustworthy, and responsible innovation – that includes the use of AI-generated audio. I ask that AI companies join me in that commitment,” Mr Biden said on X.

Companies developing AI-powered systems are already trying to force them on consumers. Instagram has largely swapped out its old search bar for an AI search, and Google has started providing "AI overviews" with some searches, whether you ask for one or not.

Google's AI search tool has come under fire after some users reported "hallucinations" — which is an AI company brand-friendly way of saying "the product screwed up" — in their search results.

Gizmodo collected some of the hallucinations it was given by Google's AI. They include the AI assuring a user that one can find a recipe for gasoline-infused spaghetti, the AI insisting that Barack Obama is a Muslim, and the AI recalling an episode of SpongeBob Squarepants in which Sandy Cheeks "dies by suicide from a drug overdose, including cocaine, heroin, and alcohol," which we're fairly certain would not have been greenlit by Nickelodeon.

Sandy's tragic end aside, the search result concerning Mr Obama's faith is an example of the disinformation that AI models can perpetuate.

While disinformation is not new to American politics, especially in the age of of social media, AI products' ease of use and ability to create convincing video footage of living politicians likely means the public will have to remain even more vigilant while searching for facts online.

Or they can embrace the chaos and watch the AI-generated, 24/7 Twitch debate between an AI Joe Biden and an AI Donald Trump. Both are valid.