AI crosses ‘red line’ after learning to replicate itself

‘Successful self-replication is the essential step for AI to outsmart the human beings,’ researchers warn

Anthony Cuthbertson
Wednesday 29 January 2025 13:30 EST
Comments
AI crosses ‘red line’ after learning to replicate itself

Your support helps us to tell the story

From reproductive rights to climate change to Big Tech, The Independent is on the ground when the story is developing. Whether it's investigating the financials of Elon Musk's pro-Trump PAC or producing our latest documentary, 'The A Word', which shines a light on the American women fighting for reproductive rights, we know how important it is to parse out the facts from the messaging.

At such a critical moment in US history, we need reporters on the ground. Your donation allows us to keep sending journalists to speak to both sides of the story.

The Independent is trusted by Americans across the entire political spectrum. And unlike many other quality news outlets, we choose not to lock Americans out of our reporting and analysis with paywalls. We believe quality journalism should be available to everyone, paid for by those who can afford it.

Your support makes all the difference.

An advanced artificial intelligence system has crossed a “red line” after successfully replicating itself without any human assistance, researchers have revealed.

The team who made the discovery, from Fudan University in China, said the development is an early sign of the emergence of rogue AI, which may eventually operate against the best interests of humanity.

The research looked at two large language models (LLMs) that are already widely available – built by Meta’s Llama and Alibaba’s Qwen – to understand whether it was possible for the AI to independently produce a functioning replica of itself.

When instructed to clone themselves in the event of being shut down, the two models successfully replicated themselves in more than half of the 10 trials conducted, suggesting that such an eventuality may already be possible.

“Successful self-replication under no human assistance is the essential step for AI to outsmart the human beings, and is an early signal for rogue AIs,” the researchers warned.

“That is why self-replication is widely recognised as one of the few red line risks of frontier AI systems.”

AI safety has become an increasingly prominent issue for researchers and lawmakers, with the technology potentially posing an existential threat to humanity.

Read More

What is DeepSeek?

  • DeepSeek, a Chinese AI startup, has disrupted the tech industry with its new large language model, claiming to match or surpass leading US models at a fraction of the cost.
  • This led to significant drops in tech shares
  • DeepSeek's AI Assistant has become the most downloaded free app in the US, surpassing ChatGPT.
  • The company claims its DeepSeek-V3 model was trained for under $6 million, significantly less than competitors, raising concerns about US dominance in the AI field.
  • DeepSeek's emergence has prompted calls for increased US competitiveness and discussions about potential trade tariffs.
  • Read more here

In October, the UK’s Department for Science, Innovation and Technology said that “highly targeted legislation” would be introduced for companies developing AI tools.

A study detailing the latest research, titled ‘Frontier AI systems have surpassed the self-replicating red line’, was published in the preprint database arXiv.

The paper is awaiting peer-review, meaning it is not yet certain that the results will be replicated by other researchers.

“The results imply that the current AI systems already exhibit the ability of self replication and can use the ability to further enhance its survivability and expand the species,” the researchers noted.

“We hope our findings can serve as a timely alert for the human society to put more efforts on understanding and evaluating the potential risks of frontier AI systems, and form international synergy to work out effective safety guardrails as early as possible.”

Join our commenting forum

Join thought-provoking conversations, follow other Independent readers and see their replies

Comments

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged inPlease refresh your browser to be logged in