ChatGPT rival passes university exam
Professor says Claude AI answers questions in law and economics test ‘better than many humans’
Your support helps us to tell the story
From reproductive rights to climate change to Big Tech, The Independent is on the ground when the story is developing. Whether it's investigating the financials of Elon Musk's pro-Trump PAC or producing our latest documentary, 'The A Word', which shines a light on the American women fighting for reproductive rights, we know how important it is to parse out the facts from the messaging.
At such a critical moment in US history, we need reporters on the ground. Your donation allows us to keep sending journalists to speak to both sides of the story.
The Independent is trusted by Americans across the entire political spectrum. And unlike many other quality news outlets, we choose not to lock Americans out of our reporting and analysis with paywalls. We believe quality journalism should be available to everyone, paid for by those who can afford it.
Your support makes all the difference.An artificial intelligence bot similar to OpenAI’s ChatGPT has achieved a passing grade in a university exam, an economics professor has revealed
The Claude AI, developed by the research firm Anthropic, earned a “marginal pass” on a blind graded law and economics test, which examiners described as “better than many human” candidates.
Professor Alex Tabarrok from George Mason University said he believed the Claude AI was an improvement on the artificial intelligence built by OpenAI, though still had notable flaws compared to the best human students.
The professor cited an example of an answer given to a question about how the law surrounding intellectual property could be improved.
The Claude AI offered a 400-word answer containing five main points detailing potential improvements to IP laws, however it reportedly lacked clear reasoning.
“The weakness of the answer was that this was mostly opinion with just a touch of support,” Professor Tabarrok said.
“A better answer would have tied the opinion more clearly to economic reasoning. Still a credible response and better than many human responses.”
The latest AI achievement comes amid a wave of hype surrounding natural language models following the public release of ChatGPT last year.
OpenAI’s technology made headlines for its ability to offer human-like responses to a vast range of queries, from explaining complex scientific concepts in simple terms, to coming up with new ideas for a business proposal.
It’s apparent abilities have led some schools and universities to ban it from computers and devices in an attempt to prevent cheating, while some fear it contains inherent biases that are common for AI algorithms trained on human-generated data sources.
OpenAI CEO and co-founder Sam Altman claims the technology is currently “incredibly limited” but good enough to “create a misleading impression of greatness”.
Mr Altman said it would be “a mistake to be relying on it for anything important right now”, however warned that future iterations will soon make ChatGPT “look like a boring toy”.
Anthropic is one of a number of competitors in a field that includes Google’s unreleased LaMDA and Sparrow chatbots.
The startup was co-founded by former employees of OpenAI, with head-to-head comparisons between ChatGPT and Claude finding that they both have strengths in different areas.
“Overall, Claude is a serious competitor to ChatGPT, with improvements in many areas,” researchers noted in a recent test of the two AI bots.
“Claude’s writing is more verbose, but also more naturalistic. Its ability to write coherently about itself, its limitations, and its goals seem to also allow it to more naturally answer questions on other subjects. For other tasks, like code generation or reasoning about code, Claude appears to be worse.”
Join our commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies
Comments