Anthropic Sets a New Gold Standard: Your Move, OpenAI

Anthropic, a startup born out of OpenAI defectors, claims its new chatbot, Claude 3, has broken new ground for the AI world. The updated AI chatbot, released on Monday, outperforms ChatGPT and Google Gemini’s most advanced models, potentially crowning a new king in the AI world. Anthropic released three tiers – Opus, Sonnet, and Haiku – which range from most to least capable.

“Opus shows us the outer limits of what’s possible with generative AI,” said Anthropic in a blog post Monday. “Claude 3 Opus is our most intelligent model, with best-in-market performance on highly complex tasks. It can navigate open-ended prompts and sight-unseen scenarios with remarkable fluency and human-like understanding.”

Anthropic’s release of Claude 3 reasserts the startup, led by siblings Dario and Daniela Amodei, as a world leader in the AI space. The new models are equipped with Anthropic’s “Constitutional AI,” which aims to make chatbots less harmful. Claude 3, however, has lower refusal rates to controversial questions than previous models and now comes at a competitively low price point.

Claude 3 most notably outperforms ChatGPT and Gemini in coding, one of AI’s most popular early use cases. Claude Opus scores an 85% success rate in zero-shot coding, compared to GPT-4’s 67% and Gemini’s 74%. Claude also outperforms the competition when it comes to reasoning, math problem-solving, and basic knowledge (MMLU). However, Sonnet and Haiku, which are cheaper and faster, are competitive with OpenAI and Google’s most advanced models as well.

Features such as Anthropic’s Constitutional AI are becoming increasingly important in the space. Last month, Google CEO Sundar Pichai paused Gemini for its controversial responses around race which he called “completely unacceptable.”

In Gizmodo’s first trials with Claude Opus (priced at $20 a month), the AI chatbot feels comparable to an industry leader. The chatbot is capable of high-level reasoning and problem-solving. Claude 3 also navigates controversial questions with more grace than Gemini and ChatGPT.

Anthropic, however, has some limitations that other companies don’t. The company doesn’t offer image generation like some of its competitors, which is becoming a standard for AI chatbots. Claude also does not have live knowledge or the ability to surf the web. Its knowledge cutoff is August 2023.

Claude 3’s low refusal rate is noticeable when using the chatbot. Unlike Gemini, Anthropic’s chatbot can answer tough questions about race, affirmative action, and hot political topics with ease. While the chatbot won’t produce an opinion, it does a better job than others at coming up with a balanced answer. These safeguards were a central focus for the founders of Anthropic and the main reason they left OpenAI.

The world of AI startups is becoming more and more competitive. Anthropic partnered with Amazon, and it offers its AI chatbot services through Amazon Bedrock, the retail giant’s AI platform for its Amazon Web Services. Just last week, Microsoft announced a new partnership with French AI startup Mistral, while continuing its much larger partnership with OpenAI.

Anthropic sent a message to the competition with the release of Claude, dethroning OpenAI as the longtime leader in the AI world. It’s been roughly a year since OpenAI released ChatGPT-4, so it may not be long until we get an answer to Claude 3 with the long-awaited GPT-5.