Anthropic Releases Claude 3, and Says Its Best Version Beats GPT-4

The company founded by former OpenAI researchers released three sizes of its chatbot. The largest, Opus, outscores OpenAI's flagship on the standard tests, according to Anthropic — the first time a rival has made that claim with numbers.

Anthropic, the San Francisco company started in 2021 by researchers who left OpenAI, released a new family of chatbots today under the name Claude 3, in three sizes: Haiku, Sonnet, and Opus. The company says the largest, Opus, beats OpenAI's GPT-4 on every standard test it ran — graduate-level reasoning, math, coding, and knowledge across dozens of subjects. GPT-4 has been the program to beat for a year, and this is the first time a competitor has claimed to have done it with a published scorecard.

All three programs can read images as well as text, and Anthropic says they refuse harmless requests less often than earlier versions, which had a reputation for being over-cautious. Opus is available to paying subscribers at $20 a month; Sonnet is free.

Anthropic was founded on the argument that AI needed to be built more carefully than it was being built, and it has spent much of its short life publishing safety research and lobbying governments. The release is a reminder that it is also in the race. The company has raised billions from Amazon and Google, and the programs it releases are the reason those companies wrote the checks.

The scores are Anthropic's own, and benchmark results have a way of flattering whoever publishes them. Independent testers will have a view within days. But the headline stands either way: there are now at least three companies — OpenAI, Google, and Anthropic — with programs at roughly the same level, and the year-long stretch in which one was clearly ahead appears to be over.