OpenAI Releases GPT-4, a Program That Scores in the Top Tenth on the Bar Exam and Can Read Pictures

The successor to the program behind ChatGPT passes a simulated bar exam around the 90th percentile, up from the bottom tenth for its predecessor, and can answer questions about an image. Paying ChatGPT subscribers get it today.

OpenAI released GPT-4 this morning, the next version of the program that powers ChatGPT. It is available today to people who pay $20 a month for ChatGPT Plus, and to companies through a waiting list. Microsoft confirmed within hours that the new Bing has been running on it for weeks.

The company's report leads with exam results, which is a telling choice. GPT-4 passed a simulated bar exam with a score around the top 10 percent of test-takers; the previous version scored around the bottom 10 percent. It scored a 5, the highest mark, on Advanced Placement tests in biology, art history, and statistics. It did poorly on AP English, which is either a comfort or a puzzle.

The other change is that it can see. Give it a photograph and it will describe what's in it, explain why a picture is funny, or — in OpenAI's demonstration — turn a pencil sketch of a website into a working web page. The image feature is not yet open to the public.

OpenAI's report is careful about what it does not include: how big the program is, what it was trained on, how it was built. The company says competition and safety are the reasons, which is a departure from a lab founded on openness. It also says the program still makes things up, still has "social biases," and can be talked into things it shouldn't do, and that it spent six months testing it before release. Sam Altman, the chief executive, said it is "still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it."

Anthropic, a company founded by former OpenAI researchers, released its own chatbot, called Claude, the same day.