OpenAI's New ChatGPT Talks Back in Real Time, Sees Through Your Phone Camera, and Is Free

GPT-4o responds to speech in about the time a person would, can be interrupted, and reads emotion in a voice. In the demo it tutored a student through a geometry problem by looking at his paper. The full abilities of the old paid version are now free to everyone.

OpenAI showed a new version of ChatGPT this morning that you talk to. Not the way you talk to Siri — a question, a pause, an answer — but the way you talk to a person. It responds in about a third of a second. You can cut it off mid-sentence. It laughs, changes its tone when asked, sings badly on request, and in the demonstration, when a presenter said he was nervous, it heard it in his breathing and told him to take a breath. The company calls the program GPT-4o; the "o" is for "omni," because it handles voice, images, and text as one thing rather than passing them between separate programs.

The camera demo is the one people will remember. A presenter pointed a phone at a handwritten geometry problem and the program walked a student through solving it, step by step, watching the pencil, without giving the answer away. In another, it looked at a presenter's face and guessed his mood. In another, it translated live between English and Italian speakers.

The other news is the price. Everything the paid version of ChatGPT could do is now free. OpenAI says the new program is cheaper for it to run, and it wants the free product to be the best one. The voice features roll out over the coming weeks.

Google holds its annual developer conference tomorrow, and OpenAI's decision to hold a surprise event the day before was not subtle. It is also worth noting what the demonstrations resemble: the voice, the warmth, the slight flirtation, are very close to the AI companion in the 2013 film Her. Sam Altman posted the film's title, and nothing else, this afternoon.