OpenAI Builds a Program That Writes Convincing Paragraphs, and Says It's Too Risky to Release

The San Francisco lab showed a program that continues any piece of writing you give it — a news story, a fantasy tale, a recipe — in the same style, for paragraphs at a time. It is releasing only a small version, citing the risk of fake news and spam.

OpenAI, a San Francisco research lab, showed off a program today called GPT-2 that can write. Give it a sentence or two — the opening of a news article, the first line of a story — and it continues in the same style, for paragraphs, with characters and quotes and details it makes up as it goes. The samples the lab published include a plausible news story about scientists discovering unicorns in the Andes, complete with a quote from a fictional biologist.

The program was trained by reading eight million web pages and learning to predict the next word. That is the entire method. Nobody taught it grammar, or facts, or how a news story is shaped. It picked all of that up from the reading.

What has people talking is not only what it does but what OpenAI decided to do with it. The lab is releasing only a much smaller version of the program, and says it is holding back the full one because of "concerns about malicious applications" — automated fake news, fake product reviews, spam that reads like a person wrote it. This is unusual. Research labs normally publish their work in full, and some researchers are already calling the decision a publicity stunt, or an exaggeration of what the program can do.

The writing is not perfect. Read a few samples and you find repetition, sudden changes of subject, and facts that are simply wrong. Sometimes it writes about fires happening underwater. But the good samples are good enough that OpenAI's own researchers say they were surprised, and the question they've raised — what do you do with a machine that can write like a person? — is likely to outlast this particular program.