Google's Medical AI Chatbot Talks With Real Patients Before Their Doctor Visits, in a Study Published in a Leading Medical Journal
In a hospital clinic, 98 patients described their symptoms to a Google AI chatbot in the days before seeing their doctor, while a physician watched every conversation live. Nobody had to step in, the doctors said the chat summaries helped them prepare, and the study now appears in The Lancet. It was small, with no comparison group, and Google paid for it.
Google and doctors at Beth Israel Deaconess Medical Center, a large teaching hospital, published a study today in The Lancet, one of the world's most respected medical journals. It tested a research chatbot called AMIE with real patients rather than actors or made-up cases. Google says it is the company's first paper in the journal's main edition. AMIE is not available to the public.
The setup was simple. Patients who had booked a non-emergency appointment at the hospital's primary care clinic about a single new problem were invited to chat by text with AMIE up to five days beforehand. The chatbot asked about their symptoms and history, the way a doctor would at the start of a visit. A physician watched every conversation live over a video link and could stop it if the patient seemed at risk, became upset, or asked to end it. In 98 conversations, the physician never had to stop one. With the patient's permission, their own doctor received the chat transcript and a summary before the appointment.
The doctors said the summaries helped them prepare in 75% of cases and changed how they approached the visit in more than half. One described a shift from gathering information to checking it. To test the chatbot's judgment, the researchers looked at each patient's records eight weeks later to see what the problem had turned out to be. The real answer was somewhere on AMIE's list of possible explanations 90% of the time, and it was the chatbot's first guess 56% of the time. Doctors who weren't involved, and didn't know which plan came from whom, rated AMIE's lists and treatment plans about as good overall as the patients' own doctors'. But the human doctors' plans were more practical and more mindful of cost. Patients came away feeling more positive about AI than before.
The researchers are clear about the limits. It was one clinic, with no comparison group of patients who skipped the chatbot, so it can't show that patients actually did better. The chatbot worked only through typed text: it couldn't see the patient, examine them, or read their medical records, which the authors say is part of why its plans were less practical. The patients also skewed younger than the clinic's usual crowd. Outside commentators note how small the numbers are, under a hundred conversations, with the doctors' feedback coming from only some of them. Google paid for the study, and the early results were first posted online in March before other experts reviewed them for the journal.
The authors call it a first step and say larger trials are needed before chatbots like this talk to patients on their own. What it shows today is narrower: with a doctor watching, an AI program can hold a safe, useful conversation with a real patient before the visit.
Sources
- Google — Study in The Lancet suggests AI could improve patient-physician relationshipsblog.google · primary
- Google Research — Exploring the feasibility of conversational diagnostic AI in a real-world clinical studyresearch.google · primary
- arXiv — A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic (preprint)arxiv.org · primary
- FourWeekMBA — Google AMIE chatbot ran 98 patient chats, zero safety stopsfourweekmba.com
- News-Medical — Google's AI medical assistant in a real clinic study (on the March preprint)news-medical.net