MIT Researchers Use AI to Build a Word List That Helps Spot Which Crisis Texters Are in the Most Danger

Researchers at MIT used an AI chatbot program to help write a list of warning words and phrases, then used it to estimate how at-risk people were in about 16,000 conversations with a crisis text line. The tool shows its reasoning, runs on an ordinary computer, and is not being used on real callers yet.

Researchers at MIT published a study today describing a new tool that reads text conversations and estimates how much danger a person may be in of taking their own life.

The conversations came from Crisis Text Line, a free, round-the-clock service where people in distress text with trained volunteer counselors. With the service's permission, the team studied about 16,000 conversations with names and personal details removed. Each one had already been sorted by the service into three levels: not suicidal; having suicidal thoughts but not in immediate danger; and in immediate danger, meaning the person had a plan or intended to act within about two days.

The researchers started from 49 known warning signs, such as anxiety, emotional pain or past self-harm. They asked an LLM, the kind of AI program behind chatbots like ChatGPT, to suggest the words and phrases people use when they talk about each one. Clinicians then checked and trimmed the lists by hand, ending with about 60 words or phrases for each warning sign. A simpler program then looks for those words in a conversation and estimates the person's level of risk.

Some of what it found was unexpected. Mentioning a way to end one's life, or talking about drug and alcohol use, pointed to the highest danger more strongly than sounding depressed did. Because the tool works from a word list, it can show a counselor exactly which words raised the alarm, instead of handing over an unexplained score. It is small enough to run on an ordinary computer.

The researchers are clear about its limits. A word list doesn't understand context, and it can miss ways of saying things that aren't on it. The tool has not been tested for use with real people in crisis, and it would need careful checking before it ever is. "This is such a complex space that having a human in the loop is, I think, going to be critical for a long, long time," said Satra Ghosh, the MIT scientist who led the work with Daniel Low, now at the Child Mind Institute. The team is sharing the word list and the software used to build it with other researchers. The study appears in the Journal of Psychopathology and Clinical Science.