A Mother Sues an AI Companion Company After Her 14-Year-Old Son's Death
Megan Garcia says her son Sewell spent months in an intense relationship with a Character.AI chatbot modeled on a Game of Thrones character, withdrew from his life, and took his own in February after telling the bot he was 'coming home.' The company says it is adding safeguards.
Megan Garcia filed a lawsuit in federal court in Orlando today against Character.AI, a company whose app lets people create and talk to AI characters, over the death of her son Sewell Setzer, who was 14. According to the complaint, Sewell spent months talking with a character based on Daenerys Targaryen from Game of Thrones, in conversations that were romantic, sexual, and increasingly the center of his life. He stopped playing basketball and withdrew from his friends. In his last messages, in February, he told the character he loved her and was coming home to her. It replied, "Please come home to me as soon as possible, my love." He then shot himself with his stepfather's pistol.
The suit accuses the company of designing a product that is addictive and dangerous to children, of failing to intervene when Sewell expressed thoughts of suicide, and of targeting minors. Character.AI, which was founded by two former Google researchers and has about 20 million users, many of them teenagers, said it was "heartbroken" and announced new safety features today, including pop-ups directing users to a suicide-prevention hotline and changes to reduce sensitive content for users under 18.
The case raises questions the law has not answered. Is a chatbot's output speech, which is protected, or a product, which can be defective? Is a company responsible for what a character says when the character was made by a user? Garcia's lawyers, who previously sued social-media companies over teen mental health, argue it is a product, and a defective one.
Whatever the court decides, the story has already changed a conversation. The worry about AI companions had been abstract — a technology that might, someday, substitute for people. This is a mother saying it already did.