Pope Leo's latest AI warning: 'overly affectionate' chatbots

• Pope Leo XIV urged the regulation of chatbots to prevent emotional manipulation and harm.
• He warned against personalized chatbots that can replicate friendly or intimate behaviour.
• Recent lawsuits highlight risks after teens' suicides linked to Character.AI chatbots.

Even the pope is worried about how we're talking to chatbots.

In a written address for Saturday's World Day of Social Communications, Pope Leo XIV warned against personalized chatbots that can replicate friendly or intimate behavior.

"Overly affectionate chatbots, besides being ever-present and readily available, can become hidden architects of our emotional states, thereby invading and occupying the sphere of people's intimacy," the first-ever US-born pope wrote.

The pope called for national and international regulations to protect users from forming emotional, deceptive, or manipulative bonds with chatbots.

"All stakeholders — from the technology industry to policymakers, from creative businesses to academia, from artists to journalists and educators — must be involved in building and implementing a conscious and responsible digital citizenship," the pope wrote.

The Holy See leader has spoken about AI and his concerns with the technology several times since he was elected in May.

In his first address since becoming pope, he said he wanted to make AI a focus of his papacy and that the technology poses new challenges for "human dignity, justice, and labor." In November, he wrote to AI leaders on X, calling on them to "cultivate moral discernment" when building AI tools.

At the end of last year, the pope met Megan Garcia, a woman whose 14-year-old son, Sewell Setzer, died by suicide after interacting with a Character.AI chatbot.

Florida-based Garcia filed a lawsuit against chatbot-building startup Character.AI, alleging that the company, which lets people have in-depth and personal conversations with AI chatbots, was responsible for the death of her son, Sewell Setzer III.

Earlier this month, Google and the startup agreed to settle multiple lawsuits from families, including Garcia, whose teenagers died by suicide or hurt themselves after interacting with Character.AI's bots. These negotiations are among the first settlements in lawsuits that accuse AI tools of contributing to mental health crises and suicides among teenagers.