Senator Kristen Gonzalez’s Leading Chatbot Liabilities Legislation Included in 25-26 State Budget

ALBANY, NY – Included in the 2025-2026 New York State Budget are new guardrails for the safe and responsible operation of companion chatbots. Provisions from two bills, S5668 and S934 first introduced in the State Legislature by Senator Kristen Gonzalez (D-59) in 2024’s legislative session. One is meant to address the unmitigated risks users face when utilizing chatbots for social interactions, as well as recent news stories that highlighted the deeply troubling consequences of these risks. 

Companion chatbots, which are designed to maintain social engagement with its users, are often used by vulnerable people of all ages looking for someone to talk to. This provision in the 2025 State Budget requires that companion chatbot operators must:

• Include a protocol that detects users’ suicidal ideation or expressions of self-harm, directing such users to crisis services, and
• Display a notification reminding users that they are not interacting with a human, at least once daily and for ongoing chats lasting more than 3 hours.

Senator Kristen Gonzalez, Chair of the Internet and Technology Committee said, “As large language models of artificial intelligence continue to develop, we’re seeing more and more people turn to chatbots for everyday needs, including basic social interactions. To embrace technological advancement, we need to ensure that vulnerable users are protected from the risks of digital companionship over direct support from real-people they can trust.”

Failure to adhere to these requirements will result in enforcement action by the Attorney General’s Office, with any related fines and penalties collected for violations committed to a suicide prevention fund within the Office of Mental Health.