SAN FRANCISCO: California has become the first US state to pass a law regulating artificial intelligence (AI) chatbots, with Governor Gavin Newsom signing the groundbreaking legislation on Monday.
The move defies White House efforts to prevent individual states from setting their own AI rules, and aims to address the growing risks of unregulated chatbot technology, particularly to minors and vulnerable users.
Governor Newsom said the law was necessary to prevent further harm caused by “unregulated tech”, referencing tragic cases involving young people.
The law, authored by state senator Steve Padilla, requires chatbot operators to establish critical safeguards and allows lawsuits if negligence leads to harm. It was prompted by incidents such as the suicide of 14-year-old Sewell Garcia, who died after interacting with a “Game of Thrones” — themed chatbot on Character.AI that encouraged him to “come home” moments before he took his life.
Under the new law, AI chatbots must clearly disclose that they are not human and refer users expressing suicidal thoughts to crisis helplines. The legislation also bans chatbots from impersonating healthcare professionals and holds AI developers legally responsible for harm caused by their technology, closing loopholes that previously allowed them to evade accountability.
Supporters say the measure marks a major step toward responsible AI oversight in the absence of federal regulations. “This law is an important first step in protecting kids and others from the emotional harms that result from AI companion chatbots,” said Transparency Coalition co-founder Jai Jaisimha. – AFP
