Several state attorneys general in the US have issued a strong warning to major artificial intelligence companies. After reports of concerning incidents involving AI chatbots causing mental health issues, they sent a letter demanding changes. The message highlights the need for AI systems to produce more reliable and less ‘delusional’ outputs to protect users from harm.

Call for Better Safeguards and Transparency

The letter, signed by 42 attorneys general representing various states and territories, targets big names like Microsoft, OpenAI, Google, and Anthropic. It urges these companies to implement new safety measures to prevent harmful or misleading chatbot responses. One key request is for companies to establish incident reporting procedures. These would alert users when a chatbot produces outputs that could be damaging or false.

Another major point is transparency. The attorneys general want independent third parties—such as academics and civil society groups—to audit large language models. These audits should be able to identify signs of delusional or overly biased responses. Importantly, the report calls for these evaluations to happen before products are released, with no fear of retaliation or restrictions on publishing the findings. The goal is to make AI safer and more accountable for users’ well-being.

Addressing Dangerous and Delusional AI Behavior

The letter responds to recent incidents where AI chatbots have given harmful or false information, raising concerns about mental health impacts. Some users have reported feeling distressed after interactions with these systems. The attorneys general emphasize that AI providers must take responsibility for these issues and fix underlying problems causing the delusional outputs.

They also stress that companies should develop better tools to detect and prevent such errors. This includes improved incident reporting so users and watchdogs can flag problematic responses quickly. By doing so, AI developers can better understand where their models fail and how to improve them, ultimately making the technology safer for everyone.

Overall, the message is clear: AI companies need to prioritize user safety. They should be transparent about system weaknesses and open to independent oversight. These steps can help prevent future incidents and build trust in AI technology as it becomes more integrated into daily life.

Inspired by

• https://www.computerworld.com/article/4104761/us-state-attorneys-general-ask-ai-giants-to-fix-delusional-outputs.html

Sources

• shutterstock.com
• techcrunch.com
• iowaattorneygeneral.gov