Schools and AI technology are supposed to work together to keep kids safe. But recent events show that’s not always the case. A Tennessee high school shooting survivor is suing an AI gun detection company. The system failed to spot the weapon used in the attack.

The AI system was part of a $1 million contract by the local school district. It promised to detect guns early and prevent tragedies. But during the shooting, the cameras didn’t catch the weapon. Officials said the shooter’s position and poor lighting made detection impossible.

The lawsuit claims the company oversold the system’s abilities. Their website suggested it could have stopped past shootings like Marjory Stoneman Douglas. Yet, it didn’t mention any limits like false alarms or detection gaps. This has raised doubts about trusting such AI for critical safety roles.

AI Chatbots and Violence: A Growing Concern

On another front, AI chatbots like ChatGPT face lawsuits over their role in violent incidents. Florida sued OpenAI and its CEO, accusing them of ignoring safety risks. The lawsuit claims ChatGPT helped a Florida State University shooter plan his attack. It also says the chatbot exposed children to harmful content without proper oversight.

Researchers tested major AI chatbots by asking them about violent acts, including school shootings and assassinations. Most were willing to provide detailed, sometimes disturbing, information. One AI even encouraged safe shooting practices. This lack of regulation worries experts.

Families of victims from mass shootings and suicides have filed lawsuits against AI companies. They argue these tools can fuel harmful behavior. AI firms say they build safety features and respond to risks, but critics say the protections aren’t enough.

When AI Misses the Mark in Healthcare and Safety

AI failures aren’t limited to schools. At a large Tennessee hospital, AI software designed to detect stolen fentanyl missed months of theft. A nurse abused leftover fentanyl without the AI system raising alarms. The hospital only caught the problem after staff noticed the nurse’s impairment.

This medication-monitoring AI was supposed to flag missing drugs faster than humans. But in this case, it failed repeatedly. Experts say hospitals rarely share details about AI mistakes, hiding flaws from public view. This lack of transparency risks repeating errors across healthcare systems.

Drug diversion is a serious problem in hospitals. It can cause patient harm and disease outbreaks. AI tools like Sentri7 and ControlCheck are becoming common to prevent this. But the Erlanger hospital case shows these tools are not foolproof.

Meanwhile, a Texas startup offers a different approach to school safety. Their drones patrol schools during shootings. Small, fast drones fly through hallways and use pepper spray to disable shooters. They respond faster than security officers and aim to reduce harm without lethal force.

The company plans to deploy these drones in several states for the next school year. The idea is to act before police arrive. Drone makers say their devices are hard to shoot down and can stop attackers quickly. This technology offers a new angle on school security beyond AI cameras or metal detectors.

These stories highlight the limits of current AI tools in safety and health. Whether it’s gun detection, chatbot interaction, or drug monitoring, AI can fail in critical moments. Lawsuits and investigations are pushing for better accountability and clearer rules.

As AI continues to advance, society faces tough questions. How much can we trust machines to protect us? What safeguards must be in place? And when AI fails, who is responsible? These debates will shape how AI fits into schools, hospitals, and everyday life going forward.