Many people hope that AI chatbots can help with medical questions, but new research suggests they’re not quite ready for prime time. A recent study from the University of Oxford found that these tools are not much better than regular search engines when it comes to health advice. In fact, they often perform worse than what you’d expect from a typical internet search.

Study Highlights Limitations of AI in Healthcare

The study involved nearly 1,300 participants in the UK. One group used AI chatbots like GPT-4, Llama 3, and Command R to evaluate symptoms and suggest next steps. The other group relied on their usual methods, such as search engines or personal knowledge. The results showed that those using AI didn’t do any better at judging how urgent a health issue was. They also struggled more to correctly identify the actual medical condition.

The researchers pointed out two main issues. First, users often found it hard to give the chatbots enough or relevant information. Second, the AI models sometimes gave conflicting advice or, worse, outright wrong answers. This makes relying on them risky, especially in sensitive situations like health emergencies.

Why AI Isn’t Ready for Medical Advice

The study also revealed that traditional tests for AI, like medical quizzes, don’t really reflect how people use these tools day-to-day. Passing a test doesn’t mean an AI can safely handle real-world health advice. When people interact with AI in real life, they often provide incomplete information, and the AI may not handle it well.

Furthermore, AI models sometimes produce inconsistent or incorrect responses, which can be dangerous. This shows that current AI chatbots lack the reliability needed for medical decision-making. The researchers emphasize that these tools are still far from being safe, dependable medical advisors for the general public.

Overall, while AI chatbots are impressive in many ways, their medical advice capabilities need significant improvement before they can replace or even supplement professional healthcare providers. For now, traditional search engines and consulting with qualified doctors remain the safest options for health concerns.

Inspired by

• https://www.computerworld.com/article/4130361/ai-chatbots-worse-than-search-engines-for-medical-advice.html

Sources

• ac.uk
• theregister.com