ChatGPT’s Advice Results in Hospitalization

A 60-year-old man seeking to reduce his table salt intake turned to ChatGPT for dietary advice, leading to an unexpected hospitalization. The AI suggested replacing sodium chloride with sodium bromide, a chemical compound toxic for human consumption. This recommendation, likely intended for non-dietary purposes such as cleaning, resulted in the man experiencing symptoms of bromism after three months of use.

Understanding Sodium Bromide and Its Risks

Sodium bromide, once used as an anticonvulsant and sedative, is now primarily utilized in cleaning and manufacturing. Its resemblance to salt can be misleading, but it poses significant health risks when ingested. The man’s symptoms included fatigue, insomnia, poor coordination, and paranoia, all indicative of bromism. His condition escalated to auditory and visual hallucinations, necessitating psychiatric intervention.

The Role of AI in Health Advice

This incident underscores the potential dangers of relying on AI for medical guidance. Researchers highlighted that AI systems like ChatGPT can generate scientifically inaccurate information and lack the ability to critically analyze results. Dr. Jacob Glanville emphasized that AI should not replace professional medical judgment, as these tools lack common sense and can produce harmful outcomes if users do not apply their own discernment.

Call for Regulation and Oversight

Experts like Dr. Harvey Castro advocate for more stringent safeguards when using large language models (LLMs) for medical information. He pointed out the “regulation gap” in AI health advice and the risks of data bias and misinformation. Castro suggested integrating medical knowledge bases and implementing risk flags to prevent similar incidents.

OpenAI’s Position on Health Advice

OpenAI clarified that ChatGPT is not intended for medical use and should not replace professional advice. They are actively working on reducing risks associated with their AI systems by encouraging users to seek expert guidance.