My Doctor, the Chatbot? – Medical Ethicist Robert Ranisch on the Benefits and Risks of Generative AI in Healthcare

The widespread adoption of generative AI chatbots, such as ChatGPT, for health-related questions is a significant development, with medical ethicist Robert Ranisch observing that almost half of the German population has already utilized these tools for health advice. This rapid integration of AI into daily life, occurring just over three years since ChatGPT's release, highlights its accessibility and the public's willingness to experiment with its capabilities across various domains, including personal health. However, the reliability of AI chatbot responses in medical contexts is complex. Early studies, conducted shortly after ChatGPT's 2022 launch, showed surprisingly accurate symptom assessments in emergency scenarios. Yet, concerns arose that these successes might stem from the AI merely recalling textbook cases present in its training data rather than genuinely assessing unique situations. This raises a critical issue regarding the AI's true understanding and diagnostic capabilities. The risks associated with relying on AI for medical advice are substantial. In the best-case scenarios, individuals might seek professional medical attention for unfounded concerns. More dangerously, incorrect AI advice could lead to patients ignoring serious symptoms, resulting in delayed or inappropriate treatment. Examples include AI suggesting rest for stroke symptoms or providing harmful dietary recommendations. Such incidents highlight the potential for generative AI to cause significant harm, especially as these tools become increasingly integrated into search engines and other widely used platforms. While acknowledging these dangers, Ranisch also points to the potential benefits of generative AI in healthcare. It could streamline administrative tasks, such as summarizing patient conversations for medical records and translating complex medical information into layperson-friendly language. The concept of an "avatar" or chatbot that interacts with patients daily to collect symptom data and provide information is also explored as a way to assist in regions with limited access to specialized medical care. Despite these potential advantages, Ranisch firmly asserts that AI chatbots cannot replicate the essential, confidential, and trusting relationship between a patient and a therapist. He stresses the importance of maintaining a healthy skepticism towards AI-generated health advice, especially given recent instances where AI-generated summaries in search results provided incorrect and dangerous health information. The ongoing entry of major AI companies like OpenAI and Anthropic into the healthcare market with specialized offerings like ChatGPT Health and Claude for Healthcare underscores the need for careful consideration of their impact and regulatory oversight, particularly concerning their deployment in Europe.