AI Chatbots Show Limitations in Handling Health Scenarios, Prompting Caution

A recent evaluation conducted by The Star newspaper on June 21, 2026, revealed that while AI chatbots have made significant strides, they still exhibit clear limitations when addressing complex user scenarios, particularly in the realm of health. The test involved presenting ChatGPT and another AI chatbot, Juma.ai, with a common health concern: a user suffering from vomiting and a headache who expressed a fear of consulting a doctor. Both chatbots demonstrated an understanding of the gravity of the situation by highlighting potential warning signs and strongly recommending that the user seek professional medical attention if their symptoms deteriorated. This aspect was lauded by Dr. Thirunavukarasu Rajoo, president of the Malaysian Medical Association, who reviewed the AI responses. He noted that this capability indicates the considerable progress made in AI technology. However, the chatbots' approaches to the scenario differed. ChatGPT adopted a more conversational style, engaging the user with follow-up questions before discussing potential treatment options. In contrast, Juma.ai provided a more structured response, initially offering a checklist of symptoms and later suggesting over-the-counter medications for the headache and nausea. Despite these varying approaches, the core message from both was to prioritize professional medical consultation for persistent or worsening conditions. The findings underscore a crucial message: while AI tools can offer preliminary information and guidance, they are not equipped to replace human medical professionals. The test serves as a timely reminder for users to exercise caution and always consult qualified healthcare providers for accurate diagnoses and treatment plans, especially when dealing with personal health issues.