According to a new study, ChatGPT can outperform real doctors in terms of diagnosing patients in emergency care. The AI chatbot was found to be as effective, if not more so, than human doctors when provided with the same doctor’s notes of a patient.
The study, conducted by Dutch researchers, implies that ChatGPT could potentially assist in reducing waiting times for patients. Dr. Hidde ten Berg, from Jeroen Bosch Hospital, noted that the AI bot performed well in generating a list of probable diagnoses and suggesting the most likely option. There was also significant overlap with the doctors’ lists of possible diagnoses, indicating that ChatGPT could suggest medical diagnoses much like a human doctor would.
ChatGPT, launched by OpenAI, uses one of the most sophisticated language models ever developed. Previous research demonstrated that it could pass the US Medical Licensing Exam and even exhibit better bedside manners than actual doctors.
The study analyzed how well ChatGPT performed in an A&E setting. It used anonymized details of 30 patients treated at Jeroen Bosch Hospital’s emergency department in 2022. The AI bot was asked to produce a list of potential diagnoses for the patients, which were then compared to lists generated by real doctors.
The researchers found approximately 60 percent overlap between the shortlists created by ChatGPT and the doctors. It was found that doctors had the correct diagnosis within their top five likely diagnoses in 87 percent of the cases, compared to 97 percent for ChatGPT version 3.5 and 87 percent for version 4.0.
Dr. ten Berg cautioned that while ChatGPT is not a medical device and there are privacy concerns when using it with medical data, there is potential for time-saving and reducing waiting times in the emergency department. It could also be beneficial for supporting less experienced doctors or aiding in the detection of rare diseases.
Despite the promising results, experts say that we are still far from using ChatGPT in clinical settings. However, the study adds to the growing body of research suggesting that AI could potentially assist doctors in hospitals in the future.
