Skip to main content
RACS ASC 2024

Employing Large Language Models for Surgical Education: An In-depth Analysis of ChatGPT-4

Verbal Presentation

Watch The Presentation

Presentation Description

Institution: Faculty of Medicine and Health, Central Clinical School, The University of Sydney, Sydney - New South Wales, Australia

Purpose – The COVID-19 pandemic has significantly disrupted surgical education, necessitating adaptations in curricula and teaching methods. Large language models (LLMs) hold potential in addressing these challenges by providing personalised feedback, facilitating engaging remote learning experiences, and offering virtual mentorship. Therefore, the aim was to evaluate the ability of LLMs to assist junior doctors in providing advice for common ward-based surgical scenarios of increasing complexity. Methodology – Utilising an instrumental case study approach, this study explored the potential of LLMs by comparing the responses of ChatGPT-4, BingAI and BARD. LLMs were prompted with common ward-based surgical reviews scenarios and tasked with assisting junior doctors with clinical decision-making. Outputs were assessed on their accuracy, safety, and effectiveness in order to determine their viability as a synergistic tool in surgical education. Results – The findings highlight the potential for incorporating LLMs into surgical education, with ChatGPT-4 demonstrating particular promise for delivery of reliable and accurate information. As assessed by our expert panel of surgeons, the advice provided by ChatGPT-4 was appropriate and safe for a doctor at the level of a first-year intern or resident. Conclusions – To optimise learning experiences and better support surgical trainees, future research should focus on refining the specificity of LLMs’ responses, exploring synergistic combinations with existing technologies, and aiming to enhance patient outcomes through the use of LLMs.

Speakers

Authors

Authors

Dr Adrian Siu - , Dr Damien Gibson - , Dr Xin Mu - , Dr Ishith Seth - , Mr Alexander Siu - , Dr Dilshad Dooreemeah - , Dr Angus Lee -