New research indicates AI can serve as a valuable tool for student development alongside clinical learning

Congratulations to UCD School of Medicine’s Dr Cathleen McCarrick, Dr Philip McEntee, Dr Patrick Boland, Associate Professor Suzanne Donnelly, Professor Yvonne O’Meara, Professor Helen Heneghan and Professor Ronan Cahill on their recently published research in the Journal of Surgical Education.

The article is titled ‘A Randomized Controlled Trial of a Deep Language Learning Model-Based Simulation Tool for Undergraduate Medical Students in Surgery’ and indicates the benefits of AI-based simulations in a controlled, educational setting.

Highlights

  • DLMs, like ChatGPT, provide scalable and personalized simulations for medical students, offering a low-cost and effective alternative to traditional patient-based simulations, while still requiring supervision to ensure quality interactions.
  • AI-based simulations offer realistic, adaptable, and creative clinical scenarios, improving communication skills in a controlled setting.
  • While DLMs show promise in enhancing medical training, further research is needed to evaluate long-term retention, explore integration with other learning methods, and address the environmental impacts of AI use in education.

Introduction

Effective communication is a critical skill for surgeons that commences often with history-taking. While simulation-based training is utilized to enhance these skills, recent advancements in artificial intelligence (AI), especially deep language learning models (DLM), offer new opportunities. This study evaluates the integration of DLM as a simulated patient (SP) into surgical history-taking training for senior medical students during clinical rotations.

Methods

A randomized controlled trial was conducted with surgery module students. Participants were divided into control and intervention groups, the former receiving standard experiential learning and the latter adding 3 structured sessions with DLM (ChatGPT, Open AI) as SP (with interaction texts submitted for tutor evaluation). All students underwent Objective Structured Clinical Examination (OSCE) of history-taking with a human SP and blinded assessor blinded by group for baseline competency ascertainment and again after either intervention or a similar time of standard learning. Intervention group students were anonymously surveyed to assess communication confidence and perspectives on DLM as SP.

Results

After initial pilot trailing, ninety students participated formally with 45 assigned to each arm via randomized cluster sampling. DLM-content was uniformly appropriate. Baseline scores were similar but significantly increased in the intervention group alone (p < 0.001, 0.37v0.19 Cohen D education effect size). 62% of students completed the survey, a majority (57%) articulating increased confidence, rich detail in DLM histories (72%) and would use again (95%).

Conclusions

DLM effectively enhanced surgical history-taking skills. These findings indicate AI can serve as a valuable tool for student development alongside clinical learning.

Read the article here.