O52: Empowering Medical Students and Early Career Surgeons in Critical Appraisal of AI Research: Design and Evaluation of a Pilot Course

Abstract Introduction As artificial intelligence (AI) plays an increasing role in surgical practice, there is a need for surgeons to understand and evaluate literature surrounding the use of AI. There is currently a lack of training in this area for medical students and surgical trainees. We designed and assessed the impact of a short educational course. Methods A two-day online course was held, consisting of lectures and small-group discussions surrounding the fundamentals and appraisal of AI literature with a focus on neurosurgery. Participants included individuals working in neurosurgery (trainees, consultants and researchers) and medical students interested in surgery. A Qualtrics survey was distributed before and after the course. Participants’ knowledge was assessed before and after using 15 multiple-choice questions. Individuals allocated to the course’s waiting list completed a control survey. Results 62 participants (33 participants, 29 waitlist controls) attended the course and completed the pre-course survey. After the course, participants significantly improved in their knowledge of AI (mean difference = 3.86, 95% CI = 2.97-4.75, p-value < 0.0001) and confidence in critically appraising AI literature (p-value = 0.002). Differences in knowledge (mean difference = 3.15, 95% CI = 1.82-4.47, p-value < 0.0001) and confidence (p-value < 0.0001) were also found when compared to the control group. Significant differences were maintained when medical students or researchers were excluded from analysis. Conclusion Short courses that are tailored to the specialty and experience of surgical trainees and medical students can improve their understanding of AI, without the need for in-depth technical knowledge or programming skills.