Standardizing Assessment of Gynecologic Brachytherapy Contours: A Pilot Study in Contour Competency Assessment for Radiation Oncology Residents

Background Radiation oncology trainees may benefit from more standardized competency-based assessment in practical skills such as contouring which are presently only assessed observationally by attendings. This is especially true in the world of gynecologic brachytherapy, given trends of declining brachytherapy utilization, lower competence reported by graduating residents, and low institutional brachytherapy volumes. We hope to develop a standardized method to objectively grade and assess contours created by radiation oncology residents for competency-based assessment in gynecologic brachytherapy. Methods A magnetic resonance and computed tomography-based cervical cancer brachytherapy contouring simulation will be designed using an interactive online platform, and radiation oncology residents will be asked to contour the gross tumor volume (GTV) and high-risk clinical target volume (HRCTV) as they would for radiation planning. These contours will be evaluated by a gynecologic brachytherapy expert using three-tier system utilized in clinical trials. In addition, three experimental methods will be investigated to determine their suitability as a surrogate for expert evaluation. These include (1) non-expert ("novice") review of contours based on a generated rubric, (2) digital evaluation using dice similarity coefficient, and (3) artificial intelligence software designed to emulate scoring provided by an expert in the field. Results This study is a work in progress, and we have applied for grant funding through the American Board of Medical Specialties Research and Education Foundation grant. We expect that one or more of the three experimental modalities will result in contour evaluations that do not differ significantly from an evaluation of an expert in the field. This study will also offer insight into the feasibility and practicality of contour evaluation using physician novice, digital metric, and artificial-intelligence-based feedback. Discussion If demonstrated to be feasible, practical, and comparable in nature to feedback given by an expert in the field, similar programs may be designed for other sites, both for brachytherapy and external beam radiotherapy. Ultimately, one or more of these experimental evaluation modalities may be considered for use in competency-based evaluation of residents, as they would be much more easily generalizable across residency programs compared to expert contour review. Future directions include implementation of such assessment modalities into formal resident evaluation and utilization of digital and artificial intelligence-based software for assessment of other practical skills including plan evaluation and approval of treatment set-up.