Predicting future meaningful autonomy using prior autonomy ratings for pediatric surgery fellows

Purpose Assessment of trainees’ operative autonomy is challenging. However, workplace-based assessment (WBA) systems have made it possible to capture longitudinal data on trainees’ operative autonomy. We evaluated the ability of prior WBA operative autonomy ratings to predict future autonomy ratings for pediatric surgery fellows. Methods WBA data from two pediatric surgery training programs were analyzed using Bayesian mixed effects models to evaluate the relationship between prior cumulative autonomous operative experiences and the probability of being granted meaningful autonomy in a subsequent operation rated using a WBA. Cumulative autonomous experience was modeled as a fixed effect, while procedure, complexity, fellow, and attending surgeon were all modeled as random effects. Marginal predictions were generated and visualized to identify the number of prior autonomous operative experiences needed to achieve a 95% probability of being granted autonomy on a subsequent rating for three procedures: laparoscopic gastrostomy, laparoscopic inguinal hernia repair, and laparoscopic pyloromyotomy. Results At site one, 843 of 1111 (76%) evaluations were rated as meaningfully autonomous for 9 fellows. At site two, 201 of 234 evaluations (86%) were rated as meaningfully autonomous for 3 fellows. Both sites identified similar expectations for autonomy based on cumulative autonomous experiences. The number of previously autonomous ratings needed to achieve a 95% probability of being granted meaningful autonomy for a subsequent procedure were 12 (site one and two) for laparoscopic gastrostomy, 10 (site one) and 14 (site two) for laparoscopic inguinal hernia repair, and 9 (site one) and 13 (site two) for laparoscopic pyloromyotomy. Conclusion Prior operative autonomy ratings appear effective in predicting the probability of being granted meaningful autonomy in subsequent procedures across multiple faculty and fellow groups. The approach demonstrated in this paper could support establishing minimum case number requirements and monitoring of fellows’ developing entrustability.