Box-Trainer Assessment System with Real-Time Multi-Class Detection and Tracking of Laparoscopic Instruments, using CNN

In Minimally Invasive Surgery (MIS), surgeons need to acquire a specific set of skills, before carrying out a "real" operation. Training with the Laparoscopic Surgical Box Trainer device helps in acquiring the needed skills for surgery residents which are traditionally not taught to them. Video recording of residents' performance and computer assisted surgical trainers for MIS provide valuable information for resident's assessment. In this paper, we propose real-time detection and tracking of a multi-class of laparoscopic instruments for an intelligent box-trainer performance assessment system using SSD-ResNet50 V1 FPN architecture in TensorFlow backend. The dataset has been extracted from various laparoscopic box training videos. Using distance measurements and evaluation criteria constraints, we present an evaluation of the surgeon's performance. Based on the experimental result, the trained model could identify each instrument at the score of 90% fidelity, in each location, within a region of interest. This research is a result of a partnership between the Department of Electrical and Computer Engineering and the Department of Surgery, of the Homer Stryker M.D. School of Medicine, at Western Michigan University.