Computer-assessed performance of psychomotor skills in endoscopic otolaryngology surgery: construct validity of the Dundee Endoscopic Psychomotor Otolaryngology Surgery Trainer (DEPOST)

Background This study was undertaken to introduce and establish the value of the Dundee Endoscopic Psychomotor Otolaryngology Surgery Trainer (DEPOST) as a customisable, objective real-time scoring system for trainee assessment. The construct validity of the system was assessed by comparing the performance of experienced otolaryngologists with that of otolaryngology trainees, junior doctors and medical students. Methods Forty two subjects (13 Consultants, 8 senior trainees, 13 junior trainees and 8 junior doctors/medical students) completed a single test on DEPOST. The test involved using a 30° rigid endoscope and a probe with position sensor, to identify a series of lights in a complex 3-dimensional model. The system scored subjects for time, success rate, and economy of movement (distance travelled). An analysis of variance and correlation analysis were used for the data analysis, with statistical significance set at 0.05. Results Increasing experience led to significantly improved performance with the DEPOST ( p < 0.01). Senior trainees’ results were significantly better than those of consultant otolaryngologists in success rate and time ( p < 0.05 & p < 0.05). Consultants were the most efficient in their movement ( p = 0.051) Conclusions The system provides an accurate and customisable assessment of endoscopic skill in otolaryngologists. The DEPOST system has construct validity, with master surgeons and senior trainees completing the tasks more accurately without sacrificing execution time, success rate or efficiency of movement.