Construct validation of a 3D printed neonatal thoracoscopic simulator: Can it measure expertise?

BACKGROUND:acquiring technical expertise for neonatal thoracoscopy is challenging. To address this, we designed a fully synthetic thoracoscopic simulator for which we established its construct validity. METHODS:three thoracoscopic tasks were assessed: ring transfer, needle pass and incision of a blind upper esophageal pouch (EA cut). Participants watched instructional videos with accompanying written instructions for each task before having their attempt video recorded. All tasks were marked by three blinded pediatric surgeons using a modified Objective Structured Assessment of Technical Skills (OSATS). Scores were assessed using appropriate statistical analysis and inter-rater reliability was analyzed by interclass correlation coefficient (ICC). RESULTS:23 participants completed the ring transfer and needle pass and 21 the EA cut: 5 experts (consultant surgeons), 5 intermediate (registrars on a training program) and 13 novices (medical students, house surgeons or non-training registrars). All three tasks distinguished between novice and intermediate/expert (ring transfer p = 0.00001, needle pass p = 0.0004 and EA cut p = 0.0014, respectively). Interrater reliability was good for ring transfer and needle pass but poor for EA cut. CONCLUSION:the tasks distinguished between novice and intermediate/expert but not between expert and intermediate. In needle pass and EA cut, there was a trend for the experts to score higher than intermediate participants. Ring transfer and needle pass tasks achieved construct validity, had good interrater reliability and were found to be useful in assessing a novice surgeon's progression towards the intermediate level. Distinguishing between intermediate and expert may require assessment of more complex tasks such as intracorporeal suturing and tying. LEVEL OF EVIDENCE:II.