Validation of an online assessment of orthopedic surgery residents' cognitive skills and preparedness for carpal tunnel release surgery.

BACKGROUND:Surgical competence requires both cognitive and technical skills. Relatively little is found in the literature regarding the value of Web-based assessments to measure surgery residents' mastery of the competencies. OBJECTIVE:To determine the validity and reliability of 2 online instruments for predicting the cognitive preparedness of residents for performing carpal tunnel release surgery. METHOD:Twenty-eight orthopedic residents and 2 medical school students responded to an online measure of their perception of preparedness and to an online cognitive skills assessment prior to an objective structured assessment of technical skills, in which they performed carpal tunnel release surgery on cadaveric specimens and received a pass/fail assessment. The 2 online assessments were analyzed for their internal reliability, external correlation with the pass/fail decision, and construct validity. RESULTS:The internal consistency of the perception of preparedness measure was high (α  =  .92) while the cognitive assessment was less strong (α  =  .65). Both instruments demonstrated moderately strong correlations with the pass/fail decision, with Spearman correlation of .606 (P  =  .000) and .617 (P  =  .000), respectively. Using logistic regression to analyze the predictive strength of each instrument, the perception of preparedness measure demonstrated a 76% probability (η(2)  =  .354) and the cognitive skills assessment a 73% probability (η(2)  =  .381) of correctly predicting the pass/fail decision. Analysis of variance modeling resulted in significant differences between levels at P < .005, supporting good construct validity. CONCLUSIONS:The online perception of preparedness measure and the cognitive skills assessment both are valid and reliable predictors of readiness to successfully pass a cadaveric motor skills test of carpal tunnel release surgery.