Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation.

Objective: Nasal endoscopy is integral to the evaluation of sinonasal disorders. However, prior studies have shown significant variability in the interrater agreement of nasal endoscopy interpretation among practicing rhinologists. The objective of the current study is to evaluate the interrater agreement of nasal endoscopy among otolaryngology residents from a single training program at baseline and following an educational intervention. Methods: Eleven otolaryngology residents completed nasal endoscopy grading forms for eight digitally recorded nasal endoscopic examinations. An instructional lecture reviewing nasal endoscopy interpretation was subsequently provided. The residents then completed grading forms for eight different nasal endoscopic examinations. Interrater agreement among residents for the pre- and postlecture videos was calculated using the unweighted Fleiss' kappa (Kt) statistic and intraclass correlation agreement (ICC). Results: Interrater agreement improved from a baseline level of fair (Kf range 0.268-0.383) to a posteducational level of moderate (Kf range 0.101-0.547) for nasal endoscopy findings of middle meatus mucosa, middle turbinate mucosa, middle meatus discharge, sphenoellmmid recess mucosa, sphenoethmoid recess discharge, and atypical lesions (ICC, P < 0.001). The baseline level of agreement for evaluation of nasal septum deviation was poor/fair and did not improve following educational intervention. Conclusions: This study demonstrates a limited baseline level of interrater agreement of nasal endoscopy interpretation among otolaryngology residents. The interrater agreement for the majority of the characteristics that were evaluated improved after educational intervention. Further study is needed to improve nasal endoscopy interpretation.