Reliability and Agreement of Ultrasonographic Measures of the Ovarian Stroma Impact of Methodology

Objectives Increased ovarian stromal area (SA), stromal-to-ovarian area ratio (S/A), and echogenicity (SEcho) on ultrasonography have been proposed as diagnostic markers for polycystic ovary syndrome. Although several methods to evaluate the stroma exist, their reproducibility has not been defined which limits clinical utility. This study aimed to determine the interrater reliability and agreement of methods to evaluate SA, S/A, and SEcho. Methods Five raters tested 3 methods to obtain SA and S/A, and one to obtain SEcho on 30 ovarian cineloops under two imaging conditions, simulating real-time (free-choice) or offline (fixed-frame) imaging. For SA, Method 1 subtracted follicular area from the ovarian area, Method 2 involved outlining the periphery of the stroma, and Method 3 represented a hybrid approach in which central follicles were subtracted from the outlined stroma. SEcho was scored on a subjective 3-tiered scale. Intraclass correlation coefficients (ICCs) and the coefficient of variation were determined for SA and S/A, and Fleiss' kappa agreement statistics (kappa) were determined for SEcho. Results Interrater reliability of SA was superior using Method 1 (ICC = 0.558 and ICC = 0.705) versus Method 2 (ICC = 0.522 and ICC = 0.230) or Method 3 (ICC = 0.429 and ICC = 0.305) under free-choice and fixed-frame imaging conditions, respectively. Interrater reliability of S/A was also moderate to poor across methods. SEcho was also not reliably assessed across raters (kappa = <0.500). Conclusions Ultrasonographic assessments of the ovarian stroma were associated with moderate to poor reproducibility. Indirect estimates of the ovarian stroma (Method 1) could be optimized to yield a reproducible approach, clarifying the clinical relevance of the stroma.