Development and evaluation of a structured guide to assess the preventability of hospital-onset bacteremia and fungemia

Objective: To assess preventability of hospital-onset bacteremia and fungemia (HOB), we developed and evaluated a structured rating guide accounting for intrinsic patient and extrinsic healthcare-related risks. Design: HOB preventability rating guide was compared against a reference standard expert panel. Participants: A 10-member panel of clinical experts was assembled as the standard of preventability assessment, and 2 physician reviewers applied the rating guide for comparison. Methods: The expert panel independently rated 82 hypothetical HOB scenarios using a 6-point Likert scale collapsed into 3 categories: preventable, uncertain, or not preventable. Consensus was defined as concurrence on the same category among >= 70% experts. Scenarios without consensus were deliberated and followed by a second round of rating. Two reviewers independently applied the rating guide to adjudicate the same 82 scenarios in 2 rounds, with interim revisions. Interrater reliability was evaluated using the kappa (kappa) statistic. Results: Expert panel consensus criteria were met for 52 scenarios (63%) after 2 rounds. After 2 rounds, guide-based rating matched expert panel consensus in 40 of 52 (77%) and 39 of 52 (75%) cases for reviewers 1 and 2, respectively. Agreement rates between the 2 reviewers were 84% overall (kappa, 0.76; 95% confidence interval [CI], 0.64-0.88]) and 87% (kappa, 0.79; 95% CI, 0.65-0.94) for the 52 scenarios with expert consensus. Conclusions: Preventability ratings of HOB scenarios by 2 reviewers using a rating guide matched expert consensus in most cases with moderately high interreviewer reliability. Although diversity of expert opinions and uncertainty of preventability merit further exploration, this is a step toward standardized assessment of HOB preventability.