Development and Validation of an Automated Method to Identify Patients Undergoing Radical Cystectomy for Bladder Cancer Using Natural Language Processing

Abstract Introduction Measurement for quality improvement relies on accurate case identification and characterization. With electronic health records now widely deployed, natural language processing, the use of software to transform text into structured data, may enrich quality measurement. Accordingly we evaluated the application of natural language processing to radical cystectomy procedures for patients with bladder cancer. Methods From a sample of 497 procedures performed from March 2013 to October 2014 we identified radical cystectomy for primary bladder cancer using the approaches of 1) a natural language processing enhanced algorithm, 2) an administrative claims based algorithm and 3) manual chart review. We also characterized treatment with robotic surgery and continent urinary diversion. Using chart review as the reference standard we calculated the observed agreement (kappa statistic), sensitivity, specificity, positive predictive value and negative predictive value for natural language processing and administrative claims. Results We confirmed 84 radical cystectomies were performed for bladder cancer, with 50.0% robotic and 38.6% continent diversions. The natural language processing enhanced and claims based algorithms demonstrated 99.8% (κ=0.993, 95% CI 0.979–1.000) and 98.6% (κ=0.951, 95% CI 0.915–0.987) agreement with manual review, respectively. Both approaches accurately characterized robotic vs open surgery, with natural language processing enhanced algorithms showing 98.8% (κ=0.976, 95% CI 0.930–1.000) and claims based 90.5% (κ=0.810, 95% CI 0.686–0.933) agreement. For urinary diversion natural language processing enhanced algorithms correctly specified 96.4% of cases (κ=0.924, 95% CI 0.839–1.000) compared with 83.3% (κ=0.655, 95% CI 0.491–0.819). Conclusions Natural language processing enhanced and claims based algorithms accurately identified radical cystectomy cases at our institution. However, natural language processing appears to better classify specific aspects of cystectomy surgery, highlighting a potential advantage of this emerging methodology.