Machine learning models in predicting failure of Helicobacter pylori treatment: A two country validation study

Background: The success rate of clarithromycin-containing Helicobacter pylori treatment had declined globally. This study aims to explore the role of different machine learning algorithms in predicting failure of H. pylori treatment. Materials and Methods: We included 84,609 adult patients who had received the first course of clarithromycin-containing triple therapy for H. pylori in Hong Kong from 2003 to 2013 as training set. Results were validated in patients who had received similar triple therapy with 27,736 Hong Kong patients between 2014 and 2017 (internal cohort); and 18,050 UK patients between 2012 and 2017 (external cohort). The performance of 11 available machine learning algorithms were used to predict the failure of triple therapy. The performance was determined by the area under receiver operating characteristic curve (AUC). Results: The treatment failure rates in the training, internal and external validation cohort was 5.9%, 9.5%, and 6.1%, respectively. In the internal validation set, Extra-Tree (ET) Classifier had the best AUC (0.88; 95% CI, 0.87-0.88), sensitivity (79.6%; 95% CI, 79.0-80.2) and specificity (79.4%; 95% CI, 79.0-79.8). In the external validation set, ET Classifier also had the best AUC (0.85; 95% CI, 0.85-0.86), sensitivity (80.1%; 95% CI, 79.5-80.9), and specificity (80.2%; 95% CI, 78.8-81.3). Top features of importance used by ET Classifier in predicting treatment failure included time interval between antibiotic use and triple therapy (48.8%), age (29.1%) and triple therapy regime (6.28%). Conclusions: Machine learning algorithm, based on simple baseline clinical parameters, could help to identify patients at high risk of failure from clarithromycin-containing triple therapy for H. pylori.