Fragility index: how fragile is the data that support the American College of Gastroenterology guidelines for the management of Crohn's disease?

Randomized controlled trials (RCTs) are the cornerstone of evidence-based medicine. However, recent literature has drawn attention to the limitations of using P-value to report statistical significance of outcomes in the clinical trials. We performed this analysis to analyze the strength of the data that supported the American College of Gastroenterology (ACG) guidelines for 'Management of Crohn's disease in adults' using fragility index (FI). We screened all the RCTs referenced in the 2018 ACG guidelines 'Management of Crohn's disease in adults'. We calculated the FI and the fragility quotient (FQ) and its correlation with P-value. Data were also collected on the patients lost to follow up, year of publication, sample size, number needed to treat (NNT), science citation index (SCI), presence of blinding and the number of centers in these studies. Of the 91 RCTs cited in this guideline, 32 RCTs met the inclusion criteria. The median values for FI for 32 trials were 3 [interquartile range (IQR) 2-6], FQ 0.026 (IQR 0.012-0.413), P-value 0.010 (IQR 0.001-0.03), lost to follow up 17 (IQR 10-39.5) and sample size 133 (IQR 74.5-281.5). There was statistically significant correlation between FI and P-value (r(s) -0.86, P <0.001) and sample size (r(s) 0.56, P = 0.002). There was no correlation found with number lost to follow up, NNT, SCI, year of publication, blinding and number of centers. The majority of the RCTs conducted in the field of Crohn's disease rely on small number of superior events for statistical significance, thus rendering the validity of their conclusion questionable. At least 18 out of 60 ACG recommendations are based on RCTs in which, number of patients lost to follow up exceeds FI, thus making reported outcomes of the trial weak. We suggest that FI and FQ should be included in clinical trials to better understand if the data are meaningful, beyond a P-value.