P-value and Bayesian analysis in randomized-controlled trials in child health research published in 2007 and 2017: a methodological review

Abstract Background Reliance on P-value of significance in clinical trials is a source of debate because of misconceptions and misinterpretations associated with it. Bayesian methods are suggested as an alternative approach. As randomized-controlled trials (RCTs) are essential in generating research evidence, we investigated the change in the use of P-values and Bayesian analysis and the clustering of P-values at key significance levels in child health RCTs published in 2007 and 2017. Methods We searched Cochrane Central Register of Controlled Trials to identify random samples of child health RCTs published in 2007 (n = 300) and 2017 (n = 300). Data on trial characteristics and analytic approaches were extracted. We analyzed the 600 RCTs using the frequentist and Bayesian methods. The change in the proportion of trials reporting P-values and Bayesian analyses was assessed using Pearson/Fisher Exact tests and non-informative Dirichlet priors. Results Of 600 RCTs, 535 (89%) used frequentist methods only versus 65 (11%) that included some Bayesian methods. Only 2 of the 65 trials used Bayesian inferential statistics. The use of frequentist methods decreased from (273, 91% to 262, 87%) while the inclusion of Bayesian analysis slightly increased from (27, 9% to 38, 13%) between 2007 and 2017. Although most RCTs were from Europe (172, 29%) and North America (133, 22%), the increase in proportion of trials by continent was most in Asia (mean difference (MD) = 0.14, 95% credible interval (CI) 0.08–0.20) with posterior probability (PP) of 1.00. Parallel (487, 81.2%) and cluster (58, 9.7%) RCTs were the most common RCT types but the increase in cluster RCT (0.06, 95%CI 0.02–0.11, PP = 0.99) was more than any other RCT types over 10 years. We found clustering of P-values at the significance level of 0.05 (437, 72.8%), which increased between 2007 (209, 69.7%) and 2017 (228, 76%). The smallest P-value reported in this review was 0.0001 (1, 0.2%). Conclusions The statistical framework in child health RCTs has not changed from the frequentist methods that is based on P-values with an unexplained clustering at the significance level of 0.05. Bayesian methods may increase the confidence in interpretation of results of RCTs