1027: Does lowering the p-value threshold render RCTs insignificant in leading OBGYN journals?

Randomized control trials (RCTs) are considered the highest level of evidence to inform clinical practice. However, the reproducibility crisis has raised concerns about the scientific rigor of published RCT findings. Some advocate for a lower p-value threshold. We aimed to review previously published RCTs in three primary OBGYN journals to determine if they withstand the rigor of re-analysis with use of a p-value threshold of 0.005. A manual search was performed to identify all RCTs published in The Green, AJOG, and BJOG between July 2017 and June 2019. Data were collected on primary outcome(s), methodology, and p-values. We determined the number of primary outcomes that would remain statistically significant with movement of the p-value threshold to 0.005 versus be re-interpreted as “suggestive” (defined as p between 0.005 < p < 0.05). We additionally compared methodologic characteristics between the subset of trials which lost significance and those that did not using Chi-square or Fisher’s exact test as appropriate. Trials without a power calculation or specified analysis type were considered at high risk of bias. 132 RCTs met inclusion criteria; 48% in OB and 51% in GYN domains. Methodologically, 31.8% did not pre-specify analysis type, 30.3% did not meet the pre-determined sample size, and 11.4% did not report an a priori sample size calculation. CONSORT guidelines were mentioned in just 37%, though 95% of trials were registered. Of the 62 primary outcomes considered significant with p< 0.05 at time of publication, 51.6% (n=32) would maintain significance (p< 0.005), while 48.4% (n=30) would be re-interpreted as suggestive. No trial characteristics were associated with a change in statistical significance by p-value threshold (Table). Adopting a p-value threshold of 0.005 would require reinterpretation of a significant proportion of RCT results in the OBGYN literature as well as the use of less binary interpretive language. Highly variable reporting quality was identified with lack of attention to CONSORT guidelines.