The Association Between The Impact Factor of A Journal And The Trueness of Findings Published Therein: A Meta-Epidemiological Study

Abstract Background: Scientists, physicians, and the general public legitimately expect scholarly publications to give true answers to study questions raised. We investigated whether findings from studies published in journals with higher Journal Impact Factors (JIFs) are more accurate than findings from studies in less-cited journals via a meta-epidemiological approach.Methods: We screened intervention reviews from the Cochrane Database of Systematic Reviews (CDSR) and sought well-appraised meta-analyses. We defined the trueness of RCT study results as their point estimates’ relative deviation from the pooled effect estimate. In addition to the JIF we considered the open-access policy of journals, the relative size of studies, and the studies’ methodical quality.Results: In 2,459 results from 446 meta-analyses, there was a positive association between a journal’s JIF and the trueness of the study results published therein (Pearson’s r for the deviation = −0.21, 95% CI −0.24 to −0.17, P < 0.001). The mean relative deviation decreased from 1.11 (SD = 0.94) in the lowest JIF quartile to 0.80 (SD = 0.75) in the top quartile (95% CI: 0.21 to 0.40, P < 0.001). Findings from larger and methodically sound studies were also more likely to be accurate. In a multiple regression analysis, the JIF proved to be the strongest determinant of trueness, independent of the covariates.Conclusions: Our results indicate that higher-impact journals act as gatekeepers in the process of science more effectively than less-cited journals. However, the fact alone that a study result is reported in a journal with a high impact factor is only a weak and impractical indicator for its accuracy.