’ s response to reviews Title : Adherence to Reporting Guidelines Increases the Number of Citations : The Argument for Including a Methodologist in the Editorial Process

Editor. Please use also acronym MQAI in the abstract after mentioning the instrument. Done. (Abstract section, page 2, line 42) Editor. ‘by adding statistical reviewers and reporting guidelines (RG) during the peer review process’: please provide more detailed information about the exact interventions that were used in those randomized studies Thank you. We have added further details in the abstract (Abstract section, page 2, lines 39 to 42). “...where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG).” Editor. Sentence ‘Now, we used the Web of Science to estimate in a post-hoc analysis the effects on number of citations of interventions devised to improve manuscript completeness.’ – should be revised into: “In this post-hoc study we analyzed whether interventions for improving reporting of manuscripts, assessed in those two randomized studies, were associated with higher number of citations.” We have mostly followed your suggestions, although we prefer to keep the word “effect” because this is the advantage of experimental over observational designs. We agree with you that, as a post-hoc hypothesis and study, the conclusion cannot be “hard”, in the sense that this is not a definitive proof because we lack a confirmatory trial powered to test such hypothesis. (Abstract section, page 2, lines 45 to 48). “In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used.” Editor. Please write more detailed results in the Abstract. This journal has generous word count policy for the Abstract. Provide the number of manuscripts that were included in those two randomized studies and that have been used for this study. We have re-written the results in the Abstract section (page 3, lines 54 to 58). “Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3% to 98%) increase in the number of citations.” Introduction Editor. ‘Although at least 24 randomized’: please provide references for all those studies Editor. ‘while only 4 analysed’: provide references for those four studies After reviewing the references, we realized that one of them was duplicated. We have corrected this in the text (Background section, page 4, lines 80 to 85) and have included 23 new references in Supplementary Material 7. “According to the systematic review published by Bruce et al. (2016)8, which we expand on in Section 7 of the Supplementary Material (SM), at least 23 randomized trials have studied some aspects of the peer review process, with the majority of them focusing on the quality of peer review as a surrogate outcome while only 315-17 analysed the completeness of reporting as an outcome.” Editor. The first reviewer in the initial revision complained two times that you have not given any literature-based rationale about why did you expect that completeness of reporting will be associated with higher citation counts. Can you please support that assumption with some references? We have added two new references for this post-hoc analysis. [20] Van der Pol CB, McInnes MDF, Petrcich W, Tunis AS, Hanna R (2015) Is Quality and Completeness of Reporting of Systematic Reviews and Meta-Analyses Published in High Impact Radiology Journals Associated with Citation Rates? PLoS ONE 10(3): e0119892. doi:10.1371/journal. pone.0119892. [21] Nieminem P, Carpenter J, Rucker G and Shumacher M (2006) The relationship between quality of research and citation frequency. BMC Med Res Methodol. 6: 42. doi: 10.1186/14712288-6-42. (Background section, page 4 line 96) We also clarified that we have previously explored the same idea with a shorter follow up: “We also previously explored this relationship with a shorter follow-up (SM section 6).” (Background section, page 4 lines 96 to 97) Discussion: Editor. The sentence: ‘Interestingly, the number of papers with zero-citations is greater in the intervention groups of both studies, raising the possibility that greater transparency avoids citations in some kinds of papers.’ I would avoid making these kinds of unwarranted conclusions. These are very small numbers of manuscripts that have been assessed. Reviewer: Line 179-180: I still do not understand why greater transparency could avoid citations in some kinds of papers. I do not see the logic of this in the context of the research conducted and reported in this paper. Thank you. This is not a result that is definitively proven in our paper. It is our interpretation of an unexpected result. In our opinion, greater transparency could facilitate a better understanding of the weaknesses of the study and show that the results might not be reliable. We have changed the sentence (Discussion section, page 9, lines 186 to 189): “The number of papers with zero-citations was also higher in the intervention groups of both studies, which raises the possibility that greater transparency deters citations for some kinds of papers. This unexpected result warrants confirmation in future studies.” Editor: Please delete this sentence: "We have definitively shown that randomised studies for improving scientific impact, measured by the number of WoS citations are feasible" – The randomized studies you are using were not originally intended to measure citation counts as an outcome. Reviewer: When the authors say "We have definitively shown that randomised studies for improving scientific impact, measured by the number of WoS citations are feasible" I still do not see the reason why this paper is providing such definitive proof. Likewise, I do not see this statement as significantly contributing to the importance of this study in particular. It is true that the 2 previous studies were not originally intended to measure citation counts as an outcome. However, in this study we have shown that it could be feasible to design, conduct and analyze such studies. Nevertheless, as this is a secondary argument, for the sake of clarity, we have deleted it. Editor: Please revise this sentence: ‘Based on a bibliographic search at Pubmed (SM 7), this is the first intervention experimentally assessed to improve the number of citations.’ – the primary aim of your intervention was not to improve citation counts. This is a post-hoc analysis. I would suggest to state: “To our best knowledge, this is the first study that shows potential association between completeness of reporting in scholarly papers with higher citation counts.” We have changed this sentence (Discussion section, page 9, lines 190 to 193) to: “To the best of our knowledge, this is the first study showing that the completeness of reporting is potentially associated with higher citation counts as a result of a specific intervention, namely: adding to the peer review process a methodological expert who ensures that the reporting guidelines are adhered to.“ Editor: Please mention very wide confidence interval as a limitation. In our opinion, this is a common situation in almost any study (unless designed to provide precise CI, a rare design). Anyway, we are happy to add (Discussion section, page 9, lines 202 to 203): “In interpreting this effect size, we should keep in mind the uncertainty reflected by the confidence intervals.” Editor: This sentence appears incomplete: ‘Our findings indicate that the number of citations was greater by 43% (95% CI from: 3% to 98%).’ Please write the more complete sentence to indicate what intervention was used to achieve this. We have changed the sentence (Conclusions section, page 10, lines 222 to 229) to: “Our findings indicate that the citation counts increased by 43% (95% CI from: 3% to 98%) after including in the editorial process a methodologist who ensures the proper reporting of checklist items. As our original studies were originally designed to test those hypotheses for a different outcome, this present study was not powered to test this post-hoc analysis; therefore, our results should not be interpreted as definitive and they need to be confirmed in properly powered designs. We invite journals to perform their own studies to ascertain whether or not scientific impact is increased, first, by adhering to reporting guidelines, and second, by further involving statisticians or methodological experts in the editorial process.”