The quality of COVID-19 systematic reviews during the coronavirus 2019 pandemic: an exploratory comparison

The unprecedented volume and speed at which COVID-19-related systematic reviews (SRs) may have been produced has raised questions regarding the quality of this evidence. It is feasible that pandemic-related factors may have led to an impairment in quality (reduced internal validity, increased risk of bias [RoB]). This may have serious implications for decision-making related to public health and individual healthcare. The primary objective was to compare the quality of SRs published during the pandemic that were related to COVID-19 with SRs published during the pandemic that were unrelated to COVID-19 (all of which were fully appraised in the KSR Evidence database of SRs in healthcare). Our secondary objective was to compare the quality of SRs published during the pandemic (regardless of research topic), with SRs published pre-pandemic. We compared all SRs related to COVID-19 to all SRs unrelated to COVID-19 that (i) were published during the pandemic (between 1st March 2020 and September 14, 2022), (ii) were included in KSR Evidence, and (iii) had been appraised using the ROBIS tool. We then compared all SRs published during the pandemic (regardless of research topic) with a pre-pandemic sample of SRs. For SRs published during the pandemic, we found there was no statistically significant difference in quality between those SRs tagged as being related to COVID-19 and those that were not [relative risk (RR) of low RoB for COVID-19 versus COVID-19-unrelated reviews: 0.94; 95