On the reliability of predictions on Covid-19 dynamics: a systematic and critical review of modelling techniques

Since the beginning of the new coronavirus 2019-nCoV disease (Covid-19) in December 2019, there has been an exponential number of studies using diverse modelling techniques to assess the dynamics of transmission of the disease, predict its future course and determine the impact of different control measures. In this study, we conducted a global systematic literature review to summarize trends in the modelling techniques used for Covid-19 from January 1st 2020 to June 30th 2020. We further examined the reliability and correctness of predictions by comparing predicted and observed values for cumulative cases and deaths. From an initial 2170 peer-reviewed articles and preprints found with our defined keywords, 148 were fully analyzed. We found that most studies on the modelling of Covid-19 were from Asia (52.70%) and Europe (25%). Most of them used compartmental models (SIR and SEIR) (57%) and statistical models (growth models and time series) (28%) while few used artificial intelligence (5%) and Bayesian approach (3%). For cumulative cases, the ratio predicted/observed values and the ratio of the amplitude of confidence interval (CI) or credibility interval (CrI) of predictions and the central value were on average larger than 1 (4.49 ± 9.98 and 1.10 ± 1.94, respectively) indicating cases of incorrect predictions, large uncertainty on predictions, and large variation across studies. There was no clear difference among models used for these two ratios. However, the ratio predicted/observed values was relatively smaller for SIR models than for SEIR models, indicating that more complex models might not be more accurate for predictions. We further found that values of both ratios decreased with the number of days covered by studies, indicating that the wider the time covered by the data, the higher the correctness and accuracy of predictions. In 21.62% of studies, observed values fall within the CI or CrI of the cumulative cases predicted by studies. Only six of the 148 selected studies (4.05%) predicted the number of deaths. For 33.3% of these predictions, the ratio of predicted to actual number of deaths was close to 1. We also found that the Bayesian model made predictions closer to reality than the compartmental and the statistical models, although these differences are only suggestive due to the small size of the data. Our findings suggest that while predictions made by the different models are useful to understand the pandemic course and guide policy-making, there should be cautious in their usage. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement The authors secured no funding for this research. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Not applicable All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes I declare that all data used in the manuscript are available online.