A systematic review of sample size estimation accuracy on power in malaria cluster randomised trials measuring epidemiological outcomes

Introduction Cluster randomised trials (CRTs) are the gold standard for measuring the community-wide impacts of malaria control tools. CRTs rely on well-defined sample size estimations to detect statistically significant effects of trialled interventions; however these are often predicted poorly by investigators. Here, we review the accuracy of predicted parameters used in sample size calculations for malaria CRTs with epidemiological outcomes. Methods We searched for published malaria CRTs using four online databases in March 2022. Eligible trials included those with malaria-specific epidemiological outcomes which randomised at least six geographical clusters to study arms. Predicted and observed sample size parameters were extracted by reviewers for each trial. Pair-wise correlation coefficients were calculated to assess the accuracy of predicted control-arm outcome estimates and desired relative reductions (effect sizes) between arms compared to what was observed. Among trials which retrospectively calculated an estimate of heterogeneity in cluster outcomes, we recalculated study power according to observed trial estimates. Results Of the 1889 records identified and screened, 108 articles were eligible and comprised of 71 malaria CRTs. Among trials that included sample size calculations (91.5%, 65/71), most estimated cluster heterogeneity using the coefficient of variation (k) (80%, 52/65) which were often predicted without using prior data (67.7%, 44/65). Predicted control-arm epidemiological outcomes correlated weakly with those observed, with 61.2% (19/31) of prevalence estimates overestimated. Among the minority of trials which retrospectively calculated cluster heterogeneity (20%, 13/65), empirical values contrasted with those used in sample size estimations and often compromised study power. Observed effect sizes were often smaller than had been predicted at the sample size stage (72.9%, 51/70) and were typically higher in the first, compared to the second, year of trials. Overall, effect sizes achieved by malaria interventions tested in trials decreased between 1995 and 2021. Conclusions Study findings reveal sample size parameters in malaria CRTs were often inaccurate and resulted in underpowered studies. Future trials must strive to obtain more representative epidemiological sample size inputs to ensure interventions against malaria are adequately evaluated. Registration This review is registered with PROSPERO (CRD42022315741). ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This research is supported by a grant to the London School of Hygiene and Tropical Medicine from the Bill & Melinda Gates Foundation (INV-038132). JDC, JH, and TC also acknowledge funding from the MRC Centre for Global Infectious Disease Analysis (reference MR/X020258/1), funded by the UK Medical Research Council (MRC). This UK funded award is carried out in the frame of the Global Health EDCTP3 Joint Undertaking. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data produced in the present work are contained in the manuscript * CRT : Cluster randomised trial ES : Effect size ICC : Intracluster correlation coefficient IRS : Indoor residual spraying ITN : Insecticide-treated bed nets k : coefficient of variation LLIN : Long-lasting insecticidal nets pa : Per annum PCR : Polymerase chain reaction py : Person-year RDT : Rapid diagnostic test WHO : World Health Organisation