Evaluating undercounts in epidemics: Response to Maruotti et al. (2022)

Several papers1-3 have promoted formulas that claim to provide bounds on the completeness of sampling of infectious disease cases, based only on case reports. We believe these approaches are fundamentally flawed, and that it is impossible to estimate undercounting from incidence data without a specialized sampling design or some kind of auxiliary information. This approach misuses the mark-recapture formulas. Cases identified at time t − 1 $t-1$ are claimed to be representative of the number of cases counted twice: why? The fact that the same individual could be counted twice in the cumulative case report (for some sampling designs) is irrelevant. How can comparing yesterday's count to today's provide information about the completeness of sampling? In principle, the number of unobserved (hidden) cases could be estimated if cases can be reidentified, or even with unmarked/unidentified cases given an appropriate sampling design.6 In practice public health case reporting rarely uses such sampling designs. Case reporting is usually exclusive (i.e., someone who has been identified as a case will not be reported again later), or anonymized so that we cannot identify which particular individuals are double-counted because they are infected, and sampled, in two different case-reporting instances. Mark-recapture methods can provide valuable public health information in specific scenarios such as contact-tracing studies, but “one needs at least two sources of information with individual case reporting and a unique personal identifier for each case”.7 This limitation is fundamental to mark-recapture methods; standard case-reporting time series, which do not identifiably resample the same individuals, provide no information with which we could estimate the fraction of the population observed. Applying a bias correction decreases the lower bound on the number of hidden cases, thus increasing the upper bound on a ˆ $\hat{a}$ . The results also depend on the overall number of reported cases, so the pattern is more complicated, but as we show below the estimated upper and lower bounds are still largely independent of the true ascertainment ratio. We ran each simulation for 100 days and used the R package asymptor11 to compute bounds on the ascertainment ratio. The authors indicated (pers. comm.) that they intended the estimator to be used at the beginning of an epidemic. Therefore we considered only sample points when the number of cases was between 5 and 500 (exclusive) and the lower bound estimator for hidden cases was greater than 1. For each simulation run (80 in total), we computed the mean and confidence intervals for the estimated lower and upper bounds of a ˆ $\hat{a}$ over time (Figure 1). The bounds on a ˆ $\hat{a}$ rarely overlap the true value, and are largely independent of the true values of a $a$ . The only noticeable signal arises from the bias-correction terms: simulations with lower overall case numbers (low r $r$ , low a , Δ t = 1 $a,{\rm{\Delta }}t=1$ ) have larger lower bounds and smaller upper bounds. The relationship between a ˆ $\hat{a}$ and the growth rate r $r$ is barely visible as increasing values of the upper bound with r $r$ for the cases with Δ t = 1 ${\rm{\Delta }}t=1$ and low true ascertainment ratio; otherwise, this pattern is swamped by the effects of noise and bias correction. In simulations without noise and with the simpler, non-bias-corrected expression for the lower bound (not shown), the lower-bound estimates of a ˆ $\hat{a}$ are completely independent of a $a$ , as expected from the mathematical argument given above. We conclude that the authors' formulas appear to work well because they lead to plausible bounds on the ascertainment ratio ( ≈ $\approx $ 0.2–0.5) for realistic values of the epidemic growth rate, but that they are in fact nearly unrelated to the true ascertainment ratio and should not be applied to estimate ascertainment ratios from disease outbreak incidence data. All authors contributed to the conceptual development of the paper. Michael Li and Benjamin M. Bolker wrote computer code for simulations and figures. Benjamin M. Bolker wrote the first draft of the paper. All authors commented and edited to produce the final version. The authors declare no conflict of interest. No data are used in the paper; all code for simulations is available at https://doi.org/10.5281/zenodo.7473422.