Epitweetr: Early Warning of Public Health Threats Using Twitter Data

Background: ECDC performs epidemic intelligence activities to systematically collate information from a variety of sources, including Twitter, to rapidly detect public health events. The lack of a freely available, customisable and automated early warning tool using Twitter data, prompted ECDC to develop epitweetr. The specific objectives are to assess the performance of the geolocation and signal detection algorithms used by epitweetr and to assess the performance of epitweetr in comparison with the manual monitoring of Twitter for early detection of public health threats. Methods: Epitweetr collects, geolocates and aggregates tweets to generate signals and email alerts. Firstly, we evaluated manually the tweet geolocation characteristics of 1,200 tweets, and assessed its accuracy in extracting the correct location and its performance in detecting tweets with available information on the tweet geolocation. Secondly, we evaluated signals generated by epitweetr between 19 October and 30 November 2020 and we calculated the positive predictive value (PPV). Then, we evaluated the sensitivity, specificity and timeliness of epitweetr in comparison with Twitter manual monitoring. Findings: The epitweetr geolocation algorithm had an accuracy of 30·1% and 25·9% at national and subnational levels, respectively. General and specific PPV of the signal detection algorithm was 3·0% and 74·6%, respectively. Epitweetr and/or manual monitoring detected 570 signals and 454 events. Epitweetr had a sensitivity of 78·6% [75·2% - 82·0%] and PPV of 74·6% [70·5% - 78·6%]; and the manual monitoring had a sensitivity of 47·9% [43·8% - 52·0%] and PPV of 97·9% [95·8% - 99·9%]. The median validation time difference between sixteen common events detected by epitweetr and manual monitoring was -48·6 hours [(-102.8) - (-23·7) hours]. Interpretation: Epitweetr has shown to have sufficient performance as an early warning tool for public health threats using Twitter data. Having developed epitweetr as a free, open-source tool with several configurable settings and a strong automated component, it is expected to increase its usability and usefulness to public health experts. Funding Statement: Not applicable. Declaration of Interests: The authors declare no conflicts of interest. Ethics Approval Statement: Epitweetr collects Twitter data using the Twitter Standard Search API which provides relevant and only publicly available tweets matching a specific query from the previous seven days. These data are similar but not identical to the results provided by the Search User Interface feature in Twitter mobile or web clients.