The futility of adverse drug event reporting systems for monitoring known safety issues: A case study of myocardial infarction with rofecoxib and other drugs

For over 50 years, spontaneous reporting systems (SRSs) were the only option for identifying unexpected effects of products once marketed.1 Their high sensitivity makes SRSs especially well-suited for identifying rare drug- or vaccine-related outcomes. However, signals for adverse events (AEs) that occur frequently in untreated populations (i.e., those with high background rates) might be harder to detect effectively.2 In addition, SRSs are not appropriate for incidence estimation, and the reporting rates of established, widely publicized issues can be dramatically skewed.1 Strategies based on real-world data (RWD), that is, electronic healthcare databases, could achieve better insight into these questions. In the digital era, numerous RWD sources suitable for conducting safety surveillance exist.3 There are little published data on the topic of costs of AE reporting and safety surveillance in general. However, the management of spontaneous reporting of AEs is far from negligible in terms of time and resource costs. For example, a small study involving a single organization reported a median data ingestion time of 69 min per spontaneous report.4 However, that estimate did not include costs accrued by multiple organizations handling the same report or sharing it with other organizations, while adhering to strict legal requirements.5 In 2021, GSK estimated a mean cost of US$33 per processed spontaneous report and similar estimations were seen across other pharmaceutical companies.6 Arrangements to comply with regulatory reporting requirements for a Phase I randomized clinical trial program can cost ≥$100 000, even when no serious adverse event occurs,7 and the same author also stated that at typical pricing structures for processing AE reports, the likely cost of processing 80 000 cases would be tens of millions of dollars. Rigorous assessments of the cost or cost-effectiveness of national pharmacovigilance systems have not been widely reported,8 although Roll Back Malaria, a national pharmacovigilance program for antimalarial agents estimated in low- and middle-income countries a start-up cost from $150 000 to $250 000 with recurrent costs of around $50 000 per year.8 Globally, there are millions of spontaneous reports in databases,1 many of which are related to either old, even discontinued products, or known adverse effects. One might question whether the resources needed to meet the regulatory requirements to process this tsunami of data can be justified by the gains obtained in terms of improved safety surveillance and benefit–risk understanding. A notorious example illustrating the need for more efficient pharmacovigilance was the discovery that rofecoxib was associated with acute myocardial infarction (AMI).9 Although not widely recognized at the time, a safety signal for marketed rofecoxib use and cardiovascular disorders (based on spontaneous reports) was already detected in October 2000 by the Netherlands Monitoring Centre (Lareb).10 After being confirmed by clinical trial data, the increased risk of cardiovascular events was added to the rofecoxib label in 2002; finally, the drug was withdrawn from the market in 2004.9 Following the rofecoxib–myocardial infarction (MI) issue, many new pharmacovigilance capabilities were developed, including the Sentinel Initiative. It was developed as a direct consequence of concerns about the United States (US) systems' capability to identify safety signals effectively in the future through SRSs and launched in 2008 by the US Food and Drug Administration (FDA).11 Sentinel is a distributed healthcare database network based primarily on transactional insurance claims records, which increasingly incorporates electronic health records, thus enabling postmarketing surveillance of RWD at scale.11, 12 There has been some methodological work exploring how Sentinel might be useful for hypothesis-free signal detection, but it is more recognized for its performance in the routine surveillance of suspected AEs where there are conceptual advantages of RWD compared with SRS data. SRSs are subject to many limitations, including notoriety bias.13 It is well established that inflated reporting of specific outcomes occurs when an issue becomes widely discussed and leads to increased volumes of data to process and negatively impacts any subsequent quantitative analysis of spontaneous reports.14 In contrast, RWD-based approaches, including Sentinel and other distributed healthcare database networks (EDHEN, CNODES, DARWIN, MID-NET, and VSD), allow for more objective recording of data, independent of suspicion of causality. These will enable estimation of incidence rates and, assuming data are updated with sufficient frequency, allow more effective monitoring of changes over time, for example, healthcare usage changes.2, 3 This facilitates the benefit–risk assessment and decision-making processes involving labeled events, where surveillance for changes in severity or frequency of AEs is important for patient safety. Furthermore, highly publicized exposures/outcomes leading to inflated spontaneous reporting, as well as those that occur frequently in an untreated population (as was the case for rofecoxib-AMI) are more accurately captured in RWD repositories than in SRSs.1 AMI is an outcome that is well captured by diagnosis codes in insurance claims data from the US, with a reported positive predictive value of ≥86% for chart review-validated AMI15, 16; evaluations involving data coded in a newer version of the coding system (the one now routinely used in Sentinel) also yielded consistent results.17 Several observational studies published after the rofecoxib label update in 2002 showed an elevated risk of AMI associated with rofecoxib in US RWD.18, 19 How much superfluous spontaneous reporting of rofecoxib-AMI, and more broadly, of rofecoxib-MI, might have been prevented if Sentinel had been in use at that time, and if the superiority of RWD repositories over SRSs in monitoring the rofecoxib-AMI association had been recognized? Moreover, as Sentinel was not in place at the time of the identification and further analysis of the rofecoxib-AMI association, we conducted a more recent analysis of the reporting of labeled MI for any marketed medicinal product since January 2016, when FDA Sentinel and other systems were available. Data were retrieved from the FDA Adverse Event Reporting System (FAERS) public dashboard20 (cutoff date: June 30, 2022). Cumulative monthly distributions were generated for 1998–2009 for rofecoxib-AMI cases (defined as MedDRA preferred term) and for AMI cases following other drugs than rofecoxib according to the report receipt date and the event occurrence date. The same graphs were generated for MI (based on the narrow standardized MedDRA queries [SMQ] definition) and for all outcomes other than MI following rofecoxib. Influence of the evolving safety issue awareness was depicted using reference lines. To evaluate the proportion of labeled MI among all MI reporting since 2016, after Sentinel became operational, the cumulative count of MI (narrow SMQ) for products with MI labeled (including labeling changes during the period) was generated. Products with MI labeled before 2016 were identified from the Side Effect Resource (SIDER) database.21 Products with first-time MI label updates added after 2016 were automatically extracted from the FDA's Drug Safety-related Labeling Changes webpage.22 Analyses were performed using customized programs in SAS version 9.4. More technical details are presented in the supporting information. With regard to SRSs, 99.8% (2076/2080) of rofecoxib-AMI (MedDRA PT) cases in the US were first received by the FDA after the rofecoxib label update (April 2002), while 51.9% (1080/2080) had a reported occurrence date before the label update. As expected, 98.9% (2058/2080) of the reports had a reported occurrence date before rofecoxib had been withdrawn from the market (September 2004); however, 93.2% (1938/2080) were first received by the FDA after the withdrawal, thus having no impact on patient safety (Figure 1A). A broader evaluation of the impact of reporting of the rofecoxib-AMI association, based on a wider definition of related AEs (narrow MedDRA SMQ), shows further increasing in reporting volumes. MI yielded similar percentage results at far larger volume (Figure 1B). Until the end of 2006, 99.9% (17 110/17 119) of rofecoxib-MI cases were reported in the US after the label update and 96.4% (16 511/17 119) after the withdrawal of rofecoxib. Worldwide, 20 239 cases of rofecoxib-MI were reported to FAERS after the label update. To assess whether superfluous reporting of known safety issues is still relevant, many years after the rofecoxib event, we examined spontaneous reports of MI cases in FAERS since the Sentinel system became fully operational (January 01, 2016) until June 30, 2022. During this period, 40.3% (13 277/32 919) of the MI cases reported in the US were associated with products that already included an MI warning in their label (Figure 2). Our findings suggest that, as we strive to improve safety surveillance, the pharmacovigilance field needs to consider what activities of minimal value should be stopped or replaced, to enable a sharper focus on patient safety. We believe that the effort to collect, structure, and share spontaneous AE reports once AEs are known does not result in added public health value, and other data sources, such as RWD, should be considered for the effective monitoring of such AEs. While we have focused on the US given the attention rofecoxib received at the time, and the direct development of Sentinel, the first regulatory use surveillance system of its kind, we note that regulator-use surveillance systems are now evolving in other geographies. While we fully expect the crucial role of spontaneous reporting in identifying safety signals to remain for the foreseeable future, we should begin to reconsider the role of SRSs for known adverse reactions in earnest. We do note that we are not for a moment deriding the value of SRSs. They are highly sensitive to the identification of emerging unexpected safety signals, and to date, there is little, if any, evidence to suggest that other data systems (including Sentinel) can outperform them for this task. However, there are extensive legislative responsibilities with regard to AE reporting, and these vary across geographies.5 Our analysis supports a position that efforts should be made to reduce the burden of reporting of labeled AEs to ensure that pharmacovigilance focus is where it is most needed. Given the existence and routine usage of RWD systems like Sentinel that are better suited to incidence estimation and therefore surveillance of known issues, we propose that, in light of the absolute volume and related burden of case processing and reporting we have observed here, consideration should be given to whether case processing and reporting of labeled AEs could be stopped when such outcomes are well captured in RWD, and more targeted approaches like surveillance conducted in RWD could be considered instead. There may be evidence to suggest reducing reporting requirements around labeled AEs in general and reducing requirements for the additional problem of replicate reporting.5 This is also justification to increase efforts to improve the timeliness, the quality, and the availability of RWD-based systems for safety surveillance. We believe that changes in the way that the field collects, shares, and analyses different data streams can much improve the field of safety surveillance. All authors made substantial contributions to the conception, analysis, and interpretation of the data and approved the submitted manuscript. The authors are grateful for Jeffery Painter's contribution in the automation of extracting drug safety-related labeling changes from the FDA website. Editorial/medical writing support was provided by OPEN Health Communications (London, UK) and the Akkodis Belgium platform (c/o GSK); the Akkodis Belgium platform also provided coordination and design support (c/o GSK). This study and the related publication were sponsored by GlaxoSmithKline Biologicals SA. All authors are employees of GSK. All authors hold shares in GSK as part of their employee remuneration and have no other financial and nonfinancial interests to declare. Supporting information Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.