Accuracy of Risk Models Used for Public Reporting of Heart Transplant Center Performance
The Journal of Heart and Lung Transplantation(2021)
Abstract
BACKGROUND: Heart transplant programs and regulatory entities require highly accurate performance metrics to support internal quality improvement activities and national oversight of transplant programs, respectively. We assessed the accuracy of publicly reported performance measures. METHODS: We used the United Network for Organ Sharing registry to study patients who underwent heart transplantation between January 1, 2016 and June 30, 2018. We used tests of calibration to compare the observed rate of 1-year graft failure to the expected risk of 1-year graft failure, which was calculated for each recipient using the July 2019 method published by the Scientific Registry of Transplant Recipients (SRTR). The primary study outcome was the joint test of calibration, which accounts for both the total number of events predicted (calibration-in-the-large) and dispersion of risk predictions (calibration slope). RESULTS: 6,528 heart transplants were analyzed. The primary test of calibration failed (p <0.0001), indicating poor accuracy of the SRTR model. The calibration-in-the-large statistic (0.63, 95% confidence interval [CI] 0.58-0.68, p < 0.0001) demonstrated overestimation of event rates while the calibration slope statistic (0.56, 95% CI 0.49-0.62, p <0.0001) indicated over-dispersion of event rates. Pre-specified subgroup analyses demonstrated poor calibration for all subgroups (each p <0.01). After recalibration, program-level observed/expected ratios increased by a median of 0.14 (p <0.0001). CONCLUSIONS: Risk models employed for publicly-reported graft survival at U.S. heart transplant centers lack accuracy in general and in all subgroups tested. The use of disease-specific models may improve the accuracy of program performance metrics. J Heart Lung Transplant 2021;40:1571-1578 (c) 2021 International Society for Heart and Lung Transplantation. All rights reserved.
MoreTranslated text
Key words
heart transplant,model calibration,risk model
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined