Quality-of-care comparison of stroke: The reliability and robustness of ranking by process or outcome measures

Background and aim Discussion on the most rational types of performance measures for care quality comparisons has received increasing attention. The important consideration is to what extent will the measure detect a genuine difference in the underlying quality. In this study, we aimed to compare the ranking of hospitals on the performance of individual indicators, composite scores (CS, that were calculated by the method of opportunity-based score on patient-level), and in-hospital outcome of acute ischemic stroke across hospitals, and determined the reliability and robustness of the three types of ranking. Methods We analyzed data from 15,090 patients diagnosed with acute ischemic stroke who were treated at 184 large tertiary hospitals from January 2014 to May 2017. We ranked the hospital effects of recombinant tissue plasminogen activator (rt-PA) and CS and independence (modified Rankin Scale <= 2) at discharge based on fixed- and random-effects regression models before and after case-mix adjustment. We assessed the time-robustness of the hospital effects and calculated the rankability by relating the uncertainty within the hospital and the total hospital variation "beyond chance." Results After case-mix and reliability adjustment, we estimated that 84.03% of the variance in CS between hospitals was due to true quality differences. The uncertainty within hospitals caused a poor (49.51%) rankability in rt-PA and moderate rankability (63.34%) in independence at discharge. The hospital rankings of CS were more robust across years compared with rt-PA and independence. Conclusions Our data indicated that CS is the optimal measure to indicate the quality-of-care variation of acute ischemic stroke between hospitals.