Evaluation of the Use of Cancer Registry Data for Comparative Effectiveness Research.

This comparative effectiveness study examines the concordance of data from an observational cancer registry with data from randomized clinical trials. Question Are the findings of comparative effectiveness analyses using cancer registry data concordant with the data from randomized clinical trials? Findings In this comparative effectiveness study replicating 141 randomized clinical trials in the National Cancer Database, concordant hazard ratios for overall survival were noted in 56% to 70% of the analyses and concordant P values were noted in 41% to 46% of the analyses. No particular clinical trial features appeared to be associated with concordant hazard ratios. Meaning The findings of this study suggest that comparative effectiveness research using the National Cancer Database often produces results discordant with randomized clinical trials. Importance Researchers often analyze cancer registry data to assess for differences in survival among cancer treatments. However, the retrospective, nonrandomized design of these analyses raises questions about study validity. Objective To examine the extent to which comparative effectiveness analyses using observational cancer registry data produce results concordant with those of randomized clinical trials. Design, Setting, and Participants In this comparative effectiveness study, a total of 141 randomized clinical trials referenced in the National Comprehensive Cancer Network Clinical Practice Guidelines for 8 common solid tumor types were identified. Data on participants within the National Cancer Database (NCDB) diagnosed between 2004 and 2014, matching the eligibility criteria of the randomized clinical trial, were obtained. The present study was conducted from August 1, 2017, to September 10, 2019. The trials included 85.118 patients, and the corresponding NCDB analyses included 1.344-536 patients. Three Cox proportional hazards regression models were used to determine hazard ratios (HRs) for overall survival, including univariable, multivariable, and propensity score-adjusted models. Multivariable and propensity score analyses controlled for potential confounders, including demographic, comorbidity, clinical, treatment, and tumor-related variables. Main Outcomes and Measures The main outcome was concordance between the results of randomized clinical trials and observational cancer registry data. Hazard ratios with an NCDB analysis were considered concordant if the NDCB HR fell within the 95% CI of the randomized clinical trial HR. An NCDB analysis was considered concordant if both the NCDB and clinical trial P values for survival were nonsignificant (P >= .05) or if they were both significant (P < .05) with survival favoring the same treatment arm in the NCDB and in the randomized clinical trial. Results Analyses using the NCDB-produced HRs for survival were concordant with those of 141 randomized clinical trials in 79 univariable analyses (56%), 98 multivariable analyses (70%), and 90 propensity score models (64%). The NCDB analyses produced P values concordant with randomized clinical trials in 58 univariable analyses (41%), 65 multivariable analyses (46%), and 63 propensity score models (45%). No clinical trial characteristics were associated with concordance between NCDB analyses and randomized clinical trials, including disease site, type of clinical intervention, or severity of cancer. Conclusions and Relevance The findings of this study suggest that comparative effectiveness research using cancer registry data often produces survival outcomes discordant with those of randomized clinical trial data. These findings may help provide context for clinicians and policy makers interpreting observational comparative effectiveness research in oncology.