A comparison between a clinical decision support system and clinicians with guidelines in breast cancer

e13551 Background: We are building a clinical decision support system (CSCO AI) for breast cancer patients to improve the efficiency of clinical decision-making. We aimed to assess cancer treatment regimens, in neoadjuvant therapy, adjuvant chemotherapy, adjuvant endocrine therapy, first line therapy and second line therapy, given by CSCO AI and clinicians. Methods: 400 breast cancer patients were screened from the CSCO database. Clinicians with similar levels were randomly assigned one of the volumes (200 cases). After that, clinicians with guidelines were asked to answer the same cases again. CSCO AI was asked to assess all cases. Three reviewers were independently asked to evaluate the regimens from clinicians and CSCO AI. Regimens were masked before evaluation. The primary outcome was the proportion of high-level conformity (HLC), which were defined as the proportions of regimens in accordance with CSCO guidelines. Results: The overall concordance between clinicians and CSCO AI was 67.4% (2350/3500). After referring to the guideline, a total of 22.6% (792/3500) regimens were modified by clinicians, 12.9% (451/3500) had a higher grades and 9.7% (341/3500) had a lower grades. In early stage, the concordance was elevated with statistical significance from 71.3% (1497/2100) to 76.1% (1598/2100, p＜0.001). In the metastatic stage, the concordance was improved form 61.7% (864/1400) to 66.0% (924/1400, p=0.018). HLC in CSCO AI was 95.8% (95%CI:94.0%-97.6%), significantly higher than that in clinicians (90.8%, 95%CI:89.8%-91.8%) and in clinicians with guidelines (92.1%, 95%CI:91.0%-93.4%). In early stage, high-level conformity in CSCO AI was 95.7%, with no statistical significance when compared with clinicians (92.7%, p=0.078) and clinicians with guidelines (92.3%, p=0.050). In metastatic stage, high-level conformity in clinicians was only 88.0%, lower than that in CSCO AI (96.0%, p=0.001). However, after referring guidelines, high-level conformity in clinicians was elevated to 91.9%, with no significant difference when compared with that in CSCO AI (p=0.058). Considering professions, the high level conformity of surgeons was 85.9%, lower than that of CSCO AI (OR=0.25,95%CI: 0.16-0.41). The most significant difference in HLC was in first-line therapy (OR=0.06, 95%CI:0.01-0.41). When clinicians were divided according to their levels, there was no statistical significance between CSCO AI and higher-level clinicians. Conclusions: Clinical decision support for breast cancer was superior for most process outcomes except for second-line therapy. The improvements in process outcomes suggest that CSCO AI can be widely used in clinical practice.