Blind validation of MSIntuit, an AI-based pre-screening tool for MSI detection from histology slides of colorectal cancer

Objective Mismatch Repair Deficiency (dMMR) / Microsatellite Instability (MSI) is a key biomarker in colorectal cancer (CRC). Universal screening of CRC patients for dMMR/MSI status is now recommended, but contributes to increased workload for pathologists and delayed therapeutic decisions. Deep learning has the potential to ease dMMR/MSI testing in clinical practice, yet no comprehensive validation of a clinically approved tool has been conducted. Design We developed an MSI pre-screening tool, MSIntuit, that uses deep learning to identify MSI status from H&E slides. For training, we used 859 slides from the TCGA database. A blind validation was subsequently performed on an independent dataset of 600 consecutive CRC patients. Each slide was digitised using Phillips-UFS and Ventana-DP200 scanners. Thirty dMMR/MSI slides were used for calibration on each scanner. Prediction was then performed on the remaining 570 patients following an automated quality check step. The inter and intra-scanner reliability was studied to assess MSIntuit’s robustness. Results MSIntuit reached a sensitivity and specificity of 97% (95% CI: 93-100%) / 46% (42-50%) on DP200 and of 95% (90-98%) / 47% (43-51%) on UFS scanner. MSIntuit reached excellent agreement on the two scanners (Cohen’s κ: 0.82) and was repeatable across multiple rescanning of the same slide (Fleiss’ κ: 0.82). Conclusion We performed a successful blind validation of the first clinically approved AI-based tool for MSI detection from H&E slides. MSIntuit reaches sensitivity comparable to gold standard methods (92-95%) while ruling out almost half of the non-MSI population, paving the way for its use in clinical practice. ### Competing Interest Statement CS, RD, OT, NL, SV, M. Sefta, MA, LG, AF are employees of Owkin Inc. J.N.K. declares consulting services for Owkin, France, for Panakeia Technologies, UK and for DoMore Diagnostics, Norway. M. Svrcek declares consulting services for Owkin, France. ### Funding Statement This study was funded by Owkin. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: All experiments were conducted in accordance with the General Data Protection Regulation (GDPR) and the French laws and regulations. Medipath data subjects have generally been informed for the re-use of their samples and data collected during the care for research purposes. Medipath has obtained an approval of the "Ministere de l'Enseignement Superieur, de la Recherche et de l'Innovation (MESRI)" for the storage of samples for research purposes and has nominatively reinformed patients for the reuse of their data for Owkin's experiments. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes The discovery TCGA dataset is publicly available at the TCGA data portal (https://portal.gdc.cancer.gov). The external validation PAIP dataset is available at https://paip2020.grand-challenge.org/. Other data produced in the present study are available upon reasonable request to the authors. The discovery TCGA dataset is publicly available at the TCGA data portal (). The external validation PAIP dataset is available at . Other data produced in the present study are available upon reasonable request to the authors.