The Critical Assessment of Genome Interpretation: A community experiment that informs use of methods for germline cancer variant impact prediction.

Interpretation of genomic variation plays an essential role in the analysis of cancer and monogenic disease, with applications ranging from basic research to clinical decisions. Yet the field lacks a clear consensus on the appropriate level of confidence to place in variant impact and interpretation methods, for both well-established oncogenes as well as less understood genes. The Critical Assessment of Genome Interpretation (CAGI, \9kā-jē\) is a community experiment to objectively assess computational methods for predicting the phenotypic impacts of genomic variation. CAGI participants are provided genetic variants from genes like BRCA1 and TP53, and make blind predictions of resulting phenotype. Independent assessors evaluate the predictions by comparing them against experimental and clinical data. Each of the five CAGI editions over the past decade has revealed new aspects of the methods, and there has been significant progress in several areas. CAGI challenges from cancer case-control studies have revealed that missense methods tend to correlate better with each other than with experiment (RAD50 breast cancer challenge), an observation also encountered in other CAGI missense challenges. Bespoke approaches can often enhance performance, as seen in the p16 challenge involving variants of unknown significance in familial malignant melanoma, while the p53 reactivation challenge showed that biophysical simulations can occasionally make exceptional predictions. Method performance can vary greatly, even within proteins of the same complex (MRN complex breast cancer challenge). CAGI shows promise meta-predictors (BRCA1 and BRCA2 challenges). Interpretation of some non-coding variants shows promise, as exemplified by the TP53 splicing challenge. Top missense prediction methods are highly statistically significant, but individual variant accuracy is limited. Data from CAGI, including the CHEK2 challenge, were recently presented within the ClinGen Sequence Variant Interpretation working group to explore increasing evidence weighting of computational methods, when those methods demonstrate sufficient reliability for a specific gene or disorder. CAGI results suggest that running multiple uncalibrated methods and considering their consensus may result in undue confidence, so we advise against this. Overall, CAGI has helped establishing the state of art in genome interpretation, encouraged new methodological developments, and informed the clinical application of computational predictors. Results from previous CAGI experiments have been described in two special issues of Human Mutation (vol. 38(9) and vol. 40(9)). Detailed information about CAGI may be found at https://genomeinterpretation.org. Citation Format: Constantina Bakolitsa, Gaia Andreoletti, Roger Hoskins, Predrag Radivojac, John Moult, Steven Brenner, CAGI participants. The Critical Assessment of Genome Interpretation: A community experiment that informs use of methods for germline cancer variant impact prediction [abstract]. In: Proceedings of the Annual Meeting of the American Association for Cancer Research 2020; 2020 Apr 27-28 and Jun 22-24. Philadelphia (PA): AACR; Cancer Res 2020;80(16 Suppl):Abstract nr LB-250.