GRADE Concept Paper 8: Judging the certainty of discrimination performance estimates of prognostic models in a body of validation studies

Background Prognostic models incorporate multiple prognostic factors to estimate the likelihood of future events for individual patients based on their prognostic factor values. Evaluating these models crucially involves conducting studies to assess their predictive performance, like discrimination. Systematic reviews and meta-analyses of these evaluation studies play an essential role in selecting models for clinical practice. Methods In this paper, we outline three thresholds to determine the target for certainty rating in the discrimination of prognostic models, as observed across a body of validation studies. Results and Conclusion We propose three thresholds when rating the certainty of evidence about a prognostic model's discrimination. The first threshold amounts to rating certainty in the model's ability to classify better than random chance. The other two approaches involve setting thresholds informed by other mechanisms for classification: clinician intuition or an alternative prognostic model developed for the same disease area and outcome. The choice of threshold will vary based on the context. Instead of relying on arbitrary discrimination cut-offs, our approach positions the observed discrimination within an informed spectrum, potentially aiding decisions about a prognostic model’s practical utility.