The relation between prediction model performance measures and patient selection outcomes for proton therapy in head and neck cancer

•All studied discrimination and calibration measures independently impacted expected risk reductions (ΔNTCP) and patient selection outcomes and should thus be assessed at external validation.•AUC had the largest impact on patient selection, and it was therefore suggested, if multiple models predicting the same outcome are compared at external validation, to select the model with the highest AUC, and consider model recalibration if needed.•Some prediction errors have more impact on clinical decision making than others and thus we wish to stress the importance of treatment decision protocols, linking predictions to treatment decisions, as they facilitate investigation of impact of prediction errors on actual clinical decisions.