ProstAttention-Net: A deep attention model for prostate cancer segmentation by aggressiveness in MRI scans

•A novel attention CNN for joint multi-class segmentation of prostate and cancer lesions by Gleason Score.•Evaluated on a heterogeneous database of 219 patients (338 lesions) based on prostatectomy.•Performance for the multi-class lesion segmentation task outperforms state-of-the art with a κ=0.418±0.138.•It is ranked first in comparison to 4 state-of-the-art deep segmentation models (Attention U-Net, U-Net, DeepLabv3+, E-Net).•Generalization performance on PROSTATEx-2 ranks among the best models, without any fine-tuning.