Cross-Modal Guidance for Hyperfluorescence Segmentation in Fundus Fluorescein Angiography.

For lesions required segmentation are very similar to other background tissues, existing methods often incorrectly segment these tissues into foregrounds. As the fact that diagnostic descriptions can improve the performance of lesion segmentation, but existing methods do not sufficiently capture the correlations between images and the corresponding diagnostic descriptions. In this paper, we propose a novel cross-modal mutual-aware network which utilizes diagnostic descriptions to guide lesion segmentation. Concretely, the proposed network integrates several mutually aware feature fusion module to learn the co-relationship between visual and linguistic features at multiple levels, yielding the final mask which are more closely to the descriptions. We have carried out experiments on an internal dataset and the experimental results show a significant performance improvement compared to other traditional and cross-modal approaches.