Prompt-driven Healthy/Diseased Image Pairs Enabling Pixel-level Chest X-ray Pathology Localization

Abstract Medical artificial intelligence (AI) offers great potential for automatic pathology interpretation, but the performance is far behind providing a practical tool in clinical settings, which demands both pixel-level accuracy and high interpretability for diagnosis. The main challenges lie in that the construction of such AI models relies on substantial training data with fine-grained labeling that is impractical in real applications. To circumvent this barrier, we propose a prompt-driven constrained generative model to produce anatomically aligned healthy/diseased image pairs and then learn a superb pathology localization model in a supervised manner. The new paradigm effectively addresses the lack of high-fidelity chest X-ray images with pathology labeling at fine scales. Benefiting from the emerging text-driven generative foundation model and the newly incorporated constraint, our model presents promising localization accuracy of the subtle pathologies, high interpretability for clinical decisions, and good transfer ability to many unseen pathological categories (e.g., new prompts and mixed pathologies). These advantageous features establish our model as a promising solution to assist chest X-ray analysis. Besides, the proposed approach is also inspiring for other tasks lacking massive training data and time-consuming manual labeling.