Vision Transformer Based Multi-class Lesion Detection in IVOCT

Cardiovascular disease is a high-fatality illness. Intravascular Optical Coherence Tomography (IVOCT) technology can significantly assist in diagnosing and treating cardiovascular diseases. However, locating and classifying lesions from hundreds of IVOCT images is time-consuming and challenging, especially for junior physicians. An automatic lesion detection and classification model is desirable. To achieve this goal, in this work, we first collect an IVOCT dataset, including 2,988 images from 69 IVOCT data and 4,734 annotations of lesions spanning over three categories. Based on the newly-collected dataset, we propose a multi-class detection model based on Vision Transformer, called G-Swin Transformer. The essential part of our model is grid attention which is used to model relations among consecutive IVOCT images. Through extensive experiments, we show that the proposed G-Swin Transformer can effectively localize different types of lesions in IVOCT images, significantly outperforming baseline methods in all evaluation metrics. Our code is available via this link. https://github.com/Shao1Fan/G-Swin-Transformer