I2CKD : Intra- and Inter-Class Knowledge Distillation for Semantic Segmentation
CoRR(2024)
Abstract
This paper proposes a new knowledge distillation method tailored for image
semantic segmentation, termed Intra- and Inter-Class Knowledge Distillation
(I2CKD). The focus of this method is on capturing and transferring knowledge
between the intermediate layers of teacher (cumbersome model) and student
(compact model). For knowledge extraction, we exploit class prototypes derived
from feature maps. To facilitate knowledge transfer, we employ a triplet loss
in order to minimize intra-class variances and maximize inter-class variances
between teacher and student prototypes. Consequently, I2CKD enables the student
to better mimic the feature representation of the teacher for each class,
thereby enhancing the segmentation performance of the compact network.
Extensive experiments on three segmentation datasets, i.e., Cityscapes, Pascal
VOC and CamVid, using various teacher-student network pairs demonstrate the
effectiveness of the proposed method.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined