Modality Translation for Object Detection Adaptation Without Forgetting Prior Knowledge
arxiv(2024)
摘要
A common practice in deep learning consists of training large neural networks
on massive datasets to perform accurately for different domains and tasks.
While this methodology may work well in numerous application areas, it only
applies across modalities due to a larger distribution shift in data captured
using different sensors. This paper focuses on the problem of adapting a large
object detection model to one or multiple modalities while being efficient. To
do so, we propose ModTr as an alternative to the common approach of fine-tuning
large models. ModTr consists of adapting the input with a small transformation
network trained to minimize the detection loss directly. The original model can
therefore work on the translated inputs without any further change or
fine-tuning to its parameters. Experimental results on translating from IR to
RGB images on two well-known datasets show that this simple ModTr approach
provides detectors that can perform comparably or better than the standard
fine-tuning without forgetting the original knowledge. This opens the doors to
a more flexible and efficient service-based detection pipeline in which,
instead of using a different detector for each modality, a unique and unaltered
server is constantly running, where multiple modalities with the corresponding
translations can query it. Code: https://github.com/heitorrapela/ModTr.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要