T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergy
CoRR(2024)
摘要
We present T-Rex2, a highly practical model for open-set object detection.
Previous open-set object detection methods relying on text prompts effectively
encapsulate the abstract concept of common objects, but struggle with rare or
complex object representation due to data scarcity and descriptive limitations.
Conversely, visual prompts excel in depicting novel objects through concrete
visual examples, but fall short in conveying the abstract concept of objects as
effectively as text prompts. Recognizing the complementary strengths and
weaknesses of both text and visual prompts, we introduce T-Rex2 that synergizes
both prompts within a single model through contrastive learning. T-Rex2 accepts
inputs in diverse formats, including text prompts, visual prompts, and the
combination of both, so that it can handle different scenarios by switching
between the two prompt modalities. Comprehensive experiments demonstrate that
T-Rex2 exhibits remarkable zero-shot object detection capabilities across a
wide spectrum of scenarios. We show that text prompts and visual prompts can
benefit from each other within the synergy, which is essential to cover massive
and complicated real-world scenarios and pave the way towards generic object
detection. Model API is now available at
.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要