ALF: Adaptive Label Finetuning for Scene Graph Generation
CoRR(2023)
Abstract
Scene Graph Generation (SGG) endeavors to predict the relationships between
subjects and objects in a given image. Nevertheless, the long-tail distribution
of relations often leads to biased prediction on coarse labels, presenting a
substantial hurdle in SGG. To address this issue, researchers focus on unbiased
SGG and introduce data transfer methods to transfer coarse-grained predicates
into fine-grained ones across the entire dataset. However, these methods
encounter two primary challenges: 1) They overlook the inherent context
constraints imposed by subject-object pairs, leading to erroneous relations
transfer. 2) Additional retraining process are required after the data
transfer, which incurs substantial computational costs. To overcome these
limitations, we introduce the first plug-and-play one-stage data transfer
pipeline in SGG, termed Adaptive Label Finetuning (ALF), which eliminates the
need for extra retraining sessions and meanwhile significantly enhance models'
relation recognition capability across various SGG benchmark approaches.
Specifically, ALF consists of two components: Adaptive Label Construction (ALC)
and Adaptive Iterative Learning (AIL). By imposing Predicate-Context
Constraints within relation space, ALC adaptively re-ranks and selects
candidate relations in reference to model's predictive logits utilizing the
Restriction-Based Judgment techniques, achieving robust relation transfer.
Supervised with labels transferred by ALC, AIL iteratively finetunes the SGG
models in an auto-regressive manner, which mitigates the substantial
computational costs arising from the retraining process. Extensive experiments
demonstrate that ALF achieves a 16
typical SGG method Motif, with only a 6
to the state-of-the-art method IETrans.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined