Selective Mixup Fine-Tuning for Optimizing Non-Decomposable Objectives
CoRR(2024)
摘要
The rise in internet usage has led to the generation of massive amounts of
data, resulting in the adoption of various supervised and semi-supervised
machine learning algorithms, which can effectively utilize the colossal amount
of data to train models. However, before deploying these models in the real
world, these must be strictly evaluated on performance measures like worst-case
recall and satisfy constraints such as fairness. We find that current
state-of-the-art empirical techniques offer sub-optimal performance on these
practical, non-decomposable performance objectives. On the other hand, the
theoretical techniques necessitate training a new model from scratch for each
performance objective. To bridge the gap, we propose SelMix, a selective
mixup-based inexpensive fine-tuning technique for pre-trained models, to
optimize for the desired objective. The core idea of our framework is to
determine a sampling distribution to perform a mixup of features between
samples from particular classes such that it optimizes the given objective. We
comprehensively evaluate our technique against the existing empirical and
theoretically principled methods on standard benchmark datasets for imbalanced
classification. We find that proposed SelMix fine-tuning significantly improves
the performance for various practical non-decomposable objectives across
benchmarks.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要