Many-Objective Multi-Solution Transport
arxiv(2024)
摘要
Optimizing the performance of many objectives (instantiated by tasks or
clients) jointly with a few Pareto stationary solutions (models) is critical in
machine learning. However, previous multi-objective optimization methods often
focus on a few number of objectives and cannot scale to many objectives that
outnumber the solutions, leading to either subpar performance or ignored
objectives. We introduce Many-objective multi-solution Transport (MosT), a
framework that finds multiple diverse solutions in the Pareto front of many
objectives. Our insight is to seek multiple solutions, each performing as a
domain expert and focusing on a specific subset of objectives while
collectively covering all of them. MosT formulates the problem as a bi-level
optimization of weighted objectives for each solution, where the weights are
defined by an optimal transport between the objectives and solutions. Our
algorithm ensures convergence to Pareto stationary solutions for complementary
subsets of objectives. On a range of applications in federated learning,
multi-task learning, and mixture-of-prompt learning for LLMs, MosT distinctly
outperforms strong baselines, delivering high-quality, diverse solutions that
profile the entire Pareto frontier, thus ensuring balanced trade-offs across
many objectives.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要