Phasic Diversity Optimization for Population-Based Reinforcement Learning
CoRR(2024)
摘要
Reviewing the previous work of diversity Rein-forcement Learning,diversity is
often obtained via an augmented loss function,which requires a balance between
reward and diversity.Generally,diversity optimization algorithms use
Multi-armed Bandits algorithms to select the coefficient in the pre-defined
space. However, the dynamic distribution of reward signals for MABs or the
conflict between quality and diversity limits the performance of these methods.
We introduce the Phasic Diversity Optimization (PDO) algorithm, a
Population-Based Training framework that separates reward and diversity
training into distinct phases instead of optimizing a multi-objective function.
In the auxiliary phase, agents with poor performance diversified via
determinants will not replace the better agents in the archive. The decoupling
of reward and diversity allows us to use an aggressive diversity optimization
in the auxiliary phase without performance degradation. Furthermore, we
construct a dogfight scenario for aerial agents to demonstrate the practicality
of the PDO algorithm. We introduce two implementations of PDO archive and
conduct tests in the newly proposed adversarial dogfight and MuJoCo
simulations. The results show that our proposed algorithm achieves better
performance than baselines.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要