An Alternative Curriculum Learning Approach with Macro Actions for Deep Reinforcement Learning
2022 3rd International Conference on Electronics, Communications and Information Technology (CECIT)(2022)
摘要
It is challenging to train deep reinforcement learning (DRL) agents for tasks with sparse rewards, especially for those that require combining discrete actions with flexible control, e.g., the scoring task for robots in RoboCup soccer. Curriculum learning has been considered as a useful tool for DRL to accelerate the learning process by splitting the task into a sequence of progressively more difficult subtasks. In this paper, we propose an alternative curriculum learning approach that firstly makes the task easier by introducing macro actions, i.e., predefined procedures of actions for solving several subtasks. Once a DRL agent has been well trained for the easier task, the approach will disable one of the macro actions and continuously train the DRL agent for the refined task. The approach continues the process until all macro actions were disabled and a DRL agent were trained for the original task. Experiments on the Half Field Offense (HFO) platform for simulated RoboCup soccer show that the approach is effective and able to learn robust policies for challenging tasks, like HFO 2v2, which has not been reported to be solved by DRL as far as the knowledge of the authors. A demonstration video is available online at https://youtu.be/e6kunbAhhaY.
更多查看译文
关键词
Reinforcement Learning,Curriculum Learning,Macro Actions,Parameterized Action Space
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要