Laying the Foundation First? Investigating the Generalization from Atomic Skills to Complex Reasoning Tasks
CoRR(2024)
摘要
Current language models have demonstrated their capability to develop basic
reasoning, but struggle in more complicated reasoning tasks that require a
combination of atomic skills, such as math word problem requiring skills like
arithmetic and unit conversion. Previous methods either do not improve the
inherent atomic skills of models or not attempt to generalize the atomic skills
to complex reasoning tasks. In this paper, we first propose a probing framework
to investigate whether the atomic skill can spontaneously generalize to complex
reasoning tasks. Then, we introduce a hierarchical curriculum learning training
strategy to achieve better skill generalization. In our experiments, we find
that atomic skills can not spontaneously generalize to compositional tasks. By
leveraging hierarchical curriculum learning, we successfully induce
generalization, significantly improve the performance of open-source LMs on
complex reasoning tasks. Promisingly, the skill generalization exhibit
effective in cross-dataset and cross-domain scenarios. Complex reasoning can
also help enhance atomic skills. Our findings offer valuable guidance for
designing better training strategies for complex reasoning tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要