xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning
CoRR(2024)
摘要
Chain-of-thought (CoT) has emerged as a powerful technique to elicit
reasoning in large language models and improve a variety of downstream tasks.
CoT mainly demonstrates excellent performance in English, but its usage in
low-resource languages is constrained due to poor language generalization. To
bridge the gap among different languages, we propose a cross-lingual
instruction fine-tuning framework (xCOT) to transfer knowledge from
high-resource languages to low-resource languages. Specifically, the
multilingual instruction training data (xCOT-INSTRUCT) is created to encourage
the semantic alignment of multiple languages. We introduce cross-lingual
in-context few-shot learning (xICL)) to accelerate multilingual agreement in
instruction tuning, where some fragments of source languages in examples are
randomly substituted by their counterpart translations of target languages.
During multilingual instruction tuning, we adopt the randomly online CoT
strategy to enhance the multilingual reasoning ability of the large language
model by first translating the query to another language and then answering in
English. To further facilitate the language transfer, we leverage the
high-resource CoT to supervise the training of low-resource languages with
cross-lingual distillation. Experimental results on previous benchmarks
demonstrate the superior performance of xCoT in reducing the gap among
different languages, highlighting its potential to reduce the cross-lingual
gap.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要