Improving Socratic Question Generation using Data Augmentation and Preference Optimization
CoRR(2024)
摘要
The Socratic method is a way of guiding students toward solving a problem
independently without directly revealing the solution to the problem. Although
this method has been shown to significantly improve student learning outcomes,
it remains a complex labor-intensive task for instructors. Large language
models (LLMs) can be used to augment human effort by automatically generating
Socratic questions for students. However, existing methods that involve
prompting these LLMs sometimes produce invalid outputs, e.g., those that
directly reveal the solution to the problem or provide irrelevant or premature
questions. To alleviate this problem, inspired by reinforcement learning with
AI feedback (RLAIF), we first propose a data augmentation method to enrich
existing Socratic questioning datasets with questions that are invalid in
specific ways. Next, we propose a method to optimize open-source LLMs such as
LLama 2 to prefer ground-truth questions over generated invalid ones, using
direct preference optimization (DPO). Our experiments on a Socratic questions
dataset for student code debugging show that a DPO-optimized 7B LLama 2 model
can effectively avoid generating invalid questions, and as a result,
outperforms existing state-of-the-art prompting methods.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要