QueryAgent: A Reliable and Efficient Reasoning Framework with Environmental Feedback-based Self-Correction
CoRR(2024)
Abstract
Employing Large Language Models (LLMs) for semantic parsing has achieved
remarkable success. However, we find existing methods fall short in terms of
reliability and efficiency when hallucinations are encountered. In this paper,
we address these challenges with a framework called QueryAgent, which solves a
question step-by-step and performs step-wise self-correction. We introduce an
environmental feedback-based self-correction method called ERASER. Unlike
traditional approaches, ERASER leverages rich environmental feedback in the
intermediate steps to perform selective and differentiated self-correction only
when necessary. Experimental results demonstrate that QueryAgent notably
outperforms all previous few-shot methods using only one example on GrailQA and
GraphQ by 7.0 and 15.0 F1. Moreover, our approach exhibits superiority in terms
of efficiency, including runtime, query overhead, and API invocation costs. By
leveraging ERASER, we further improve another baseline (i.e., AgentBench) by
approximately 10 points, revealing the strong transferability of our approach.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined