Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration.
CoRR(2023)
Abstract
Large Language Models (LLMs) have shown remarkable capabilities in general
natural language processing tasks but often fall short in complex reasoning
tasks. Recent studies have explored human-like problem-solving strategies, such
as self-correct, to push further the boundary of single-model reasoning
ability. In this work, we let a single model "step outside the box" by engaging
multiple models to correct each other. We introduce a multi-agent collaboration
strategy that emulates the academic peer review process. Each agent
independently constructs its own solution, provides reviews on the solutions of
others, and assigns confidence levels to its reviews. Upon receiving peer
reviews, agents revise their initial solutions. Extensive experiments on three
different types of reasoning tasks show that our collaboration approach
delivers superior accuracy across all ten datasets compared to existing
methods. Further study demonstrates the effectiveness of integrating confidence
in the reviews for math reasoning, and suggests a promising direction for
human-mimicking multi-agent collaboration process.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined