SuperCLUE-Math6: Graded Multi-Step Math Reasoning Benchmark for LLMs in Chinese
CoRR(2024)
Abstract
We introduce SuperCLUE-Math6(SC-Math6), a new benchmark dataset to evaluate
the mathematical reasoning abilities of Chinese language models. SC-Math6 is
designed as an upgraded Chinese version of the GSM8K dataset with enhanced
difficulty, diversity, and application scope. It consists of over 2000
mathematical word problems requiring multi-step reasoning and providing natural
language solutions. We propose an innovative scheme to quantify the reasoning
capability of large models based on performance over problems with different
reasoning steps. Experiments on 12 representative Chinese models demonstrate a
clear stratification of reasoning levels, with top models like GPT-4 showing
superior performance. SC-Math6 fills the gap in Chinese mathematical reasoning
benchmarks and provides a comprehensive testbed to advance the intelligence of
Chinese language models.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined