GPT-4 vs. Human Translators: A Comprehensive Evaluation of Translation Quality Across Languages, Domains, and Expertise Levels
arxiv(2024)
摘要
This study comprehensively evaluates the translation quality of Large
Language Models (LLMs), specifically GPT-4, against human translators of
varying expertise levels across multiple language pairs and domains. Through
carefully designed annotation rounds, we find that GPT-4 performs comparably to
junior translators in terms of total errors made but lags behind medium and
senior translators. We also observe the imbalanced performance across different
languages and domains, with GPT-4's translation capability gradually weakening
from resource-rich to resource-poor directions. In addition, we qualitatively
study the translation given by GPT-4 and human translators, and find that GPT-4
translator suffers from literal translations, but human translators sometimes
overthink the background information. To our knowledge, this study is the first
to evaluate LLMs against human translators and analyze the systematic
differences between their outputs, providing valuable insights into the current
state of LLM-based translation and its potential limitations.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要