What is 'Typological Diversity' in NLP?
CoRR(2024)
摘要
The NLP research community has devoted increased attention to languages
beyond English, resulting in considerable improvements for multilingual NLP.
However, these improvements only apply to a small subset of the world's
languages. Aiming to extend this, an increasing number of papers aspires to
enhance generalizable multilingual performance across languages. To this end,
linguistic typology is commonly used to motivate language selection, on the
basis that a broad typological sample ought to imply generalization across a
broad range of languages. These selections are often described as being
'typologically diverse'. In this work, we systematically investigate NLP
research that includes claims regarding 'typological diversity'. We find there
are no set definitions or criteria for such claims. We introduce metrics to
approximate the diversity of language selection along several axes and find
that the results vary considerably across papers. Furthermore, we show that
skewed language selection can lead to overestimated multilingual performance.
We recommend future work to include an operationalization of 'typological
diversity' that empirically justifies the diversity of language samples.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要