The Effect of Data Partitioning Strategy on Model Generalizability: A Case Study of Morphological Segmentation
arxiv(2024)
摘要
Recent work to enhance data partitioning strategies for more realistic model
evaluation face challenges in providing a clear optimal choice. This study
addresses these challenges, focusing on morphological segmentation and
synthesizing limitations related to language diversity, adoption of multiple
datasets and splits, and detailed model comparisons. Our study leverages data
from 19 languages, including ten indigenous or endangered languages across 10
language families with diverse morphological systems (polysynthetic, fusional,
and agglutinative) and different degrees of data availability. We conduct
large-scale experimentation with varying sized combinations of training and
evaluation sets as well as new test data. Our results show that, when faced
with new test data: (1) models trained from random splits are able to achieve
higher numerical scores; (2) model rankings derived from random splits tend to
generalize more consistently.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要