Different Tokenization Schemes Lead to Comparable Performance in Spanish Number Agreement
CoRR(2024)
Abstract
The relationship between language model tokenization and performance is an
open area of research. Here, we investigate how different tokenization schemes
impact number agreement in Spanish plurals. We find that
morphologically-aligned tokenization performs similarly to other tokenization
schemes, even when induced artificially for words that would not be tokenized
that way during training. We then present exploratory analyses demonstrating
that language model embeddings for different plural tokenizations have similar
distributions along the embedding space axis that maximally distinguishes
singular and plural nouns. Our results suggest that morphologically-aligned
tokenization is a viable tokenization approach, and existing models already
generalize some morphological patterns to new items. However, our results
indicate that morphological tokenization is not strictly required for
performance.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined