A Context-Contrastive Inference Approach To Partial Diacritization
CoRR(2024)
摘要
Diacritization plays a pivotal role in improving readability and
disambiguating the meaning of Arabic texts. Efforts have so far focused on
marking every eligible character (Full Diacritization). Comparatively
overlooked, Partial Diacritzation (PD) is the selection of a subset of
characters to be marked to aid comprehension where needed. Research has
indicated that excessive diacritic marks can hinder skilled readers – reducing
reading speed and accuracy. We conduct a behavioral experiment and show that
partially marked text is often easier to read than fully marked text, and
sometimes easier than plain text. In this light, we introduce
Context-Contrastive Partial Diacritization (CCPD) – a novel approach to PD
which integrates seamlessly with existing Arabic diacritization systems. CCPD
processes each word twice, once with context and once without, and diacritizes
only the characters with disparities between the two inferences. Further, we
introduce novel indicators for measuring partial diacritization quality,
essential for establishing this as a machine learning task. Lastly, we
introduce TD2, a Transformer-variant of an established model which offers a
markedly different performance profile on our proposed indicators compared to
all other known systems.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要