Sebastian, Basti, Wastl?! Recognizing Named Entities in Bavarian Dialectal Data
CoRR(2024)
Abstract
Named Entity Recognition (NER) is a fundamental task to extract key
information from texts, but annotated resources are scarce for dialects. This
paper introduces the first dialectal NER dataset for German, BarNER, with 161K
tokens annotated on Bavarian Wikipedia articles (bar-wiki) and tweets
(bar-tweet), using a schema adapted from German CoNLL 2006 and GermEval. The
Bavarian dialect differs from standard German in lexical distribution,
syntactic construction, and entity information. We conduct in-domain,
cross-domain, sequential, and joint experiments on two Bavarian and three
German corpora and present the first comprehensive NER results on Bavarian.
Incorporating knowledge from the larger German NER (sub-)datasets notably
improves on bar-wiki and moderately on bar-tweet. Inversely, training first on
Bavarian contributes slightly to the seminal German CoNLL 2006 corpus.
Moreover, with gold dialect labels on Bavarian tweets, we assess multi-task
learning between five NER and two Bavarian-German dialect identification tasks
and achieve NER SOTA on bar-wiki. We substantiate the necessity of our
low-resource BarNER corpus and the importance of diversity in dialects, genres,
and topics in enhancing model performance.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined