SP-BERT: A Language Model for Political Text in Scandinavian Languages.

Language models are at the core of modern Natural Language Processing. We present a new BERT-style language model dedicated to political texts in Scandinavian languages. Concretely, we introduce SP-BERT, a model trained with parliamentary speeches in Norwegian, Swedish, Danish, and Icelandic. To show its utility, we evaluate its ability to predict the speakers’ party affiliation and explore language shifts of politicians transitioning between Cabinet and Opposition.