EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answering
PROCEEDINGS OF THE 2020 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP)(2020)
Abstract
We propose E chi alpha mu s - a new benchmark dataset for cross-lingual and multilingual question answering for high school examinations. We collected more than 24,000 highquality high school exam questions in 16 languages, covering 8 language families and 24 school subjects from Natural Sciences and Social Sciences, among others. E chi alpha mu s offers a fine-grained evaluation framework across multiple languages and subjects, which allows precise analysis and comparison of various models. We perform various experiments with existing top-performing multilingual pre-trained models and we show that E chi alpha mu s offers multiple challenges that require multilingual knowledge and reasoning in multiple domains. We hope that E chi alpha mu s will enable researchers to explore challenging reasoning and knowledge transfer methods and pretrained models for school question answering in various languages which was not possible before. The data, code, pre-trained models, and evaluation are available at http://github.com/mhardalov/exams-qa.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined