A Qualitative Case Study on the Validation of Automatically Generated Multiple-Choice Questions from Science Textbooks

In a concept learning scenario, any technology-supported learning system must provide students with mechanisms that help them with the acquisition of the concepts to be learned. For the technology-supported learning systems to be successful in this task, the development of didactic material is crucial—a hard task that could be alleviated by means of an automation process. In this proposal, two systems which have been previously developed, ArikIturri and DOM-Sortze, are combined to automatically generate multiple-choice questions, based on pedagogically relevant information gathered in textbooks. Originally, the former was able to generate multiple-choice questions from plain texts; and the latter was able to elicit learning objects based on didactic material explicitly represented in electronic textbooks, i.e., definitions, examples, and exercises. This article presents an approach for the automatic generation of multiple-choice questions from learning objects extracted from textbooks. Specifically, ArikIturri uses as input the texts gathered in the learning objects elicited by DOM-Sortze and, using natural language processing techniques, generates multiple-choice questions. This way, considering domain-relevant information from the textbooks, test-type exercises which were not previously elicited by DOM-Sortze are created. In summary, this new approach is able to enrich domain modules of technology-supported learning systems. The proposal has been tested with a textbook which is written in the Basque language and the results show that the generated exercises are suitable to be used in science learning scenarios at secondary school.