Automatic Content Analysis of Student Moral Discourse in a Collaborative Learning Activity

In most computer supported collaborative learning activities, the teacher monitors and/or reviews data generated by students and groups as they complete the learning tasks, in order to provide guidance and feedback. Without appropriate technological means that support the processes of collection and selection of students' generated responses, these duties can result in a high cognitive load for teachers, especially if students generate textual, qualitative content that requires real-time reviewing. In this research we deal with EthicApp, a collaborative application in which this problem is apparent, as students analyze a given ethics case individually and in small groups and deliver written judgements in each phase of the activity. We present a solution to the problem, based on enhancing EthicApp's teacher's interface with automated content analysis capabilities. This includes a dashboard that automatically displays students' most relevant contributions, and cluster visualizations that permit identifying groups of students with similar responses to activity tasks. Validation of the approach was based on a dataset comprising 4,366 comments about an academic ethics case, which were written by 520 students divided into 19 class groups. Expert judgement was applied to evaluate content analysis effectiveness at selecting comments that are both meaningful and representative of students' different views. More than 80% of comment selections were found valuable, according to experts' analysis.