Content-Learning Correlations in Spoken Tutoring Dialogs at Word, Turn, and Discourse Levels

We study correlations between dialog content and learn- ing in a corpus of human-computer tutoring dialogs. Using an online encyclopedia, we first extract domain- specific concepts discussed in our dialogs. We then ex- tend previously studied shallow dialog metrics by in- corporating content at three levels of granularity (word, turn and discourse) and also by distinguishing between students' spoken and written contributions. In all ex- periments, our content metrics show strong correlations with learning, and outperform the corresponding shal- low baselines. Our word-level results show that al- though verbosity in student writings is highly associated with learning, verbosity in their spoken turns is not. On the other hand, we notice that content along with con- ciseness in spoken dialogs is strongly correlated with learning. At the turn-level, we find that effective tutor- ing dialogs have more content-rich turns, but not neces- sarily more or longer turns. Our discourse-level analysis computes the distribution of content across larger dialog units and shows high correlations when student contri- butions are rich but unevenly distributed across dialog segments.