Extracting Short Stories from Large Data Sets

In this position paper, we discuss the challenge of processing large data sets to extract short stories that an analyst can use to understand and communicate effectively. We argue that many analytic tools, while valuable for aggregating data and showing the big picture, result in abstracting away the key pieces of information required to explain the behaviors and interactions of agents. Instead, we propose tools that extract and present information as short stories within the data. We present a simple classification for such stories with respect to behaviors of agents and mode of observation. Using this approach, we describe four case studies in which extracting short stories enables improved understanding of complex agent behaviors. We conclude with a discussion of how this approach might be applied in future research.