An automated approach for the recognition of intended messages in grouped bar charts.

Information graphics (bar charts, line graphs, grouped bar charts, etc) often appear in popular media such as newspapers and magazines. In most cases, the information graphic is intended to convey a high-level message. This message plays a role in facilitating the discourse purpose of the document but is seldom repeated in the document's text, headlines, or captions. We present a methodology and an implemented system for recognizing the intended message of a grouped bar chart. The recognition system relies on the following components: (1) a linguistic classifier that processes text in the graphic and predicts the most linguistically salient entity from those that are mentioned in text, (2) a cognitive model that estimates the relative perceptual effort required for an individual to recognize some high-level message in a graph, and (3) a Bayesian network that captures the probabilistic relationship between the high-level intended message of a graphic and its communicative signals. This research contributes to three applications: accessibility of information graphics for sight-impaired individuals, retrieval of information graphics from a digital library, and summarization of multimodal documents.