Entity Skeletons for Visual Storytelling

We are enveloped by stories of visual interpretations in our everyday lives. Story narration often comprises of two stages, which are, forming a central mind map of entities and then weaving a story around them. In this paper, we address these two stages of introducing the right entities at seemingly reasonable junctures and also referring them coherently in the context of visual storytelling. The building blocks of the central mind map, also known as entity skeleton are entity chains including nominal and coreference expressions. We establish a strong baseline for skeleton informed generation and propose a glocal hierarchical attention model that attends to the skeleton both at the sentence (local) and the story (global) levels. We observe that our proposed models outperform the baseline in terms of automatic evaluation metric, METEOR. We also conduct human evaluation from which it is concluded that the visual stories generated by our model are preferred 82% of the times.