Umwelt: Accessible Structured Editing of Multimodal Data Representations
arxiv(2024)
摘要
We present Umwelt, an authoring environment for interactive multimodal data
representations. In contrast to prior approaches, which center the visual
modality, Umwelt treats visualization, sonification, and textual description as
coequal representations: they are all derived from a shared abstract data
model, such that no modality is prioritized over the others. To simplify
specification, Umwelt evaluates a set of heuristics to generate default
multimodal representations that express a dataset's functional relationships.
To support smoothly moving between representations, Umwelt maintains a shared
query predicated that is reified across all modalities – for instance,
navigating the textual description also highlights the visualization and
filters the sonification. In a study with 5 blind / low-vision expert users, we
found that Umwelt's multimodal representations afforded complementary overview
and detailed perspectives on a dataset, allowing participants to fluidly shift
between task- and representation-oriented ways of thinking.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要