Bluefish: A Relational Grammar of Graphics

The Grammar of Graphics (GoG) has become a popular format for specifying visualizations because it unifies different chart types into a consistent, modular, and customizable framework. But its benefits have not yet reached the broader class of data-driven graphic representations -- from annotated charts and hierarchical visualizations to molecular structure diagrams, Euclidean geometry, and mathematical formulae. These graphics are still developed using rigid typologies, monolithic tools, or specialized grammars that lack the customizability and generality of the GoG. In response, we present Bluefish, a relational grammar of graphics that extends the benefits of the GoG to this larger domain. Bluefish provides two key abstractions: user-extensible, domain-specific elements (e.g., mathematical expressions, chemical atoms, or program state stack frames); and perceptual groupings (also known as Gestalt relations) like proximity, nesting, and linking. Users compose these primitives within a Bluefish specification, which the language runtime compiles to a relational scenegraph: a formal representation of a graphic that, compared to traditional tree-based scenegraphs, better preserves semantic relationships between visual elements. To illustrate its flexibility, we show that Bluefish can represent data-driven graphic representations across a diverse range of domains while closely aligning with domain-specific vocabulary. Moreover, to demonstrate the affordances of Bluefish's relational scenegraph, we develop a prototype screen reader tool that allows blind and low-vision users to traverse a diagram without significant additional scaffolding.