A Usable API for Multi-Surface Systems

A multi-surface system brings together into a single application a wide variety of different devices. Interactions for these systems must be designed to span across all these devices – which could include tabletops, large format displays and mobile phones. This integration allows users to take advantage of the unique capabilities of each device in ways that would not be possible using those devices separately. However, creating usable interactions for moving content and control between such devices has proven a difficult problem. Gestural interactions, especially those informed by the spatial layout of the room as well as the people and devices in it, might provide a solution to this problem. But such systems are difficult and tedious to build and represent too large an investment of time and effort for developers to bear. To decrease the cost of developing such systems, we have created an API – called MSE-API – that allows developers to quickly and efficiently add gestural interactions to multi-surface applications. In this work we present the API, its uses cases and its structure.