THAW: Tangible Interaction with See-Through Augmentation for Smartphones on Computer Screens.

ABSTRACTIn this paper, we present a novel interaction system that allows a collocated large display and small handheld devices to seamlessly work together. The smartphone acts both as a physical interface and as an additional graphics layer for near-surface interaction on a computer screen. Our system enables accurate position tracking of a smartphone placed on or over any screen by displaying a 2D color pattern that is captured using the smartphone's back-facing camera. The proposed technique can be implemented on existing devices without the need for additional hardware.