MM-Tap: Adaptive and Scalable Tap Localization on Ubiquitous Surfaces With mm-Level Accuracy

Transforming physical surfaces into virtual interfaces can extend the interaction capability of many exciting metaverse applications in the future. Recent advances in vibration-based tap sensing show promise for this vision using passive vibration signals. However, current approaches based on Time-Difference-of-Arrival (TDoA) triangulation suffer the impact of fluctuant wave velocity due to the dispersive and heterogeneous nature of solid mediums, failing to meet the performance requirement for practical use. In this article, we present MM-Tap, a vibration-based tap localization system that can transform ubiquitous surfaces into virtual touch screens with low overhead. A novel localization scheme is proposed based on the finding of spatiotemporal mapping between tap locations and TDoA values, which pushes the accuracy limits of vibration-based tap sensing from unstable cm-level to mm-level. We investigate the geometry of the sensor layout and design a model-based method to synthesize tap data, which enables MM-Tap to adapt to various surface materials and respond to arbitrary sensing scales after a few seconds of calibration. We combine MM-Tap with a COTS projector and facilitate a digitally augmented surface where users can play video games with low latency.