Multisensory Pseudo-Haptics for Rendering Manual Interactions with Virtual Objects

Recent advances in extended reality (XR) technologies make seeing and hearing virtual objects commonplace, yet strategies for synthesizing haptic interactions with virtual objects continue to be limited. Two design principles govern the rendering of believable and intuitive haptic feedback: movement through open space must feel "free" while contact with virtual objects must feel stiff. Herein, a novel multisensory approach that conveys proprioception and effort through illusory visual feedback and refers to the wrist, via a bracelet interface, discrete and continuous interaction forces that would otherwise occur at the hands and fingertips, is presented. Results demonstrate that users reliably discriminate the stiffness of virtual buttons when provided with multisensory pseudohaptic feedback, comprising tactile pseudohaptic feedback (discrete vibrotactile feedback and continuous squeeze cues in a bracelet interface) and visual pseudohaptic illusions of touch interactions. Compared to the use of tactile or visual pseudohaptic feedback alone, multisensory pseudohaptic feedback expands the range of physical stiffnesses that are intuitively associated with the rendered virtual interactions and reduces individual differences in physical-to-virtual stiffness mappings. This multisensory approach, which leaves users' hands unencumbered, provides a flexible framework for synthesizing a wide array of touch-enabled interactions in XR, with great potential for enhancing user experiences.