EyeClick: A Robust Two-Step Eye-Hand Interaction for Text Entry in Augmented Reality Glasses
UIST '23 Adjunct: Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology(2023)
Abstract
Efficient text entry is a crucial aspect of the user experience for augmented reality (AR) head-mounted displays (HMD). Eye-tracking for virtual keyboard interaction is a popular choice for AR text entry, as it is intuitive, privacy-preserving, and socially acceptable. However, as AR HMDs move toward more consumer-friendly form factors and price points, technical constraints necessitate trade-offs that result in a limited field of view (FoV) and reduced eye-tracking accuracy. To address these challenges, we develop EyeClick, a novel two-step eye-hand interaction method utilizing a modified QWERTY keyboard for AR HMDs. To type, users first employ gaze to select a large area containing multiple characters, then press a button on a handheld controller to choose a single character from the selected area. We demonstrated competitive speed and reduced corrected error rate (CER) with two variants of the EyeClick design: ColType (7.30 WPM) and RowType (9.41 WPM). We also integrated ChatGPT with the EyeClick keyboard to build an interactive information retrieval AR application. We demonstrate that the EyeClick keyboard can help users efficiently extract information about surroundings by typing to ChatGPT in mixed reality.
MoreTranslated text
Key words
text entry,text input,augmented reality,gaze detection,eye-tracking
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined