Neurogoggles for Multimodal Augmented Reality.

We present a neurogoggles system that offers truly immersive augmented and virtual reality with unique biofeedback based on brain and bodily signals. The system consists of a wearable head-mounted display that integrates two stereoscopic color cameras for video-through experience, a depth sensor for object tracking, an inertial sensor for head tracking and electrophysiological measurements for brain and bodily signals. The system is designed to ensure low-latency synchronization across all functional modules, which is a key element for multimodal real-time data analysis. We showcase this technology with an immersive experience combining multimodal data and artistic digital content using Kudan augmented reality engine. In this experience, the user is presented with an interactive world behind a brain image wall poster, which reacts to user's brain state.