Coactive Design of Human-UGV Teamwork Using Augmented Reality

Human and unmanned ground vehicle (human-UGV) team is useful in dangerous and dynamic tasks such as military reconnaissance, disaster rescue, nuclear waste processing and anti-terrorist operations. However, it is still difficult for a human to monitor, understand and control a UGV using traditional interaction user interfaces such as a touch screen or a remote handheld controller due to the task complexity. In this paper, we propose to use an augmented reality (AR) headset, i.e., Hololens2, for constructing the human-UGV team prototype system with AR interaction methods. We have considered three types of human-UGV collaboration scenarios in urban warfare, i.e., the UGV is within the human's field of vision (In-FOV), the UGV is out of the human's field of vision (Out-FOV), and the navigation of the UGV with human in the loop control. We have used the coactive design method to analyze the cognitive capabilities required for the three tasks. Specifically, we have analyzed the task-oriented interdependence of the UGV and the human, according to the principles of observability, predictability and directability. We have demonstrated the effectiveness of the proposed system through real-world experiments.