3D Autocomplete: Enhancing UAV Teleoperation with AI in the Loop

Manually teleoperating a flying robot can be a demanding task, especially for users with limited levels of experience. This is primarily due to the non-linear properties of such robots in addition to the difficulty of controlling various degrees of freedom at the same time. 3D Autocomplete helps mitigate such limitations by assisting the users in teleoperation. It aids in teleoperating 3D motions, such as helical motions, which are more challenging to the users. The proposed framework uses Artificial Intelligence (AI) to predict just-in-time the user's intended motion and then, if the user accepts, completes it autonomously in 3D. The AI component of 3D Autocomplete was presented in our previous work, where we introduced a deep learning model and an algorithm to predict as early as possible the user's desired motion. Moving forward in this work, we focus on synthesizing and completing the user-intended motion autonomously. Also, we introduce a Mixed Reality (MR) user interface for better human-robot interaction. Finally, we evaluate our system subjectively and objectively through human-subject experiments. Autocomplete outperformed traditional method on all criteria with at least 30% improvement in all objective measures.