Research on Multi-modal Interactive Control for Quadrotor UAV

Due to the changeable tasks and environments of unmanned aerial vehicle (UAV), the single modal interactive control is unable to satisfy users' requirements. To solve this problem, a multi-modal interactive control method for Quadrotor UAV is proposed in this paper, which integrates eye movement, gesture and voice. Firstly, single modal interactive control of UAV is realized. SMI eye tracker, Kinect gesture recognizer and iFlytek voice input software are used to capture the interactive information respectively. Then, a seamless modal switching method between three interaction types is proposed. Experimental results show that the system guarantees high accuracy of single modal interactive control. Besides, the system can achieve seamless switching between the three channels, which brings more natural and efficient interactive experience to operators.