Off-screen handwriting recognition
enriches the handwriting interaction paradigm for mobile devices. However, the existing approaches are only applicable to the specific environment and equipment conditions. In this paper, we propose
AcouWrite
, a general, scalable and real-time handwriting recognition system based on active acoustic sensing. In detail, AcouWrite relies on
active acoustic sensing
using only a pair of microphones and speakers on the smartphone to capture real-time handwriting input. Particularly, we extract the
short-time dCIR (st-dCIR)
to monitor the changes in the acoustic transmission channel resulting from finger movement. Technically, we use a
CNN-GRU
classifier to complete the recognition task in AcouWrite. Moreover, we use data augmentation and spelling error correction methods to improve AcouWrite's robustness. To improve the generalization of our AcouWrite for new characters, we incorporate the transfer learning module into our AcouWrite. In various real-world environments, experiments demonstrate that AcouWrite achieves a mean recognition accuracy of 97.62%, a word accuracy (WA) of 96.4% and a character error rate (CER) of 1.5% for 100 common words, and an average response time of 94 milliseconds.
更多
查看译文
关键词
Active acoustic sensing,differential channel impulse response (dCIR),handwriting recognition
