Speech interaction in a multimodal tool for handwritten text transcription.

ABSTRACTSTATE is a multimodal tool for document processing and text transcription. Its graphical front-end can be easily connected to different text recognition back-ends. New features and improvements are presented in this work: the interactive correction of one word in the transcribed line has been improved to reestimate the entire transcription line using the user feedback and speech input has been integrated in the multimodal interface enabling the user to also utter the word to be corrected, giving the user the possibility to use the interface according to her preferences or the task at hand. Thus, at the current version of STATE, the user can type, write on the screen with a stylus, or utter the incorrectly recognized word, and then, the system uses the user feedback in any of the proposed modalities to reestimate the transcribed line so as to hopefully correct other errors which could be caused by the mistaken word the user has corrected.