S.H.E.R.P.A (Super Helpful Engine Recognizing People’s Audio)

Hands-free technology could be an integral tool for those who are differently abled, the elderly or just someone who wishes to perform another task while accessing technology. Speech to text to command programs are comprised of an ASR (automatic speech recognition) module and a command implementation module. Their purpose is to provide a means for a user to navigate their technology using only spoken words. We propose the implementation of OpenAI’s Whisper model in conjunction with a Graphical User Interface (GUI) and a command library. Utilizing the pretrained model we were able to create a program that allows for hands-free user interaction and command with certain browser, email, and desktop applications.