John Malkin – Vocal Joystick

Chris Pirillo — Thu, 23 Oct 2008 08:59:06 +0000

Add to iTunes | Add to YouTube | Add to Google | RSS Feed

Jon Malkin is a PhD candidate in Electrical Engineering at the University of Washington. Jon’s current project, and the topic of his talk, the Vocal Joystick (VJ), an assistive device for individuals with motor impairment. The VJ provides speakers with continuous control of devices such as mouse pointers or simple robotic arms using non-speech vocalizations.

The goal of this project is to develop a novel system that we call the Vocal Joystick (VJ). This device will enable individuals with motor impairments to use vocal parameters to control objects on a computer screen (buttons, sliders, etc.) and ultimately electro-mechanical instruments (e.g., robotic arms, wireless home automation devices).

Standard spoken language can be quite inefficient for such continuous control tasks and is often recognized poorly by automatic speech recognizers. The VJ system, in contrast, will allow users to exploit a large and varied set of vocalizations for both continuous and discrete motion control, and its selection will be optimized for high discrimination ability and low communication bandwidth. Furthermore, the users are able to perceive visualized feedbacks from the system and make adjustments on the fly. This may include regular speech sounds, such as vowels and consonants, but the primary focus will be on the variation of individual acoustic-phonetic parameters like pitch, energy, vowel quality and voice quality.

Want to embed this video on your own site, blog, or forum? Use this code or download the video:

John Malkin – Vocal Joystick

Chris Pirillo » vocal-joystick

John Malkin – Vocal Joystick