Title:

Hand Gesture Recognition Using Hidden Markov Models

Affiliation:

Joseph Picone, Shuang Lu and Amir Harati
Institute for Signal and Information Processing
Temple University

Abstract:

Advanced gaming interfaces such as Microsoft's Kinect have generated renewed interest in hand gesture recognition as an ideal interface for human computer interaction. In this talk, we will discuss a specific application of gesture recognition - fingerspelling in American Sign Language. Fingerspelling is widely used for communication amongst signers. Signer-independent (SI) fingerspelling alphabet recognition is a very challenging task due to a number of factors including the large number of similar gestures, hand orientation and cluttered background. We propose a novel framework that uses a two-level hidden Markov model (HMM) that can recognize each gesture as a sequence of sub-units and performs integrated segmentation and recognition. We present results on signer-dependent (SD) and signer-independent (SI) tasks for the ASL Fingerspelling Dataset: error rates of 2.0% and 46.8% respectively. The SI results improved the best previously published results by 18.2% absolute (28.0% relative). A live demo will also be presented.

Biography:

Joseph Picone received his Ph.D. in Electrical Engineering in 1983 from
the Illinois Institute of Technology. He is currently a Professor in the Department of Electrical and Computer Engineering at Temple
University. His primary research interests are currently machine
learning approaches to acoustic modeling in speech recognition. His
research group is known for producing many innovative open source
materials for signal processing including a public domain speech
recognition system. He is a Senior Member of the IEEE and has been
active in several professional societies related to human language
technology. He has authored numerous papers on the subject and holds
several patents in this field.