Paper Reading 20 – A Multimodal Labeling Interface for Wearable Computing

Comments
Paola Garza
Patrick Frith

Reference Information
A Multimodal Labeling Interface for Wearable Computing
 - Shanqing Li and Yunde Jia
 - IUI 2010, Hong Kong, China

The different parts that make up the labeling system.

Summary
A different way of labeling items and features in the field was discussed.  The current way of performing these tasks is with a keyboard and mouse.  The mouse is used to circle an area or object and the keyboard can easily be used to type in the label.  However, this can be cumbersome and very inconvenient for many environments.  The authors proposed a new way to achieve these tasks with a wearable computer.

The device is comprised of the different elements outlined in the picture to the right.  The person needs to wear a computer that is in a backpack of sorts.  Then they also have a camera on their head and a Bluetooth microphone.  In order to label an object, the person simply reaches out their finger and encircles what they would like to label.  The camera picks up on this and registers encircled object.  Then the microphone is used to label the object through speech recognition.  Encircling objects with a finger was found to be much quicker than with a mouse and could be done in a fraction of the time.  Their speech recognition was also found to be 98% accurate in labeling properly.  Overall, their system was extremely successful in the applicable environments.

The labeling system in action.

Discussion
This idea was pretty interesting.  I had never previously thought about situations that would need a device like this.  But now I can certainly see where it could be useful.  If these labeling actions are needed, using a portable keyboard and mouse seem extremely inconvenient.  It just makes a lot more sense to have a system where you can simply gesture around an area and speak the name, and then move on to the next item.  I don’t really have much else to say about this.  This idea seems very useful for its environment and it was a fantastic solution to the problem they presented.