Reach Out and Touch Someone
The technology, called Skinput, was developed by Chris Harrison, a third-year Ph.D. student in Carnegie Mellon University's Human-Computer Interaction Institute (HCII), along with Desney Tan and Dan Morris of Microsoft Research. Harrison will describe the technology in a paper to be presented on at CHI 2010, the Association for Computing Machinery's annual Conference on Human Factors in Computing Systems in Atlanta, Ga.
Skinput (www.chrisharrison.net/projects/skinput/) could help people take better advantage of the tremendous computing power now available in compact devices that can be easily worn or carried. The diminutive size that makes smart phones, MP3 players and other devices so portable, however, also severely limits the size and utility of the keypads, touchscreens and jog wheels typically used to control them.
"With Skinput, we can use our own skin — the body's largest organ — as an input device," Harrison said "It's kind of crazy to think we could summon interfaces onto our bodies, but it turns out to make a lot of sense. Our skin is always with us, and makes the ultimate interactive touch surface"
In a prototype developed while Harrison was an intern at Microsoft Research last summer, acoustic sensors are attached to the upper arm. These sensors capture sound generated by such actions as flicking or tapping fingers together, or tapping the forearm. This sound is not transmitted through the air, but by transverse waves through the skin and by longitudinal, or compressive, waves through the bones.
In a trial involving 20 subjects, the system was able to classify the inputs with 88 percent accuracy overall. Accuracy depended in part on proximity of the sensors to the input; forearm taps could be identified with 96 percent accuracy when sensors were attached below the elbow, 88 percent accuracy when the sensors were above the elbow. Finger flicks could be identified with 97 percent accuracy.
"There's nothing super sophisticated about the sensor itself," Harrison said, "but it does require some unusual processing. It's sort of like the computer mouse — the device mechanics themselves aren't revolutionary, but are used in a revolutionary way." The sensor is an array of highly tuned vibration sensors — cantilevered piezo films.
The prototype armband includes both the sensor array and a small projector that can superimpose colored buttons onto the wearer's forearm, which can be used to navigate through menus of commands. Additionally, a keypad can be projected on the palm of the hand. Simple devices, such as MP3 players, might be controlled simply by tapping fingertips, without need of superimposed buttons; in fact, Skinput can take advantage of proprioception — a person's sense of body configuration — for eyes-free interaction.
Testing indicates the accuracy of Skinput is reduced in heavier, fleshier people and that age and sex might also affect accuracy. Running or jogging also can generate noise and degrade the signals, the researchers report, but the amount of testing was limited and accuracy likely would improve as the machine learning programs receive more training under such conditions.
Harrison, who delights in "blurring the lines between technology and magic," is a prodigious inventor. Last year, he launched a company, Invynt LLC, to market a technology he calls "Lean and Zoom," which automatically magnifies the image on a computer monitor as the user leans toward the screen. He also has developed a technique to create a pseudo-3D experience for video conferencing using a single webcam at each conference site. Another project explored how touchscreens can be enhanced with tactile buttons that can change shape as virtual interfaces on the touchscreen change.
Skinput is an extension of an earlier invention by Harrison called Scratch Input, which used acoustic microphones to enable users to control cell phones and other devices by tapping or scratching on tables, walls or other surfaces. Such technologies could be beneficial for space missions, where reducing the amount and size of equipment is essential in reducing launch costs. In the future, technologies like Skinput and Scratch Input might help reduce the amount of computer devices, as well as increase computer capabilities, for astronauts exploring beyond the atmosphere of Earth.