Controlling a computer with a gesture

Laura S. Marshall, laura.marshall@laurin.com

A very simple principle is at the heart of Dr. Manolya Kavakli’s high-tech computer interfacing technologies: Less is more.

Kavakli, founder of the Virtual and Interactive Simulations of Reality Research Group at Macquarie University, did not start off in the virtual reality field. Trained as an architect at Istanbul Technical University in Turkey, she was an associate professor of architectural design at her alma mater for 10 years.

“I have always had an interest in using high technology in the field of design,” she said. When she officially switched fields, she began by looking for ways to make computers more intelligent and more creative, and she soon realized that the definition of intelligence was unclear and creativity not yet understood.

Kavakli’s goal became making it easier for anyone to be creative. “I wanted to design platforms to support everyone to express their creativity freely.”

Teaching architectural design, she learned what kinds of tools inspire creativity and how novice designers learn to use them. “When I was a postdoctoral fellow in the UK and Australia,” she said, “I explored the nature of the design process, especially how designers externalize their design ideas and what tools would best support this early conceptual design phase without interfering with their creativity and cognitive processing.

“As we all know, motor skills are quite important in this externalization, which puts some highly creative people in a disadvantaged situation, just because they do not have the necessary drawing or sketching skills. This also led me to think about the physically disabled people, who are more disadvantaged than all of the others.

“Existing tools are too complex to use, and they all require a considerable amount of training.”

So, for the past five years, Kavakli and her research team have been working on interaction and visualization using virtual reality technology, always keeping the “less is more” mantra in mind.

The DesIRe system allows for precise writing of 3-D text.

“The best user interface is a nonexistent one,” she said. “The user interface has to be intuitive and transparent. Interaction shouldn’t interfere with our cognitive processing. As a natural consequence of this, I came up with the idea of gesture recognition that would remove the interface totally from the scene.”

She and her team developed two gesture-recognition systems: DESigning In virtual Reality, or DesIRe, and DRiving for disabled, aka DRive.

With DesIRe, users can communicate with game characters.

“DesIRe allows any user to control dynamically in real-time simulators or other programs” using finger movements, Kavakli said. To operate DesIRe, the user dons a data glove with LEDs, and two pairs of webcams capture the gestures.

“Using one hand, they can draw, and using the other, they can give commands and perform a number of transformations on the object designed,” she noted.

“DRive allows a quadriplegic person to control a car interface using input from just two LEDs on an over-shoulder garment,” she said. “We have tested DRive in a virtual environment similar to the ones used for racing games, and it works well. We have plans to integrate a sensor jacket with this system in the near future.”

The interface used in the DRive system has been tested in a game-like virtual environment.

There are numerous applications for the DesIRe system.

“Many design professions, including architecture, fashion and engineering, can benefit from the outcomes of this project, since they heavily rely on sketching in the conceptual design process,” Kavakli said. “Other areas, such as films, computer games, user interface design that involves storyboarding and visualization, may also benefit from sketching in virtual reality.”