Wave a hand, create a touch screen

Caren B. Les, caren.les@photonics.com

Smartphone apps can take care of everything but walking the dog (although there may be an app for that, too). It’s convenient, sure, but you have to dig your phone out of your pocket, and you’re stuck with the interface designed by an app’s creator. Wouldn’t you rather draw your own virtual touch-screen interface on any given surface and tell it what to do?

The technology is in development at Carnegie Mellon University in Pittsburgh and could be as easy to install as a lightbulb.

Called the WorldKit system – and created by Robert Xiao, Chris Harrison and Scott E. Hudson at the university’s Human-Computer Interaction Institute – it is basically a ceiling-mounted projector and depth camera.

Using WorldKit technology, doctoral student Robert Xiao glides his hand to instantly create a touch screen near an office door. The ad hoc screen could serve as an interactive calendar or messaging unit.

In earlier research, depth camera systems such as Kinect have been combined with projectors to turn surfaces into touch-driven interfaces. The CMU team’s work has shown that these interfaces can be created with hand gestures. “Users can ‘instantiate’ – create and define – interfaces by swiping their hands over surfaces, sort of like a digital paintbrush,” Harrison said.

The depth camera sees the room in 3-D, and by having that information, it can determine when a person’s hand touches a table, wall or other surface. The wide-angle projector allows users to render the interactive interfaces and content onto the surfaces. This means that a user could create a personalized television remote on the arm of a sofa or “paint” an interactive calendar on an office wall, all with the wave of a hand.

“Apps you ‘paint’ are already hooked up to functions,” Harrison said. “So, for example, before sitting down, you could say ‘Remote.’ Then you can sit down, lasso an area with your hands, and boom – there’s a remote control there. It’s sort of like how you can download a Facebook app to your smartphone, and it’s connected to Facebook.”

The prototype WorldKit 2.0 device for creating touch screens on everyday surfaces could be installed like a lightbulb.

People have envisioned futuristic offices, hospitals and classrooms for a very long time, but retrofitting these rooms with interactive capability is very expensive, Harrison added; WorldKit could sidestep the complexity of creating smart environments.

The findings were presented in April at the Conference on Human Factors in Computing Systems.

More work is needed before a wide release: The team must bring down the cost, integrate everything tightly and make it much smaller, Harrison said. “If we can achieve a lightbulblike experience, where you just screw it in and, bam, there is touch interaction everywhere – that would be the dawn of a new era of computing.”