Projection Systems that Interact

A system called SideBySide enables animated images from two handheld projectors to interact with each other on the same surface.

The SideBySide concept – a self-contained, full-color, handheld projection device allowing multiuser interaction in almost any space. Pictured is an illustration of the game Gorilla, an application that demonstrates the technology. In Gorilla, one player uses a plane and a net to catch the other player's gorilla. (Images: Disney Research, Pittsburgh)

SideBySide was developed by teams at Disney Research, Pittsburgh and Carnegie Mellon University.

The system, which is suitable for games, education and a variety of other applications, is self-contained in special handheld devices. External cameras or other sensors are not required.

The researchers say that SideBySide can be used to exchange contact information, or even to share data files. The technology could spur a more participatory and intimate style of interaction than is possible with computers or overhead projectors.

"Smartphones have made it possible for us to communicate, play games and retrieve information from the Web wherever we might be, but our interaction with the devices remains a largely solitary, single-user experience," said researcher Karl D.D. Willis.

Two-player boxing using the SideBySide system.

“Now that handheld projectors have become a reality, we finally have a technology that allows us to create a new way for people to interact in the real world.”

The handheld projectors are hybrid devices that emit both visible and infrared light; they also contain a camera for monitoring the projected images, a ranging sensor and an inertial measurement unit.

The infrared channel plays a key role in enabling interaction. It is used to project markers that help the system recognize when the images are moving or overlapping and to communicate information between the devices.

The researchers have developed a number of applications to demonstrate the capabilities of the technology. Games include Boxing, in which matches are performed without a ring; Cannon, in which players knock a stack of bricks off a platform by firing a cannonball from one screen to another; and Gorilla, in which one player uses a plane and a net to catch the other player's gorilla.

The SideBySide system overview.

They also have developed a 3-D viewer, which allows two users to control and explore a 3-D model together, and applications for exchanging contact information and transferring files. In addition, a question-and-answer application was created to teach basic vocabulary to young children.

Carnegie Mellon also has announced its development of the TapSense touch-screen technology, which could enhance touch interaction on smartphones and tablet computers. Taking greater advantage of the finger's anatomy and dexterity, the technology distinguishes taps on the screen by different parts of the finger.

For more information, visit: www.disneyresearch.com or www.cmu.edu