Turning Motion Capture Technology Inside Out

In the film industry, traditional motion capture techniques use cameras to meticulously record the movements of actors inside studios, enabling those movements to be translated into digital models. But by turning the cameras around — mounting almost two dozen, outward-facing cameras on the actors themselves — scientists at Disney Research, Pittsburgh (DRP) and Carnegie Mellon University have shown that motion capture can occur almost anywhere outside of a studio.

Motion capture makes possible scenes such as those in “Rise of the Planet of the Apes,” where the movements of actor Andy Serkis were translated into a digitally created Caesar with highly expressive, yet apelike, characteristics. But body-mounted cameras enable capture of motions, such as running outside or swinging on ropes between trees, that would be difficult — if not impossible — otherwise, said Takaaki Shiratori of DRP.

A system developed by Disney Research, Pittsburgh, and Carnegie Mellon University uses body-mounted cameras to capture motion for use in animation. Cameras attached to a runner outdoors (left) record motions used to render an animated runner (right). (Images: Carnegie Mellon University)

“This could be the future of motion capture,” he said in advance of a presentation on the new technique at SIGGRAPH 2011, the International Conference on Computer Graphics and Interactive Techniques, in Vancouver. As video cameras become ever smaller and cheaper, “I think anyone will be able to do motion capture in the not-so-distant future.”

The wearable camera system makes it possible to reconstruct the relative and global motions of an actor, thanks to a process called structure from motion (SfM). Takeo Kanade, a Carnegie Mellon professor, developed SfM 20 years ago as a means of determining the 3-D structure of an object by analyzing the images from a camera as it moves around the object, or as the object moves past the camera.

In this application, SfM is not used primarily to analyze objects in a person’s surroundings, but to estimate the pose of the cameras on the person. Researchers mounted 20 lightweight cameras on the limbs and trunk of each subject. Each camera was calibrated with respect to a reference structure, and then each person performed a range-of-motion exercise that allowed the system to automatically build a digital skeleton and to estimate positions of cameras with respect to that skeleton.

The quality of motion capture from body-mounted cameras does not yet match the fidelity of traditional motion capture, Shiratori said, but will improve as the resolution of small video cameras continues to improve.

For more information, visit: http://drp.disneyresearch.com/projects/mocap