Wearable Camera Enables Human Motion Capture

Facebook X LinkedIn Email
TOKYO, Oct. 23, 2020 — Researchers at Tokyo Institute of Technology and Carnegie Mellon University developed a human motion capture system called MonoEye, which consists of a single ultrawide fisheye camera mounted on the user’s chest. The system’s simplicity lends itself to a wide range of applications, in sports, medicine, and entertainment.
MonoEye is based on a single ultra-wide fisheye camera worn on the user's chest, enabling activity capture in everyday life. Courtesy of the Association of Computer Machinery.
MonoEye is based on a single ultrawide fisheye camera worn on the user's chest, enabling activity capture in everyday life. Courtesy of the Association of Computer Machinery.

The system features a 280° field of view and is able to capture the user’s limbs, face, and surrounding environment. To achieve robust multimodal motion capture, the system was designed with three deep neural networks capable of estimating 3D body pose, head pose, and camera pose in real time.

The neural networks have been trained with an extensive synthetic data set consisting of 680,000 renderings of people with a range of body shapes, clothing, actions, background, and lighting conditions, as well as 16,000 frames of photorealistic images.

Due to the domain gap between synthetic and real-world data sets, MonoEye’s developers plan to continue expanding their data set with more photorealistic images to improve accuracy. The researchers said they envision that the chest-mounted camera could be transformed into a small, wearable accessory such as a tie clip, brooch, or sports gear. 

The research was presented at the 33rd Association of Computer Machinery Symposium on User Interface Software and Technology.

Published: October 2020
machine vision
Machine vision, also known as computer vision or computer sight, refers to the technology that enables machines, typically computers, to interpret and understand visual information from the world, much like the human visual system. It involves the development and application of algorithms and systems that allow machines to acquire, process, analyze, and make decisions based on visual data. Key aspects of machine vision include: Image acquisition: Machine vision systems use various...
computer vision
Computer vision enables computers to interpret and make decisions based on visual data, such as images and videos. It involves the development of algorithms, techniques, and systems that enable machines to gain an understanding of the visual world, similar to how humans perceive and interpret visual information. Key aspects and tasks within computer vision include: Image recognition: Identifying and categorizing objects, scenes, or patterns within images. This involves training algorithms...
A lens is a transparent optical device that focuses or diverges light, allowing it to pass through and form an image. Lenses are commonly used in optical systems, such as cameras, telescopes, microscopes, eyeglasses, and other vision-correcting devices. They are typically made of glass or other transparent materials with specific optical properties. There are two primary types of lenses: Convex lens (converging lens): This type of lens is thicker at the center than at the edges. Convex...
fish-eye lens
A type of wide-angle lens that has an angular field above 140° and that exhibits barrel distortion. The most commonly used fish-eye lenses have a field of about 180°, though they are manufactured up to 200°.
Research & Technologymachine visioncomputer visionmotion captureAsia-PacificTokyo Institute of TechnologyCarnegie Mellon Universityneural networkslensfisheyefisheye lensfish-eye lenswide anglewide angle lens

We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.