Simple Camera Setup Enables 3D Human Shape Reconstruction

Facebook X LinkedIn Email
KAUNAS, Lithuania, Feb. 18, 2022 — Researchers from Kaunas University of Technology (KTU) proposed a deep-learning-based method for the three-dimensional human shape reconstruction when the original figure is only partly visible. The method is relatively low cost, provides high compression of the images obtained, and is easily integrated with existing virtual reality tools. The method was developed using a real-world data set. A clinical trial is pending.

Currently used 3D image reconstruction solutions for VR contain complex configurations of multiple cameras and require high levels of computational power to process the image. This can make full object reconstruction impractical and prohibitively expensive, the researchers said.
Computer scientists at KTU are building technology to enable physicians to monitor patients at their homes using virtual reality tools. Courtesy of Bermix Studios.
Computer scientists at KTU are building technology to enable physicians to monitor patients at their homes using virtual reality tools. Courtesy of Bermix Studios.

To address these issues, the team of computer scientists led by Rytis Maskeliunas, chief researcher in the KTU Department of Multimedia Engineering, proposed a deep-learning-based method that can reconstruct a full human posture point cloud from a depth view. They applied a three-stage adversarial deep neural network to deal with depth sensor noise and perform the refining of depth sensor data for full 3D human shape reconstruction.

The team used recordings of multiple subjects performing physical rehabilitation exercises as the data set for the experiment. Two commercially available depth cameras were used to film the subjects from the front and from the side.

“A camera sees only a part of the image: If it is filming the frontal view, the view from the back is invisible; if something is blocking the view, the camera cannot see what’s behind. Therefore, we employ artificial intelligence which reconstructs the invisible parts of the image,” Maskeliunas said.

A five-stage training approach was adopted for training artificial intelligence. Results were validated via expert testimony, which observed that the network reconstructed the result with only a few flaws, mainly near the end of the limbs.

The proposed solution is the continuation of several applications that Maskeliunas and his team are currently developing for the medical field.

According to Maskeliunas, in health care, the 3D image of the person is crucial when there is a need to diagnose various traumas related to spinal injuries and for various other purposes.

“For example, a doctor may ask their patient to perform a simple task, such as touching their nose or rotating their shoulder. To fully see how the person bends, twists, and in how their posture is changing, the physician needs to see them as a three-dimensional subject, to be able to look at them from all the sides and angles,” Maskeliunas said. He added that the availability of the tools and the variety of applications with which the proposed solution can easily be integrated can make the developed method a preferred approach for 3D image reconstruction.

The research was published in IEEE Sensors Journal (

Published: February 2022
machine learning
Machine learning (ML) is a subset of artificial intelligence (AI) that focuses on the development of algorithms and statistical models that enable computers to improve their performance on a specific task through experience or training. Instead of being explicitly programmed to perform a task, a machine learning system learns from data and examples. The primary goal of machine learning is to develop models that can generalize patterns from data and make predictions or decisions without being...
neural network
A computing paradigm that attempts to process information in a manner similar to that of the brain; it differs from artificial intelligence in that it relies not on pre-programming but on the acquisition and evolution of interconnections between nodes. These computational models have shown extensive usage in applications that involve pattern recognition as well as machine learning as the interconnections between nodes continue to compute updated values from previous inputs.
point cloud
A point cloud is a set of data points in a three-dimensional coordinate system, where each point represents a specific location in space. These points are typically obtained through various sensing techniques such as lidar (light detection and ranging), photogrammetry, structured light scanning, or 3D scanning. Each point in a point cloud is defined by its spatial coordinates (x, y, z), representing its position in three-dimensional space, as well as potentially additional attributes such as...
Research & TechnologyImagingSensors & Detectorsmachine learningneural network3D imagingpoint cloudreconstructionimage recognitiondepth cameradepth sensingEuropeIEEE Sensors Journalocclusion

We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.