Search Menu
Photonics Media Photonics Marketplace Photonics Spectra BioPhotonics Vision Spectra Photonics Showcase Photonics ProdSpec Photonics Handbook

AI Aids in Real-Time Generation of 3D Holograms

Facebook Twitter LinkedIn Email
A method for generating holograms uses an artificial intelligence program that a consumer-grade laptop is capable of running, giving implications in VR and 3D printing. A team at MIT introduced the method, which generates holograms almost instantly.

The process of generating holograms via computer typically necessitates a supercomputer device to run the necessary physics simulations. The process is slow even on a supercomputer and often delivers subpar results. By comparison, the new method enables a consumer-grade computer to generate real-time 3D holographic images in milliseconds.
This figure shows the experimental demonstration of 2D and 3D holographic projection. The left photograph is focused on the mouse toy (in yellow box) closer to the camera, and the right photograph is focused on the perpetual desk calendar (in blue box). Courtesy of Liang Shi, Wojciech Matusik, et al.
This figure shows the experimental demonstration of 2D and 3D holographic projection. The left photograph is focused on the mouse toy (in yellow box) closer to the camera, and the right photograph is focused on the perpetual desk calendar (in blue box). Courtesy of Liang Shi, Wojciech Matusik, et al.

“People previously thought that with existing consumer-grade hardware it was impossible to do real-time 3D holography computations,” said lead author Liang Shi, a Ph.D. student in MIT’s Department of Electrical Engineering and Computer Science. “It’s often been said that commercially available holographic displays will be around in 10 years, yet this statement has been around for decades.”

Shi believes the new approach, “tensor holography,” will bring the goal within reach.

Ultimately, the difference between a photograph and a hologram lies in the hologram’s encoding of the brightness and phase of each lightwave. This allows a hologram to portray a more life-like representation of a scene’s parallax and depth. To optically capture a hologram, a laser beam is split, with half used to illuminate the subject and the other half used as a reference for the lightwaves’ phase. The reference generates a sense of depth. These holograms, however, which were developed in the mid-20th century, were static and therefore unable to capture motion. And the method only produced one hard copy.

Computer-generated holography is designed to bypass these challenges by simulating the optical setup. Because each point in the scene is of a different depth, though, one cannot apply common operations for each point.

“That increases the complexity significantly,” Shi said.

A supercomputer running these simulations could take up to several minutes to generate a single holographic image. Existing algorithms also do not model occlusion with photorealistic precision.

The MIT team used deep learning and designed a convolutional network in a method that uses a series of tensors to mimic the way humans process visual information. Training a neural network typically requires a large, high-quality data set, which the team had to assemble on its own.

The custom database contained 4000 pairs of computer-generated images, each matching a picture — including color and depth information for each pixel — with its corresponding hologram. The researchers created the database holograms with scenes that included complex and varying shapes and colors, with an evenly distributed depth of pixels from the background to the fore.

To address occlusion, they also provided a new set of physics-based calculations.

The algorithm, with a photorealistic training data set, optimized its own calculations, successfully enhancing its ability to generate holograms. The network operated orders of magnitude faster than physics-based calculations.

The method is able to generate holograms in milliseconds from images with depth information — provided by typical computer generated images and can be calculated with a multicamera setup or a lidar sensor. The compact tensor network requires less than 1 MB of memory.

“It’s negligible, considering the tens and hundreds of gigabytes available on the latest cellphone,” researcher Wojciech Matusik said.

In VR, the team believes the technology could provide more realistic scenery and eliminate eyestrain and other side effects of long-term VR use. The technology could also see use in displays capable of modulating the phase of lightwaves.

“It’s a considerable leap that could completely change people’s attitudes toward holography,” Matusik said. “We feel like neural networks were born for this task.”

The work was published in Nature (


Photonics Spectra
May 2021
An acronym of light detection and ranging, describing systems that use a light beam in place of conventional microwave beams for atmospheric monitoring, tracking and detection functions. Ladar, an acronym of laser detection and ranging, uses laser light for detection of speed, altitude, direction and range; it is often called laser radar.
artificial intelligence
The ability of a machine to perform certain complex functions normally associated with human intelligence, such as judgment, pattern recognition, understanding, learning, planning and problem solving.
The optical recording of the object wave formed by the resulting interference pattern of two mutually coherent component light beams. In the holographic process, a coherent beam first is split into two component beams, one of which irradiates the object, the second of which irradiates a recording medium. The diffraction or scattering of the first wave by the object forms the object wave that proceeds to and interferes with the second coherent beam, or reference wave at the medium. The resulting...
An interference pattern that is recorded on a high-resolution plate, the two interfering beams formed by a coherent beam from a laser and light scattered by an object. If after processing, the plate is viewed correctly by monochromatic light, a three-dimensional image of the object is seen.
Research & Technologylidarartificial intelligenceholographyhologramdeep learningcomputer-generated hologramcomputer-generated hologramscomputer-generated holographyMITLiang ShiWojciech MatusikNatureAmericasTech Pulse

back to top
Facebook Twitter Instagram LinkedIn YouTube RSS
©2023 Photonics Media, 100 West St., Pittsfield, MA, 01201 USA, [email protected]

Photonics Media, Laurin Publishing
x Subscribe to Photonics Spectra magazine - FREE!
We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.