3D Holography Integrated Glasses Could Unlock Mixed Reality

The three dimensional shape of holographic images grants them real depth, whereas monitors can only simulate depth on a 2D screen. Because humans perceive the world in three dimensions, holographic images could be integrated seamlessly into the normal view of the everyday world.

With this in mind, researchers from Princeton University and Facebook creator Meta are working towards mixing the real and virtual worlds using high-definition 3D holographic images. The team has developed a spatial light modulator capable of projecting these images while fitting on a standard pair of glasses. The small optical device could be the foundation for a virtual and augmented reality display that is fully immersive to the human eye.

This would differ from conventional VR and AR technologies that rely on a 2D screen in headsets or phones to project a 3D world. While immersive, the user is aware of a break between simulated and actual reality when using these devices because the image can be lost when in motion.

Princeton and Meta researchers have created a small optical device that makes holographic images larger and clearer. The device is small enough to be fitted in a pair of glasses, as shown, and has the potential to enable VR without the need of a traditional headset. Courtesy of Princeton University.

According to Felix Heide, assistant professor of computer science at Princeton, to have a similar experience using 3D holographic images versus a 2D plane, a monitor would have to be the size of the average cinema screen with the user sitting right in front of it. Because the proposed technology would be using holographic image projections and can fit on a pair of glasses, the researchers believe that they can bypass the use of VR headset hardware entirely. This could make VR more accessible for many applications.

“Holography could make virtual and augmented reality displays easily usable, wearable, and ultrathin,” said Heide. The researchers say that the technology could transform how humans interact with their environments, from getting directions while driving, to monitoring a patient during surgery, to accessing plumbing instructions while doing a home repair.

One of the challenges faced by the researchers was image quality, as spatial light modulators, which create holographic images, could only create high-definition images on a small scale. This tradeoff between image size and clarity results in a narrow field of view, so the team had to create another device to improve image quality and potentially solve this problem. They built a second optical element to work in tandem with the spatial light modulator, filtering the light from the spatial light modulator to expand the field of view while preserving the stability and fidelity of the image. The result was the creation of a larger image with only a minimal drop in quality.

Holographic images made by spatial light modulators (left) are high definition but are too small to be used in an immersive setting. Using a spatial light modulator in conjunction with the team’s newly developed optical element (right) allows the same image to be enlarged without it losing clarity. Courtesy of Princeton University.

The new optical element is similar to a small custom-built piece of frosted glass, said Heide. The pattern etched into the frosted glass is the key. Designed using AI and optical techniques, the etched surface scatters light created by the spatial light modulator in a very precise way, pushing some elements of an image into frequency bands that are not easily perceived by the human eye. This improves the quality of the holographic image and expands the field of view.

According to Heide, hurdles to making a working holographic display remain as the image quality is not yet perfect and the fabrication process for the optical elements needs to be improved. “A lot of technology has to come together to make this feasible,” he said. “But this research shows a path forward.”

The findings were published in Nature Communications (www.doi.org/10.1038/s41467-024-46915-3).