Over the last several years, head-mounted devices (HMDs) for virtual and augmented reality have been coming to consumer markets in different sizes and weights, and with different functionality and experiences to offer, from the lightweight Google Glass to the fully immersive HTC Vive. We are at the early stages of a huge revolution that is changing the interaction between humans and information. However, on both the hardware and software sides, more innovation is needed to reduce the size, weight, and portability of HMDs. Hardware improvements are actually driven by photonics, as HMDs consist of two main elements — optics and image displays. From the optical point of view, the fundamental difference between augmented reality (AR) and virtual reality (VR) is that AR lenses are reflective, while VR lenses are transparent. AR challenges AR has seen an increase in the production of headsets, which are worn to improve the gaming experience. But this rise in acceptance to the market has, to date, been incremental, with no sign of the huge point of inflection seen with 3D sensors, for example, which have really taken off in 2018 thanks mainly to the introduction of lidar, especially in the automotive industry. For AR, a significant market upswing is not expected soon (Figure 1). The main obstacle is that although the technical demands for optics and displays are now well-defined in terms of both field of view and resolution, the current state-of-the-art devices offer solutions that tend to be too costly for the consumer market; we are very much in a chicken and egg situation. Figure 1. Market figures showing the gradual adoption of augmented reality headsets. Courtesy of Yole Développement. Courtesy of the European Photonics Industry Consortium. Choosing the right optics, or in this case, optical waveguides, is key for AR. Right now, different solutions are available on the market, from the holographic waveguide used by Sony’s SmartEyeglass displays, the polarized waveguide of the Lumus AR glasses (adding extra complexity with a partially reflective polarized surface), and the reflective waveguide of the Google Glass concepts, to what seems at this moment to be the technology of choice — the diffractive waveguide of the Vuzix displays and Microsoft’s HoloLens. The HoloLens has stacked RGB waveguides with slanted subwavelength surface relief gratings (SRGs) that act as in/out couplers operating both in transmission and reflection modes. In contrast, the Vuzix View the Future has controllable waveguides for near-eye display applications. Figure 2. A holographic waveguide solution based on ThinEyes freeform optics by Limbak. TTL: through the lens. Courtesy of Limbak. Courtesy of the European Photonics Industry Consortium. The holographic waveguide is the simplest of the cases, as it consists merely of two lenses for incoupling and outcoupling the light in and out of a waveguide made of two planar reflective surfaces. Due to its simplicity, it has perhaps the best chance of becoming standard in the short term, and especially with future solutions for miniaturization, such as those enabled by Limbak’s ThinEyes (Figure 2) or by VRgineers’ XTAL lenses (Figure 3). Figure 3. XTAL lenses by VRgineers. Courtesy of VRgineers. Courtesy of the European Photonics Industry Consortium. Challenges for fabricating SRGs It is in the fabrication of SRGs where photonics manufacturing encounters some of the key challenges, including the need to reduce cost and complexity of fabrication. Cost is largely a function of volume. To achieve high volumes, advances in roll-to-roll manufacturing using nanoimprint lithography are required, which are currently being developed by EPIC member companies such as EVGroup and Obducat AB. Additionally, high-volume production requires a mold stamping process; the challenge there is not only in fabricating the molds but also having a comprehensive supply chain in place to provide components such as detectors, light sources, microdisplays, and others to very high specifications. The need for such a supply chain is applicable not only to the AR and VR worlds but to all photonics-enabled devices in general. Microdisplays Another big requirement for advancement is the production of microdisplays that are around 1 sq in. and positioned close to the retina. While there are several European companies developing solutions for microdisplays — CEA-Leti and Aledia, among them — the challenge is to develop what will be the standard for microdisplays in the future. Existing state-of-the-art technology already offers fully mature technologies, including LCDs and digital light projectors (DLPs), which each comprise two million individually controlled micromirrors, corresponding to a 2-MP image projected onto the retina. With the popularity of micro-optics, many European companies and initiatives see a niche for the market. It is expected, for example, that there will be a large demand for freeform micro-optics. Manufacturers, including Fraunhofer IPT, offer a range of micro-optics solutions (Figure 4). Figure 4. A portfolio of micro-optics solutions offered by Fraunhofer IPT. Courtesy of Fraunhofer IPT. Courtesy of the European Photonics Industry Consortium. As the market demands further miniaturization, other solutions are quickly emerging, including microdisplays based on OLED technology, as well as those based on liquid crystal on silicon (LCOS), which is currently used by Microsoft’s HoloLens, Google Glass, and the Magic Leap One. There are two key approaches for OLED-on-silicon displays — WOLED (white OLED) and True RGB OLED. WOLED displays (the backlight in most OLED TVs) need a reference white pixel and filters to transform the white pixel into different colors. In contrast, RGB OLEDs (mostly used in smartphones) emit red, green, and blue light directly. While RGB OLEDs have a higher efficiency and higher color range, they suffer from complexity and differential aging of the three colors. This gives WOLEDs the advantage. Micro-LED displays All of these technologies represent only an intermediate step toward the final goal of using micro-LED technology to create very high-resolution microdisplays, which will lead to a huge paradigm shift in the AR and VR markets (Figure 5). Figure 5. The projected adoption rate of micro-LEDs in the AR market. Source: Displays and optical vision systems for VR/AR/MR report, Yole Développement, 2018. Courtesy of Yole Développement. Courtesy of the European Photonics Industry Consortium. EPIC members, including Aledia, are developing this type of technology. And in the near future we expect a boom in the demand for micro-LED displays not only for AR and VR but also for the automotive, avionics, and smartwatch sectors, along with others where small-size, very high-resolution displays are required. As market data indicates, by 2027, micro-LEDs are projected to account for around 30 percent of market share. But to achieve this, a number of value chain problems will need to be solved. Specifically, the issue of volume testing that must be done at wafer level. One main challenge is the complexity and number of light sources needed, which makes tests complex, expensive, and slow, thus limiting the production of micro-LEDs. VR standalone glasses Retail market data, as detailed in market analysis by GfK Retail & Technology GmbH, indicates that the VR hardware market is currently dominated by mobile phone gaming, game consoles, and high-end computers as the principle platforms. While gamers are continually seeking a better gaming experience, the market for standalone VR glasses is still negligible at only 1 percent. According to a GfK gamer survey, the main reason for slow market acceptance is that most headsets currently on the market are heavy, bulky, difficult to wear, and uncomfortable for use over long periods. GfK studies predict that with these problems resolved, standalone headsets will see a growth potential of several hundred percent. To create a new generation of VR and AR devices that enhance the user experience, new types of optical and photonics devices need to be generated. Specifically, a new generation of smaller, freeform optics is needed to reduce the size and weight of the headsets. Additionally, smart designs and use of new materials are required, not only to reduce size and weight, but also to handle thermal management due to the high level of integration between the optics and the electronics. Future applications Although gaming is currently the big market driver, the near future should see healthy growth in the virtual home market, along with growth in virtual travel, sports, and tourism, especially for venues such as art museums. The EPIC Meeting on Photonics for AR/VR in Stuttgart, Germany, brought together the whole value chain of photonics technologies. Courtesy of the European Photonics Industry Consortium. However, the success of smart speaker systems, including Amazon Echo, Google Home, and Apple HomePod, which react to voice commands and provide a range of skills and connectivity to domestic appliances, indicates a move away from glasses and headsets for accessing VR. The next generation of devices for interaction with machines will inevitably supplement voice commands with the ability to interpret gestures, hand movements, and body language. Because these devices will need to incorporate a range of photonics technologies — for example, 3D IR sensors, microdisplays, and lidar — the photonics industry is well-positioned to play an important role in this new world. Meet the authors Jose Pozo, Ph.D., is the director of technology and innovation at the European Photonics Industry Consortium (EPIC). He has a 15-year background in photonics technology and market knowledge, and a large network within the industrial and academic photonics landscape. Pozo is a member of the board of the IEEE Photonics Society Benelux. He has a doctorate in electrical engineering from the University of Bristol in England and bachelor’s and master’s degrees in telecom engineering. Ana Belén González Guerrero, Ph.D., is a project leader at EPIC. Her expertise lies in the development of systems based on integrated photonic circuits, packaging and assembly, and the investigation of applications such as chemical/biological sensing and Datacom. In addition, she has been involved in technology transfer and business development processes. She has a bachelor’s degree in chemistry from the University Autonomous of Barcelona and a doctorate from the Catalan Institute of Nanoscience and Nanotechnology. Acknowledgments EPIC wishes to thank the companies participating at the EPIC Meeting on Photonics for AR/VR held at the R&D facilities of Sony in Stuttgart, Germany, for the great discussion and technical inputs. Namely: Bosch Sensortec, Carl Zeiss, CEA-Leti, Coherent, Comptek Solutions, Corning Laser Technologies, DigitArena, Eclipse Optics, Exalos, Focuz Manufacturing, Fogale Nanotech, Fraunhofer FEP, Fraunhofer IOF, Fraunhofer IPT, GfK Retail & Technology, Hitachi High-Technologies Europe, Huawei, ICFO, IMASENIC, Inkron, Jenoptik, Joanneum Research, Jolt Capital, KIT, LIGHT TEC, Limbak, LioniX International, MicroscopeIT, Multiphoton Optics, Nanoscribe, NIL Technologies, Oplatek, Optics Balzers, Oxford Instruments, Physik Instrumente (PI), Q.ant, SABIC Innovative Plastics, SCHOTT, Sony, Synopsys, Tarkas Venture, TEMATYS, TNO, Tobii, TOPTICA Photonics, the VR/AR Association, VRgineers, Yelo, and especially Yole Développement for its market input.