Improving the manufacturability, reliability and size of hardware components for 3-D sensing systems opens up new applications for the technology. Today, a growing number of smart TVs and operating systems respond to gestured commands by changing channels, zooming and scrolling. Sensors detect motion and surface qualities on manufacturing lines, controlling how items flow through processes. Designers can scan complex shapes for replication with 3-D printers, and surveillance systems can track animals and humans to keep them out of dangerous areas. As the technology evolves, however, sensors are detecting ever-finer motions and characteristics. Modern sensors can not only detect a head nod, but also precisely identify to whom the head belongs. Besides identity, they can detect heartbeats and the subtle facial characteristics that communicate emotions. And 3-D sensing has moved outdoors to environments that previously were too bright or had complex lighting that precluded useful surveillance. The market for gesture recognition and touchless sensing will see growth of 79 percent over the next few years, according to a recent report from MarketsandMarkets;1 CompaniesandMarkets.com2 sees a 50.7 percent compound annual growth rate and $7.15 billion spent through 2018; and IHS Automotive3 predicts more than 38 million units in automotive applications by 2023. This spectacular market growth and the increasing number of applications adopting 3-D sensing are enabled by advancements in illumination and optical technologies – advances that have further miniaturized components and optimized manufacturability to dramatically reduce costs. The basics Three-dimensional sensing is simply tracking changes in an object’s characteristics and location over time. To do this, humans use two image sensors equipped with optical lenses and filters (eyes) and a sophisticated processor (brain) that work together to monitor changes in 390- to 700-nm light patterns. 3-D sensing devices work pretty much the same way, typically adding an illumination source to provide known lighting. In a computer game where a player swings a golf club in front of a 3-D sensor, illumination sources in the device flood the player and surrounding area with usually invisible NIR light. The light bounces off the player and reflects back to the device. Optical filters screen out spurious and ambient light, letting only the NIR spectra through to the light sensor. Interpreting changes in light patterns over time, firmware creates digital 3-D mapping of the action and continuously feeds it to the game program. 3-D sensing systems use various configurations and software approaches to do the job, but they all share the same basic hardware component list: illumination sources (LEDs or laser diodes typically generate IR or NIR light); controlling optics (optical lenses help optimally illuminate the environment and focus reflected light onto a detector surface, and bandpass filters allow only reflected light matching the illuminating frequency to reach the light sensor, eliminating ambient and other stray light); depth camera (an optical receiver detects the reflected, filtered light, converting it to an electrical signal for processing by the firmware); and firmware (high-speed application-specific integrated circuit or digital signal processing chips process the information received and turn it into a format that can be understood by the end-user application). Figure 1 shows basic 3-D sensing components. Figure 1. 3-D sensing systems all share the same basic hardware components. Because of their spectral precision and electrical-to-optical conversion efficiency, diode lasers are the preferred light source for consumer electronics applications. These applications are characterized by a limited source of electrical power and a high density of components – factors that drive the need to minimize dissipated thermal power. The lasers are used with optical filters and sensors that are wavelength-sensitive, requiring tight wavelength control over a wide temperature range. LED-based illumination systems are disadvantaged by the limited selection of wavelengths and poor illumination accuracy. The quantum efficiency of sensors is much higher at the lower wavelengths used by low-wavelength lasers. Commercial, off-the-shelf LEDs have broad illumination cones that often direct as much as 50 percent of the light outside the volume of interest. These two factors, coupled with the benefits highlighted in Table 1, can result in a five- to tenfold improvement in overall system efficiency with laser-based designs. Table 1. Laser diode/LED performance comparison. Single-mode diode lasers feature an index-guided waveguide to provide high power, low astigmatism, narrow spectral width and a single-spatial-mode Gaussian far field. They also operate at high powers with low operating current. These devices are extremely reliable at temperatures and powers as high as 60 °C at 200 mW. Fabry-Perot vertical-cavity surface-emitting lasers (VCSELs) have extreme power densities (~1-W single-transverse mode) and high efficiencies without compromised reliability. These attributes make them suitable for applications where overall efficiency and high power densities are critical, such as 3-D sensing. Importantly, VCSELs are amenable to wafer-scale testing and assembly, and they are scalable for extremely high volume applications. Optical filters for 3-D sensing systems typically are narrow-bandpass NIR filters with low signal-to-noise ratios in the desired band over a large angle range and with thorough blocking elsewhere. The optical filtering component may be either a camera lens coating or a stand-alone element. Limiting the light that reaches the sensor eliminates data unrelated to the 3-D-sensing task at hand. Along with noise-suppressing functionality coded into application software, this approach dramatically reduces the processing load on firmware. Evolving technologies Three factors define a component’s success in the 3-D sensing marketplace: size, efficiency and cost. Miniaturization is a prerequisite for emerging applications such as wearable devices – obviously, an eyeglass frame has a limited capacity for carrying an illumination source, a sensor, a processor and a power source to drive them all. Component efficiency enables miniaturization as well as reduced power requirements. Manufacturability – keeping costs sufficiently low – is what drives availability for high-volume consumer applications. Recent developments in laser diode technologies have reduced diode thicknesses to 2 to 3 mm. This level of miniaturization – after all, a human retina is about 0.5 mm thick – means that size alone is simply no longer much of an issue. Designs also have recorded extraordinary gains in efficiency. Higher efficiency – getting the most light from the least power – affects 3-D-sensing systems in two important ways. Lower power requirements from illumination sources mean that more power can be dedicated to processing, the heart of the 3-D equation. Batteries can be smaller, and processing power can be greater. The other effect of high efficiency is the option for less heat dissipation. Designers now have a greater choice when weighing illumination needs against power requirements. The best performance will come when a laser diode operates at full power, providing bright illumination for sensors with finer and finer resolution. However, if design considerations dictate a smaller power source, the latest diodes will still offer adequate illumination. Some new laser diodes have considerably more stable wavelengths, and matching them with appropriate filters is much easier. When it comes to optimizing systems to variations in ambient light, tuning laser/filter wavelengths is the key, particularly when it comes to applications working in the dark. Figure 2. High-volume laser diode packaging speeds up manufacturing and lowers costs, ultimately driving down pricing. Lastly, new laser chip designs no longer require hermetic sealing, enabling the use of lower-cost, off-the-shelf integrated-chip and LED packaging instead of customized packaging. This development substantially speeds manufacturing and reduces costs, ultimately driving down end-user pricing. Figure 2 shows laser diode packaging. With the advent of greater sensor resolution and more powerful illumination, maintaining a maximum signal-to-noise ratio with the lowest possible angle shift is imperative. Mobile devices in particular tend to use large-field-of-view thin-film interference filters that are sensitive to angle shift, and with a bandpass that changes throughout the wavelength spectrum. Proprietary developments in thin-film deposition techniques have resulted in filters that substantially lessen performance variations due to changes in angles. These increases in refractive indices mean improved 3-D sensing performance at the edges of the monitored area. This also decreases ambient light in outdoor applications, expanding the effective reach of 3-D systems (Figure 3). Figure 3. Low and standard angle-shift design: New thin-film deposition techniques bring filters that substantially lessen performance variations due to changes in angles. These improved techniques can apply coatings on discrete filters and directly onto CMOS sensors. Depositing directly on sensor lenses is more costly but reduces size. Although the latest manufacturing techniques can mass-produce filters on glass as thin as 0.2 mm, every little bit counts when shrinking the optical path. Resolution Detecting gestures like arm and hand movements doesn’t require the finest sensor resolution. However, facial recognition does: Recognizing heartbeats and eye movements enables another level of intelligence – and for this, sensors need much smaller pixels. The newest high-volume consumer smartphones offer up to 41-megapixel resolution, which provides more than adequate detail for any current 3-D-sensing application. The gating factor for resolution is the processing power needed to handle the data. Pushing processor technology typically means increasing cost and power, and both these factors count against high-volume applications, particularly in consumer electronics. Software developers have created sophisticated new algorithms that can make more efficient use of whatever resolution data is available. Yet for applications such as fixed surveillance systems, where neither cost nor power is an issue, the combination of the latest laser diodes, filters and high-resolution sensors makes for powerful 3-D-sensing capabilities. Evolving applications Recent improvements in how 3-D systems deal with ambient light are critical to automotive applications. Inside the vehicle, the primary sensing area is directly beneath a windshield and can be bathed in bright sunlight or spotlighted by headlights. Subtle, touchless control of automotive “infotainment” displays under any lighting conditions has already been demonstrated, and implementation is likely to steadily increase across all price ranges, from economy to luxury vehicles, because it is not merely convenience, but a solid improvement in safety. Parking assistance and precrash detection also are growing applications, as 3-D sensors are used to map the vehicle’s surroundings. 3-D facial recognition, which measures the geometry of rigid facial features, rivals fingerprints in accuracy. On a practical level, this means that tedious logins will soon be a thing of the past, as 3-D-sensing enabled computers immediately recognize and validate users even for banking applications like ATMs. Indeed, the whole world of “locked” access will change – applications, doors and information sources will simply swing open for the authorized. Wearable devices stand to benefit considerably from increasing miniaturization. If 3-D sensing were to be implemented on a near-to-eye display, users would be able to find their way quickly through complex buildings as easily as they navigate roads. For the visually impaired, the potential benefits are huge. Component manufacturing Imagining applications for 3-D sensing is relatively easy, but translating one’s imagination into effective systems, prototypes and even low-volume units is a considerable challenge. Then, scaling those systems and delivering a reliable product consistently and in volume is another challenge that is an order of magnitude even more complex; a demonstrated ability to consistently manufacture high volumes of reliable miniaturized illumination sources and optical filters (Figure 4) is a key element of any credible effort to scale in 3-D sensing. Figure 4. Optical filters must be consistently manufactured in high volumes; they also must be consistently reliable. High-data-rate systems such as those used for 3-D sensing require components that operate with low failure rates and with minimal degradation over time; a gating issue with these components has been manufacturability. With any high-volume, low-margin consumer electronics component, the cost profile can make the difference between an inventor’s fantasy and a revenue-generating reality. While a number of suppliers may demonstrate the ability to deliver high-performance proof-of-concept or even prototype parts in low volumes, only a global manufacturer with proven engineering, development and production resources dedicated to 3-D sensing can credibly and economically produce the parts in the millions to the necessarily tight performance standards demanded by the leaders in this evolving segment of the consumer electronics market. Meet the authors Dr. Andre Wong is the director of product line management for commercial diode lasers at JDSU; email: andre.wong@jdsu.com. Markus Bilger is a senior product line manager for optical filters at JDSU; email: markus.bilger@jdsu.com. References 1. MarketsandMarkets. Gesture Recognition & Touch-Less Sensing Market by Technology: Global Forecasts and Analysis to 2013-2020. 2. CompaniesandMarkets.com (April 5, 2013). Gesture Recognition & Touchless Sensing Market: 2013-2018. 3. IHS Automotive. Emerging Technologies: New Human-Machine Interface Trends, 2013. Basic hardware for 3-D sensing systems: • Illumination sources (LEDs or laser diodes) • Controlling optics (lenses and bandpass filters) • Depth camera • Firmware