Search
Menu

Driving with shrimp vision

Facebook X LinkedIn Email
RESHAWNA MAINE [email protected]

Mantis shrimp — a distant relative of the prawn that hang off the edges of cocktail glasses — were the inspiration for a new type of camera that could make autonomous cars even safer.

The camera, featuring a high-dynamic-range polarization imaging sensor, was developed by a team at the University of Illinois at Urbana-Champaign. Its dynamic range is about 10,000× higher than commercial cameras currently being used. By mimicking the vision of a mantis shrimp, the sensor enables the camera to better spot hazards in challenging imaging conditions, such as transitions from a dark area (a garage, for example) to a bright area (outside on a sunny day), and in hazy or foggy conditions. Compared to the color cameras currently used in cars, this new camera can detect hazards 3× farther.

Mantis shrimp (Odontodactylus scyllarus).


Mantis shrimp (Odontodactylus scyllarus).


“In a recent crash involving a self-driving car, the car failed to detect a semitruck because [the truck’s] color and light intensity blended with that of the sky in the background,” said research team leader Viktor Gruev, an associate professor in the electrical engineering department at Illinois. “Our camera can solve this problem because its high dynamic range makes it easier to detect objects that are similar to the background, and the polarization of a truck is different than that of the sky.”

Polarization imaging can act like memory foam in that it “remembers” and encodes information such as 3D shape, surface roughness, and material or tissue structure composition. Although most vertebrate species, including humans, are not equipped with the ability to distinguish polarization of light, invertebrates have evolved with high-dynamic-range photosensitive cells to detect polarization and utilize it in visually guided behavior.

The mantis shrimp was the inspiration for a new type of camera that could make autonomous cars even safer. Courtesy of iStock/vectorfun.


The mantis shrimp was the inspiration for a new type of camera that could make autonomous cars even safer. Courtesy of iStock/vectorfun.


Among the various vision systems within the animal kingdom, that of the mantis shrimp is most complex, with capabilities to detect 16 spectral channels, as well as four linear and two circular polarization channels. Also, individual photoreceptors have logarithmic responses to incident light intensity, which equips the creatures with high-dynamic-range imaging capabilities.


The polarization imaging sensors (polarimeters) currently used in a number of applications are outmatched by digital color cameras for frame rate, resolution, noise, and dynamic range. Such optoelectronic limitations, along with limited dynamic range, have hindered the industrial integration of polarization technology. To address them, the researchers took to the sea, imitating the complex visual system of the mantis shrimp in two ways — first, by utilizing four different pixelated polarization filters that are offset by 45°; second, by operating the sensor’s underlying photodiodes in a forward bias mode rather than the traditional reverse bias mode.

Sample image showing the co-registered color and polarization. Regular color image (a). Degree of linear polarization (DoLP) represented in a false-color map, where red and blue indicate fully polarized and unpolarized light, respectively (b). Angle of polarization (AoP) represented in a false-color map, where red and light blue indicate horizontally and vertically polarized light, respectively (c). Courtesy of The Optical Society of America (OSA).


Sample image showing the co-registered color and polarization. Regular color image (a). Degree of linear polarization (DoLP) represented in a false-color map, where red and blue indicate fully polarized and unpolarized light, respectively (b). Angle of polarization (AoP) represented in a false-color map, where red and light blue indicate horizontally and vertically polarized light, respectively (c). Courtesy of The Optical Society of America (OSA).


These combined advancements enabled the researchers to create an imager that achieves a 30-fps, 140-dB dynamic range, along with a 61-dB maximum signal-to-noise ratio across 384 × 288 pixels equipped with logarithmic photodiodes.

The resulting bioinspired polarization imager is compact, with low fabrication cost, potentially making it an integral part of automotive and remote sensing applications.


Published: April 2019
Glossary
polarization
Polarization refers to the orientation of oscillations in a transverse wave, such as light waves, radio waves, or other electromagnetic waves. In simpler terms, it describes the direction in which the electric field vector of a wave vibrates. Understanding polarization is important in various fields, including optics, telecommunications, and physics. Key points about polarization: Transverse waves: Polarization is a concept associated with transverse waves, where the oscillations occur...
high dynamic range
High dynamic range (HDR) refers to a technology that allows for a broader and more dynamic range of luminosity in visual content, such as images or videos. It is particularly associated with displays like TVs, monitors, and cameras. The dynamic range is the ratio between the darkest and brightest parts of an image. In the context of imaging, HDR techniques capture and display a wider range of brightness levels, enabling more details to be visible in both the shadows and highlights of an...
mantis shrimpautonomous carspolarizationImagingSensors & DetectorsUniversity of Illinois at Urbana-Champaign3Dhigh dynamic rangephotodiodesPostscripts

We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.