Close

Search

Search Menu
Photonics Media Photonics Buyers' Guide Photonics Spectra BioPhotonics EuroPhotonics Vision Spectra Photonics Showcase Photonics ProdSpec Photonics Handbook
More News

Driving with shrimp vision

Facebook Twitter LinkedIn Email Comments

Mantis shrimp — a distant relative of the prawn that hang off the edges of cocktail glasses — were the inspiration for a new type of camera that could make autonomous cars even safer.

The camera, featuring a high-dynamic-range polarization imaging sensor, was developed by a team at the University of Illinois at Urbana-Champaign. Its dynamic range is about 10,000× higher than commercial cameras currently being used. By mimicking the vision of a mantis shrimp, the sensor enables the camera to better spot hazards in challenging imaging conditions, such as transitions from a dark area (a garage, for example) to a bright area (outside on a sunny day), and in hazy or foggy conditions. Compared to the color cameras currently used in cars, this new camera can detect hazards 3× farther.

Mantis shrimp (Odontodactylus scyllarus).


Mantis shrimp (Odontodactylus scyllarus).


“In a recent crash involving a self-driving car, the car failed to detect a semitruck because [the truck’s] color and light intensity blended with that of the sky in the background,” said research team leader Viktor Gruev, an associate professor in the electrical engineering department at Illinois. “Our camera can solve this problem because its high dynamic range makes it easier to detect objects that are similar to the background, and the polarization of a truck is different than that of the sky.”

Polarization imaging can act like memory foam in that it “remembers” and encodes information such as 3D shape, surface roughness, and material or tissue structure composition. Although most vertebrate species, including humans, are not equipped with the ability to distinguish polarization of light, invertebrates have evolved with high-dynamic-range photosensitive cells to detect polarization and utilize it in visually guided behavior.

The mantis shrimp was the inspiration for a new type of camera that could make autonomous cars even safer. Courtesy of iStock/vectorfun.


The mantis shrimp was the inspiration for a new type of camera that could make autonomous cars even safer. Courtesy of iStock/vectorfun.


Among the various vision systems within the animal kingdom, that of the mantis shrimp is most complex, with capabilities to detect 16 spectral channels, as well as four linear and two circular polarization channels. Also, individual photoreceptors have logarithmic responses to incident light intensity, which equips the creatures with high-dynamic-range imaging capabilities.

The polarization imaging sensors (polarimeters) currently used in a number of applications are outmatched by digital color cameras for frame rate, resolution, noise, and dynamic range. Such optoelectronic limitations, along with limited dynamic range, have hindered the industrial integration of polarization technology. To address them, the researchers took to the sea, imitating the complex visual system of the mantis shrimp in two ways — first, by utilizing four different pixelated polarization filters that are offset by 45°; second, by operating the sensor’s underlying photodiodes in a forward bias mode rather than the traditional reverse bias mode.

Sample image showing the co-registered color and polarization. Regular color image (a). Degree of linear polarization (DoLP) represented in a false-color map, where red and blue indicate fully polarized and unpolarized light, respectively (b). Angle of polarization (AoP) represented in a false-color map, where red and light blue indicate horizontally and vertically polarized light, respectively (c). Courtesy of The Optical Society of America (OSA).


Sample image showing the co-registered color and polarization. Regular color image (a). Degree of linear polarization (DoLP) represented in a false-color map, where red and blue indicate fully polarized and unpolarized light, respectively (b). Angle of polarization (AoP) represented in a false-color map, where red and light blue indicate horizontally and vertically polarized light, respectively (c). Courtesy of The Optical Society of America (OSA).


These combined advancements enabled the researchers to create an imager that achieves a 30-fps, 140-dB dynamic range, along with a 61-dB maximum signal-to-noise ratio across 384 × 288 pixels equipped with logarithmic photodiodes.

The resulting bioinspired polarization imager is compact, with low fabrication cost, potentially making it an integral part of automotive and remote sensing applications.


BioPhotonics
May/Jun 2019
GLOSSARY
polarization
With respect to light radiation, the restriction of the vibrations of the magnetic or electric field vector to a single plane. In a beam of electromagnetic radiation, the polarization direction is the direction of the electric field vector (with no distinction between positive and negative as the field oscillates back and forth). The polarization vector is always in the plane at right angles to the beam direction. Near some given stationary point in space the polarization direction in the beam...
mantis shrimpautonomous carspolarizationimagingSensors & DetectorsUniversity of Illinois at Urbana-Champaign3Dhigh dynamic rangephotodiodesPost Scripts

Comments
back to top
Facebook Twitter Instagram LinkedIn YouTube RSS
©2019 Photonics Media, 100 West St., Pittsfield, MA, 01201 USA, info@photonics.com

Photonics Media, Laurin Publishing
x Subscribe to BioPhotonics magazine - FREE!
We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.