Lensless Neural Network Opens Up Machine Vision Possibilities

Researchers led by Hongwei Chen of the Beijing National Research Center for Information Science and Technology and Tsinghua University have developed a lensless optoelectronic neural network architecture for computer vision tasks. The system uses a passive mask inserted in the imaging light path to perform convolution operations in the optical field.

The system addresses the challenge of processing incoherent and broadband light signals in natural scenes, and it is designed to overcome the bandwidth bottlenecks encountered by electrical convolutional neural networks while at the same time avoiding problems encountered by optical neural networks (ONNs).

ONNs require a coherent laser as the light source for computation, and there is a need for an alternative for ONNs to work in combination with a mature machine vision systems in natural light scenes. This has led to interest in the development of optoelectronic hybrid neural networks in which the front end is optical and the back end is electrical. These lens-based systems increase the difficulty of use in edge devices, such as autonomous vehicles.
Schematic diagram of the optical mask replacing the convolutional layer of the network. Courtesy of Tsinghua University.

Schematic diagram of the optical mask replacing the convolutional layer of the network. Courtesy of Tsinghua University.

Schematic of the optical mask replacing the convolutional layer of the network. Courtesy of Tsinghua University.

Compared to the hardware architecture in conventional machine vision systems, the researchers proposed an optical mask positioned close to the image sensor to replace the lenses. According to geometrical optics theory, that light propagates in a straight line, the scenes can be regarded as sets of point light sources, and the optical signal is spatially modulated by the mask to realize the convolution operation of shift and superposition on the image sensor.

QPC Lasers Inc. - QPC Lasers is LIDAR 4-24 MR

To perform object classification tasks like handwritten digit recognition, the team built a lightweight network for real-time recognition to verify the performance of the optical convolution in the architecture. While using a single convolution kernel, the recognition accuracy reached 93.47%. When the multichannel convolution operation is implemented by arranging multiple kernels in parallel on the mask, the classification accuracy was improved to 97.21%. Compared with traditional machine vision links, the systems was shown to save about 50% of energy consumption.

Further, by expanding the dimension of the optical mask, the image is convolved in the optical domain. The sensor then captured an image that is unrecognizable to the human eye, which enabled natural encryption of private information without computational consumption.

The team confirmed the performance of the optical encryption with a facial recognition task. Compared with the random maximum length sequence pattern, the recognition accuracy of the mask jointly optimized by an end-to-end network was improved by more than 6%.

The researchers envision the technology having application in autonomous driving, smart homes, and smart security.

The research was published in Light: Science & Applications (www.doi.org/10.1038/s41377-022-00809-5).

Published: May 2022

Glossary

mask: 1. A framelike structure that serves to restrict the viewing area of the screen when placed before a television picture tube. 2. In photolithography, a photomask (or mask) is typically a patterned transparent plate or an opaque plate with patterned holes or transparencies that uses a laser light source to transfer and print the pattern by an etching process onto a substrate that is typically a silicon wafer used in integrated circuitry.
machine vision: Machine vision, also known as computer vision or computer sight, refers to the technology that enables machines, typically computers, to interpret and understand visual information from the world, much like the human visual system. It involves the development and application of algorithms and systems that allow machines to acquire, process, analyze, and make decisions based on visual data. Key aspects of machine vision include: Image acquisition: Machine vision systems use various...
convolutional neural network: A powerful and flexible machine-learning approach that can be used in machine vision to help solve difficult problems. Inspired by biological processes, multiple layers of neurons process portions of an image to arrive at a classification model. The network of neurons is trained by a set of input images and the output classification (e.g., picture A is of a dog, picture B is of a cat, etc.) and the algorithm trains the neuron connection weights to arrive close to the desired classification. At...

Browse Cameras & Imaging, Lasers, Optical Components, Test & Measurement, and more.