Search
Menu
Deposition Sciences Inc. - Difficult Coatings - LB - 8/23

On-Chip Neural Network Classifies Images Almost Instantaneously

Facebook X LinkedIn Email
An optical deep neural network integrated on a chip could be a significant advancement for image classification technology. Its on-chip neural network can recognize and classify an image in less than a nanosecond.

Further, the photonic neural network, developed at the University of Pennsylvania, is scalable to allow it to classify images of increasing complexity.

The speed of the on-chip deep neural network derives from its ability to directly process the light it receives from an object of interest. It does not need to convert optical signals to electrical signals or change input data to a binary format before it can recognize images.
Using a deep neural network of optical waveguides, the researchers’ chip — smaller than a square centimeter — can detect and classify an image in less than a nanosecond, without the need for a separate processor or memory unit. Courtesy of Ella Maru Studio.

 


Using a deep neural network of optical waveguides, the researchers’ chip — smaller than a square centimeter — can detect and classify an image in less than a nanosecond, without the need for a separate processor or memory unit. Courtesy of Ella Maru Studio.
Waveguides connect the chip’s optical neurons, forming a network of multiple layers similar to the neuronal layers in the human brain. As information moves through the layers of the network, the input image is classified with increasing accuracy into a category that the network has previously learned.

The researchers ensured that the training for the network was specific enough to result in accurate image classifications, but general enough to be useful to the network when presented with new data sets. The network can be scaled up by adding neural layers. As layers are added, the network’s ability to read data in more complex images, with higher resolution, grows.

Though current on-chip image classification technology can perform billions of computations per second, the computing speed is limited by a linear, clock-based processing schedule that in traditional systems requires computation steps to be performed one after another.

In contrast, the on-chip deep neural network directly processes optical waves as they propagate through the network’s layers. The nonlinear activation function is realized optoelectronically and allows a classification time of under 570 ps.

“Our chip processes information through what we call ‘computation-by-propagation,’ meaning that unlike clock-based systems, computations occur as light propagates through the chip,” said Farshid Ashtiani, a researcher on the work. “We are also skipping the step of converting optical signals to electrical signals because our chip can read and process optical signals directly, and both of these changes make our chip a significantly faster technology.”
Firooz Aflatouni, associate professor in electrical and systems engineering at the University of Pennsylvania. Courtesy of University of Pennsylvania.
Firooz Aflatouni, associate professor in electrical and systems engineering at the University of Pennsylvania. Courtesy of University of Pennsylvania.
The researchers showed that the chip can perform an entire image classification in 0.5 ns — the time it takes traditional digital computer chips to complete just one computation step. The on-chip network demonstrated two-class and four-class classification of handwritten letters with accuracies higher than 93.8% and 89.8%, respectively.

Hamamatsu Corp. - Earth Innovations MR 2/24

“To understand just how fast this chip can process information, think of a typical frame rate for movies,” professor Firooz Aflatouni said. “A movie usually plays between 24 and 120 frames per second. This chip will be able to process nearly 2 billion frames per second.”

In addition to eliminating analog-to-digital conversion, the chip’s direct, clockless optical data processing removes the need for a memory module, allowing faster and more energy-efficient neural networks. “When current computer chips process electrical signals, they often run them through a graphics processing unit, or GPU, which takes up space and energy,” Ashtiani said. “Our chip does not need to store the information, eliminating the need for a large memory unit.”

The elimination of a memory module can also increase data privacy. “With chips that read image data directly, there is no need for photo storage and thus, a data leak does not occur,” Aflatouni said.
As a proof of concept, the chip was tested on data sets containing either two or four types of handwritten characters. It achieved classification accuracies higher than 93.8% and 89.8%, respectively. Courtesy of the University of Pennsylvania.
As a proof of concept, the chip was tested on data sets containing either two or four types of handwritten characters. It achieved classification accuracies higher than 93.8% and 89.8%, respectively. Courtesy of the University of Pennsylvania.
By speeding up image classification, the on-chip deep neural network could improve applications such as face recognition and lidar sensing in self-driving cars. The on-chip photonic deep neural network can be used for applications across a range of data types.

“We already know how to convert many data types into the electrical domain — images, audio, speech, and many other data types,” Aflatouni said. “Now, we can convert different data types into the optical domain and have them processed almost instantaneously using this technology.”

The next steps for the team will be to further investigate the scalability of the chip and explore 3D object classification.

The research was published in Nature (www.doi.org/10.1038/s41586-022-04714-0).

Published: June 2022
Glossary
integrated photonics
Integrated photonics is a field of study and technology that involves the integration of optical components, such as lasers, modulators, detectors, and waveguides, on a single chip or substrate. The goal of integrated photonics is to miniaturize and consolidate optical elements in a manner similar to the integration of electronic components on a microchip in traditional integrated circuits. Key aspects of integrated photonics include: Miniaturization: Integrated photonics aims to reduce the...
optoelectronics
Optoelectronics is a branch of electronics that focuses on the study and application of devices and systems that use light and its interactions with different materials. The term "optoelectronics" is a combination of "optics" and "electronics," reflecting the interdisciplinary nature of this field. Optoelectronic devices convert electrical signals into optical signals or vice versa, making them crucial in various technologies. Some key components and applications of optoelectronics include: ...
artificial intelligence
The ability of a machine to perform certain complex functions normally associated with human intelligence, such as judgment, pattern recognition, understanding, learning, planning, and problem solving.
deep learning
Deep learning is a subset of machine learning that involves the use of artificial neural networks to model and solve complex problems. The term "deep" in deep learning refers to the use of deep neural networks, which are neural networks with multiple layers (deep architectures). These networks, often called deep neural networks or deep neural architectures, have the ability to automatically learn hierarchical representations of data. Key concepts and components of deep learning include: ...
machine vision
Machine vision, also known as computer vision or computer sight, refers to the technology that enables machines, typically computers, to interpret and understand visual information from the world, much like the human visual system. It involves the development and application of algorithms and systems that allow machines to acquire, process, analyze, and make decisions based on visual data. Key aspects of machine vision include: Image acquisition: Machine vision systems use various...
nonlinear optics
Nonlinear optics is a branch of optics that studies the optical phenomena that occur when intense light interacts with a material and induces nonlinear responses. In contrast to linear optics, where the response of a material is directly proportional to the intensity of the incident light, nonlinear optics involves optical effects that are not linearly dependent on the input light intensity. These nonlinear effects become significant at high light intensities, such as those produced by...
Research & TechnologyeducationUniversity of PennsylvaniaAmericasImagingintegrated photonicsLight SourcesOpticsoptoelectronicsSensors & Detectorsartificial intelligencedeep learningneural networksautomotiveBiophotonicsConsumermachine visioninductrialcamerassemiconductorsimage classificationon-chip technologiesdefensenonlinear opticsTechnology NewsThe News Wire

We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.