Search
Menu

Photonic-Electronic NIC Serves Live Machine Learning Requests

Facebook X LinkedIn Email
A burgeoning growth in machine learning applications is occurring alongside shrinking computing resources. As computers approach the limits of their power according to Moore’s law, the need to design platforms that support the computational demands of machine learning applications intensifies.

Lightning, a hybrid computing platform developed by an MIT research team, is the first photonic computing system to serve machine learning inference requests in real time. Lightning is a reconfigurable, photonic-electronic, smart network interface card (smartNIC) that is both fast and energy-efficient.

Unlike their electronic counterparts, photonic computing devices lack the memory or instructions to control dataflows. Lightning removes this obstacle and ensures that data moves smoothly between a computer’s electronic and photonic components. It uses a fast data path to feed traffic from the NIC into the photonic domain without creating bottlenecks.
MIT researchers introduce Lightning, a reconfigurable, photonic-electronic, smartNIC that serves real-time deep neural network inference requests at 100 Gbps. Courtesy of Alex Shipps/MIT CSAIL via Midjourney.
MIT researchers introduced Lightning, a reconfigurable, photonic-electronic, smartNIC that serves real-time deep neural network inference requests at 100 Gbit/s. Courtesy of Alex Shipps/MIT CSAIL via Midjourney.

To achieve a fast, smooth data path, the system leverages a reconfigurable count-action abstraction that keeps track of the required computation operations of each inference packet. The count-action abstraction controls access to the data moving through the system. It counts the number of operations in each task and triggers the execution of the next task without interrupting the dataflow.

The count-action abstraction connects electronics and photonics. The information carried by electrons is translated into photons that work at light speed to assist in the completion of an inference task. Then, the photons are converted back to electrons to relay the information to the computer.

According to researcher Zhizhen Zhong, photonic computing provides significant advantages when it comes to bulky linear computing tasks such as matrix multiplication, but for other tasks, it needs an assist from electronics. This, he said, creates a significant amount of data to be exchanged between the photonic components and the electronic components in order to complete tasks such as machine learning inference requests.

By combining the speed of photonics with the dataflow control capabilities of electrons, Lightning can serve real-time, deep neural network inference requests at an impressive speed of 100 Gbit/s.

“Controlling this dataflow between photonics and electronics was the Achilles’ heel of past state-of-the-art photonic computing works,” Zhong said. “Even if you have a superfast photonic computer, you need enough data to power it without stalls. Otherwise, you’ve got a supercomputer just running idle without making any reasonable computation.”


The researchers said that previous attempts to develop a photonic-electronic computing platform used a “stop-and-go” approach. In that approach, control software makes all the decisions about the movement of data, slowing the dataflow.

“Building a photonic computing system without a count-action programming abstraction is like trying to steer a Lamborghini without knowing how to drive,” said professor Manya Ghobadi. “You probably have a driving manual in one hand, then press the clutch, then check the manual, then let go of the brake, then check the manual, and so on. This is a stop-and-go operation because, for every decision, you have to consult some higher-level entity to tell you what to do.

“But that’s not how we drive,” Ghobadi continued. “We learn how to drive and then use muscle memory without checking the manual or driving rules behind the wheel. Our count-action programming abstraction acts as the muscle memory in Lightning. It seamlessly drives the electrons and photons in the system at runtime.”

The photons used by Lightning move faster and generate less heat than electrons, allowing Lightning to operate at a faster frequency and more efficiently than nonhybrid computers.

To measure the system’s energy efficiency, the researchers compared it to standard graphics processing units, data processing units, smartNICs, and other accelerators by synthesizing a Lightning chip. They found that Lightning was more efficient than the other accelerators when completing inference requests.

“Our synthesis and simulation studies show that Lightning reduces machine learning inference power consumption by orders of magnitude, compared to state-of-the-art accelerators,” researcher Mingran Yang said.

In all, the researchers evaluated the system’s performance using four platforms: a hybrid photonic-electronic prototype, an emulation environment, chip synthesis, and large-scale simulation. The Lightning prototype demonstrated the feasibility of performing 8-bit photonic, multiply-accumulate operations with 99.25% accuracy.

Machine learning services like ChatGPT and BERT require heavy computing resources. As a fast, cost-effective option for serving real-time, deep neural network inference requests, Lightning offers a potential upgrade for data centers that wish to reduce their machine learning model’s carbon footprint, while accelerating the inference response time for users.

The team presented its work at the Association for Computing Machinery’s Special Interest Group on Data Communication (SIGCOMM), held at Columbia University, New York, Sept. 10-14, 2023. The research paper is available at (https://lightning.mit.edu/sites/default/files/Lightning_SIGCOMM_2023_final_version.pdf).

Published: October 2023
Glossary
network interface card
A network interface card (NIC), also known as a network adapter or LAN adapter, is a hardware component that allows computers to connect to a local area network (LAN) and communicate with other devices on the network. The NIC serves as the interface between the computer's internal bus (such as PCI or PCIe) and the network cable, facilitating the transmission and reception of data over the network. Some features and functions of a network interface card include: Physical connection: NICs...
artificial intelligence
The ability of a machine to perform certain complex functions normally associated with human intelligence, such as judgment, pattern recognition, understanding, learning, planning, and problem solving.
machine learning
Machine learning (ML) is a subset of artificial intelligence (AI) that focuses on the development of algorithms and statistical models that enable computers to improve their performance on a specific task through experience or training. Instead of being explicitly programmed to perform a task, a machine learning system learns from data and examples. The primary goal of machine learning is to develop models that can generalize patterns from data and make predictions or decisions without being...
integrated photonics
Integrated photonics is a field of study and technology that involves the integration of optical components, such as lasers, modulators, detectors, and waveguides, on a single chip or substrate. The goal of integrated photonics is to miniaturize and consolidate optical elements in a manner similar to the integration of electronic components on a microchip in traditional integrated circuits. Key aspects of integrated photonics include: Miniaturization: Integrated photonics aims to...
optoelectronics
Optoelectronics is a branch of electronics that focuses on the study and application of devices and systems that use light and its interactions with different materials. The term "optoelectronics" is a combination of "optics" and "electronics," reflecting the interdisciplinary nature of this field. Optoelectronic devices convert electrical signals into optical signals or vice versa, making them crucial in various technologies. Some key components and applications of optoelectronics include: ...
machine vision
Machine vision, also known as computer vision or computer sight, refers to the technology that enables machines, typically computers, to interpret and understand visual information from the world, much like the human visual system. It involves the development and application of algorithms and systems that allow machines to acquire, process, analyze, and make decisions based on visual data. Key aspects of machine vision include: Image acquisition: Machine vision systems use various...
Research & TechnologyeducationAmericasMITMassachusetts Institute of TechnologyMIT CSAILnetwork interface cardartificial intelligencemachine learningintegrated photonicsOpticsoptoelectronicsneural networksSoftwaremachine visionmachine learning inferenceChat GPTprogramming abstractionphotonic computingTechnology News

We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.