Single-Pixel Camera Streams in Real Time at 100 fps

Facebook X LinkedIn Email
The speed of most single-pixel imaging (SPI) systems is hampered by the refresh rates of digital micromirror devices (DMDs) and the time-consuming iterations of compressed-sensing (CS)-based reconstruction. The data acquisition rate of most SPI systems is limited to below the megahertz bandwidths of typical photodetectors, due to the kilohertz pattern refresh rates of DMDs.

Consequently, the 2D imaging speed in most SPI systems is ultimately restricted to under 100 frames per second (fps).

Researchers at the Énergie Matériaux Télécommunications Research Centre of the Institut national de la recherche scientifique (INRS, Quebec) have developed an SPI system that overcomes these limitations to speed to stream real-time video at 100 fps, and up to 12,000 fps offline. The researchers’ Single-Pixel Imaging Accelerated via Swept Aggregate Patterns (SPI-ASAP) system could be used to provide insights into industrial processes, as well as for applications that rely on self-luminescent events, such as optical property characterization of nanomaterials or the monitoring of neural activities.

Previous efforts to overcome the speed limitations of SPI, such as the use of fast-moving physical encoding masks for light modulation, suffer from low reconfigurability and reduced accuracy. The researchers addressed these challenges by linking a DMD with laser scanning hardware to achieve a highly flexible prototype camera. Laser scanning allowed the researchers to use individual encoding masks as optically selected subregions of larger aggregate patterns that are displayed via the DMD. Pattern aggregation enhanced SPI data acquisition rates by more than two orders of magnitude above the limitations of DMD-only modulation, while retaining a high pixel count and flexible pattern display.
SPI-ASAP operates in real-time and can be adapted for different scenes, frame resolutions, and frame rates. Courtesy of professor Jinyang Liang and researcher Patrick Kilcullen/Institut national de la recherche scientifique.
SPI-ASAP operates in real time and can be adapted for different scenes, frame resolutions, and frame rates. Courtesy of professor Jinyang Liang and researcher Patrick Kilcullen/Institut national de la recherche scientifique.
The researchers also developed a fast CS reconstruction algorithm that is tightly integrated with the pattern deployment architecture and fully compatible with parallel computing. As a result, SPI-ASAP would not be restricted by the iterative optimization approach used in conventional CS-based image reconstruction, according to the researchers.

SPI-ASAP showed the flexibility to operate at different spatial resolutions, different imaging speeds, and different modes. The researchers demonstrated the high-speed, real-time imaging capabilities of SPI by recording various dynamic scenes in both reflection and transmission modes across various frame sizes. In both the reflection and transmission modes, SPI-ASAP exhibited flexibility regarding frame size and frame rate, resilience to strong ambient light, and the capability to perform ultrahigh-speed imaging at up to 12 kfps.

Further, SPI-ASAP retains the practical advantages of DMDs, including flexible and reconfigurable pattern projection, while extending the rate of pattern deployment. It could have broad application, especially in the nonvisible spectrum — a range that, according to the researchers, currently lacks suitable imaging options.

With its ability to provide dynamic imaging under strong ambient light conditions, additional applications for SPI-ASAP could include the study of combustion phenomena, and SPI-ASAP combined with 3D profilometry could provide scene relighting on high-speed 3D objects. SPI-ASAP could also be used to induce patterned regions of high conductivity in semiconductor materials for high-speed 2D terahertz imaging.

The team has patented the technique used for SPI-ASAP and is seeking collaborators to work on commercialization of the camera.

The research was published in Nature Communications (

Published: January 2023
frames per second
The number of separate images exposed by a cine camera in a second or the number illuminated by a cine projector in a second. By varying the number of images recorded per second, films can be created that show fast or slow motion as well as normal motion when projected.
ImagingcamerasindustrialcomponentsResearch & TechnologyeducationAmericasINRSimage sensorsimage sensor2D imagingsingle-pixel imagingsingle-pixelsingle-pixel cameradigital micromirror devicesframes per secondTechnology NewsThe News Wire

We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.