Photonics HandbookTechnology News

Multispectral Terahertz Imaging System Incorporates Neural Network

Facebook X LinkedIn Email
Researchers at UCLA have developed a focal plane array (FPA) capable of supporting real-time multispectral terahertz (THz) video in 3D. According to the team, led by UCLA professors Mona Jarrahi and Aydogan Ozcan, this is the first time a terahertz imaging system has been able to achieve real-time multispectral imaging with video capability while maintaining a high signal-to-noise ratio (SNR).

The terahertz spectrum is desirable due to its ability to penetrate opaque materials and provide unique spectral information on various chemicals, but practical implementation of this spectral region has been limited due to the slow speeds, high costs, and general complexity of current imaging systems. The problem, the researchers said, is the lack of suitable FPA detectors — components that contain radiation detectors used by the imaging system.

Current single pixel systems use raster scanning, which builds images pixel by pixel. Previous attempts to bypass raster scanning have employed nonlinear crystals, which also require bulky complicated setups and provide very low SNR levels. Another method used a time-varying spatial modulator to enable image reconstruction using acquired spatial information, but the technique proved to be slow, bulky, and costly.
A focal-plane array developed at UCLA captures multispectral terahertz images in real time, aided by a trained neural network to enhance the resolution. The system is video-capable and is being commercialized by UCLA spinoff Lookin Inc. Courtesy of Terahertz Electronics Laboratory/UCLA.
A focal-plane array (FPA) developed at UCLA captures multispectral terahertz (THz) images in real time, aided by a trained neural network to enhance the resolution. The system is video-capable and is being commercialized by UCLA spinoff Lookin Inc. Courtesy of Terahertz Electronics Laboratory/UCLA.

In contrast, the THz-FPA developed by the UCLA team bypasses the need for raster scanning by directly providing spatial amplitude and phase distributions, as well as an object’s temporal and spectral data. Additionally, the resolution of the captured images is enhanced in real time by a machine learning-trained neural network.

The THz-FPA consists of a 2D array of 283,500 plasmonic nanoantennas, engineered to detect broadband terahertz radiation with a high SNR when used in a terahertz time-domain spectroscopy (THz-TDS) system. To simplify data readout from the THz-FPA, the plasmonic nanoantennas are grouped into 7 × 9 clusters, and the collective response of all the nanoantenna clusters is electronically captured at each temporal point to simultaneously resolve their time-domain response.

The system extracts the amplitude and phase responses of the THz-FPA outputs from the time-domain data over a 3-THz bandwidth. It captures the THz-FPA outputs at each temporal point in 164 μs using an electronic readout, enabling time-domain THz video capture at 16 fps.

The researchers demonstrated the multispectral nature of the data captured by the plasmonic nanoantennas by using it to image different objects, including super-resolved etched patterns in a silicon substrate and defects in battery electrodes. By eliminating the need for raster scanning and spatial terahertz modulation, the THz-FPA provides a more than 1000-fold increase in imaging speed compared with current state-of-the-art terahertz imagers.

The team used a deep learning-trained convolutional neural network to enhance the resolution of the captured images in real time and achieve pixel superresolution (PSR). The PSR-enhanced THz-FPA increases the spatial resolution and the effective number of pixels in the final image reconstructions, compared with the physical pixel size of the FPA.

The deep learning-driven terahertz imaging system uses the amplitude and phase information of the spectral components detected in an imaged object to enhance the spatial resolution by four-fold in each lateral direction, increasing the space-bandwidth product of the FPA by 16-fold, with more than 1 kilopixel in each reconstructed image. In addition, the THz-FPA can quantify subwavelength material thickness variations as small as 10 µm, with a mean thickness accuracy of ~1 µm.

To demonstrate the significance of the multispectral operation of the THz-FPA, the team compared the performance of its PSR framework with multispectral input data against the use of single-frequency input data. The researchers trained six image reconstruction deep neural networks that used the amplitude and phase outputs of the THz-FPA at a single frequency of 0.50, 0.75, 1, 1.25, 1.60, and 2 THz. Although the single-frequency systems showed the images of the input objects to some extent, the reconstructed images exhibited severe artifacts and distortions compared with the much clearer multispectral PSR results of the THz-FPA.

The high-speed acquisition of spatial, ultrafast temporal, spectral, amplitude, and phase information presents a variety of implementation opportunities, such as reconstructing 3D images of multilayered objects and chemical identification. Additionally, the system could be integrated with diffractive optical networks for feature detection and object classification.

Potential applications include nondestructive biomedical imaging, security screening, quality control of pharmaceutical, industrial, and agricultural products, and cultural heritage conservation.

The technology is being commercialized by Lookin Inc., a startup spun off from Jarrahi’s research group.

The research was published in Nature Photonics (

Published: January 2024
deep learning
Deep learning is a subset of machine learning that involves the use of artificial neural networks to model and solve complex problems. The term "deep" in deep learning refers to the use of deep neural networks, which are neural networks with multiple layers (deep architectures). These networks, often called deep neural networks or deep neural architectures, have the ability to automatically learn hierarchical representations of data. Key concepts and components of deep learning include: ...
machine vision
Machine vision, also known as computer vision or computer sight, refers to the technology that enables machines, typically computers, to interpret and understand visual information from the world, much like the human visual system. It involves the development and application of algorithms and systems that allow machines to acquire, process, analyze, and make decisions based on visual data. Key aspects of machine vision include: Image acquisition: Machine vision systems use various...
Terahertz (THz) refers to a unit of frequency in the electromagnetic spectrum, denoting waves with frequencies between 0.1 and 10 terahertz. One terahertz is equivalent to one trillion hertz, or cycles per second. The terahertz frequency range falls between the microwave and infrared regions of the electromagnetic spectrum. Key points about terahertz include: Frequency range: The terahertz range spans from approximately 0.1 terahertz (100 gigahertz) to 10 terahertz. This corresponds to...
An SI prefix meaning one billionth (10-9). Nano can also be used to indicate the study of atoms, molecules and other structures and particles on the nanometer scale. Nano-optics (also referred to as nanophotonics), for example, is the study of how light and light-matter interactions behave on the nanometer scale. See nanophotonics.
Plasmonics is a field of science and technology that focuses on the interaction between electromagnetic radiation and free electrons in a metal or semiconductor at the nanoscale. Specifically, plasmonics deals with the collective oscillations of these free electrons, known as surface plasmons, which can confine and manipulate light on the nanometer scale. Surface plasmons are formed when incident photons couple with the conduction electrons at the interface between a metal or semiconductor...
focal plane array
A linear or two-dimensional matrix of individual detector elements, typically used at the focus of an imaging system.
signal-to-noise ratio
The ratio of the power in a desired signal to the undesirable noise present in the absence of a signal.
Referring to the bandwidth and spectrum location of the signal produced by television or radar scanning.
Research & TechnologyAmericasUCLAAydogan OzcancommercializationImagingLight SourcesOpticsSensors & Detectorsspectroscopydeep learningneural networksBiophotonicsmachine visionagricultureterahertzterahertz imagingfocal plane arraysnanoplasmonicsterahertz time-domain spectroscopyindustrialpharmaceuticalbiomedical imagingnondestructiveTHzreal-time imagingfocal plane arrayFPAsignal-to-noise ratiohigh SNR3DVideoLookin IncMona JarrahiTerahertz Electronics LaboratoryTechnology News

We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.