Advancements in biomedical 3D imaging promise to improve research findings and clinical outcomes, thereby producing widespread benefits. In research, a combination of techniques will enable high-speed visual 3D imaging effectively below the diffraction limit, allowing scientists to better track what goes on in the brain or to examine other tissues and organs. The results of these advancements could be discoveries about how the brain works and how diseases progress. Fruit fly brain, color-coded according to 3D depth. Courtesy of Boyden Group/MIT. For clinical applications, the widening use of 3D imaging could lead to faster and more precise minimally invasive surgery. What’s more, continuing developments could lead to systems for use in surgery that offer improved imaging, with 3D 4K resolution that is better than high definition. But there are obstacles, such as how to make imaging systems smaller and how best to image inside the body. These advancements are important to laparoscopic surgery, a minimally invasive procedure that has been around since the early 1990s with surgeons using 2D imaging and displays. Over the years, both imaging and displays have gotten better, offering angled views and 4K ultrahigh-definition technology. “So, better clarity and quality, but you never had depth perception,” said Dr. Kent Bowden, a surgeon at Munson Healthcare in Cadillac, Mich. But the restrictions of flat-picture imaging changed with the introduction of robotic laparoscopic systems, which offered 3D imaging and therefore depth perception. A nerve and artery may appear side by side when viewed using a 2D camera and display, Bowden said. Yet, with 3D imaging, a surgeon could see, say, a 4- to 8-mm gap between the two. The ability to see that gap could speed up surgery and make it more precise. Robotic laparoscopic surgery systems are not cheap, however, with a cost reported to be in the millions and perhaps as much as 10× that of a standard manual setup. Fortunately, other biomedical 3D imaging approaches are significantly less expensive, such as a 3D endoscope from KARL STORZ of Tuttlingen, Germany. A relatively new stereoscopic 3D imaging system intended for laparoscopy is also available from Center Valley, Pa.-based Olympus America Inc. Achieving 3D imaging inside the body presents challenges, said Brian MacDonald, a product manager with Olympus. For laparoscopy, the system must fit through small ports, eliminating the use of some 3D imaging methods. In the Olympus solution, they opted for a design with two 5-mm laparoscopes mounted on the end of an articulating tip, along with a xenon light source. The latter provides a bright white light, creating an image with contrast and color similar to what surgeons would see during traditional open surgery, in which access to the operation site is gained through one or more large incisions. When processing the images received from the sensor chips, the system collects, mixes, and displays them on a monitor. Polarized 3D glasses passively filter the output from the monitor into right and left views. 3D imaging during minimally invasive surgery. An instrument with two sensors creates a stereoscopic image, which the surgical team sees in 3D by looking at a monitor through polarized glasses. This 3D imaging capability may speed up operations and improve patient outcomes. Courtesy of Olympus America. “This design emulates natural stereoscopic vision, where each human eye provides slightly different images to the human brain for interpretation,” MacDonald said. Resolution, contrast, brightness, sharpness, and color reproduction are all important, he added. The next step in the 3D imaging technology would be 4K ultra- high definition, which would quadruple the number of pixels over the current product. However, the hardware to do such imaging stereoscopically in 3D is still too large for laparoscopy. MacDonald said Olympus has partnered with Sony, a leader in the technology used in the sensor chips in current laparoscopes, to develop the necessary imaging products. He said there are already microscopes available for traditional open surgery that offer this 3D 4K imaging capability. Like products for laparoscopy, these systems magnify images, enhancing anatomic visibility and detail. Regarding the future of 3D imaging in a clinical setting, Munson Healthcare’s Bowden said that today any hospital with an operating room has video capability. Those without robotic systems are often confined to 2D imaging. Bowden’s experience has been that operations performed using 2D imaging take longer than those done with 3D imaging. This results, in part, from surgeons needing more time to process what they see on the screen and translate it into the actual layout of nerves and arteries inside the body. Given this delay and the outcomes Bowden has seen with his patients, he predicted that the days of 2D-only imaging in biomedical settings will come to an end within a few years. “Within the next five years, I would think every OR would or should be 3D capable because it makes that much difference in the visibility and the efficiency of the surgeon and the safety of the patient,” he said. In addition to advancements in 3D imaging in clinical settings, innovations are also affecting research. A team that includes researchers from MIT and the Howard Hughes Medical Institute’s (HHMI’s) Janelia Research Campus has demonstrated a new way to image the brain much more quickly and at higher resolution than possible before. In just days, the technique can achieve resolutions that may have taken many months using electron microscopy. Dual optics and sensor chips in a minimally invasive surgery instrument for 3D imaging. Processing images into left and right views enables depth perception. Courtesy of Olympus America. In a January Science paper1, the group outlined how they accomplished this improvement through a combination of two techniques: lattice light sheet microscopy and tissue expansion. The first is a 3D imaging method developed by HHMI scientist Eric Betzig that illuminates a sample from the side using a structured light sheet. Detectors then capture high-resolution images of the entire section of tissue through which the light sheet passes, with the achievable imaging depth set by the working distance of the objectives. This approach allows the sample to be imaged more rapidly and at a lower light exposure than with confocal microscopy, which shines a light from above and achieves 3D imaging by moving the focal point through the sample. Importantly, lowering the light level reduces photobleaching, which dims the fluorescence often used to tag biomolecules being studied. Less light also means less phototoxicity, the detrimental effect on living cells that receive too much illumination. Tissue expansion, the second technique, originated with professor Ed Boyden’s group at MIT. It involves embedding a tissue sample, which has specific proteins labeled by fluorescent antibodies, in a hydrogel. Researchers then break down the proteins and their connections that bind the tissue together and make the gel swell by bathing it in water. They end up with an enlarged version of the original tissue, and the fluorophores present throughout this expanded replica can be imaged using optical microscopes. Mouse cortex neurons expressing a fluorescent protein that fills the cells with a blue color to show their shape. Such 3D imaging could help researchers understand how the brain works and diseases progress. Courtesy of Boyden Group/MIT. A single instance of expansion results in a four-fold physical magnification, or an effective resolution of 60 nm. This is far below the several hundred nanometers set by the traditional diffraction limit. In the Science study, the researchers used one round of expansion. “In principle, we can keep repeating this process and achieve an even better effective resolution,” said Ruixuan Gao, an MIT postdoctoral researcher and co-lead author of the Science paper. “In reality, however, we have practical limitations in, for example, labeling density, physical size of the labels, and other limitations of microscopy.” The best result achieved to date is two rounds of expansion, which translates to 25 nm-resolution imaging in the original sample. Research is underway, however, that investigates the means to overcome the limitations and achieve an even smaller effective resolution. Tracing neural circuitry requires imaging at high resolution over long distances, and this cannot be achieved quickly using conventional microscopes, according to Shoh Asano, another of the paper’s co-lead authors. Now a senior scientist at Pfizer Inc., Asano was an MIT postdoctoral researcher when the work for the paper was done.?While traditional microscopes struggle, light sheet microscopy is well suited to such high-resolution, rapid 3D imaging, and it minimizes photobleaching. What’s more, the researchers found that tissue expansion removed optical aberrations and scattering present in the original sample, thereby increasing the effective resolution of the novel microscopy technique. “The two methods perfectly complemented each other,” Asano said. The combination of techniques can be performed significantly faster than electron microscopy for some research studies, he added. The new method is also compatible with genetic and multicolor labeling of biomolecules, a task that is difficult with electron microscopy, and a capability that could be important in uncovering neuronal connections. An entire fruit fly brain, with neurons color-coded according to its various brain domains. A new approach significantly speeds up this type of 3D imaging. Courtesy of Boyden Group/MIT. Imaging whole tissue samples, such as an entire insect or fish brain, produces a lot of data. For their paper, the researchers had to develop 3D stitching software that could merge thousands of image tiles together, with the individual tiles being analyzed and processed on a workstation. In the future, advancements in cloud computing could make it possible to rent computing resources as needed. One application of the combined technique approach would be high-throughput imaging of brain tissue over, say, a good portion of a rodent or human brain. This creates the possibility of tracing whole-brain wiring diagrams, or connectomes, for various species. “Importantly, the scalability of our method could enable cross-brain comparison of such connectomes, and the analysis of disease states, revealing how brain circuits change in Alzheimer’s, schizophrenia, and other conditions,” Asano said. Reference 1. R. Gao et al. (Jan. 18, 2019). Cortical column and whole-brain imaging with molecular contrast and nanoscale resolution. Science, Vol. 363, Issue 6424, www.doi.org/10.1126/science.aau8302.