In its drive to explore outer space, NASA is developing optical technology that tests the limits of the laws of physics and promises to turn neurosurgeons into pioneers in innerspace. While minimally invasive neurosurgery is far from being the final frontier in medical science, many neurosurgeons who perform this delicate procedure have been handicapped by technology when exploring the inner realms of the human head. In the course of attempting to locate and remove a tumor via a 15-mm incision — rather than through a partial skull removal — these surgeons have mostly seen a world that was flat. Behind these 2D imaging limitations are single-lens endoscopes. Usually, two lenses are needed to afford surgeons the stereoscopic vision that provides depth perception. And while there are plenty of stereo endoscopes on the market, the need for two lenses made many of the devices too bulky for minimally invasive neurosurgery1. Figure 1. NASA’s Jet Propulsion Laboratory (JPL) has developed the Multi-Angle Rear Viewing Endoscopic Tool (MARVEL) with a diameter of 4 mm, making it ideal for minimally invasive neurosurgery. On the screen in the background is a 3D image taken by MARVEL of a walnut’s interior. Courtesy NASA/JPL-Caltech/Skull Base Institute. However, NASA’s Jet Propulsion Laboratory (JPL) in Pasadena, Calif., in collaboration with the Skull Base Institute in Los Angeles, has found a way to achieve stereoscopic imaging with a miniature single lens endoscope. Its name: the Multi-Angle Rear Viewing Endoscopic Tool (MARVEL), and it highlights just one of the areas — size — in which manufacturers are pushing the limits for the next generation of endoscopes. Other areas include IR fluorescence resolution and 4K resolution. MARVEL JPL’s next generation 3D HD stereoscope has a diameter of 4 mm and a rotating tip that provides panoramic views spanning 120° (Figure 1). MARVEL is based on a single lens, dual aperture system. The device allows light to alternately penetrate each aperture at a rate of 30 frames per second. Three-dimensional imaging is then achieved through the overlapping of images in accordance to a time-multiplex scheme, according to Skull Base Institute Director Dr. Hrayr Shahinian (Figure 2). Figure 2. MARVEL is based on a single lens, dual aperture system. The device allows light to alternately (a and b) penetrate each aperture at a rate of 30 frames per second. Three-dimensional imaging is then achieved through the overlapping of images in accordance to a time-multiplex scheme. Courtesy of NASA/JPL-Caltech/Skull Base Institute. “In order to perform surgery in a 3D platform within the brain cavity or the sinus cavities or minor joints, you need a device that is smaller than maximum diameter of 4 mm. The dual lens systems are unable to miniaturize to that level based on the basics of optical physics; therefore, MARVEL does open the door to a whole series of surgical procedures that were unsafe and unavailable to us previously,” said Shahinian. MARVEL highlights just how far the field of endoscopy has come since the German physician, Dr. Philipp Bozzini, in 1804 presented his Lichtleiter, or “light conductor,” which was designed to view the larynx2. MARVEL, which the Skull Base Institute licenses from JPL’s manager, the California Institute of Technology, is still years away from commercialization, though Shahinian said manufacturing partnerships have already been forged. NASA is also eyeing using MARVEL on robots exploring other worlds to obtain 3D images of geological features. “MARVEL is on the cusp or the margins of what is feasible based on the laws of physics. We believe that any future miniaturization will come at the cost of loss of resolution. It will be possible to make smaller 3D devices using this technology, but the image quality will be sub-HD,” said Shahinian. Stereopsis Stereopsis is the ability to view an object from two slightly different angles, resulting in depth perception. As MARVEL illustrates, stereo or binocular vision can be achieved with a single optical channel. Philadelphia-based Visionsense Corp. has also developed a 3D HD stereoscopic endoscope that relies on a single channel. Providing what the company calls a “bee’s eye view,” its VSiii System is modeled after an insect’s compound eye consisting of hundreds of thousands of visual units that look in different directions (Figure 3). Software then converts this visual data into left eye and right eye images3. Figure 3. Visionsense’s 3D HD endoscope is modeled after an insect’s compound eye and is able to achieve stereoscopic vision, despite relying on a single channel. In 2009, the Emory Pituitary Center at Emory University Hospital began using Visionsense’s 3D stereoscopic vision system to remove tumors from the pituitary, a pea-sized gland at the brain’s base. Five years later, Emory became the first medical center in the country to use Visionsense’s 3D HD stereoscopic system. Dr. Nelson Oyesiku, Emory Pituitary Center’s co-director, stressed that Visionsense’s technology provides the advantages of both surgical microscopes, such as stereoscopic vision, and endoscopes, such as more light and wider fields of view. Figure 4. Companies with 3D endoscopes and laparoscopes slightly larger than MARVEL include Olympus, which offers the ENDOEYE FLEX 3D with a 10-mm diameter (a), and Intuitive Surgical, which offers 8.5-mm and 12-mm endoscopes for the da Vinci Surgical System (b). Courtesy of Olympus Corp. and Intuitive Surgical Inc., respectively. Visionsense’s smallest endoscope is currently 4 mm in diameter, and the company has developed a 5 mm laparoscope prototype. CEO Alex Chanin said Visionsense’s endoscope could go below the 4-mm mark, but he questioned how strong demand would be for such a small device. Other companies have developed slightly larger 3D HD endoscopes and laparoscopes (Figures 4a and 4b). For example, Tokyo-based Olympus Corp.’s ENDOEYE FLEX 3D, a dual lens, articulating 3D HD laproscope, has a 10-mm diameter (Figure 5). Sunnyvale, Calif.-based Intuitive Surgical Inc. also offers 8.5-mm and 12-mm 3D stereo endoscopes for the da Vinci Surgical System, a robotic surgical system. Figure 5. An artist’s rendering of the 3D imaging capabilities of Olympus’ ENDOEYE FLEX 3D, which in 2013 became the world’s first 3D HD articulated system for laparoscopy. Courtesy of Olympus Corp. IR fluorescence imaging is another feature being added to 3D HD endoscopy systems to help visually assess blood flow and related tissue perfusion. In 2011 Intuitive began offering a Firefly Fluorescence Imaging platform for its da Vinci systems that provides real-time endoscopic visible and near-infrared fluorescence imaging4. Visionsense last year also rolled out an IR fluorescence imaging platform called Iridium. 4K As the instrumentation gets smaller, and as 3D continues to slowly make inroads in the endoscope market, manufacturers are already looking beyond HD. “With ongoing demand for smaller instruments, the technology for stereoscopic imaging has to follow, without loss of image quality,” said Dr. Shawn Tsuda, an associate professor of surgery at the University of Nevada School of Medicine and co-chair of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) Technology Committee. Highlighting this emphasis on image quality, this fiscal year Olympus plans to roll out a 2D Ultra-HD, 4K (3840 × 2160 pixels or more) surgical endoscopy system developed by Sony Olympus Medical Solutions Inc., the medical business joint venture between Olympus and Sony Corp. With the 4K images, surgeons will also see a range of colors that is broader than what they see with HD (Figure 6). That should help with distinguishing the edges and details of blood vessels, nerves and lymphatic vessels, according to Olympus’ announcement for the new system. The system, however, will be limited to 2D, consisting of only one optical channel, according to Keith Griffis, Olympus’ executive director of marketing for surgical endoscopy & systems integration. Figure 6. Endoscopic 4K imaging provides a broader color range than the full HD color range. Courtesy of Olympus Corp. “The 4K system is four times the resolution of HD. … Current technology limits the ability to achieve this [3D imaging], but as technology evolves this may be a possibility to explore,” Griffis said. Visionsense, however, plans to roll out a 3D 4K endoscope system in 2017, according to Chanin. “Endoscopy is going to 4K. That’s pretty clear,” Chanin said. The rollout of Visionsense’s 4K endoscope system will come 10 years after the company unveiled what it called “‘the first and only miniature 3D stereoscopic surgical camera.” During that time, only a minority of surgeons have adopted 3D endoscopes. Chanin estimates only 10 percent of the endoscope market has gone to 3D. The transition from 2D to 3D has been slow because most surgeons, aside from those using robotic systems, train with 2D endoscopes — something Chanin likens to “driving with one eye closed.” “It takes some time to move the market and to get them to open both eyes,” he said. References 1. M. Shearn et al. (March 2012) Multi-angle rear-viewing endoscopic tool (MARVEL) for minimally invasive neurosurgeries. J Med Devices, Vol. 6, Issue 1, p. 017540 2. D. Mirota and G. Hager. (2011) Vision-based navigation in image-guided interventions. Annu Rev Biomed Eng, Vol. 13, pp. 297-319. 3. Visionsense, A unique bee’s eye view. http://www.visionsense.com/our-technology/. 4. Intuitive Surgical. Intuitive Surgical Receives Firefly FDA Clearance for da Vinci Xi Surgical System. Press release, Aug. 21, 2014. 5. H. Marcus et al. (October 2014). Endoscopic and keyhole endoscope-assisted neurosurgical approaches: a qualitative survey on technical challenges and technological solutions. Br J Neurosurg, Vol. 28, pp. 606–610. 6. E. Kari et al. (2012). Comparison of traditional 2-dimensional endoscopic pituitary surgery with new 3-dimensional endoscopic technology: intraoperative and early postoperative factors. Int Forum Allergy Rhinol, Vol. 2, pp. 2-8. 7. T. Fujii et al. Superiority of 3D imaging in endoscopic surgery training. Society of American Gastrointestinal and Endoscopic Surgeons 2014 Abstracts. www.sages.org/meetings/annual-meeting/abstracts-archive/superiority-of-3d-imaging-in-endoscopic-surgery-training/.W 3D versus 2D endoscopy Three-dimensional endoscopy is already standard practice in some procedures, particularly those involving robotic surgical systems. However, 2D endoscopy has not lost its luster. For instance, in a recent survey of members of the Society of British Neurological Surgeons on technical challenges of endoscopic and keyhole endoscope-assisted neurosurgical approaches, only 49 percent identified a need for intra-operative visualization and improvements in neuroendoscopy. More specifically, 34 percent of these respondents identified a need for 3D endoscopy and 17 percent identified a need for HD endoscopy55. “It is still a question of whether any surgery ‘demands’ a stereoscopic view — there is data showing that surgeons may prefer it, or feel more comfortable with it, but more data is needed to show definitely if it makes surgery in the clinical realm — not just the laboratory — more safe and efficient,” said Dr. Shawn Tsuda, an associate professor of surgery at the University of Nevada School of Medicine and co-chair of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) Technology Committee. For abdominal and gastrointestinal surgery, Tsuda said 3D endoscopy is “not necessarily a must.” He noted, however, that “stereoscopic views definitely help with more complex operations,” such as cancer removals of the kidney, adrenal glands, pancreas, liver and prostate. Keith Griffis, Olympus’ executive director of marketing for surgical endoscopy & systems integration, said, 3D “is gaining adoption across laparoscopic specialties including bariatric, colorectal, thoracic and general surgery. Gynecology has taken specific interest and utilizes 3D to improve intercorporeal suturing, without the major expense incurred to utilize 3D in robotic surgery.” Studies vary as to whether 3D endoscopic technology is superior to 2D endoscopic technology. Much depends on the procedure. For example, in one 2012 study of 58 patients who underwent endoscopic pituitary surgery, with a 2D endoscopic system used on 32 patients and a 3D endoscopic system used on the rest, “3D endoscopy does not result in significantly different perioperative or postoperative outcomes versus 2D endoscopic surgery6.” Another study in 2014 found surgeons in training using 3D endoscopes were able to perform a suture and ligations significantly faster than similarly skilled surgeons using 2D endoscopes. There was no significant difference, however, between skilled surgeons using 3D and 2D endoscopes7. Alex Chanin, CEO of the 3D endoscope system maker Visionsense in Philadelphia, pointed out that many skilled surgeons have grown accustomed to operating with 2D vision, the same way a driver adapts to driving with one eye.