AR Visor Delivers Sci-Fi Capabilities

AUTUM C. PYLANT, NEWS EDITOR, autum.pylant@photonics.com

Think of them as safety glasses — with an emphasis on safety — for the surgeon and for the patient. A photonics device that surgeons wear during a procedure can superimpose patients’ x-rays over their bodies in perfect unison with their anatomy, and it can give the wearer instant access to a patient’s medical data. This device could reduce surgery times and improve the accuracy of needed surgical interventions.

The device merges two existing approaches to augmented reality headsets. It combines a video-see through (VST) system and an optical see-through (OST) system, ultimately creating the video optical see-through augmented reality surgical system, known as VOSTARS.

Pictorial Concept of VOSTARS. Courtesy of Photonics21.

Fabrizio Cutolo, engineer and expert in wearable augmented reality systems, believes neither VST nor OST alone is suitable for operating on a live patient.

“For something as critical as an operation, we had to have the naturalistic feel of OST, while having the fluid interaction of the VST,” said Cutolo.

The resulting hybrid medical visor is a head-mounted display system that conveniently projects a patient’s anesthetic data, heart rate, body temperature, blood pressure and breathing rates into the surgeon’s field of vision. The hope is that the surgeon can focus on the operation without having to look away for critical information.

The device uses a small, high-luminous microdisplay, LED optical waveguide and a micro-projector array to project the x-ray image in front of the user. Devices like this, once only depicted in science fiction flicks, could soon become mainstream.

In the same way that a facing camera on a smartphone films moving images, the VOSTARS system works by capturing what the surgeon sees from a head-mounted camera. The system then merges this footage of reality with the patient’s medical images from x-rays, CT, MRI or 3DUS scans.

The central processor, using the most advanced registration techniques available for surgical navigation, then presents a real-time hybrid image on the visor dashboard to the surgeon.

“The AR view will be shown when the surgeon requires it,” project coordinator Vincenzo Ferrari told Photonics Media. “Furthermore, it will show only the information required for the particular surgical task. The AR view should be useful, to improve surgery, not as a high-tech fashion gadget.”

Ferrari, who is a biomedical engineering researcher at the Department of Information Engineering at the University of Pisa, said the VOSTARS system can be activated with gestures, voice commands and pedals. He described it as a navigation system of sorts in the operating room.

“Imagine driving with a sat-nav; we know how to drive and roughly where to go, but with real-time information, like speed, distance and time presented to us, we can take the most efficient route in the quickest time and perhaps more safely,” he said.

VOSTARS aims to have a working prototype of the hybrid device ready in May 2018. Initial trials will include a number of procedures to the head, including maxillofacial surgery, neurosurgery, ENT and orthopedic surgery. The surgeon-driven navigation system could be available to end users in three years.