Augmented reality helps surgeons to ‘see through’ tissue

A series of procedures carried out by surgeons at St Mary’s Hospital and researchers at Imperial College London have shown for the first time how surgeons can use computer-generated augmented reality imaging while operating on patients undergoing reconstructive lower limb surgery.

Photo
CGI is helping surgeons to ‘see through’ tissue during reconstructive surgery at St Mary’s

The Microsoft HoloLens headset is a self-contained computer headset that enables the wearer to interact with ‘holograms’ – computer-generated objects made visible through the headset. The Imperial team used the technology to overlay images of CT scans – including the position of bones and key blood vessels – onto each patient’s leg, in effect enabling the surgeon to ‘see through’ the limb during surgery.

According to the team trialling the technology, the approach can help surgeons locate and reconnect key blood vessels during reconstructive surgery, which could improve outcomes for patients.

Following a car accident or severe trauma, patients may have tissue damage or open wounds that require reconstructive surgery. This is where tissue, which is taken from elsewhere on the body and includes the skin and blood vessels, is used to cover the wound and enable it to close and heal properly. A vital step in the process is connecting the blood vessels of the ‘new’ tissue with those at the site of the wound, so oxygenated blood can reach the new tissue and keep it alive.

Utilising a team composed of Mr Jon Simmons, a plastic and reconstructive surgeon and Dr Dimitri Amiras a consultant radiologist, both from Imperial College Healthcare NHS Trust and Dr Philip Pratt, a Research Fellow in the department of surgery and cancer, several procedures using the HoloLens headset and augmented reality models were carried out. The cases ranged from a 41-year-old man who had sustained leg injuries during a car accident, to an 85-year-old woman with a compound fractured ankle. The surgical teams reported the HoloLens to be a powerful tool in the theatre, with the approach being more reliable and less time-consuming than the ultrasound method of locating blood vessels.

Photo
The surgeon’s view.
Source: Philip Pratt, et al. Eur Radiol Exp, 2018’

In the procedures used to trial the technology, five patients requiring reconstructive surgery on their legs underwent CT scans to map the structure of the limb, including the position of bones and the location and course of blood vessels. Images from the scans were then segmented into bone, muscle, fatty tissue and blood vessels by Dr Amiras and loaded into intermediary software to create 3D models of the leg. 

Dr Amiras, said: “As St Mary’s Hospital is a major trauma centre, giving us the opportunity to try and improve the pre-operative planning for reconstructive tissue. After seeing some videos of the HoloLens I contacted Dr Pratt to see if we could use this technology to bring the information from the CT directly to the operating theatre. Over time, the scanning protocol has been optimised to give excellent images of the anatomy, however, at first we had to rely on rough measurements of anatomical landmarks taken from 3D CT reconstructions to guide surgery.”

Mr Simmons, said: “The application of AR technology in the operating theatre has some really exciting possibilities. It could help to simplify and improve the accuracy of some elements of reconstructive procedures. While the technology can’t replace the skill and experience of the clinical team, it could potentially help to reduce the time a patient spends under anaesthetic and reduce the margin for error. We hope that it will allow us to provide more tailored surgical solutions for individual patients.”

“We are one of the first groups in the world to use the HoloLens successfully in the operating theatre,” said Dr Philip Pratt, lead author of the study, published in European Radiology Experimental. “Through this initial series of patient cases we have shown that the technology works and that it can provide a benefit to the surgical team. With the HoloLens, you look at the leg and essentially see inside of it. You see the bones, the course of the blood vessels, and can identify exactly where the targets are located.”

Subscribe to our newsletter

Related articles

“XR is fusing surgical reality with medical images”

“XR is fusing surgical reality with medical images”

Egidijus Pelanis, a medical doctor at Oslo University Hospital, explains how extended realities is applied in the operating room.

First AR minimally-invasive spine surgery

First AR minimally-invasive spine surgery

Dr. Frank Phillips, Professor and Director of the Division of Spine Surgery and the Section of Minimally Invasive Spine Surgery at Rush University Medical Center, completed the first augmented reality (AR) minimally invasive spine surgery.

Ocutrx unveiled 8K AR surgery system

Ocutrx unveiled 8K AR surgery system

Ocutrx Vision Technologies has released a new system that provides the most modern options for surgery visualization designed to make it easier for surgeons to perform procedures.

AR may assist cardiologists perform complex procedures

AR may assist cardiologists perform complex procedures

AR offers a new platform to help physicians better visualize complex medical data, particularly before and during medical procedures.

Holographic images increase neurosurgeons performance

Holographic images increase neurosurgeons performance

Researchers have developed a new technique of external ventricular drain insertion that involves the use of a mixed-reality holographic computer headset.

Artificial intelligence betters holographic displays

Artificial intelligence betters holographic displays

Researchers are developing new techniques for improving 3D displays for virtual and augmented reality technologies.

Tactile holograms could help train surgeons

Tactile holograms could help train surgeons

Engineers have developed a new way to create the sensation of physically interacting with holographic projections.

Robotics expands in spinal surgery

Robotics expands in spinal surgery

Increasing numbers of hospitals and spine surgeons are adopting robotics and computer navigation, which aim to enhance precision, accuracy and predictability.

Mixed reality enhances brain surgery

Mixed reality enhances brain surgery

A research program explores the use of mixed reality for enabling surgeons to identify brain tumors quickly and precisely.

Popular articles

Subscribe to Newsletter