Augmented reality helps surgeons to ‘see through’ tissue

A series of procedures carried out by surgeons at St Mary’s Hospital and researchers at Imperial College London have shown for the first time how surgeons can use computer-generated augmented reality imaging while operating on patients undergoing reconstructive lower limb surgery.

Photo
CGI is helping surgeons to ‘see through’ tissue during reconstructive surgery at St Mary’s

The Microsoft HoloLens headset is a self-contained computer headset that enables the wearer to interact with ‘holograms’ – computer-generated objects made visible through the headset. The Imperial team used the technology to overlay images of CT scans – including the position of bones and key blood vessels – onto each patient’s leg, in effect enabling the surgeon to ‘see through’ the limb during surgery.

According to the team trialling the technology, the approach can help surgeons locate and reconnect key blood vessels during reconstructive surgery, which could improve outcomes for patients.

Following a car accident or severe trauma, patients may have tissue damage or open wounds that require reconstructive surgery. This is where tissue, which is taken from elsewhere on the body and includes the skin and blood vessels, is used to cover the wound and enable it to close and heal properly. A vital step in the process is connecting the blood vessels of the ‘new’ tissue with those at the site of the wound, so oxygenated blood can reach the new tissue and keep it alive.

Utilising a team composed of Mr Jon Simmons, a plastic and reconstructive surgeon and Dr Dimitri Amiras a consultant radiologist, both from Imperial College Healthcare NHS Trust and Dr Philip Pratt, a Research Fellow in the department of surgery and cancer, several procedures using the HoloLens headset and augmented reality models were carried out. The cases ranged from a 41-year-old man who had sustained leg injuries during a car accident, to an 85-year-old woman with a compound fractured ankle. The surgical teams reported the HoloLens to be a powerful tool in the theatre, with the approach being more reliable and less time-consuming than the ultrasound method of locating blood vessels.

Photo
The surgeon’s view.
Source: Philip Pratt, et al. Eur Radiol Exp, 2018’

In the procedures used to trial the technology, five patients requiring reconstructive surgery on their legs underwent CT scans to map the structure of the limb, including the position of bones and the location and course of blood vessels. Images from the scans were then segmented into bone, muscle, fatty tissue and blood vessels by Dr Amiras and loaded into intermediary software to create 3D models of the leg. 

Dr Amiras, said: “As St Mary’s Hospital is a major trauma centre, giving us the opportunity to try and improve the pre-operative planning for reconstructive tissue. After seeing some videos of the HoloLens I contacted Dr Pratt to see if we could use this technology to bring the information from the CT directly to the operating theatre. Over time, the scanning protocol has been optimised to give excellent images of the anatomy, however, at first we had to rely on rough measurements of anatomical landmarks taken from 3D CT reconstructions to guide surgery.”

Mr Simmons, said: “The application of AR technology in the operating theatre has some really exciting possibilities. It could help to simplify and improve the accuracy of some elements of reconstructive procedures. While the technology can’t replace the skill and experience of the clinical team, it could potentially help to reduce the time a patient spends under anaesthetic and reduce the margin for error. We hope that it will allow us to provide more tailored surgical solutions for individual patients.”

“We are one of the first groups in the world to use the HoloLens successfully in the operating theatre,” said Dr Philip Pratt, lead author of the study, published in European Radiology Experimental. “Through this initial series of patient cases we have shown that the technology works and that it can provide a benefit to the surgical team. With the HoloLens, you look at the leg and essentially see inside of it. You see the bones, the course of the blood vessels, and can identify exactly where the targets are located.”

Subscribe to our newsletter

Related articles

First AR minimally-invasive spine surgery

First AR minimally-invasive spine surgery

Dr. Frank Phillips, Professor and Director of the Division of Spine Surgery and the Section of Minimally Invasive Spine Surgery at Rush University Medical Center, completed the first augmented reality (AR) minimally invasive spine surgery.

Ocutrx unveiled 8K AR surgery system

Ocutrx unveiled 8K AR surgery system

Ocutrx Vision Technologies has released a new system that provides the most modern options for surgery visualization designed to make it easier for surgeons to perform procedures.

AR may assist cardiologists perform complex procedures

AR may assist cardiologists perform complex procedures

AR offers a new platform to help physicians better visualize complex medical data, particularly before and during medical procedures.

Holographic images increase neurosurgeons performance

Holographic images increase neurosurgeons performance

Researchers have developed a new technique of external ventricular drain insertion that involves the use of a mixed-reality holographic computer headset.

Augmented Reality in the OR: matching man and machine

Augmented Reality in the OR: matching man and machine

One of the crucial future technologies in surgery is Augmented Reality. Most experts agree that AR will increase safety and efficiency, improve surgical training and decrease costs.

Multimodal mixed reality visualisation for surgical guidance

Multimodal mixed reality visualisation for surgical guidance

Researchers have developed a MR visualisation platform which projects multiple imaging modalities to assist intraoperative surgical guidance.

Augmented reality brings ultrasound into a new dimension

Augmented reality brings ultrasound into a new dimension

Augmented reality ultrasound has, for the first time, made it possible to superimpose topographical representations of ultrasound images directly on a patient, with the examiner seeing the sectional image in AR glasses.

Augmedics launches AR guidance system for surgery

Augmedics launches AR guidance system for surgery

Groundbreaking AR system allows surgeons to visualize anatomy in real time – as if they have “x-ray vision” – and accurately guide their instruments and implants.

Promising applications of mixed realities in medicine

Promising applications of mixed realities in medicine

At ECR 2019, researchers talked about the practical applications of mixed realities in medical education and training as well as preprocedural planning and visualization during a surgery.

Popular articles