Multimodal mixed reality visualisation for surgical guidance

Researchers at Imperial College London have developed a MR visualisation platform which projects multiple imaging modalities to assist intraoperative surgical guidance.

Photo
Source: Imperial College London

A Hamlyn-led project that introduced a novel Mixed reality (MR) visualisation platform for intra-operative surgical guidance has won the IPCAI 2020 Audience Award. MR is emerging as a vital tool in surgery as it enables the surgeon to visualise subsurface anatomical structures in 3D. In the last decade, there has been a great effort to bring MR into the operating room to assist surgeons intra-operatively.

The virtual objects, showing a patient’s data, can be consulted by a surgeon for decision-making and can also be moved around the operating room using mid-air hand gestures. Therefore, MR allows a surgeon to consult data when and where needed, making it a valuable tool for surgery.

Regardless of numerous research efforts contributed to MR technology development for clinical applications, the progress of this type of development is still at an early stage.

Collaborated with Department of Surgery and Cancer, the researchers at the Hamlyn Centre proposed a MR visualisation platform for the Microsoft HoloLens, aiming to assist surgeons during the surgery in the operating room. The platform contains three visualisation components, namely a 3D organ model, volumetric data, and tissue morphology captured with intra-operative imaging modalities. Furthermore, a set of novel interactive functionalities have been designed including scrolling through volumetric data and adjustment of the virtual objects’ transparency.

The research team invited a group of surgeons to participate in this project. The participants were allowed to interact with the visualisation components and test the different functionalities. The majority of the surgeons found the proposed visualisation platform intuitive and would use it in their operating rooms for intra-operative surgical guidance.

This pilot study verified the potential of the proposed visualisation platform and its usability in the operating room. The future work our research team plans to carry out is to focus on enhancing the platform by incorporating the surgeons’ suggestions and conducting extensive evaluation on a large group of surgeons.

The research was published in International Journal of Computer Assisted Radiology and Surgery.

Subscribe to our newsletter

Related articles

Ocutrx unveiled 8K AR surgery system

Ocutrx unveiled 8K AR surgery system

Ocutrx Vision Technologies has released a new system that provides the most modern options for surgery visualization designed to make it easier for surgeons to perform procedures.

AR may assist cardiologists perform complex procedures

AR may assist cardiologists perform complex procedures

AR offers a new platform to help physicians better visualize complex medical data, particularly before and during medical procedures.

Microsoft HoloLens cleared for medical use

Microsoft HoloLens cleared for medical use

The OpenSight® Augmented Reality System is the first AR medical solution for Microsoft HoloLens cleared by the FDA .

Holographic images increase neurosurgeons performance

Holographic images increase neurosurgeons performance

Researchers have developed a new technique of external ventricular drain insertion that involves the use of a mixed-reality holographic computer headset.

Augmented reality helps surgeons to ‘see through’ tissue

Augmented reality helps surgeons to ‘see through’ tissue

A series of procedures have shown how surgeons can use computer-generated augmented reality imaging while operating on patients undergoing reconstructive lower limb surgery.

Augmented Reality in the OR: matching man and machine

Augmented Reality in the OR: matching man and machine

One of the crucial future technologies in surgery is Augmented Reality. Most experts agree that AR will increase safety and efficiency, improve surgical training and decrease costs.

Mouth and throat cancer: Robotic surgery may improve outcomes

Mouth and throat cancer: Robotic surgery may improve outcomes

Robotic surgery for patients with early stage, oropharyngeal squamous cell cancer is associated with improved health outcomes, including better long-term survival.

Using holograms during cardiac procedure

Using holograms during cardiac procedure

Physicians have shown that using a holographic display improves physician accuracy when performing a procedure to treat irregular heartbeat.

First AR minimally-invasive spine surgery

First AR minimally-invasive spine surgery

Dr. Frank Phillips, Professor and Director of the Division of Spine Surgery and the Section of Minimally Invasive Spine Surgery at Rush University Medical Center, completed the first augmented reality (AR) minimally invasive spine surgery.

Popular articles