With the help of a Mixed Reality headset, a 3D hologram of a patient’s liver...
With the help of a Mixed Reality headset, a 3D hologram of a patient’s liver scan is projected into space, allowing Dr Gao Yujia, NUHS, to control and view the holographic image from different angles.
Source: National University Hospital

Mixed reality enhances brain surgery

The National University Health System in Singapore (NUHS) has started a research and development program to explore the use of mixed reality (MR) technology for, among other things, enabling surgeons to identify brain tumors quickly and precisely.

The research programme hopes to support the development of next-generation clinical applications and improve patient safety. This would augment clinical processes, and enhance both undergraduate and postgraduate education. While the use of holographic technology in operating theatres is still nascent, NUHS hopes to apply it in multiple fields of surgery.

"Holographic technology may radically transform the way we practice medicine. Early adoption will place NUHS at the forefront of medical mixed reality research and position us as a pioneer in the clinical use of this technology," said Associate Professor Ngiam Kee Yuan, NUHS's Group Chief Technology Officer, who is overseeing the research and development of the holomedicine programme in NUHS.

While holomedicine is new in the field of medicine, it has become increasingly prominent in the past year. "It leverages the concept of MR not only to augment our physical environment, but permits interaction with virtual objects superimposed onto the real world. The virtual objects can also be manipulated relative to the real world using natural hand gestures," explained Dr Gao Yujia, Associate Consultant with the Division of Hepatobiliary & Pancreatic Surgery, National University Hospital (NUH), and the programme lead for holomedicine at NUHS.

Potential of holographic technology in brain surgery

A team of neurosurgeons at NUH has initiated a study to assess the feasibility of using holographic technology to spatially locate brain tumours when operating on patients.

Using holographic visors, a three-dimensional (3D) hologram of a patient's brain scan is projected into space and superimposed onto the patient's head during surgery. This hologram is generated from the patient's own brain CT scan using a 3D medical software created by apoQlar, called Virtual Surgery Intelligence (VSI). When the hologram is set in place on a patient's head, surgeons are able to view the 3D holographic images of the brain from different angles.

They can also pull up information and alter the images in the visor by gesture and speech recognition, allowing them to interact with, and control the holographic image that is superimposed onto the patient.

Clinical Associate Professor Yeo Tseng Tsai, Head and Senior Consultant with the Division of Neurosurgery, NUH said, "With this holographic technology, you are able to see inside the brain. You will be able to see the blood vessels and most importantly identify the tumour quickly and precisely, as well as to know which angle and the exact location to make the incision. For over thirty years now, we have been using a handheld navigation system to navigate and identify the location of the tumour. In comparison, this new mixed reality system is more intuitive as we can now see inside the patient's head without the need to look up and refer to a computer screen while performing a procedure."

With the potential of higher precision and quicker localisation, patient care and safety can be improved. Furthermore, with the MR headset weighing only around 500 grams, it may one day replace existing bulky operation theatre equipment and even reduce the exposure to radiation in procedures such as spine operations, where X-rays are currently used to guide the insertion of metallic implants.

NUHS has completed the first phase of the research programme involving a proof of concept study, where various disciplines such as neurosurgery, plastics surgery and ophthalmology at NUH have evaluated the MR devices and brainstormed on future areas of development. The headsets were used for evaluation purposes to identify use cases, understanding of clinicians' acceptance level, system evaluation, as well as assessing its long-term deployment sustainability.

NUHS has developed a holistic holomedicine roadmap, including short- and long-term research projects, integration with existing hospital systems, and procedures to onboard users and enhance the hospital's infrastructure to support the system. Some potential use cases include streaming of live data from image acquisition machines, and the employment of artificial intelligence and machine learning for advanced image processing and predictive analytics. In addition, the HoloLens 2 device may also be used by patients to help them better understand the procedures they are to undergo. This is achieved by projecting a 3D rendering of a patient's scan to better illustrate the steps of their procedure.

Clinical validation studies and trials will be necessary prior to the adoption of this holomedicine solution as a primary clinical method. This involves comparing MR technology to current gold standards of clinical practice and outcomes measured, including data on accuracy, stability, and potential risks and limitations of this capability. Registration with the necessary governing bodies including the Health Science Authority (HSA) will also be needed before this solution can be used in a direct patient intervention role.

Collaboration with industry

The NUHS team, in collaboration with the Engineering Design & Innovation Centre under the National University of Singapore's (NUS) Faculty of Engineering, has also been awarded a grant of $100,000 from the Engineering in Medicine Grant under NUHS and NUS in March 2021. The grant will enable the team to embark on a project on real-time volumetric rendering and positioning of ultrasound scans.

"Holomedicine has the potential to sustainably revolutionise medical standards in most - if not all - medical fields. We are very excited to join forces with NUHS, a world-renowned academic health system, to achieve our mission of further improving patient outcomes and safety," said Mr Sirko Pelzl, CEO, apoQlar.

The software by apoQlar has successfully obtained HSA's Class A License and the Singapore Standard SS620 certification in April 2021. NUHS hopes to promulgate holographic technology to all its hospitals and institutes within the healthcare cluster in the near future.

"The advancements in technology, especially in the field of MR, has opened new horizons to the medical profession. By merging other technologies such as artificial intelligence, real-time image recognition, and predictive modelling, these MR devices offer clinicians capabilities that were once thought impossible," said A/Prof Ngiam Kee Yuan.

Subscribe to our newsletter

Related articles

“XR is fusing surgical reality with medical images”

“XR is fusing surgical reality with medical images”

Egidijus Pelanis, a medical doctor at Oslo University Hospital, explains how extended realities is applied in the operating room.

Multimodal mixed reality visualisation for surgical guidance

Multimodal mixed reality visualisation for surgical guidance

Researchers have developed a MR visualisation platform which projects multiple imaging modalities to assist intraoperative surgical guidance.

Holographic images increase neurosurgeons performance

Holographic images increase neurosurgeons performance

Researchers have developed a new technique of external ventricular drain insertion that involves the use of a mixed-reality holographic computer headset.

Tactile holograms could help train surgeons

Tactile holograms could help train surgeons

Engineers have developed a new way to create the sensation of physically interacting with holographic projections.

World-leading pioneers speak at Medical XR congress

World-leading pioneers speak at Medical XR congress

At Shift Medical, more than 60 leading medical XR experts will present and discuss the latest developments on the use of digital technologies such as virtual reality, augmented reality and mixed reality in medicine.

Augmented reality provides insights during liver surgery

Augmented reality provides insights during liver surgery

Researchers are developing 3D technologies for surgeons. The aids for surgery planning are particularly advanced.

4D-simulator breakthrough in brain surgery

4D-simulator breakthrough in brain surgery

A new training technology will improve patient safety during future procedures: a new 4D simulator enables the planning, testing and optimization of the procedure.

Creating an accessible surgery simulator

Creating an accessible surgery simulator

Researchers are developing a simulator that could be used to train both surgeons who are in the early stages of their career and those who are more experienced.

Augmented Reality in the OR: matching man and machine

Augmented Reality in the OR: matching man and machine

One of the crucial future technologies in surgery is Augmented Reality. Most experts agree that AR will increase safety and efficiency, improve surgical training and decrease costs.

Popular articles

Subscribe to Newsletter