Augmented reality brings ultrasound into a new dimension

Augmented reality ultrasound has, for the first time, made it possible to superimpose topographical representations of ultrasound images directly on a patient, with the examiner seeing the sectional image in AR glasses. The process, developed by Fraunhofer IGD, is intended to improve the success of ultrasound-guided biopsy and surgery.

Photo
The AR technologies allow ultrasound images to be viewed directly on the patient.
Source: Fraunhofer IGD

Researchers at the Fraunhofer Institute for Computer Graphics Research IGD have developed and process that combines medical ultrasound with augmented reality (AR). “AR ultrasound” will make it possible for the first time for doctors to view mapped ultrasound images with the help of AR glasses, allowing them to look directly at the patient and not have to spend the entire time looking at the ultrasound monitor.

The technology will improve the success rate of ultrasound-guided biopsies in the future. “The sectional planes through the body appear directly over the represented structures on the patient, allowing the doctor to see the needle in the correct position and adjust the puncture channel more easily,” explained Matthias Noll, deputy head of the Visual Healthcare Technologies Competence Center at Fraunhofer IGD. The safety of surgical procedures will also increase in that surgeons can use the new technology to plan the procedures in more detail. Soon, even physical anomalies or tissue changes will be directly visible in the AR image through techniques such as color-highlighting.

The topographical representation of ultrasound images is based on an external optical ultrasound probe/AR glasses tracking system. Fraunhofer IGD’s software analyzes their positions and uses various calibrations to compute the positions of both objects in relation to one another. The ultrasound image is then displayed in the examiner’s field of view on the AR glasses.

In the next three months, AR ultrasound will be tested in a hospital setting. “In order be able to tap into the ultrasound machine’s signal, we need an interface with the ultrasound software or we use the monitor cable,” said Noll. The only other requirements are a conventional ultrasound machine, the AR glasses, the tracking system and a laptop to make the AR calculations.

AR visualization of ultrasound images is the latest development from Fraunhofer IGD, a world leader in applied visual computing research. The institute has been developing new ultrasound applications since the turn of the millennium. Another of Fraunhofer IGD’s emphases is AR, a technology that has been increasing in importance in recent years. Most recently, developers have created a process that uses AR to help surgeons better and more quickly localize metastatic lymph nodes.

Subscribe to our newsletter

Related articles

Towards an AI diagnosis like the doctor's

Towards an AI diagnosis like the doctor's

Researchers show how they can make an AI show how it's working, as well as let it diagnose more like a doctor, thus making AI-systems more relevant to clinical practice.

First AR minimally-invasive spine surgery

First AR minimally-invasive spine surgery

Dr. Frank Phillips, Professor and Director of the Division of Spine Surgery and the Section of Minimally Invasive Spine Surgery at Rush University Medical Center, completed the first augmented reality (AR) minimally invasive spine surgery.

Promising applications of mixed realities in medicine

Promising applications of mixed realities in medicine

At ECR 2019, researchers talked about the practical applications of mixed realities in medical education and training as well as preprocedural planning and visualization during a surgery.

MURAB: Roboter-assisted diagnostic of breast cancer

MURAB: Roboter-assisted diagnostic of breast cancer

The Murab project is developing technology that will make it possible to take more accurate biopsies and diagnose cancer and other illnesses faster.

Holographic images increase neurosurgeons performance

Holographic images increase neurosurgeons performance

Researchers have developed a new technique of external ventricular drain insertion that involves the use of a mixed-reality holographic computer headset.

Augmented reality helps surgeons to ‘see through’ tissue

Augmented reality helps surgeons to ‘see through’ tissue

A series of procedures have shown how surgeons can use computer-generated augmented reality imaging while operating on patients undergoing reconstructive lower limb surgery.

Smart surgery - from technical fascination to clinical reality

Smart surgery - from technical fascination to clinical reality

Artificial intelligence is developing at an enormous speed and intelligent instruments will profoundly change surgery and medical interventions.

Tool with smart sensors advances cardiac surgery

Tool with smart sensors advances cardiac surgery

A new class of medical instruments equipped with an advanced soft electronics system could improve the diagnoses and treatments of a number of cardiac diseases and conditions.

Augmented Reality in the OR: matching man and machine

Augmented Reality in the OR: matching man and machine

One of the crucial future technologies in surgery is Augmented Reality. Most experts agree that AR will increase safety and efficiency, improve surgical training and decrease costs.

Popular articles