Brendan Gallagher, a biological sciences graduate student at Carnegie Mellon...
Brendan Gallagher, a biological sciences graduate student at Carnegie Mellon University, uses virtual reality to examine 3D biological data gathered using expansion microscopy.
Source: Carnegie Mellon University

Microscopy and VR illuminate new ways to treat disease

The development of these technologies, a two-step process funded at $200,000 through Grand Challenges, an initiative of the Bill & Melinda Gates Foundation, will accelerate researchers’ understanding of infectious and autoimmune diseases and enhance their ability to develop disease diagnostics and prevention and treatment methods.

Yongxin (Leon) Zhao, an assistant professor of biological sciences at Carnegie Mellon’s Mellon College of Science, has been developing the expansion microscopy technique to physically magnify a biopsy, allowing researchers to see fine details in biological samples using standard microscopes.

Zhao makes biopsy samples grow in size by chemically transforming them into water-soluble hydrogels. He then applies a treatment that loosens the tissues and allows them to expand more than 100 times in volume. The tissues and molecules within the sample can then be labeled, imaged and compiled into a complex set of data, to be used to study interactions among cells and their structures.

However, a limitation of the technology is that it extracts two to three orders of magnitudes more data than current techniques are able to interpret. To help solve that problem, the Gates Foundation grant pairs expansion microscopy with a virtual reality technique developed at the Benaroya Research Institute at Virginia Mason (BRI).

Through VR technology developed specifically for this purpose, researchers will be able to see and manipulate the originally 2D expansion microscopy images in 3D, giving them a 360 degree view of tissue and protein organizations and interactions. “At BRI, we’ll prepare the live infectious and autoimmune disease samples,” said Caroline Stefani, senior postdoctoral research associate. “We’ll send those to Carnegie Mellon, where they will enlarge the samples and send images back to BRI to be viewed in VR.”

“This is the future of how scientists can handle complex data,” Zhao said. “It’s an immersive experience, just like you are sitting inside your data. You have the freedom to explore your data from every angle and every spot.”

The virtual reality technology was developed by Tom Skillman, BRI’s former director of research technology, who has since founded a VR company, Immersive Science. “My role in this grant is to develop a software tool that will allow scientists studying disease a way to understand large amounts of data through a computational technique called ‘immersive science,’” Skillman said. “Bringing all that data into VR not only allows the scientist to see their 2D microscope images in full 3D, but to interact with the data, selecting channels, adjusting the views, colors and contrast, and grabbing and rotating the images to quickly identify key aspects of the image that are coupled back to the disease under study.”

The eventual goal is for the VR tool, called ExMicroVR™, to be shared on open platforms with other researchers along with expansion microscopy so that they too can view new details of disease processes and understand larger, more complex sets of data.

The system to convert expansion microscopy data into VR 3D images will be affordable and easily accessible to researchers and physicians in developing countries. It will also allow for up to six people to collaborate and view the same sample remotely at the same time.

Subscribe to our newsletter

Related articles

Hydrogel contact lenses for therapeutic use

Hydrogel contact lenses for therapeutic use

Researchers at the Terasaki Institute have developed prototypes of contact lenses that can assist with tear sampling for diagnostic purposes.

Towards an AI diagnosis like the doctor's

Towards an AI diagnosis like the doctor's

Researchers show how they can make an AI show how it's working, as well as let it diagnose more like a doctor, thus making AI-systems more relevant to clinical practice.

Combination of AI and radiologists accurately identified breast cancer

Combination of AI and radiologists accurately identified breast cancer

An AI tool identified breast cancer with approximately 90 percent accuracy when combined with analysis by radiologists.

Neurodegenerative diseases: Nanodiamonds in the brain

Neurodegenerative diseases: Nanodiamonds in the brain

Scientists have now produced tiny diamonds, so-called "nanodiamonds", which could serve as a platform for both the therapy and diagnosis of brain diseases.

Ingestible pill monitors the stomach for up to a month

Ingestible pill monitors the stomach for up to a month

Engineers have designed an ingestible pill that quickly swells to the size of a soft, squishy ping-pong ball big enough to stay in the stomach for an extended period of time.

Virtual reality enhances physicians’ treatment planning

Virtual reality enhances physicians’ treatment planning

VR brings medical images to life on screen, showing interventional radiologists a patient’s unique internal anatomy to help physicians effectively prepare and tailor their approach to complex treatments.

Olfactory VR show promise for mental health practices

Olfactory VR show promise for mental health practices

New research on Olfactory Virtual Reality (OVR) paints a clearer picture for clinical psychiatrists about how it could be used to safely and effectively help mental health and mood disorders.

AI enhances efficacy of sleep disorder treatments

AI enhances efficacy of sleep disorder treatments

Based on 20,000 nights of sleep, researchers have developed an algorithm that can improve the diagnosis, treatment and overall understanding of sleep disorders.

COVID-19: AIs shortcuts lead to misdiagnosis

COVID-19: AIs shortcuts lead to misdiagnosis

Researchers discovered that AI models have a tendency to look for shortcuts. In the case of AI-assisted disease detection, these shortcuts could lead to diagnostic errors if deployed in clinical settings.

Popular articles

Subscribe to Newsletter