The DeepMReye software uses artificial intelligence to directly predict eye...
The DeepMReye software uses artificial intelligence to directly predict eye position and eye movements from MRI images.
Source: Pexels

Neural network predicts eye movements

Scientists at the Max Planck Society have developed a software that can be used in combination with MRI data for research and diagnosis.

A large amount of information constantly flows into our brain via the eyes. Scientists can measure the resulting brain activity using magnetic resonance imaging (MRI). The precise measurement of eye movements during an MRI scan can tell scientists a great deal about our thoughts, memories and current goals, but also about diseases of the brain. 

Researchers from the Max Planck Institute for Human Cognitive and Brain Sciences (MPI CBS) in Leipzig and the Kavli Institute for Systems Neuroscience in Trondheim have now developed software that uses artificial intelligence to directly predict eye position and eye movements from MRI images. The method opens up rapid and cost-effective research and diagnostic possibilities, for example, in neurological diseases that often manifest as changes in eye-movement patterns.

To record eye movements, research institutions typically use a so-called eye tracker—a sensor technology in which infrared light is projected onto the retina, reflected, and eventually measured. "Because an MRI has a very strong magnetic field, you can't use standard cameras in it. You need special MRI-compatible equipment, which is often not feasible for clinics and small laboratories due to the high costs and time-intensive use" says study author Matthias Nau, who developed the new alternative together with Markus Frey and Christian Doeller. The high cost of these cameras and the experimental effort involved in their use have so far prevented the widespread use of eye-tracking in MRI examinations. That could now change. The scientists from Leipzig and Trondheim recently presented their free and easy-to-use software "DeepMReye" in Nature Neuroscience.

With it, it is now possible to track participants' viewing behavior even without a camera during an MRI scan. "The neural network we use detects specific patterns in the MRI signal from the eyes. This allows us to predict where the person is looking. Artificial intelligence helps a lot here, because we often don't know exactly which patterns to look for as scientists" Markus Frey explains. He and his colleagues have trained the neural network with their own and publicly available data from study participants in such a way that it can now perform eye tracking even in data the software has not been trained on. This opens up many possibilities. For example, it is now possible to study the gaze behavior of participants and patients even in existing MRI data, which were originally acquired without eye-tracking. In this way, scientists could use older studies and data sets to answer entirely new questions.

Moreover, the software can also predict when eyes are open or closed and tracks eye movements even when the eyes remain closed. This may allow to perform eye tracking even when study participants are asleep. "I can imagine that the software will also be used in the clinical field, for example, in the sleep lab to study eye movements in different sleep stages," says Matthias Nau. In addition, for blind patients, the traditional eye-tracking cameras have rarely been used because an accurate calibration was very cumbersome. "Here too, studies can be carried out more easily with DeepMReye, as the artificial intelligence can be calibrated with the help of healthy subjects and then be applied in examinations of blind patients." The software could thus enable a variety of applications in research and clinical settings, perhaps even leading to eye-tracking finally becoming a standard in MRI studies and everyday clinical practice.

Subscribe to our newsletter

Related articles

Deep learning helps visualize X-ray data in 3D

Deep learning helps visualize X-ray data in 3D

Scientists have leveraged artificial intelligence to train computers to keep up with the massive amounts of X-ray data taken at the Advanced Photon Source.

AI measures fat around heart to predict diabetes

AI measures fat around heart to predict diabetes

Researchers have developed a new artificial intelligence tool that is able to automatically measure the amount of fat around the heart from MRI scan images.

COVID-19: AIs shortcuts lead to misdiagnosis

COVID-19: AIs shortcuts lead to misdiagnosis

Researchers discovered that AI models have a tendency to look for shortcuts. In the case of AI-assisted disease detection, these shortcuts could lead to diagnostic errors if deployed in clinical settings.

Biomedical research: deep learning outperforms machine learning

Biomedical research: deep learning outperforms machine learning

Deep-learning methods have the potential to offer substantially better results, generating superior representations for characterizing the human brain.

Deep learning-based image segmentation

Deep learning-based image segmentation

Scientists have presented a new method for configuring self-learning algorithms for a large number of different imaging datasets – without the need for specialist knowledge or very significant computing power.

Machine learning algorithm detects early stages of Alzheimer's

Machine learning algorithm detects early stages of Alzheimer's

An artificial intelligence-based detects early stages of Alzheimer’s through functional magnetic resonance imaging.

Deep learning platform accurately diagnoses dystonia

Deep learning platform accurately diagnoses dystonia

Researchers have developed a unique diagnostic tool that can detect dystonia from MRI scans, the first technology of its kind to provide an objective diagnosis of the disorder.

AI system to prevent imaging cyberthreats

AI system to prevent imaging cyberthreats

Researchers have developed an AI technique that will protect medical devices from malicious operating instructions in a cyberattack.

AI accelerates blood flow MRI

AI accelerates blood flow MRI

Researchers have presented a method that could greatly accelerate dynamic magnetic resonance imaging of blood flow.

Popular articles

Subscribe to Newsletter