AIs detect diabetic eye disease inconsistently

Although some artificial intelligence software tested reasonably well, only one met the performance of human screeners, researchers at University of Washington Health Sciences (UW Medicine) found.

Photo

Diabetes continues to be the leading cause of new cases of blindness among adults in the United States. But the current shortage of eye-care providers would make it impossible to keep up with demand to provide the requisite annual screenings for this population. A new study looks at the effectiveness of seven artificial intelligence-based screening algorithms to diagnose diabetic retinopathy, the most common diabetic eye disease leading to vision loss.

Researchers compared the algorithms against the diagnostic expertise of retina specialists. Five companies produced the tested algorithms - two in the United States (Eyenuk, Retina-AI Health), one in China (Airdoc), one in Portugal (Retmarker), and one in France (OphtAI).

The researchers deployed the algorithm-based technologies on retinal images from nearly 24,000 veterans who sought diabetic retinopathy screening at the Veterans Affairs Puget Sound Health Care System and the Atlanta VA Health Care System from 2006 to 2018.

The researchers found that the algorithms don't perform as well as they claim. Many of these companies are reporting excellent results in clinical studies. But their performance in a real-world setting was unknown. Researchers conducted a test in which the performance of each algorithm and the performance of the human screeners who work in the VA teleretinal screening system were all compared to the diagnoses that expert ophthalmologists gave when looking at the same images. Three of the algorithms performed reasonably well when compared to the physicians' diagnoses and one did worse. But only one algorithm performed as well as the human screeners in the test.

"It's alarming that some of these algorithms are not performing consistently since they are being used somewhere in the world," said lead researcher Aaron Lee, assistant professor of ophthalmology at the University of Washington School of Medicine.

Differences in camera equipment and technique might be one explanation. Researchers said their trial shows how important it is for any practice that wants to use an AI screener to test it first and to follow the guidelines about how to properly obtain images of patients' eyes, because the algorithms are designed to work with a minimum quality of images.

The study also found that the algorithms' performance varied when analyzing images from patient populations in Seattle and Atlanta care settings. This was a surprising result and may indicate that the algorithms need to be trained with a wider variety of images.

The research was published in Diabetes Care.

Subscribe to our newsletter

Related articles

AI screens and diagnoses glaucoma

AI screens and diagnoses glaucoma

Scientists have developed a novel method that uses artificial intelligence to screen for glaucoma.

AI test predicts eye disease years before symptoms

AI test predicts eye disease years before symptoms

A new eye test may predict wet age-related macular degeneration, a leading cause of severe sight loss, three years before symptoms develop.

Deep learning enables screening for eye disease

Deep learning enables screening for eye disease

Researchers created a novel deep learning method that makes automated screenings for eye diseases such as diabetic retinopathy more efficient.

Retinal screening in diabetes: diagnosis by robot

Retinal screening in diabetes: diagnosis by robot

Physicians have been using automatic digital retinal screening, without assistance from an ophthalmologist, to detect diabetic retinal disease.

Enabling AI-driven advances without sacrificing privacy

Enabling AI-driven advances without sacrificing privacy

Secure AI Labs is expanding access to encrypted health care data to advance AI-driven innovation in the field.

Harnessing AI to discover new drugs

Harnessing AI to discover new drugs

Artificial intelligence can recognise the biological activity of natural products in a targeted manner.

AI enhances efficacy of sleep disorder treatments

AI enhances efficacy of sleep disorder treatments

Based on 20,000 nights of sleep, researchers have developed an algorithm that can improve the diagnosis, treatment and overall understanding of sleep disorders.

Making AI understandable – constructing explanation processes

Making AI understandable – constructing explanation processes

Human-machine interaction is complex. Researchers investigate a new form of interaction between humans and machines.

Would we be able to control superintelligent machines?

Would we be able to control superintelligent machines?

Using theoretical calculations, scientists showed that it would not be possible to control a superintelligent AI.

Popular articles

Subscribe to Newsletter