Eliminating bias from AI critical to improve equity

Artificial intelligence (AI)-driven healthcare has the potential to transform medical decision-making and treatment, but these algorithms must be thoroughly tested and continuously monitored to avoid unintended consequences to patients.

Photo
Source: CC0 Public Domain

Regenstrief Institute President and Chief Executive Officer and Indiana University School of Medicine Associate Dean for Informatics and Health Services Research Peter Embí, M.D., M.S., strongly stated the importance of algorithmovigilance to address inherent biases in healthcare algorithms and their deployment in a JAMA Network Open Invited Commentary. Algorithmovigilance, a term coined by Dr. Embí, can be defined as the scientific methods and activities relating to the evaluation, monitoring, understanding, and prevention of adverse effects of algorithms in healthcare.

"We wouldn't think of treating patients with a new pharmaceutical or device without first ensuring its efficacy and safety," said Dr. Embí. "In the same way, we must recognize that algorithms have the potential for both great benefit and harm and, therefore, require study. Also, compared with drugs or devices, algorithms often have additional complexities and variations, such as how they are deployed, who interacts with them, and the clinical workflows where interactions with algorithms take place."

The commentary was in response to a study from IBM scientists evaluating different approaches to debiasing healthcare algorithms developed to predict postpartum depression. Dr. Embí stated the study suggests that debiasing methods can help address underlying disparities represented in the data used to develop and deploy the AI approaches. He also said the study demonstrates that the evaluation and monitoring of these algorithms for effectiveness and equity is necessary and even ethically required.

"Algorithmic performance changes as it is deployed with different data, different settings and different human-computer interactions. These factors could turn a beneficial tool into one that causes unintended harm, so these algorithms must continually be evaluated to eliminate the inherent and systemic inequities that exist in our healthcare system," Dr. Embí continued. "Therefore, it's imperative that we continue to develop tools and capabilities to enable systematic surveillance and vigilance in the development and use of algorithms in healthcare."

Subscribe to our newsletter

Related articles

AI identified decision-relevant patterns in EEG of coma patients

AI identified decision-relevant patterns in EEG of coma patients

A research team has succeeded in identifying specific patterns in Electro-Encephalogram (EEG) analyses that the deep learning network uses for making prognosis decisions.

A contact aware robot design

A contact aware robot design

Researchers have developed a new method to computationally optimize the shape and control of a robotic manipulator for a specific task.

Artificial intelligence for emergency management

Artificial intelligence for emergency management

A consortium aims to develop a platform that will serve as the basis for novel services and test the use of new artificial intelligence tools.

Harnessing AI to discover new drugs

Harnessing AI to discover new drugs

Artificial intelligence can recognise the biological activity of natural products in a targeted manner.

Smart biomarkers to empower drug development

Smart biomarkers to empower drug development

Researchers aim to speed up developing drugs against brain diseases through cutting-edge technology. They are generating an innovative technology platform based on high-density microelectrode arrays and 3D networks of human neurons.

Intelligent algorithms for movement apps

Intelligent algorithms for movement apps

With LTech, the Lindera Software Development Kit, health tech company Lindera brings innovation and AI technology to the fitness industry.

AI enhances efficacy of sleep disorder treatments

AI enhances efficacy of sleep disorder treatments

Based on 20,000 nights of sleep, researchers have developed an algorithm that can improve the diagnosis, treatment and overall understanding of sleep disorders.

AI makes great microscopes better than ever

AI makes great microscopes better than ever

Machine learning helps some of the best microscopes to see better, work faster, and process more data.

AI makes no-cath forecast

AI makes no-cath forecast

Researchers use AI software to predict coronary artery plaque composition and significance without the risks of invasive procedures.

Popular articles

Subscribe to Newsletter