Dr. Dan Nguyen, a clinical instructor at the UW School of Medicine, uses the...
Dr. Dan Nguyen, a clinical instructor at the UW School of Medicine, uses the team's smart speaker prototype (white box in lower left corner) to demonstrate how the system works.
Source: Mark Stone/University of Washington

Smart speakers contactlessly monitors heart rhythms

Researchers at University of Washington have developed system for smart speakers to monitor both regular and irregular heartbeats without physical contact.

Smart speakers, such as Amazon Echo and Google Home, have proven adept at monitoring certain health care issues at home. For example, researchers at the University of Washington have shown that these devices can detect cardiac arrests or monitor babies breathing.

But what about tracking something even smaller: the minute motion of individual heartbeats in a person sitting in front of a smart speaker?

UW researchers have developed a new skill for a smart speaker that for the first time monitors both regular and irregular heartbeats without physical contact. The system sends inaudible sounds from the speaker out into a room and, based on the way the sounds are reflected back to the speaker, it can identify and monitor individual heartbeats. Because the heartbeat is such a tiny motion on the chest surface, the team's system uses machine learning to help the smart speaker locate signals from both regular and irregular heartbeats.

When the researchers tested this system on healthy participants and hospitalized cardiac patients, the smart speaker detected heartbeats that closely matched the beats detected by standard heartbeat monitors. "Regular heartbeats are easy enough to detect even if the signal is small, because you can look for a periodic pattern in the data," said co-senior author Shyam Gollakota, a UW associate professor in the Paul G. Allen School of Computer Science & Engineering. "But irregular heartbeats are really challenging because there is no such pattern. I wasn't sure that it would be possible to detect them, so I was pleasantly surprised that our algorithms could identify irregular heartbeats during tests with cardiac patients."

While many people are familiar with the concept of a heart rate, doctors are more interested in the assessment of heart rhythm. Heart rate is the average of heartbeats over time, whereas a heart rhythm describes the pattern of heartbeats.

For example, if a person has a heart rate of 60 beats per minute, they could have a regular heart rhythm - one beat every second - or an irregular heart rhythm - beats are randomly scattered across that minute but they still average out to 60 beats per minute.

"Heart rhythm disorders are actually more common than some other well-known heart conditions. Cardiac arrhythmias can cause major morbidities such as strokes, but can be highly unpredictable in occurrence, and thus difficult to diagnose," said co-senior author Dr. Arun Sridhar, assistant professor of cardiology at the UW School of Medicine. "Availability of a low-cost test that can be performed frequently and at the convenience of home can be a game-changer for certain patients in terms of early diagnosis and management."

The teams smart speaker prototype compares signals from its multiple...
The team's smart speaker prototype compares signals from its multiple microphones (visible through the holes of this box) to identify the elusive heartbeat signal.
Source: Mark Stone/University of Washington

The key to assessing heart rhythm lies in identifying the individual heartbeats. For this system, the search for heartbeats begins when a person sits within 1 to 2 feet in front of the smart speaker. Then the system plays an inaudible continuous sound, which bounces off the person and then returns to the speaker. Based on how the returned sound has changed, the system can isolate movements on the person -- including the rise and fall of their chest as they breathe.

"The motion from someone's breathing is orders of magnitude larger on the chest wall than the motion from heartbeats, so that poses a pretty big challenge," said lead author Anran Wang, a doctoral student in the Allen School. "And the breathing signal is not regular so it's hard to simply filter it out. Using the fact that smart speakers have multiple microphones, we designed a new beam-forming algorithm to help the speakers find heartbeats."

The team designed what's called a self-supervised machine learning algorithm, which learns on the fly instead of from a training set. This algorithm combines signals from all of the smart speaker's multiple microphones to identify the elusive heartbeat signal.

"This is similar to how Alexa can always find my voice even if I'm playing a video or if there are multiple people talking in the room," Gollakota said. "When I say, 'Hey, Alexa,' the microphones are working together to find me in the room and listen to what I say next. That's basically what's happening here but with the heartbeat."

The heartbeat signals that the smart speaker detects don't look like the typical peaks that are commonly associated with traditional heartbeat monitors. The researchers used a second algorithm to segment the signal into individual heartbeats so that the system could extract what is known as the inter-beat interval, or the amount of time between two heartbeats. "With this method, we are not getting the electric signal of the heart contracting. Instead we're seeing the vibrations on the skin when the heart beats," Wang said.

Recommended article

The researchers tested a prototype smart speaker running this system on two groups: 26 healthy participants and 24 hospitalized patients with a diversity of cardiac conditions, including atrial fibrillation and heart failure. The team compared the smart speaker's inter-beat interval with one from a standard heartbeat monitor. Of the nearly 12,300 heartbeats measured for the healthy participants, the smart speaker's median inter-beat interval was within 28 milliseconds of the standard monitor. The smart speaker performed almost as well with cardiac patients: of the more than 5,600 heartbeats measured, the median inter-beat interval was within 30 milliseconds of the standard.

Currently this system is set up for spot checks: If a person is concerned about their heart rhythm, they can sit in front of a smart speaker to get a reading. But the research team hopes that future versions could continuously monitor heartbeats while people are asleep, something that could help doctors diagnose conditions such as sleep apnea.

"If you have a device like this, you can monitor a patient on an extended basis and define patterns that are individualized for the patient. For example, we can figure out when arrhythmias are happening for each specific patient and then develop corresponding care plans that are tailored for when the patients actually need them," Sridhar said. "This is the future of cardiology. And the beauty of using these kinds of devices is that they are already in people's homes."

T research was published in Communications Biology.

Subscribe to our newsletter

Related articles

The first contactless cardiac arrest AI system for smart speakers

The first contactless cardiac arrest AI system for smart speakers

Researchers at the University of Washington have developed a new tool to monitor people for cardiac arrest while they’re asleep without touching them.

AI makes no-cath forecast

AI makes no-cath forecast

Researchers use AI software to predict coronary artery plaque composition and significance without the risks of invasive procedures.

AI algorithm to help manage diabetes

AI algorithm to help manage diabetes

Researchers, using artificial intelligence and automated monitoring, have designed a method to help people with type 1 diabetes better manage their glucose levels.

Necklace detects abnormal heart rhythm

Necklace detects abnormal heart rhythm

A necklace which detects abnormal heart rhythm will be showcased for the first time on EHRA Essentials 4 You, a scientific platform of the European Society of Cardiology (ESC).

AI identified decision-relevant patterns in EEG of coma patients

AI identified decision-relevant patterns in EEG of coma patients

A research team has succeeded in identifying specific patterns in Electro-Encephalogram (EEG) analyses that the deep learning network uses for making prognosis decisions.

Smart wristband monitors heart rates and physical activity

Smart wristband monitors heart rates and physical activity

Engineers have created biosensor technology with a wireless connection to smartphones that will enable a new wave of personal health.

‘Smart stent‘ detects narrowing of arteries

‘Smart stent‘ detects narrowing of arteries

Engineers have developed a so-called smart stent that detects changes in blood flow through an artery.

A contact aware robot design

A contact aware robot design

Researchers have developed a new method to computationally optimize the shape and control of a robotic manipulator for a specific task.

Artificial intelligence for emergency management

Artificial intelligence for emergency management

A consortium aims to develop a platform that will serve as the basis for novel services and test the use of new artificial intelligence tools.

Popular articles

Subscribe to Newsletter