A three-step process helps standardize smartwatches and other devices that...
A three-step process helps standardize smartwatches and other devices that collect biometric data. Image shows Jessilyn Dunn and Brinnae Bent.
Source: Duke University

Verifying and validating the clinical usefulness of wearables

A collaborative team of researchers from the Digital Medicine Society (DiMe) and biomedical engineers at Duke University have developed a framework that will help data scientists and other researchers use better digital health tools for clinical purposes.

As smartwatches and other wearable technologies are becoming more popular, researchers are exploring how they can use biometric data collected by these tools to create beneficial health insights about the users. Although some of these devices are marketed as being clinically validated, there are currently no standards to ensure that the data from digital medicine tools is evaluated and fit for clinical purposes.

In a new paper, Jessilyn Dunn, an assistant professor of biomedical engineering at Duke University, worked with an interdisciplinary, international team of 16 researchers from the DiMe community to propose a three-step framework that evaluates and documents the clinical usefulness of these tools, addressing shortcomings with the current approach to evaluating digital health tools.

The project is one of several collaborations between Dunn and the expert DiMe community that include co-participation in a World Economic Forum initiative focused on managing epidemics with consumer wearables. "In the last decade alone we've seen digital biomarker research increase by more than 325 percent, but we haven't caught up with this growth in terms of standards and evaluation of digital medicine tools," said Brinnae Bent, a PhD student in the Dunn lab and one of the authors of the paper. "Our main goal was to develop a common framework for evaluating these new technologies, but we also wanted to create a unifying language for the field so there's structure as it grows."

During the evaluation of more traditional medical devices, engineers will verify the software and sensor technology of a product and validate that the end product can accurately mreasure what it claims to measure, like heart rate or activity levels.

But there are no set definiitions for what clinically validated means for wearable devices. For examaple, companies may advertise their heart rate measurements as accurate and clinically validated, but they may have only been tested across a limited range of environments and body and activity types.

These testing discrepancies can lead to varying rates of data accuracy, which Dunn and her lab discovered in their February 2020 Nature Digital Medicine paper. In their new paper, lead author Jennifer Goldsack, the executive director at DiMe, and Dunn introduce the term Biometric Monitoring technologies, or BioMeTs, as a standardized description for all technologies that combine sensors and other hardware with software to collect biometric data.

The first step of their "V3" framework is verification, where hardware manufacturers will tet sample-level sensor outputs of the devices. In the next step, analytical validation, engineers, data scieitists and physiologists will evaluate the algorithms that process data from the sensor to produce physiological metrics.

The third and final step, clinical validation, is the largest departure from more traditional approaches to medical technology development. During this critical step, a technology vendor or clinical expert will demonstrate that the BioMeT can acceptably identify, measure or predict the clinical, biological, physical or functional experience that it was intended to capture in the target population of users. "It is essential to make sure tha tthe BioMeTs are indeed measuring what manufacturers claim they are measuring, and do it in a manner that is open and trustworhty," said Will Wang, a PhD student in the Dunn lab and one of the co-authors of the paper. "If such information is not clinically validated, users and researchers alike will be led to unjustified conclusions."

Now, the DiMe team, including Dunn, are working to advance teh framework with the Food and Drug Administration to make it the standard methodology for evaluating all BioMeTs. "The development of this V3 framework for evaluating BioMeTs characterizes DiMe's committment to building solutions to advance the safe, efficient and effective use of digital technologies to optimize health," Goldsack said. "We're incredibly proud of this seminal work in digital medicine that was completed by 16 experts from our diverse membership. We're grateful for Jessilyn's scientific expertise, her team's important contribition, and their commitment to collaboration."

Subscribe to our newsletter

Related articles

Humanising wearable health technologies

Humanising wearable health technologies

Designer Leah Heiss considers her work as creating “emotional technologies”, i.e. wearable devices based on human-centred design principles. For her, empathy is everything!

Smart insole monitors foot health for diabetic patients

Smart insole monitors foot health for diabetic patients

The smart insole can be inserted into a sneaker or dress shoe to passively monitor the foot health of a person living with diabetes.

Smart wristband alerts about dangerous health conditions

Smart wristband alerts about dangerous health conditions

Researchers are developing a smart wrist-worn device for monitoring of atrial fibrillation – a condition, which if left untreated can lead to serious health complications and even death.

Smart wristband monitors heart rates and physical activity

Smart wristband monitors heart rates and physical activity

Engineers have created biosensor technology with a wireless connection to smartphones that will enable a new wave of personal health.

Patches detect when a viral disease is getting worse

Patches detect when a viral disease is getting worse

Xsensio has been awarded CHF 1.8 million in EU funding to adapt its Lab-on-Skin sensing patches so that they can detect when a viral illness like the flu or COVID-19 is about to get worse.

Smartwatch tracks medication levels to personalize treatments

Smartwatch tracks medication levels to personalize treatments

Engineers have demonstrated that drug levels inside the body can be tracked in real time using a custom smartwatch that analyzes the chemicals found in sweat.

Wearable sensor tracks biochemical data

Wearable sensor tracks biochemical data

Scientist are developing a patch that monitors the sweat of high performance athletes for medical information.

Progress in capture of biosignals for EEG

Progress in capture of biosignals for EEG

New electrode technology and AI analytics solve challenges in neurological emergency, acute and intensive care medicine.

Smartwatch turns into biochemical monitoring system

Smartwatch turns into biochemical monitoring system

Engineers have designed a thin adhesive film that could upgrade a consumer smartwatch into a powerful health monitoring system.

Popular articles