Glove translates sign language into speech in real time

UCLA bioengineers have designed a glove-like device that can translate American Sign Language into English speech in real time through a smartphone app.

Photo
The system includes gloves with thin, stretchable sensors that run the length of each of the five fingers.
Source: Jun Chen Lab/UCLA

“Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,” said Jun Chen, an assistant professor of bioengineering at the UCLA Samueli School of Engineering and the principal investigator on the research. “In addition, we hope it can help more people learn sign language themselves.”

The system includes a pair of gloves with thin, stretchable sensors that run the length of each of the five fingers. These sensors, made from electrically conducting yarns, pick up hand motions and finger placements that stand for individual letters, numbers, words and phrases.

The device then turns the finger movements into electrical signals, which are sent to a dollar-coin–sized circuit board worn on the wrist. The board transmits those signals wirelessly to a smartphone that translates them into spoken words at the rate of about a one word per second.

The researchers also added adhesive sensors to testers’ faces — in between their eyebrows and on one side of their mouths — to capture facial expressions that are a part of American Sign Language. Previous wearable systems that offered translation from American Sign Language were limited by bulky and heavy device designs or were uncomfortable to wear, Chen said.

Photo
A digital rendering of the system that helps convert sign language into speech.

The device developed by the UCLA team is made from lightweight and inexpensive but long-lasting, stretchable polymers. The electronic sensors are also very flexible and inexpensive.

In testing the device, the researchers worked with four people who are deaf and use American Sign Language. The wearers repeated each hand gesture 15 times. A custom machine-learning algorithm turned these gestures into the letters, numbers and words they represented. The system recognized 660 signs, including each letter of the alphabet and numbers 0 through 9.

UCLA has filed for a patent on the technology. A commercial model based on this technology would require added vocabulary and an even faster translation time, Chen said.

The research was published in the journal Nature Electronics.

Subscribe to our newsletter

Related articles

Patches detect when a viral disease is getting worse

Patches detect when a viral disease is getting worse

Xsensio has been awarded CHF 1.8 million in EU funding to adapt its Lab-on-Skin sensing patches so that they can detect when a viral illness like the flu or COVID-19 is about to get worse.

Smartwatch turns into biochemical monitoring system

Smartwatch turns into biochemical monitoring system

Engineers have designed a thin adhesive film that could upgrade a consumer smartwatch into a powerful health monitoring system.

ELSAH – A ‘Smart Patch’ to determine biomarkers

ELSAH – A ‘Smart Patch’ to determine biomarkers

At the start of 2019 the EU project ELSAH began with the objective of designing a wearable within four years that enables the continuous determination of biomarker concentrations.

Update Apple Heart Study: Wearables can detect aFib

Update Apple Heart Study: Wearables can detect aFib

The clinical trial to determine whether a smartwatch app that analyzes pulse-rate data can screen for a heart-rhythm disorder has enrolled more than 400,000 participants.

Medical technology 2020 – a review

Medical technology 2020 – a review

Covid-19 gave many of these predictions for 2020 an entirely new spin: while some of the hyped trends turned out to play only bit-parts others became box-office hits in the new normal.

Fitness trackers boost physical activity levels

Fitness trackers boost physical activity levels

Smartphone fitness apps and wearable activity trackers do help boost physical activity levels, finds a review and pooled data analysis of the available evidence.

Earable computing: A new research area in the making

Earable computing: A new research area in the making

CSL's Systems and Networking Research Group (SyNRG) is defining a new sub-area of mobile technology that they call "earable computing."

Sense Glucose Earring for managing diabetes

Sense Glucose Earring for managing diabetes

A product design graduate has developed a discreet item of wearable technology that monitors blood sugar levels and delivers feedback in real-time.

Wearable shines light on benefits of outdoor lighting

Wearable shines light on benefits of outdoor lighting

A device could help scientists better understand the health benefits of outdoor lighting and lead to wearables that could nudge users to get more outdoor time.

Popular articles