A deep learning e-skin decodes complex human motion

A deep learning e-skin decodes complex human motion

A deep learning powered single-strained electronic skin sensor can capture human motion from a distance. The single strain sensor placed on the wrist decodes complex five-finger motions in real time with a virtual 3D hand that mirrors the original motions.

The deep neural network boosted by rapid situation learning (RSL) ensures stable operation regardless of its position on the surface of the skin. Conventional approaches require many sensor networks that cover the entire curvilinear surfaces of the target area. Unlike conventional wafer-based fabrication, this laser fabrication provides a new sensing paradigm for motion tracking.

The research team, led by Professor Sungho Jo from the School of Computing, collaborated with Professor Seunghwan Ko from Seoul National University to design this new measuring system that extracts signals corresponding to multiple finger motions by generating cracks in metal nanoparticle films using laser technology. The sensor patch was then attached to a user’s wrist to detect the movement of the fingers.

Photo
Deep Learned Sensor collecting epicentral motion.
Source: KAIST

The concept of this research started from the idea that pinpointing a single area would be more efficient for identifying movements than affixing sensors to every joint and muscle. To make this targeting strategy work, it needs to accurately capture the signals from different areas at the point where they all converge, and then decoupling the information entangled in the converged signals. To maximize users’ usability and mobility, the research team used a single-channeled sensor to generate the signals corresponding to complex hand motions.

The RSL system collects data from arbitrary parts on the wrist and automatically trains the model in a real-time demonstration with a virtual 3D hand that mirrors the original motions. To enhance the sensitivity of the sensor, researchers used laser-induced nanoscale cracking.

This sensory system can track the motion of the entire body with a small sensory network and facilitate the indirect remote measurement of human motions, which is applicable for wearable VR/AR systems.

Photo
RSL system based on transfer learning
Source: KAIST

The research team said they focused on two tasks while developing the sensor. First, they analyzed the sensor signal patterns into a latent space encapsulating temporal sensor behavior and then they mapped the latent vectors to finger motion metric spaces.

Professor Jo said, “Our system is expandable to other body parts. We already confirmed that the sensor is also capable of extracting gait motions from a pelvis. This technology is expected to provide a turning point in health-monitoring, motion tracking, and soft robotics.”

Subscribe to our newsletter

Related articles

E-skin: Engineers imitate hands to make better sensors

E-skin: Engineers imitate hands to make better sensors

Researchers have developed “electronic skin” sensors capable of mimicking the dynamic process of human motion.

Rubbery semiconductor for medical robotic hands

Rubbery semiconductor for medical robotic hands

Researchers have designed and produced a smart electronic skin and a medical robotic hand capable of assessing vital diagnostic data.

Electronic skin reacts to pain like human skin

Electronic skin reacts to pain like human skin

Researchers have developed electronic artificial skin that reacts to pain just like real skin, opening the way to better prosthetics, smarter robotics and non-invasive alternatives to skin grafts.

A robotic skin with a sense of touch

A robotic skin with a sense of touch

Scientists have developed a sensory integrated artificial brain system that mimics biological neural networks, which can run on a power-efficient neuromorphic processor.

Integrate micro chips for electronic skin

Integrate micro chips for electronic skin

First fully integrated flexible electronics made of magnetic sensors and organic circuits opens the path towards the development of electronic skin.

3D printing helps form wearable sensor

3D printing helps form wearable sensor

Researchers have developed a highly sensitive wearable pressure sensor for health monitoring applications and early diagnosis of diseases.

Exceptional sensitive e-skin for prosthetics

Exceptional sensitive e-skin for prosthetics

Researchers have developed an e-skin that may soon have a sense of touch equivalent to, or better than, the human skin with the Asynchronous Coded Electronic Skin (ACES).

Spray coated tactile sensor for robots and prosthetics

Spray coated tactile sensor for robots and prosthetics

Robots will be able to conduct a wide variety of tasks as well as humans if they can be given tactile sensing capabilities.

Mini-brains help robots recognise pain

Mini-brains help robots recognise pain

Using a brain-inspired approach, scientists have developed a way for robots to have the AI to recognise pain and to self-repair when damaged.

Popular articles