The intelligent carpet inferrs 3D pose from tactile signals.
The intelligent carpet inferrs 3D pose from tactile signals.
Source: MIT Computer Science & Artificial Intelligence Lab

Smart carpet gives insight into human poses

The sentient magic carpet from 'Aladdin' might have a new competitor. While it can't fly or speak, a new tactile sensing carpet from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) can estimate human poses without using cameras, in a step towards improving self-powered personalized healthcare, smart homes, and gaming.

Many of our daily activities involve physical contact with the ground: walking, exercising, or resting. These embedded interactions contain a wealth of information that help us better understand people's movements. Previous research has leveraged use of single RGB cameras, (think Microsoft Kinect), wearable omnidirectional cameras, and even plain old off the shelf webcams, but with the inevitable byproducts of camera occlusions and privacy concerns.

The CSAIL team's system only used cameras to create the dataset the system was trained on, and only captured the moment of the person performing the activity. To infer the 3D pose, a person would simply have to get on the carpet, perform an action, and then the team's deep neural network, using just the tactile information, could determine if the person was doing sit-ups, stretching, or doing another action.

"You can imagine leveraging this model to enable a seamless health monitoring system for high-risk individuals, for fall detection, rehab monitoring, mobility, and more," says Yiyue Luo, a lead author on a paper about the carpet.

The carpet itself, which is low cost and scalable, was made of commercial, pressure-sensitive film and conductive thread, with over nine thousand sensors spanning thirty six by two feet. (Most living room rug sizes are eight by ten or nine by twelve.) Each of the sensors on the carpet convert the human's pressure into an electrical signal, through the physical contact between people's feet, limbs, torso, and the carpet. The system was specifically trained on synchronized tactile and visual data, such as a video and corresponding heatmap of someone doing a pushup.

The model takes the pose extracted from the visual data as the ground truth, uses the tactile data as input, and finally outputs the 3D human pose. This might look something like, when, after stepping onto the carpet, and doing a set up of pushups, the system is able to produce an image or video of someone doing a push-up.

In fact, the model was able to predict a person's pose with an error margin (measured by the distance between predicted human body key points and ground truth key points) by less than ten centimeters. For classifying specific actions, the system was accurate 97 percent of the time. "You may envision using the carpet for workout purposes. Based solely on tactile information, it can recognize the activity, count the number of reps, and calculate the amount of burned calories," says Yunzhu Li, a co-author on the paper.

Since much of the pressure distributions were prompted by movement of the lower body and torso, that information was more accurate than the upper body data. Also, the model was unable to predict poses without more explicit floor contact, like free-floating legs during sit-ups, or a twisted torso while standing up.

While the system can understand a single person, the scientists, down the line, want to improve the metrics for multiple users, where two people might be dancing or hugging on the carpet. They also hope to gain more information from the tactical signals, such as a person's height or weight.

Subscribe to our newsletter

Related articles

Tracking muscle activity through clothes

Tracking muscle activity through clothes

Printing metals onto cloth makes for comfortable, low-cost, and effective biosensors.

Making motion sensing devices more personal

Making motion sensing devices more personal

An electrical impedance tomography toolkit lets users design and fabricate health and motion sensing devices.

Smart shoe sensors to detect health issues

Smart shoe sensors to detect health issues

A smart shoe with inbuilt sensors could improve the quality of life of older people through the early detection of dementia and diabetic ulcers.

‘Smart’ shirt keeps tabs on the heart

‘Smart’ shirt keeps tabs on the heart

A flexible carbon nanotube fibers can be incorporated into clothing to function as wearable health monitors.

Touch-sensing glove detects and maps tactile stimuli

Touch-sensing glove detects and maps tactile stimuli

Engineers have designed a new touch-sensing glove that can “feel” pressure and other tactile stimuli. The design could help restore motor function after stroke.

Face mask detects Covid-19 infection

Face mask detects Covid-19 infection

Engineers have designed a novel face mask that can diagnose the wearer with Covid-19 within about 90 minutes.

Forget wearables: smart fabrics to monitor health

Forget wearables: smart fabrics to monitor health

Engineers have developed a method to transform existing cloth items into battery-free wearables resistant to laundry. These smart clothes are powered wirelessly through a flexible, silk-based coil sewn on the textile.

Smart textiles: a programmable digital fiber

Smart textiles: a programmable digital fiber

In a first, the digital fiber contains memory, temperature sensors, and a trained neural network program for inferring physical activity.

E-textiles made with new cellulose thread

E-textiles made with new cellulose thread

Researchers have developed a thread made of conductive cellulose, which offers practical possibilities for electronic textiles.

Popular articles

Subscribe to Newsletter