Children with the autism often fail to recognise basic facial emotions, which make social interactions and developing friendships even more difficult to sustain. Now, a pilot study by researchers at the Stanford University School of Medicine looked at the effects and feasibility of using a wearable called Superpower Glass.
The therapy uses an Stanford-designed app that provides real-time cues about other people’s facial expressions to a child wearing Google Glass. The device, which was linked with a smartphone through a local wireless network, consists of a glasses-like frame equipped with a camera to record the wearer’s field of view, as well as a small screen and a speaker to give the wearer visual and audio information. As the child interacts with others, the app identifies and names their emotions through the Google Glass speaker or screen. After one to three months of regular use, parents reported that children with autism made more eye contact and related better to others. And their autism got better with some dropping from severe to moderate. “We have too few autism practitioners,” said the study’s senior author, Dennis Wall, PhD, associate professor of pediatrics and of biomedical data science. Early autism therapy has been shown to be particularly effective, but many children aren’t treated quickly enough to get the maximum benefit, he said. “The only way to break through the problem is to create reliable, home-based treatment systems. It’s a really important unmet need.”
Typically developing children learn to recognize emotions by engaging with people around them. For children with autism, it’s different. “They don’t pick those things up without focused treatment,” Wall said. The researchers named the new therapy “Superpower Glass” to help make it appealing to children. The therapy is based on applied behavior analysis, a well-studied autism treatment in which a clinician teaches emotion recognition using structured exercises such as flash cards depicting faces with different emotions. Although traditional applied behavior analysis helps children with autism, it has limitations: It must be delivered one-on-one by trained therapists, flash cards can’t always capture the full range of human emotion and children may struggle to transfer what they learn to their daily lives.
Eight core facial expressions
Wall’s team decided to try using applied behavior analysis principles in a way that would bring parents and everyday situations into the treatment process. They built a smartphone app that uses machine learning to recognize eight core facial expressions: happiness, sadness, anger, disgust, surprise, fear, neutral and contempt. The app was trained with hundreds of thousands of photos of faces showing the eight expressions, and also had a mechanism to allow people involved in the study to calibrate it to their own “neutral” faces if necessary.
The researchers designed three ways to use the face-recognizing program: In “free play,” children wear Google Glass while interacting or playing with their families, and the software provides the wearer with a visual or auditory cue each time it recognizes an emotion on the face of someone in the field of view. There are also two game modes. In “guess my emotion,” a parent acts out a facial expression corresponding to one of the eight core emotions, and the child tries to identify it. The game helps families and researchers track children’s improvement at identifying emotions. In “capture the smile,” children give another person clues about the emotion they want to elicit, until the other person acts it out, which helps the researchers gauge the children’s understanding of different emotions.
In the study, 14 families tested the Superpower Glass setup at home for an average of 10 weeks each. Each family had a child between the ages of 3 and 17 with a clinically confirmed autism diagnosis. The families used the therapy for at least three 20-minute sessions per week. At the start and end of the study, parents completed questionnaires to provide detailed information about their child’s social skills. In interviews, parents and children also gave feedback about how the program worked for their families. Parents’ comments in interviews such as “a switch has been flipped; my child is looking at me” or “suddenly the teacher is telling me that my child is engaging in the classroom” was “really heartwarming and super-encouraging for us to hear,” Prof Wall said.