Example of ‘skeleton’ extraction using OpenPose on an image extracted from...
Example of ‘skeleton’ extraction using OpenPose on an image extracted from the recording of a social scene: (A) OpenPose keypoints overlaid on the video recording (B). OpenPose keypoints over a black background, preserving the bare minimum of scene features necessary to identify the social interaction.
Source: Kojovic, Natraj, Mohanty, Maillart, Schaer

Using AI and video for early detection of autism

Using artificial intelligence, a team from the University of Geneva (UNIGE), Switzerland, has developed a device for the early detection of autism spectrum disorder in children.

Individuals affected by autism spectrum disorder often present communication issues and difficulties in social interactions. Although very frequent, this disorder is challenging to diagnose before the age of five. However, early care can compensate for these difficulties by providing specific behavioural intervention focused on the development of skills affected by autism. This is why an interdisciplinary team at UNIGE has developed an artificial intelligence (AI) algorithm based on the automated analysis of videos, making it possible to study children’s non-verbal communication in an anonymous and standardised manner. Easy to use, this technology correctly classified 80% of cases from short videos showing a child with or without autism under 5 playing with an adult. These results pave the way for a tool for the early detection of autistic disorder.

Autism Spectrum Disorder affects one in 54 children and is characterised by difficulties in social interactions, altered communication skills, and by the presence of repetitive behaviours and restricted interests. Consequently, children with autism often have difficulty following a standard school curriculum. “However, if the diagnosis is made before the age of 3, it is often possible to make up for these developmental delays. Specific behavioural interventions may indeed completely change their skills acquisition trajectories and allow them to integrate a mainstream school”, notes Marie Schaer, professor in the Department of Psychiatry at UNIGE Faculty of Medicine and senior author of the study. The challenge therefore resides in the early diagnosis, as autism remains too often diagnosed too late, after the age of 3.

Using automated video analysis

Autism is characterised by a non-verbal communication that differs from that of a typically- developing child. “It differs on several points, such as the difficulty in establishing eye-contact, smiling, pointing to objects or the way they are interested in what surrounds them”, explains Nada Kojovic, a researcher in Marie Schaer’s team and first author of the study. “This is why we designed an algorithm using artificial intelligence that analyses the children’s movements on video and identifies whether or not they are characteristic of autism spectrum disorder.”

Over a period of three years, the scientists, supported by the Swiss National Centre of Comptence in Research (NCCR) Synapsy and the Fondation Pole Autisme, developed this algorithm, which aims to classify videos based solely on the child’s movements when interacting with another person. 

To do this, they first used a technology named OpenPose, developed at Carnegie Mellon University. This computer vision technology extracts the skeletons of moving people as captured in a video and allows the analysis of gestures by removing all characteristics that could be discriminating (age, sex, setting, etc.), keeping only the relationships of skeletons in space and time. 

The UNIGE research team then developed their AI algorithm tailored for detecting autism and tested it on 68 typically developing children and 68 children with autism, all under 5 years. “We divided each group into two: the first 34 in each group ‘trained’ our AI to differentiate the non-verbal behaviour of children with and without autism. The others then helped us test its accuracy. We also carried out an assessment on 101 other children”, explains Thomas Maillart, a researcher at the Institute of Information Sciences and a faculty member at the Geneva School Economics and Management (GSEM) and of the University Centre for Informatics (CUI) at UNIGE.

A convincing result in 10 minutes

The AI sifted through videos of children playing freely with an adult. “There is no pre-established scenario. It is a matter of analysing the children’s non-verbal behaviour freely, while providing them with various objects that will make it possible to determine the presence or absence of an autistic disorder”, notes Nada Kojovic. The results show that the AI makes accurate autism classification in more than 80% of cases. “This is an excellent result, enthuses Marie Schaer. In 10 minutes, we can indeed obtain a first screening accessible to anyone, wherever they live. This would allow parents worried about their young children to obtain an initial automated assessment of the symptoms of autism. It is of course not perfect, but could constitute a first step to be confirmed by a consultation with a specialist.” Studies show that it often takes more than a year between the parents’ first concerns and their referral to a specialised consultation.

In addition, this automated video analysis offers complete anonymity. “This is precious, not only for the exchange of videos between specialists to refine a diagnosis, but also for training students”, continues the Geneva researcher.

It should be noted that this technology does not require any direct intervention on the child. “The installation of movement sensors is time-consuming and sensitive; it can also disturb the children and influence the results. Here, the computer vision-based analysis is non-invasive”, emphasises Nada Kojovic. Moreover, as it does not require any specific setup, the algorithm can be used to analyse videos recorded in the past, a clear advantage for research purposes.

The aim of the multidisciplinary team is now to make this AI available to everyone. “We now wish to develop an application that would allow such as analysis with only a 10 minutes video filmed with a smartphone”, concludes Thomas Maillart.

The research was published in Scientific Reports.

Subscribe to our newsletter

Related articles

Robots benefit special education students

Robots benefit special education students

Researchers have discovered that primary school children in both regular and special needs schools make strides when they learn together with a robot.

Robin, the AI  social robot

Robin, the AI social robot

In response to the COVID-19 pandemic, UCLA Mattel Children’s Hospital launches an innovative project to support the emotional needs of children.

Robot toys provide clues about infant development

Robot toys provide clues about infant development

New research shows that using robots could help detect early signs of autism or ADHD in infants.

Video game to treat children with ADHD

Video game to treat children with ADHD

Researchers have developed advanced brain-computer interface technology that harnesses machine learning to personalise brain-training for children with ADHD.

AI may help spot autism early

AI may help spot autism early

A machine learning algorithm can spot abnormalities in pupil dilation that are predictive of autism spectrum disorder in mouse models.

AI detects a new class of mutations behind autism

AI detects a new class of mutations behind autism

Using artificial intelligence, researchers have decoded the functional impact of genome mutations in people with autism spectrum disorder.

Autism: Google Glass helps kids improve socialization skills

Autism: Google Glass helps kids improve socialization skills

Children with autism improved measurably on a test of socialization and learning when their therapy included an at-home intervention with Google Glass.

AI can detect depression in a child’s speech

AI can detect depression in a child’s speech

Researchers have used artificial intelligence to detect hidden depression in young children, a condition that can lead to increased risk of substance abuse and suicide later in life if left untreated.

VR therapy treats autism phobias

VR therapy treats autism phobias

Immersive virtual reality has been shown to help children with autism with nearly 45% remaining free from their fears and phobias six months after treatment.

Popular articles

Subscribe to Newsletter