Using VR headsets, healthcare professionals are able to conduct interviews with...
Using VR headsets, healthcare professionals are able to conduct interviews with the avatars.
Source: Roberto Schirdewahn

Mentally ill avatars

To teach medical students, scientists have created avatars that feature typical symptoms of different mental disorders as well as psychiatric and psychotherapeutic treatment situations.

The diagnosis of mental disorders depends not only on the answers patients provide in the interview with a healthcare professional. The way they behave is likewise relevant: how does their voice sound? Do they look the other person in the eye? What’s their posture like? Are they pacing restlessly? Is the face stony or animated? “An experienced healthcare professional monitors and assesses all these aspects while talking to a patient, noticing changes to the facial expression within milliseconds,” outlines Associate Professor Paraskevi Mavrogiorgou from the RUB clinic for psychiatry, psychotherapy and preventive medicine.

Medical students have to learn how to do that, which isn’t always easy. “At the university clinic, we mainly see patients who are seriously ill,” says the physician. “It is difficult to motivate them to take part in training sessions with the students. Plus, they are not the kind of patients whom the future doctors will most likely be seeing in the course of their career.”

In order to enable students to practise with patients presenting the full range of symptoms and degrees of severity and to broaden the panorama, the clinic has launched a project focusing on creating avatars that feature typical symptoms of different mental disorders as well as psychiatric and psychotherapeutic treatment situations. Students wearing a virtual reality headset (VR headset) will be able to meet those avatars in a three-dimensional space – or, as experts call it, “exploration”.

The clinic has already established rooms and prepared them structurally. Here, small cubicles will be set up that students will enter with their VR headsets in order to talk with their virtual patients. “As far as we know, this has never been done anywhere yet; we are pioneering the field,” says Professor Dr. Georg Juckel, Medical Director at the clinic.

The interaction with the avatars will be taking place in small cubicles, in...
The interaction with the avatars will be taking place in small cubicles, in order to make students feel like they’re in a real interview setting.
Source: Roberto Schirdewahn

Difficulty: designing a dialogue flow

The preparations have taken two to three years; now, the virtual characters are being fleshed out. “They need a biography, a case history,” explains Paraskevi Mavrogiorgou. “And they must be able to answer specific key questions in accordance with their respective mental disorder.” If the attending doctor asks an avatar suffering from schizophrenia whether he hears voices, the avatar should say he does. If the attending doctor asks a question that doesn’t help find the correct diagnosis, the avatar mustn’t simply remain silent; rather, he has to give a plausible, evasive answer.

“At the moment, this is the greatest difficulty: designing a dialogue flow,” says Georg Juckel. “We have compiled hundreds of pages with instructions and dialogues,” points out Paraskevi Mavrogiorgou. For the dialogue to work, high-performance speech recognition software is necessary to ensure that the computer in the background understands the questions and can provide relevant answers.

In order to create avatars with plausible physical features, the medical professionals intend to collaborate with a commercial enterprise specialised in visualisation for medical applications – after Arthur Berns, young computer engineer from Göttingen, has designed the logical framework. The avatars’ body language is to be adapted to the type of mental disorders they represent. They might be sitting, standing, lying or pacing during the interview. Their facial expressions, too, are supposed to tell the trained observer something about the underlying disorder. “The programmer has to design each of these facial expressions individually,” as Georg Juckel elaborates the immense complexity of the task. “It’s the same as with Walt Disney’s animated movies, or with computer games today.”

The avatars are to be built up segmentally, “or else the whole thing would soon become quite boring for the students,” says Juckel. Thanks to the modular system, the difficulty level of the exploration can be regulated. The selected avatar represents a specific psychiatric diagnosis or treatment situation. However, the way students get there, the questions they ask, is not predefined: this is what they will be learning under guidance and supervision.

The project team expects that the avatars will be taking their first steps in spring 2020. In summer semester 2020, students might begin conducting the first interviews with them. “We’re hoping that this approach will catch on with the students – especially in view of today’s leisure activities,” concludes Paraskevi Mavrogiorgou. They are definitely curious to see what the virtual patients are all about.

Subscribe to our newsletter

Related articles

Exposure therapy in VR to prevent disorders

Exposure therapy in VR to prevent disorders

VR can have significant impact on the validity of remote health appointments for those with eating disorders, through called Virtual Reality Exposure Therapy (VRET).

Virtual reality: Avatars against obesity

Virtual reality: Avatars against obesity

A collaborative project develops virtual reality methods to positively affect the body perception of obese patients.

A mental health app for young people

A mental health app for young people

A computer cames graduate has helped a tech start-up to develop a new augmented reality app which provides round-the-clock mental health support to young people.

Olfactory VR show promise for mental health practices

Olfactory VR show promise for mental health practices

New research on Olfactory Virtual Reality (OVR) paints a clearer picture for clinical psychiatrists about how it could be used to safely and effectively help mental health and mood disorders.

Researchers use virtual reality to reach youth

Researchers use virtual reality to reach youth

Researchers found that patients participating in VR sessions experienced reduced levels of anxiety and depression.

Fortnite and Zelda can up your surgical game

Fortnite and Zelda can up your surgical game

Video games offer students obvious respite from the stresses of studies and, now, a study has found they could benefit surgical skills training.

Virtual reality helps to treat fear of heights

Virtual reality helps to treat fear of heights

Researchers have developed a VR app to reduce fear of heights. Now, they have conducted a clinical trial to study its efficacy.

Exercise caution after working out in VR

Exercise caution after working out in VR

Researchers have investigated the aftereffects of playing one of the most popular VR exergames on aspects of vision, cognition, and self-reported VR sickness.

Creating an accessible surgery simulator

Creating an accessible surgery simulator

Researchers are developing a simulator that could be used to train both surgeons who are in the early stages of their career and those who are more experienced.

Popular articles

Subscribe to Newsletter