Using VR headsets, healthcare professionals are able to conduct interviews with...
Using VR headsets, healthcare professionals are able to conduct interviews with the avatars.
Source: Roberto Schirdewahn

Mentally ill avatars

To teach medical students, scientists have created avatars that feature typical symptoms of different mental disorders as well as psychiatric and psychotherapeutic treatment situations.

The diagnosis of mental disorders depends not only on the answers patients provide in the interview with a healthcare professional. The way they behave is likewise relevant: how does their voice sound? Do they look the other person in the eye? What’s their posture like? Are they pacing restlessly? Is the face stony or animated? “An experienced healthcare professional monitors and assesses all these aspects while talking to a patient, noticing changes to the facial expression within milliseconds,” outlines Associate Professor Paraskevi Mavrogiorgou from the RUB clinic for psychiatry, psychotherapy and preventive medicine.

Medical students have to learn how to do that, which isn’t always easy. “At the university clinic, we mainly see patients who are seriously ill,” says the physician. “It is difficult to motivate them to take part in training sessions with the students. Plus, they are not the kind of patients whom the future doctors will most likely be seeing in the course of their career.”

In order to enable students to practise with patients presenting the full range of symptoms and degrees of severity and to broaden the panorama, the clinic has launched a project focusing on creating avatars that feature typical symptoms of different mental disorders as well as psychiatric and psychotherapeutic treatment situations. Students wearing a virtual reality headset (VR headset) will be able to meet those avatars in a three-dimensional space – or, as experts call it, “exploration”.

The clinic has already established rooms and prepared them structurally. Here, small cubicles will be set up that students will enter with their VR headsets in order to talk with their virtual patients. “As far as we know, this has never been done anywhere yet; we are pioneering the field,” says Professor Dr. Georg Juckel, Medical Director at the clinic.

The interaction with the avatars will be taking place in small cubicles, in...
The interaction with the avatars will be taking place in small cubicles, in order to make students feel like they’re in a real interview setting.
Source: Roberto Schirdewahn

Difficulty: designing a dialogue flow

The preparations have taken two to three years; now, the virtual characters are being fleshed out. “They need a biography, a case history,” explains Paraskevi Mavrogiorgou. “And they must be able to answer specific key questions in accordance with their respective mental disorder.” If the attending doctor asks an avatar suffering from schizophrenia whether he hears voices, the avatar should say he does. If the attending doctor asks a question that doesn’t help find the correct diagnosis, the avatar mustn’t simply remain silent; rather, he has to give a plausible, evasive answer.

“At the moment, this is the greatest difficulty: designing a dialogue flow,” says Georg Juckel. “We have compiled hundreds of pages with instructions and dialogues,” points out Paraskevi Mavrogiorgou. For the dialogue to work, high-performance speech recognition software is necessary to ensure that the computer in the background understands the questions and can provide relevant answers.

In order to create avatars with plausible physical features, the medical professionals intend to collaborate with a commercial enterprise specialised in visualisation for medical applications – after Arthur Berns, young computer engineer from Göttingen, has designed the logical framework. The avatars’ body language is to be adapted to the type of mental disorders they represent. They might be sitting, standing, lying or pacing during the interview. Their facial expressions, too, are supposed to tell the trained observer something about the underlying disorder. “The programmer has to design each of these facial expressions individually,” as Georg Juckel elaborates the immense complexity of the task. “It’s the same as with Walt Disney’s animated movies, or with computer games today.”

The avatars are to be built up segmentally, “or else the whole thing would soon become quite boring for the students,” says Juckel. Thanks to the modular system, the difficulty level of the exploration can be regulated. The selected avatar represents a specific psychiatric diagnosis or treatment situation. However, the way students get there, the questions they ask, is not predefined: this is what they will be learning under guidance and supervision.

The project team expects that the avatars will be taking their first steps in spring 2020. In summer semester 2020, students might begin conducting the first interviews with them. “We’re hoping that this approach will catch on with the students – especially in view of today’s leisure activities,” concludes Paraskevi Mavrogiorgou. They are definitely curious to see what the virtual patients are all about.

Subscribe to our newsletter

Related articles

Virtual reality: Avatars against obesity

Virtual reality: Avatars against obesity

A collaborative project develops virtual reality methods to positively affect the body perception of obese patients.

Exposure therapy in VR to prevent disorders

Exposure therapy in VR to prevent disorders

VR can have significant impact on the validity of remote health appointments for those with eating disorders, through called Virtual Reality Exposure Therapy (VRET).

Virtual rehab for stroke patients

Virtual rehab for stroke patients

Researchers have been putting virtual reality rehabilitation for stroke survivors to the test.

VR trains public to reverse opioid overdoses

VR trains public to reverse opioid overdoses

Researchers developed a virtual reality immersive video aimed at training and motivating people to save lives from opioid overdoses.

VR training improves physical and cognitive functions

VR training improves physical and cognitive functions

Researchers have developed an innovative training protocol that, utilizing immersive virtual reality (IVR), leads to real physical and cognitive benefits.

Gamification: Intervention app may improve mental health

Gamification: Intervention app may improve mental health

A new randomized control trial has found that turning mobile mental health intervention into a smartphone game can potentially improve well-being.

Augmented Reality in the OR: matching man and machine

Augmented Reality in the OR: matching man and machine

One of the crucial future technologies in surgery is Augmented Reality. Most experts agree that AR will increase safety and efficiency, improve surgical training and decrease costs.

Closer threats inspire a more primitive kind of fear

Closer threats inspire a more primitive kind of fear

Using VR to make threats appear near or far is what makes it harder to extinguish the fear of a close-up threat and more likely that you’ll have some long-term stress from the experience.

Virtual reality takes the 'work' out of workouts

Virtual reality takes the 'work' out of workouts

If you want to exercise harder, enjoy it more and feel it less, pull on a VR headset and plug in some upbeat tunes.

Popular articles