Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study.

Steppan, Martin; Zimmermann, Ronan; Fürer, Lukas; Southward, Matthew; Koenig, Julian; Kaess, Michael; Kleinbub, Johann Roland; Roth, Volker; Schmeck, Klaus (2024). Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study. Psychopathology, 57(3), pp. 159-168. Karger 10.1159/000534811

[img]
Preview
Text
000534811.pdf - Published Version
Available under License Creative Commons: Attribution-Noncommercial (CC-BY-NC).

Download (661kB) | Preview

BACKGROUND

New advances in the field of machine learning make it possible to track facial emotional expression with high resolution, including micro-expressions. These advances have promising applications for psychotherapy research, since manual coding (e.g., the Facial Action Coding System), is time-consuming.

PURPOSE

We tested whether this technology can reliably identify in-session emotional expression in a naturalistic treatment setting, and how these measures relate to the outcome of psychotherapy.

METHOD

We applied a machine learning emotion classifier to video material from 389 psychotherapy sessions of 23 patients with borderline personality pathology. We validated the findings with human ratings according to the Clients Emotional Arousal Scale (CEAS) and explored associations with treatment outcomes.

RESULTS

Overall, machine learning ratings showed significant agreement with human ratings. Machine learning emotion classifiers, particularly the display of positive emotions (smiling and happiness), showed medium effect size on median-split treatment outcome (d = 0.3) as well as continuous improvement (r = 0.49, p < 0.05). Patients who dropped out form psychotherapy, showed significantly more neutral expressions, and generally less social smiling, particularly at the beginning of psychotherapeutic sessions.

CONCLUSIONS

Machine learning classifiers are a highly promising resource for research in psychotherapy. The results highlight differential associations of displayed positive and negative feelings with treatment outcomes. Machine learning emotion recognition may be used for the early identification of drop-out risks and clinically relevant interactions in psychotherapy.

Item Type:

Journal Article (Original Article)

Division/Institute:

04 Faculty of Medicine > University Psychiatric Services > University Hospital of Child and Adolescent Psychiatry and Psychotherapy

UniBE Contributor:

Koenig, Julian, Kaess, Michael

Subjects:

600 Technology > 610 Medicine & health

ISSN:

1423-033X

Publisher:

Karger

Language:

English

Submitter:

Pubmed Import

Date Deposited:

29 Nov 2023 12:09

Last Modified:

05 Jun 2024 00:12

Publisher DOI:

10.1159/000534811

PubMed ID:

38011846

Uncontrolled Keywords:

Adolescents Borderline personality disorder Emotions Facial expressions classifiers Psychotherapy

BORIS DOI:

10.48350/189488

URI:

https://boris.unibe.ch/id/eprint/189488

Actions (login required)

Edit item Edit item
Provide Feedback