Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study.

Steppan, Martin; Zimmermann, Ronan; Fürer, Lukas; Southward, Matthew; Koenig, Julian; Kaess, Michael; Kleinbub, Johann Roland; Roth, Volker; Schmeck, Klaus (2024). Machine Learning Facial Emotion Classifiers in Psychotherapy Research: A Proof-of-Concept Study. Psychopathology, 57(3), pp. 159-168. Karger 10.1159/000534811

Preview

Text 000534811.pdf - Published Version Available under License Creative Commons: Attribution-Noncommercial (CC-BY-NC). Download (661kB) | Preview

BACKGROUND

New advances in the field of machine learning make it possible to track facial emotional expression with high resolution, including micro-expressions. These advances have promising applications for psychotherapy research, since manual coding (e.g., the Facial Action Coding System), is time-consuming.

PURPOSE

We tested whether this technology can reliably identify in-session emotional expression in a naturalistic treatment setting, and how these measures relate to the outcome of psychotherapy.

METHOD

We applied a machine learning emotion classifier to video material from 389 psychotherapy sessions of 23 patients with borderline personality pathology. We validated the findings with human ratings according to the Clients Emotional Arousal Scale (CEAS) and explored associations with treatment outcomes.

RESULTS

Overall, machine learning ratings showed significant agreement with human ratings. Machine learning emotion classifiers, particularly the display of positive emotions (smiling and happiness), showed medium effect size on median-split treatment outcome (d = 0.3) as well as continuous improvement (r = 0.49, p < 0.05). Patients who dropped out form psychotherapy, showed significantly more neutral expressions, and generally less social smiling, particularly at the beginning of psychotherapeutic sessions.

CONCLUSIONS

Machine learning classifiers are a highly promising resource for research in psychotherapy. The results highlight differential associations of displayed positive and negative feelings with treatment outcomes. Machine learning emotion recognition may be used for the early identification of drop-out risks and clinically relevant interactions in psychotherapy.

Interest & Impact

Downloads

69 since deposited on 29 Nov 2023
69 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item