Machine Learning Distinguishes Familiar from Unfamiliar Pairs of Subjects Performing an Eye Contact Task: A Systemic Physiology Augmented Functional Near-Infrared Spectroscopy Hyperscanning Study.

Guglielmini, S; Bopp, G; Marcar, V L; Scholkmann, Felix; Wolf, M (2022). Machine Learning Distinguishes Familiar from Unfamiliar Pairs of Subjects Performing an Eye Contact Task: A Systemic Physiology Augmented Functional Near-Infrared Spectroscopy Hyperscanning Study. In: Oxygen Transport to Tissue XLIII. Advances in Experimental Medicine and Biology: Vol. 1395 (pp. 177-182). Springer 10.1007/978-3-031-14190-4_30

Text 978-3-031-14190-4_30.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (410kB) | Request a copy

BACKGROUND

Eye contact is an important aspect of human communication and social interactions. Changes in brain and systemic physiological activity associated with interactions between humans can be measured with systemic physiology augmented functional near-infrared spectroscopy (SPA-fNIRS) hyperscanning, enabling inter-brain and inter-body synchronisation to be determined. In a previous study, we found that pairs of subjects that are socially connected show higher brain and body synchrony.

AIM

To enable a deeper understanding, our aim was to build and automatically detect the best set of features to distinguish between two different groups (familiar and unfamiliar pairs).

MATERIAL AND METHODS

We defined several features based on the Spearman correlation and wavelet transform coherence (WTC) of biosignals measured on 23 pairs of subjects (13 familiar and 10 unfamiliar pairs) during eye contact for 10 min. Additional custom features that identify the maximum brain-to-body coupling instants between pairs were generated.

RESULTS

After testing on combinations of different feature extraction methods, four subsets of features with the strongest discrimination power were taken into account to train a decision tree (DT) machine learning (ML) algorithm. We have obtained 95.65% classification accuracy using a leave-one-out cross-validation. The coupling features which represent the two maximum mean values resulting from the sum of 7 time-dependent WTC signals (oxyhaemoglobin concentration of the right prefrontal region, total haemoglobin concentration of the left and right prefrontal region, heart rate, electrodermal activity on the left and right wrist, and skin temperature on the right wrist) played an essential role in the classification accuracy.

CONCLUSION

Training the DT-ML algorithm with combined brain and systemic physiology data provided higher accuracy than training it only with brain or systemic data alone. The results demonstrate the power of the SPA-fNIRS hyperscanning approach and the potential in applying ML to investigate the strength of social bonds in a wide range of social interaction contexts.

Interest & Impact

Downloads

1 since deposited on 20 Dec 2022
1 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item