Is a single model enough? Lessons learned from systematically comparing automated classifications of populist radical right content in German

Makhortykh, Mykola; de León Williams, Ernesto Emiliano; Christner, Clara; Sydorova, Maryna; Urman, Aleksandra; Adam, Silke; Maier, Michaela; Gil-Lopez, Teresa (2022). Is a single model enough? Lessons learned from systematically comparing automated classifications of populist radical right content in German (Unpublished). In: ECPR General Conference. Innsbruck. 22-26 August 2022.

Full text not available from this repository. (Request a copy)

The rise of populist radical right (PRR) ideas stresses the importance of understanding how individuals are exposed to and engage with PRR content in high-choice information environments. However, this task is complicated by the multitude of channels via which such exposure can take place. This prompts the need for developing automated approaches for identifying PRR content. In this paper, we share insights from our experience of developing automated classifiers for differentiating between PRR and non-PRR textual content in the German language. By training and comparing 66 dictionary-, supervised machine learning-, deep learning-, and transformer-based classification models, we offer a systematic comparison of their performance on three validation sets of PRR textual items and examine the impact of different pre-processing steps (e.g., stemming and lemmatization) on models’ performance. We also discuss the use of synthetic models (i.e., combining individual classification models in the ensemble form) for PRR classification based on a comparison of 396 model combinations. Our findings demonstrate that transformer- and supervised machine learning-based models show the best performance on average among the individual models and it can further be improved using synthetic models which combine supervised machine learning- and dictionary-based approaches.

Item Type:	Conference or Workshop Item (Paper)
Division/Institute:	03 Faculty of Business, Economics and Social Sciences > Social Sciences > Institute of Communication and Media Studies (ICMB)
UniBE Contributor:	Makhortykh, Mykola, de León Williams, Ernesto Emiliano, Sydorova, Maryna, Urman, Aleksandra, Adam, Silke
Subjects:	000 Computer science, knowledge & systems 300 Social sciences, sociology & anthropology 300 Social sciences, sociology & anthropology > 320 Political science
Language:	English
Submitter:	Mykola Makhortykh
Date Deposited:	17 Aug 2022 08:18
Last Modified:	05 Dec 2022 16:22
Uncontrolled Keywords:	methodology, neural networks, machine learning, automated text classification, populism, radical right
URI:	https://boris.unibe.ch/id/eprint/171970