Panning for gold: Lessons learned from the platform-agnostic automated detection of political content in textual data

Makhortykh, Mykola; de León, Ernesto; Urman, Aleksandra; Christner, Clara; Sydorova, Maryna; Adam, Silke; Maier, Michaela; Gil-Lopez, Teresa (2022). Panning for gold: Lessons learned from the platform-agnostic automated detection of political content in textual data (Submitted) (arXiv). Cornell University 10.48550/arxiv.2207.00489

Preview

Text 2207.00489v1.pdf - Published Version Available under License Creative Commons: Attribution (CC-BY). Download (1MB) | Preview

The growing availability of data about online information behaviour enables new possibilities for political communication research. However, the volume and variety of these data makes them difficult to analyse and prompts the need for developing automated content approaches relying on a broad range of natural language processing techniques (e.g. machine learning- or neural network-based ones). In this paper, we discuss how these techniques can be used to detect political content across different platforms. Using three validation datasets, which include a variety of political and non-political textual documents from online platforms, we systematically compare the performance of three groups of detection techniques relying on dictionaries, supervised machine learning, or neural networks. We also examine the impact of different modes of data preprocessing (e.g. stemming and stopword removal) on the low-cost implementations of these techniques using a large set (n = 66) of detection models. Our results show the limited impact of preprocessing on model performance, with the best results for less noisy data being achieved by neural network- and machine-learning-based models, in contrast to the more robust performance of dictionary-based models on noisy data.

Interest & Impact

Downloads

9 since deposited on 24 Jun 2024
9 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item