Auditing the effect of search personalisation on the visibility of COVID- and Holocaust-related misinformation on Google in Switzerland

Makhortykh, Mykola; Adam, Silke; Keller, Franziska; Sydorova, Maryna; Urman, Aleksandra (6 September 2023). Auditing the effect of search personalisation on the visibility of COVID- and Holocaust-related misinformation on Google in Switzerland (Unpublished). In: ECPR 2023. 4-8 September 2023.

Full text not available from this repository. (Request a copy)

Web search engines, such as Google, are important components of today’s digital ecosystem. Intensively used and highly trusted by individuals (e.g. Schultheiß & Lewandowski, 2021), search engines help their users tackle information overload by ranking content in response to user queries. This process involves prioritising specific interpretations of societally relevant phenomena, ranging from racial or gender stereotypes (Noble, 2018; Urman & Makhortykh, 2022) to elections (Unkel & Haim, 2021; Trielli & Diakopoulos, 2022), and determining which of these interpretations individuals are exposed to.

Despite the importance of search engines for understanding political communication phenomena, such as selective information exposure (Slechten et al., 2022) or agenda-setting (Ragas et al., 2014), studying them remains a challenging task. The selection and quality of search outputs are influenced by a broad range of user-side - e.g., how individuals search for information - and system-side factors - e.g., whether search results are personalised or randomised. Furthermore, the degree to which these factors influence search outputs can vary depending on the subject and the design logic behind a particular search engine.

In this paper, we investigate how search personalisation influences the visibility of COVID- and Holocaust-related misinformation on Google, which is a monopolist on the Swiss search engine market. Using agent-based algorithm audits (e.g., Ulloa et al., 2022), we simulate the browsing activity of a large number of Swiss Internet users across three periods of time in the summer of 2022 to examine how Google retrieves information in response to Germanophone queries dealing with recent and more established types of misinformation. Specifically, we compare outputs for queries implying interest in specific misinformation narratives (e.g. whether gas chambers in Holocaust camps are fake or COVID is harmless) and queries inquiring about similar types of information, but without an explicit interest in their conspiratorial aspects (e.g. information about Holocaust concentration camps or health impacts of COVID).

Building on the concept of search personas - i.e. sequences of browsing actions (Haim et al., 2018) - we investigate how earlier history of visits to websites with different political leanings (e.g. websites of right- and left-leaning Swiss political parties and journalistic media) affects the visibility of misinformation in search results. To analyse data, we extracted all organic search results from the first page of Google search in response to the above-mentioned queries and coded them to identify the type of source from which the result is coming, its stance on disinformation (e.g. debunking, mentioning, or promoting), and the exact type of disinformation which the result is related to. Our preliminary findings demonstrate the limited impact of search personalisation with only minor differences between results for specific search personas. At the same time, we observed substantial variation in the selection of sources between Holocaust- and COVID-related queries as well as the tendency of Google to prioritise journalistic sources in response to conspiratorial queries. We also identified higher visibility of outputs promoting conspiracies in response to the COVID-related queries which is concerning considering that these outputs appear in the top 10 search results.

Item Type:	Conference or Workshop Item (Paper)
Division/Institute:	03 Faculty of Business, Economics and Social Sciences > Social Sciences > Institute of Communication and Media Studies (ICMB)
UniBE Contributor:	Makhortykh, Mykola, Adam, Silke, Keller, Barbara Franziska, Sydorova, Maryna, Urman, Aleksandra
Subjects:	000 Computer science, knowledge & systems 300 Social sciences, sociology & anthropology 900 History 900 History > 940 History of Europe
Language:	English
Submitter:	Mykola Makhortykh
Date Deposited:	02 Oct 2023 09:42
Last Modified:	02 Oct 2023 09:42
Uncontrolled Keywords:	algorithm, audit, AI, Holocaust, COVID, misinformation, Google, web search, search engine
URI:	https://boris.unibe.ch/id/eprint/186814