The application of factorial surveys to study recruiters’ hiring intentions: comparing designs based on hypothetical and real vacancies

Gutfleisch, Tamara; Samuel, Robin; Sacchi, Stefan (2020). The application of factorial surveys to study recruiters’ hiring intentions: comparing designs based on hypothetical and real vacancies. Quality and quantity, 55(3), pp. 775-804. Springer Netherlands 10.1007/s11135-020-01012-7

Gutfleisch-2020.pdf - Published Version
Available under License Creative Commons: Attribution (CC-BY).

Download (793kB) | Preview

Factorial survey experiments have been widely used to study recruiters’ hiring intentions. Respondents are asked to evaluate hypothetical applicant descriptions, which are experimentally manipulated, for hypothetical job descriptions. However, this methodology has been criticized for putting respondents in hypothetical situations that often only partially correspond to real-life hiring situations. It has been proposed that this criticism can be overcome by sampling real-world vacancies and the recruiters responsible for filling them. In such an approach, only the applicants’ descriptions are hypothetical; respondents are asked about a real hiring problem, which might increase internal and external validity. In this study, we test whether using real vacancies triggers more valid judgments compared to designs based on hypothetical vacancies. The growing number of factorial survey experiments conducted in employer studies makes addressing this question relevant, both for methodological and practical reasons. However, despite the potential implications for the validity of data, it has been neglected so far. We conducted a factorial survey experiment in Luxembourg, in which respondents evaluated hypothetical applicants referring either to a currently vacant position in their company or to a hypothetical job. Overall, we found little evidence for differences in responses by the design of the survey experiment. However, the use of real vacancies might prove beneficial depending on the research interest. We hope that our comparison of designs using real and hypothetical vacancies contributes to the emerging methodological inquiry on the possibilities and limits of using factorial survey experiments in research on hiring.

Item Type:

Journal Article (Original Article)


03 Faculty of Business, Economics and Social Sciences > Social Sciences > Institute of Sociology

UniBE Contributor:

Sacchi, Stefan


300 Social sciences, sociology & anthropology
300 Social sciences, sociology & anthropology > 310 Statistics
300 Social sciences, sociology & anthropology > 330 Economics
300 Social sciences, sociology & anthropology > 370 Education




Springer Netherlands




Stefan Sacchi

Date Deposited:

14 Aug 2020 17:04

Last Modified:

16 May 2021 02:53

Publisher DOI:


Uncontrolled Keywords:

Factorial survey experiments Hiring intentions Real vacancies Hypothetical vacancies




Actions (login required)

Edit item Edit item
Provide Feedback