Evaluating Special Techniques for Surveying Sensitive Topics: An Approach that Detects False Positives

Höglinger, Marc; Diekmann, Andreas (22 August 2016). Evaluating Special Techniques for Surveying Sensitive Topics: An Approach that Detects False Positives (Unpublished). In: 111th Annual Meeting of the American Sociological Association - "Rethinking Social Movements: Can Changing the Conversation Change the World?". Seattle, WA. 20.-23. Aug. 2016.

Preview

Slideshow ASA20016-presentation.pdf - Presentation Available under License BORIS Standard License. Download (3MB) | Preview

The valid measurement of sensitive issues such as norm-violations or stigmatizing traits through self-reports in surveys is often problematic. Special sensitive question techniques such as the Randomized Response Technique (RRT, Warner 1965) and among its variants the recent Crosswise Model RRT (CM, Yu, Tian and Tang 2008) should generate more honest answers by providing full response privacy. However, the RRT sometimes does not work as expected and evaluating whether particular implementations actually improve data validity is essential before their application in substantive surveys. To this end, studies so far mostly compared prevalence estimates of sensitive traits and behaviors of different techniques. Assuming that respondents only falsely deny but never falsely admit a sensitive trait or behavior, higher prevalence estimates were interpreted as more valid estimates. However, if false positives occur, i.e., if respondents are misclassified as bearing a sensitive trait although they actually do not, conclusions drawn under this assumption are likely wrong. False positives occurred knowingly in one variant of the Crosswise Model RRT. Other RRT variants might be affected too, but studies so far have largely neglected this possibility and did not test for it. We show an evaluation approach that detects systematic false positives without the need of a validation criterion – which is often unavailable. Results from an application in a survey on “Organ donation and health” (N = 1’686) show, that the CM RRT produced false positives to a non-ignorable extent. This finding is in line with a previous validation study, but has not been revealed by several comparative evaluation studies that did not consider false positives.

Interest & Impact

Downloads

77 since deposited on 28 Jun 2017
4 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item