Advertisement

Quality & Quantity

, Volume 46, Issue 5, pp 1601–1616 | Cite as

Asking sensitive questions: the impact of forgiving wording and question context on social desirability bias

  • Anatol-Fiete Näher
  • Ivar KrumpalEmail author
Article

Abstract

Sensitive questions are prone to systematic measurement error due to the respondents’ social desirability concerns. Literature on empirical social research often recommends either positive “loading” of sensitive questions, e.g. using “forgiving” wording, or manipulating the question context to reduce social desirability bias. We derive theoretical explanations of how manipulations of question wording and context could elicit more socially undesirable answers in sensitive surveys. In an experimental online survey (N = 1,176), we evaluate the effects of (1) forgiving wording and (2) question context on social desirability bias in different sensitive questions. The empirical evidence on the assumed bias-reducing effects shows inconsistent results. It is indicated however, that the perceived social norm has the strongest and most consistent effect on the respondents’ propensity to self-report socially undesirable behavior.

Keywords

Social desirability Sensitive questions Context effects 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aquilino W.S., Wright D.L., Supple A.J.: Response effects due to bystander presence in casi and paper-and-pencil surveys of drug use and alcohol use. Subst. Use Misuse 35, 845–867 (2000)CrossRefGoogle Scholar
  2. Aronson E.: Dissonance, hypocrisy and the self-concept. In: Harmon-Jones, E., Mills, J. (eds) Cognitive Dissonance: Progress on a Pivotal Theory in Social Psychology, pp. 103–106. APA, Washington (1999)CrossRefGoogle Scholar
  3. Aronson E., Fried C.B., Stone J.: Overcoming denial and increasing the intention to use condoms through the induction of hypocrisy. Am. J. Public Health 81, 1636–1638 (1991)CrossRefGoogle Scholar
  4. Barton A.H.: Asking the embarrassing question. Public Opin. Q. 22, 67–68 (1958)CrossRefGoogle Scholar
  5. Becker R.: Selective response to questions on delinquency. Qual. Quant. 40, 483–498 (2006)CrossRefGoogle Scholar
  6. Becker R., Günther R.: Selektives Antwortverhalten bei Fragen zum delinquenten Handeln. Eine empirische Studie über die Wirksamkeit der “sealed envelope technique” bei selbstberichteter Delinquenz mit Daten des ALLBUS 2000. ZUMA-Nachrichten 54, 39–59 (2004)Google Scholar
  7. Belli R.F., Traugott M.W., Beckmann M.N.: What leads to voting overreports? Contrasts of overreporters to validated votes and admitted nonvoters in the American national election studies. J. Off. Stat. 21, 287–308 (2001)Google Scholar
  8. Belli R.F., Moore S.E., VanHoewyk J.: An experimental comparison of question forms used to reduce vote overreporting. Elect. Stud. 25, 751–759 (2006)CrossRefGoogle Scholar
  9. Beuer-Krüssel M., Krumpal I.: Der Einfluss von Häufigkeitsformaten auf die Messung von subjektiven Wahrscheinlichkeiten. Methoden, Daten, Analysen: Zeitschrift für empirische Sozialforschung 3, 31–58 (2009)Google Scholar
  10. Beyer H., Krumpal I.: “Aber es gibt keine Antisemiten mehr”: Eine experimentelle Studie zur Kommunikationslatenz antisemitischer Einstellungen. Kölner Zeitschrift für Soziologie und Sozialpsychologie 62, 681–705 (2010)CrossRefGoogle Scholar
  11. Blair E., Sudman S., Bradburn N.M., Stocking C.: How to ask questions about drinking and sex: response effects in measuring consumer behavior. J. Mark. Res. 14, 316–321 (1977)CrossRefGoogle Scholar
  12. Cantril H., Wilks S.S.: Problems and techniques. Public Opin. Q. 4, 330–338 (1940)CrossRefGoogle Scholar
  13. Catania J.A., Binson D., Canchola J., Pollack L.M., Hauck W.: Effects of interviewer gender, interviewer choice, and item wording on responses to questions concerning sexual behavior. Public Opin. Q. 60, 345–375 (1996)CrossRefGoogle Scholar
  14. Coutts E., Jann B.: Sensitive questions in online surveys: experimental results for the Randomized Response Technique (RRT) and the Unmatched Count Technique (UCT) Working paper. Working paper, ETH Zürich (2008)Google Scholar
  15. Coutts E., Jann B.: Sensitive questions in online surveys: experimental results for the Randomized Response Technique (RRT) and the Unmatched Count Technique (UCT). Sociol. Methods Res. 40, 169–193 (2011)CrossRefGoogle Scholar
  16. Crowne D.P., Marlowe D.: A new scale of social desirability independent of psychopathology. J. Consult. Psychol. 24, 349–354 (1960)CrossRefGoogle Scholar
  17. Diekmann A.: Empirische Sozialforschung. Grundlagen, Methoden, Anwendungen. Rowohlt Taschenbuch, Reinbek (2003)Google Scholar
  18. Edwards A.L.: The relationship between the judged desirability of a trait and the probability that the trait will be endorsed. J. Appl. Psychol. 2, 90–93 (1953)CrossRefGoogle Scholar
  19. Festinger L.: A Theory of Cognitive Dissonance. Stanford University Press, Stanford (1957)Google Scholar
  20. Fointiat V.: Being together in a situation of induced hypocrisy. Curr. Res. Soc. Psychol. 13, 145–153 (2008)Google Scholar
  21. Fowler F.J.: Improving Survey Questions. Design and Evaluation. Sage Publications, London (1995)Google Scholar
  22. Frey D., Gaska A.: Die Theorie der kognitiven Dissonanz. In: Frey, D., Irle, M. (eds) Theorien der Sozialpsychologie, pp. 274–324. Hans Huber, Bern (1993)Google Scholar
  23. Fu, H., Darroch, J.E., Henshaw, S.K., Kolb, E.: Measuring the extent of abortion underreporting in the 1995 National Survey of Family Growth. Fam. Plan. Perspect. 30, 128–133, 138 (1995)Google Scholar
  24. Groves R.M., Fowler F.J., Couper M.P., Lepkowski J.M., Singer E., Tourangeau R.: Survey Methodology. Wiley, Hoboken (2004)Google Scholar
  25. Hartmann P.: Response behavior in interview settings of limited privacy. Int. J. Public Opin. Res. 7, 383–390 (1995)CrossRefGoogle Scholar
  26. Holtgraves T., Eck J., Lasky B.: Face management, question wording and social desirability. J. Appl. Soc. Psychol. 27, 1650–1671 (1997)CrossRefGoogle Scholar
  27. Huang C., Liao H., Chang S.-H.: Social desirability and the clinical self-report inventory: methodological reconsideration. J. Clin. Psychol. 54, 517–528 (1998)CrossRefGoogle Scholar
  28. Jann, B., Jerke, J., Krumpal, I.: Asking sensitive questions using the crosswise model: an experimental survey measuring plagiarism. Public Opin. Q. (forthcoming)Google Scholar
  29. Johnson T.J., van de ijver F.J.: Social desirability in cross-cultural research. In: Harness, J., van de ijver, F.J., Mohler, P. (eds) Cross-Cultural Survey Methods, pp. 193–202. Wiley, New York (2002)Google Scholar
  30. Krumpal, I.: Evaluation of the effectiveness of the Randomized Response Technique and the Item Count Method in the telephone survey mode. In: Balbi, S., Scepi, G., Russolillo, G., Stawinoga, A. (eds.) Proceedings of the 7th International Conference on Social Science Methodology, RC33—Logic and Methodology in Sociology (ISA). Jovene Editore, Napoli (2008)Google Scholar
  31. Krumpal, I., Rauhut, H., Böhr, D., Naumann. E.: The framing of risks and the communication of subjective probabilities for victimizations. Qual. Quant. (forthcoming)Google Scholar
  32. Nicotera A.M.: An assessment of the argumentativeness scale for social desirability bias. Commun. Rep. 9, 23–26 (1996)CrossRefGoogle Scholar
  33. Opp K.-D.: Norms. In: Baltes, P.B., Smelser, N.J. (eds) International Encyclopedia of Social and Behavioral Sciences, pp. 10714–10720. Elsevier, Amsterdam (2001a)CrossRefGoogle Scholar
  34. Opp K.-D.: Social networks and the emergence of protest norms. In: Hechter, M., Opp, K.-D. (eds) Social Norms, pp. 234–273. Russell Sage Foundation, New York (2001b)Google Scholar
  35. Paulhus D.L.: Two-component models of socially desirable responding. J. Personal. Soc. Psychol. 46, 598–609 (1984)CrossRefGoogle Scholar
  36. Paulhus D.L.: Socially desirable responding: the evolution of a construct. In: Braun, H.I., Jackson, D.N., Wiley, D.E. (eds) The Role of Constructs in Psychological and Educational Measurement, pp. 49–69. Erlbaum, Mahwah (2002)Google Scholar
  37. Preisendörfer, P.: Heikle Fragen in mündlichen Interviews: Ergebnisse einer Methodenstudie im studentischen Milieu. Johannes Gutenberg-University, Mainz. Institute for Sociology. Working Paper (2008)Google Scholar
  38. Presser S.: Can changes in context reduce vote overreporting in surveys?. Public Opin. Q. 54, 586–593 (1990)CrossRefGoogle Scholar
  39. Rauhut H., Krumpal I.: Die Durchsetzung sozialer Normen in Low-Cost- und High-Cost-Situationen. Z. für Soziol. 37, 380–402 (2008)Google Scholar
  40. Reuband K.H.: Unerwünschte Dritte beim Interview: Erscheinungsformen und Folgen. Z. für Soziol. 16, 303–308 (1987)Google Scholar
  41. Reuband K.H.: On 3rd persons in the interview situation and their impact on responses. Int. J. Public Opin. Res. 4, 269–274 (1992)CrossRefGoogle Scholar
  42. Schwarz N., Bless H.: Scandals and the public trust in politicians: assimilation and contrast effects. Personal. Soc. Psychol. Bull. 18, 574–579 (1992)CrossRefGoogle Scholar
  43. Schwarz N., Bless H.: Mental construal processes: the inclusion/exclusion model. In: Stapel, D.A., Suls, J. (eds) Assimilation and Contrast in Social Psychology, pp. 119–141. Psychology Press, Philadelphia (2007)Google Scholar
  44. Stapel D.A., Koomen W.: Impact of interpretation versus comparison mindsets on knowledge accessibility effects. J. Exp. Soc. Psychol. 37, 134–149 (2001)CrossRefGoogle Scholar
  45. Stapel D.A., Koomen W., van der ligt J.: Categories of category accessibility: the impact of trait concept versus exemplar priming on person judgements. J. Exp. Soc. Psychol. 33, 47–76 (1998)CrossRefGoogle Scholar
  46. Stocké V.: Deutsche Kurzskala des Bedürfnisses nach sozialer Anerkennung. In: Glöckner-Rist, A. (eds) Elektronisches Handbuch sozialwissenschaftlicher Erhebungsinstrumente. ZIS-Version 11.0, GESIS, Bonn (2007)Google Scholar
  47. Stocké V., Hunkler C.: Measures of desirability beliefs and their validity as indicators for socially desirable responding. Field Methods 19, 313–336 (2007)CrossRefGoogle Scholar
  48. Stone J., Wiegand A.W., Cooper J., Aronson E.: When exemplification fails: hypocrisy and the motive for self-integrity. J. Personal. Soc. Psychol. 72, 54–65 (1997)CrossRefGoogle Scholar
  49. Strack F., Schwarz N., Wänke M.: Semantic and pragmatic aspects of context effects in social and psychological research. Soc. Cogn. 9, 111–125 (1991)CrossRefGoogle Scholar
  50. Sudman S., Bradburn N.M.: Asking Questions: A Practical Guide to Questionnaire Design. Jossey-Bass, San Francisco (1982)Google Scholar
  51. Sudman S., Bradburn N.M., Schwarz N.: Thinking about Answers. The Application of Cognitive Processes to Survey Methodology. Jossey-Bass, San Francisco (1996)Google Scholar
  52. Tourangeau R., Smith T.W.: Asking sensitive questions: the impact of data collection mode, question format, and question context. Public Opin. Q. 60, 275–304 (1996)CrossRefGoogle Scholar
  53. Tourangeau R., Yan T.: Sensitive questions in surveys. Psychol. Bull. 133, 859–883 (2007)CrossRefGoogle Scholar
  54. Tourangeau R., Rasinski K., Bradburn N.M., D’Andrade R.: Carry-over effects in attitude surveys. Public Opin. Q. 53, 495–524 (1989)CrossRefGoogle Scholar
  55. Tourangeau R., Rips L.J., Rasinski K.A.: The Psychology of Survey Response. Cambridge University Press, Cambridge (2000)CrossRefGoogle Scholar
  56. Traugott M.P., Katosh J.P.: Response validity in surveys of voting behavior. Public Opin. Q. 43, 359–377 (1979)CrossRefGoogle Scholar
  57. Voss T.: Game-theoretical perspectives on the emergence of social norms. In: Hechter, M., Opp, K.-D. (eds) Social Norms, pp. 105–136. Russell Sage Foundation, New York (2001)Google Scholar
  58. Wolter, F.: Sensitive questions in surveys: an evaluation of the Randomized Response Technique in face-to-face-interviews. Johannes Gutenberg University, Mainz. Institute for Sociology. Working Paper (2008)Google Scholar
  59. Wyner, G.A.: Response errors in self-reported number of arrests. Sociol. Methods Res. 9: 161–177 (1980)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.Department of SociologyUniversity of LeipzigLeipzigGermany

Personalised recommendations