Advertisement

Detecting nonadherence without loss in efficiency: A simple extension of the crosswise model

  • Daniel W. Heck
  • Adrian Hoffmann
  • Morten Moshagen
Article

Abstract

In surveys concerning sensitive behavior or attitudes, respondents often do not answer truthfully, because of social desirability bias. To elicit more honest responding, the randomized-response (RR) technique aims at increasing perceived and actual anonymity by prompting respondents to answer with a randomly modified and thus uninformative response. In the crosswise model, as a particularly promising variant of the RR, this is achieved by adding a second, nonsensitive question and by prompting respondents to answer both questions jointly. Despite increased privacy protection and empirically higher prevalence estimates of socially undesirable behaviors, evidence also suggests that some respondents might still not adhere to the instructions, in turn leading to questionable results. Herein we propose an extension of the crosswise model (ECWM) that makes it possible to detect several types of response biases with adequate power in realistic sample sizes. Importantly, the ECWM allows for testing the validity of the model’s assumptions without any loss in statistical efficiency. Finally, we provide an empirical example supporting the usefulness of the ECWM.

Keywords

Randomized response Measurement model Sensitive questions Survey design Social desirability 

References

  1. Böckenholt, U., Barlas, S., & van der Heijden, P. G. M. (2009). Do randomized-response designs eliminate response biases? An empirical study of non-compliance behavior. Journal of Applied Econometrics, 24, 377–392. doi: https://doi.org/10.1002/Jae.1052 CrossRefGoogle Scholar
  2. Chang, H.-J., Wang, C.-L., & Huang, K.-C. (2004). Using randomized response to estimate the proportion and truthful reporting probability in a dichotomous finite population. Journal of Applied Statistics, 31, 565–573. doi: https://doi.org/10.1080/02664760410001681819 CrossRefGoogle Scholar
  3. Clark, S. J., & Desharnais, R. A. (1998). Honest answers to embarrassing questions: Detecting cheating in the randomized response model. Psychological Methods, 3, 160–168. doi: https://doi.org/10.1037/1082-989X.3.2.160 CrossRefGoogle Scholar
  4. Dawes, R. M., & Moore, M. (1980). Die Guttman-Skalierung orthodoxer und randomisierter Reaktionen [Guttman scaling of orthodox and randomized reactions]. In F. Petermann (Ed.), Einstellungsmessung, Einstellungsforschung [Attitude measurement, attitude research] (pp. 117–133). Göttingen: Hogrefe.Google Scholar
  5. Gupta, S., Gupta, B., & Singh, S. (2002). Estimation of sensitivity level of personal interview survey questions. Journal of Statistical Planning and Inference, 100, 239–247. doi: https://doi.org/10.1016/S0378-3758(01)00137-9 CrossRefGoogle Scholar
  6. Heck, D. W., & Moshagen, M. (in press). RRreg: An R package for correlation and regression analyses of randomized response data. Journal of Statistical Software.Google Scholar
  7. Hilbig, B. E., Moshagen, M., & Zettler, I. (2015). Truth will out: Linking personality, morality, and honesty through indirect questioning. Social Psychological and Personality Science, 6, 140–147. doi: https://doi.org/10.1177/1948550614553640 CrossRefGoogle Scholar
  8. Hoffmann, A., Diedenhofen, B., Verschuere, B. J., & Musch, J. (2015). A strong validation of the crosswise model using experimentally induced cheating behavior. Experimental Psychology, 62, 403–414. doi: https://doi.org/10.1027/1618-3169/a000304 CrossRefPubMedGoogle Scholar
  9. Hoffmann, A., & Musch, J. (2016). Assessing the validity of two indirect questioning techniques: A stochastic lie detector versus the crosswise model. Behavior Research Methods, 48, 1032–1046. doi: https://doi.org/10.3758/s13428-015-0628-6 CrossRefPubMedGoogle Scholar
  10. Hoffmann, A., Waubert de Puiseau, B., Schmidt, A. F., & Musch, J. (2017). On the comprehensibility and perceived privacy protection of indirect questioning techniques. Behavior Research Methods, 49, 1470–1483. doi: https://doi.org/10.3758/s13428-016-0804-3 CrossRefPubMedGoogle Scholar
  11. Höglinger, M., & Jann, B. (2016). More is not always better: An experimental individual-level validation of the randomized response technique and the crosswise model (University of Bern Social Sciences Working Paper 18). Retrieved from http://econpapers.repec.org/paper/bsswpaper/18.htm
  12. Höglinger, M., Jann, B., & Diekmann, A. (2016). Sensitive questions in online surveys: An experimental evaluation of different implementations of the randomized response technique and the crosswise model. Survey Research Methods, 10, 171–187.Google Scholar
  13. Jann, B., Jerke, J., & Krumpal, I. (2012). Asking sensitive questions using the crosswise model. Public Opinion Quarterly, 76, 32–49. doi: https://doi.org/10.1093/Poq/Nfr036 CrossRefGoogle Scholar
  14. Korndörfer, M., Krumpal, I., & Schmukle, S. C. (2014). Measuring and explaining tax evasion: Improving self-reports using the crosswise model. Journal of Economic Psychology, 45, 18–32. doi: https://doi.org/10.1016/j.joep.2014.08.001 CrossRefGoogle Scholar
  15. Krumpal, I. (2013). Determinants of social desirability bias in sensitive surveys: A literature review. Quality and Quantity, 47, 2025–2047. doi: https://doi.org/10.1007/s11135-011-9640-9 CrossRefGoogle Scholar
  16. Kundt, T. C., Misch, F., & Nerré, B. (2013). Re-assessing the merits of measuring tax evasions through surveys: Evidence from Serbian firms (ZEW Discussion Papers, No. 13-047). Retrieved Dec 12th, 2013, from http://hdl.handle.net/10419/78625
  17. Landsheer, J. A., van der Heijden, P. G. M., & van Gils, G. (1999). Trust and understanding, two psychological aspects of randomized response—A study of a method for improving the estimate of social security fraud. Quality and Quantity, 33, 1–12. doi: https://doi.org/10.1023/A:1004361819974 CrossRefGoogle Scholar
  18. Lensvelt-Mulders, G. J. L. M., Hox, J. J., van der Heijden, P. G. M., & Maas, C. J. M. (2005). Meta-analysis of randomized response research: Thirty-five years of validation. Sociological Methods and Research, 33, 319–348. doi: https://doi.org/10.1177/0049124104268664 CrossRefGoogle Scholar
  19. Mangat, N. S. (1994). An improved randomized-response strategy. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 56, 93–95.Google Scholar
  20. Moshagen, M., Hilbig, B. E., Erdfelder, E., & Moritz, A. (2014). An experimental validation method for questioning techniques that assess sensitive issues. Experimental Psychology, 61, 48–54. doi: https://doi.org/10.1027/1618-3169/a000226 CrossRefPubMedGoogle Scholar
  21. Moshagen, M., Hilbig, B. E., & Musch, J. (2011). Defection in the dark? A randomized-response investigation of cooperativeness in social dilemma games. European Journal of Social Psychology, 41, 638–644. doi: https://doi.org/10.1002/ejsp.793 CrossRefGoogle Scholar
  22. Moshagen, M., & Musch, J. (2012). Surveying multiple sensitive attributes using an extension of the randomized-response technique. International Journal of Public Opinion Research, 24, 508–523.CrossRefGoogle Scholar
  23. Moshagen, M., Musch, J., & Erdfelder, E. (2012). A stochastic lie detector. Behavior Research Methods, 44, 222–231. doi: https://doi.org/10.3758/s13428-011-0144-2 21858604 CrossRefPubMedGoogle Scholar
  24. Moshagen, M., Musch, J., Ostapczuk, M., & Zhao, Z. (2010). Reducing socially desirable responses in epidemiologic surveys: An extension of the randomized-response technique. Epidemiology, 21, 379–382. doi: https://doi.org/10.1097/Ede.0b013e3181d61dbc CrossRefPubMedGoogle Scholar
  25. Nakhaee, M. R., Pakravan, F., & Nakhaee, N. (2013). Prevalence of use of anabolic steroids by bodybuilders using three methods in a city of Iran. Addict Health, 5, 77–82.PubMedPubMedCentralGoogle Scholar
  26. Ostapczuk, M., Moshagen, M., Zhao, Z., & Musch, J. (2009). Assessing sensitive attributes using the randomized response technique: Evidence for the importance of response symmetry. Journal of Educational and Behavioral Statistics, 34, 267–287. doi: https://doi.org/10.3102/1076998609332747 CrossRefGoogle Scholar
  27. Ostapczuk, M., Musch, J., & Moshagen, M. (2009). A randomized-response investigation of the education effect in attitudes towards foreigners. European Journal of Social Psychology, 39, 920–931. doi: https://doi.org/10.1002/ejsp.588 CrossRefGoogle Scholar
  28. Ostapczuk, M., Musch, J., & Moshagen, M. (2011). Improving self-report measures of medication non-adherence using a cheating detection extension of the randomised-response-technique. Statistical Methods in Medical Research, 20, 489–503. doi: https://doi.org/10.1177/0962280210372843 CrossRefPubMedGoogle Scholar
  29. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (Vol. 1, pp. 17–59). San Diego: Academic Press.CrossRefGoogle Scholar
  30. Read, T. R., & Cressie, N. A. (1988). Goodness-of-fit statistics for discrete multivariate data. New York: Springer.CrossRefGoogle Scholar
  31. Schröter, H., Studzinski, B., Dietz, P., Ulrich, R., Striegel, H., & Simon, P. (2016). A comparison of the cheater detection and the unrelated question models: A randomized response survey on physical and cognitive doping in recreational triathletes. PLoS ONE, 11, e155765:1–11. doi: https://doi.org/10.1371/journal.pone.0155765 Google Scholar
  32. Thielmann, I., Heck, D. W., & Hilbig, B. E. (2016). Anonymity and incentives: An investigation of techniques to reduce socially desirable responding in the Trust Game. Judgment and Decision Making, 11, 527–536.Google Scholar
  33. Tian, G.-L., & Tang, M.-L. (2014). Incomplete categorical data design: Non-randomized response techniques for sensitive questions in surveys. Boca Raton: CRC Press, Taylor & Francis Group.Google Scholar
  34. Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859–883. doi: https://doi.org/10.1037/0033-2909.133.5.859 17723033 CrossRefPubMedGoogle Scholar
  35. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131. doi: https://doi.org/10.1126/science.185.4157.1124 CrossRefPubMedGoogle Scholar
  36. Ulrich, R., Schröter, H., Striegel, H., & Simon, P. (2012). Asking sensitive questions: A statistical power analysis of randomized response models. Psychological Methods, 17, 623–641. doi: https://doi.org/10.1037/A0029314 CrossRefPubMedGoogle Scholar
  37. van den Hout, A., Böckenholt, U., & van der Heijden, P. (2010). Estimating the prevalence of sensitive behaviour and cheating with a dual design for direct questioning and randomized response. Journal of the Royal Statistical Society, Series C: Applied Statistics, 59, 723–736. doi: https://doi.org/10.1111/j.1467-9876.2010.00720.x PubMedCentralGoogle Scholar
  38. Warner, S. L. (1965). Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60, 63–69.CrossRefPubMedGoogle Scholar
  39. Wolter, F., & Preisendörfer, P. (2013). Asking sensitive questions: An evaluation of the randomized response technique versus direct questioning using individual validation data. Sociological Methods & Research, 42, 321–353. doi: https://doi.org/10.1177/0049124113500474 CrossRefGoogle Scholar
  40. Yu, J.-W., Tian, G.-L., & Tang, M.-L. (2008). Two new models for survey sampling with sensitive characteristic: design and analysis. Metrika, 67, 251–263. doi: https://doi.org/10.1007/s00184-007-0131-x CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Daniel W. Heck
    • 1
  • Adrian Hoffmann
    • 2
  • Morten Moshagen
    • 3
  1. 1.Department of PsychologyUniversity of MannheimMannheimGermany
  2. 2.Department of Experimental PsychologyUniversity of DuesseldorfDuesseldorfGermany
  3. 3.Psychological Research MethodsUlm UniversityUlmGermany

Personalised recommendations