Advertisement

Quality & Quantity

, Volume 51, Issue 2, pp 941–958 | Cite as

Comparing acquiescent and extreme response styles in face-to-face and web surveys

  • Mingnan LiuEmail author
  • Frederick G. Conrad
  • Sunghee Lee
Article
First Online:

Abstract

Likert scales are popular for measuring attitudes, but response style, a source of measurement error associated with this type of question, can result in measurement bias. This study investigates the effect of data collection mode on both types of response styles using data from the 2012 American National Election Studies (ANES). 2012 was the 1 year in which ANES conducted two parallel surveys, one through face-to-face interviews and another through Web, using two independent national probability samples and an identical questionnaire. We used three sets of balanced Likert scales from the survey to measure the acquiescent and extreme response styles. Using the latent class analysis modeling approach, we find that: (1) both acquiescent and extreme response styles exist in both face-to-face and Web survey modes; (2) face-to-face respondents demonstrate more acquiescent and extreme response styles than Web respondents; (3) the mode effect is similar for white, black and Hispanic respondents.

Keywords

Response style Face-to-face survey Web survey Latent class analysis 
This is a preview of subscription content, log in to check access.

References

  1. Aday, L.A., Chiu, G.Y., Andersen, R.: Methodological issues in health care surveys of the Spanish heritage population. Am. J. Public Health 70(4), 367–374 (1980)CrossRefGoogle Scholar
  2. Bachman, J.G., O’Malley, P.M.: Black-white differences in self-esteem: are they affected by response styles? Am. J. Sociol. 90(3), 624–639 (1984a)CrossRefGoogle Scholar
  3. Bachman, J.G., O’Malley, P.M.: Yea-saying, nay-saying, and going to extremes: black-white differences in response styles. Public Opin. Q. 48(2), 491–509 (1984b)CrossRefGoogle Scholar
  4. Bachman, J. G., O’Malley, P. M.: Response styles revisited: racial/ethnic and gender differences in extreme responding. Retrieved from http://monitoringthefuture.org/pubs/occpapers/occ72.pdf (2010)
  5. Baumgartner, H., Steenkamp, J.-B.E.M.: Response styles in marketing research: a cross-national investigation. J. Mark. Res. 38(2), 143–156 (2001)CrossRefGoogle Scholar
  6. Billiet, J.B., Davidov, E.: Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociol. Methods Res. 36(4), 542–562 (2008). doi: 10.1177/0049124107313901 CrossRefGoogle Scholar
  7. Billiet, J.B., McClendon, M.J.: Modeling acquiescence in measurement models for two balanced sets of items. Struct. Equ. Model. 7(4), 608–628 (2000). doi: 10.1207/S15328007SEM0704_5 CrossRefGoogle Scholar
  8. Bolt, D.M., Newton, J.R.: Multiscale measurement of extreme response style. Educ. Psychol. Meas. 71(5), 814–833 (2011). doi: 10.1177/0013164410388411 CrossRefGoogle Scholar
  9. Carr, L.G.: The srole items and acquiescence. Am. Sociol. Rev. 36(2), 287–293 (1971)CrossRefGoogle Scholar
  10. Clarke III, I.: Extreme response style in cross-cultural research. Int. Mark. Rev. 18(3), 301–324 (2001)CrossRefGoogle Scholar
  11. Couch, A., Keniston, K.: Yeasayers and naysayers: agreeing response set as a personality variable. J. Abnorm. Soc. Psychol. 60(2), 151–174 (1960)CrossRefGoogle Scholar
  12. Cronbach, L.J.: Studies of acquiescence as a factor in the true-false test. J. Educ. Psychol. 33(6), 401–415 (1942)CrossRefGoogle Scholar
  13. Davis, R.E., Resnicow, K., Couper, M.P.: survey response styles, acculturation, and culture among a sample of mexican american adults. J. Cross-Cult. Psychol. 42(7), 1219–1236 (2010). doi: 10.1177/0022022110383317 CrossRefGoogle Scholar
  14. De Leeuw, E.D.: Data quality in mail, telephone and face to face surveys. ERIC. Retrieved from http://eric.ed.gov/?id=ED374136 (1992)
  15. De Leeuw, E.D.: To mix or not to mix data collection modes in surveys. J. Off. Stat. 21(5), 233–255 (2005)Google Scholar
  16. de Spinoza, B., Shirley, S., Feldman, S.: The Ethics and Selected Letters. Hackett Pub. Co, Indianapolis (1982)Google Scholar
  17. Gibbons, J.L., Zellner, J.A., Rudek, D.J.: Effects of language and meaningfulness on the use of extreme response style by Spanish–English bilinguals. Cross-Cult. Res. 33(4), 369–381 (1999)CrossRefGoogle Scholar
  18. Gilbert, D.T.: How mental systems believe. Am. Psychol. 46(2), 107–119 (1991)CrossRefGoogle Scholar
  19. Heerwegh, D., Loosveldt, G.: Face-to-face versus web surveying in a high-internet-coverage population differences in response quality. Public Opin. Q. 72(5), 836–846 (2008). doi: 10.1093/poq/nfn045 CrossRefGoogle Scholar
  20. Heerwegh, D., Loosveldt, G.: Assessing mode effects in a national crime victimization survey using structural equation models: social desirability bias and acquiescence. J. Off. Stat. 27(1), 49 (2011)Google Scholar
  21. Hui, C.H., Triandis, H.C.: Effects of culture and response format on extreme response style. J. Cross-Cult. Psychol. 20(3), 296–309 (1989). doi: 10.1177/0022022189203004 CrossRefGoogle Scholar
  22. Jordan, L.A., Marcus, A.C., Reeder, L.G.: Response styles in telephone and household interviewing: a field experiment. Public Opin. Q. 44(2), 210–222 (1980)CrossRefGoogle Scholar
  23. Kankaraš, M., Moors, G.: Measurement equivalence in solidarity attitudes in Europe insights from a multiple-group latent-class factor approach. Int. Sociol. 24(4), 557–579 (2009). doi: 10.1177/0268580909334502 CrossRefGoogle Scholar
  24. Kieruj, N.D., Moors, G.: variations in response style behavior by response scale format in attitude research. Int. J. Public Opin. Res. 22(3), 320–342 (2010). doi: 10.1093/ijpor/edq001 CrossRefGoogle Scholar
  25. Kieruj, N.D., Moors, G.: Response style behavior: question format dependent or personal style? Qual. Quant. 47(1), 193–211 (2013). doi: 10.1007/s11135-011-9511-4 CrossRefGoogle Scholar
  26. Kiesler, S., Sproull, L.S.: Response effects in the electronic survey. Public Opin. Q. 50(3), 402–413 (1986). doi: 10.1086/268992 CrossRefGoogle Scholar
  27. Knowles, E.S., Condon, C.A.: Why people say yes: a dual-process theory of acquiescence. J. Pers. Soc. Psychol. 77(2), 379–386 (1999)CrossRefGoogle Scholar
  28. Krosnick, J.A.: Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl. Cogn. Psychol. 5(3), 213–236 (1991)CrossRefGoogle Scholar
  29. Leary, M.R., Kowalski, R.M.: Impression management: a literature review and two-component model. Psychol. Bull. 107(1), 34–47 (1990)CrossRefGoogle Scholar
  30. Madsen, W.: Mexican-Americans of South Texas. Retrieved from http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=ED021661 (1964)
  31. Marin, G., Gamba, R.J., Marin, B.V.: extreme response style and acquiescence among hispanics the role of acculturation and education. J. Cross-Cult. Psychol. 23(4), 498–509 (1992). doi: 10.1177/0022022192234006 CrossRefGoogle Scholar
  32. Marin, G., Marin, B.V.: Research with Hispanic Populations. 2455 Teller Road, Newbury Park California 91320 United States of America: SAGE Publications, Inc. Retrieved from http://srmo.sagepub.com/view/research-with-hispanic-populations/SAGE.xml (1991)
  33. Moors, G.: Diagnosing response style behavior by means of a latent-class factor approach. Socio-demographic correlates of gender role attitudes and perceptions of ethnic discrimination reexamined. Qual. Quant. 37(3), 277–302 (2003). doi: 10.1023/A:1024472110002 CrossRefGoogle Scholar
  34. Moors, G.: Exploring the effect of a middle response category on response style in attitude measurement. Qual. Quant. 42(6), 779–794 (2008). doi: 10.1007/s11135-006-9067-x CrossRefGoogle Scholar
  35. Moors, G.: ranking the ratings: a latent-class regression model to control for overall agreement in opinion research. Int. J. Public Opin. Res. 22(1), 93–119 (2010). doi: 10.1093/ijpor/edp036 CrossRefGoogle Scholar
  36. Moors, G., Kieruj, N.D., Vermunt, J.K.: The effect of labeling and numbering of response scales on the likelihood of response bias. Sociol. Methodol. (2014). doi: 10.1177/0081175013516114 Google Scholar
  37. Morren, M., Gelissen, J.P.T.M., Vermunt, J.K.: dealing with extreme response style in cross-cultural research: a restricted latent class factor analysis approach: extreme response style in cross-cultural research. Sociol. Methodol. 41(1), 13–47 (2011). doi: 10.1111/j.1467-9531.2011.01238.x CrossRefGoogle Scholar
  38. Naemi, B.D., Beal, D.J., Payne, S.C.: Personality predictors of extreme response style. J. Personal. 77(1), 261–286 (2009). doi: 10.1111/j.1467-6494.2008.00545.x CrossRefGoogle Scholar
  39. Paulhus, D. L.: Measurement and control of response bias. Measures of Personality and Social Psychological Attitudes. Measures of Social Psychological Attitudes, Vol. 1. Retrieved from http://doi.apa.org/psycinfo/1991-97206-001 (1991)
  40. Ross, C.E., Mirowsky, J.: Socially-desirable response and acquiescence in a cross-cultural survey of mental health. J. Health Soc. Behav. 25(2), 189–197 (1984)CrossRefGoogle Scholar
  41. Saris, W.E., Revilla, M., Krosnick, J.A., Shaeffer, E.M.: Comparing questions with agree/disagree response options to questions with item-specific response options. Surv. Res. Methods 4(1), 61–79 (2010)Google Scholar
  42. Schlenker, B. R.: Impression Management: the Self-Concept, Social Identity, and Interpersonal Relations. Brooks/Cole Publishing Company, Monterey. Retrieved from http://www.getcited.org/pub/102061888 (1980)
  43. Thomas, T.D., Abts, K., Weyden, P.V.: Response styles and the rural-urban divide. Educ. Psychol. Meas. 74(1), 97–115 (2014). doi: 10.1177/0013164413502487 CrossRefGoogle Scholar
  44. Vannieuwenhuyze, J.T.A., Revilla, M.: Relative mode effects on data quality in mixed-mode surveys by an instrumental variable. Surv. Res. Methods 7(3), 157–168 (2013). doi: 10.18148/srm/2013.v7i3.5137 Google Scholar
  45. Van Vaerenbergh, Y., Thomas, T.D.: Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int. J. Public Opin. Res. 25(2), 195–217 (2013). doi: 10.1093/ijpor/eds021 CrossRefGoogle Scholar
  46. Warnecke, R.B., Johnson, T.P., Chávez, N., Sudman, S., O’rourke, D.P., Lacey, L., Horm, J.: Improving question wording in surveys of culturally diverse populations. Ann. Epidemiol. 7(5), 334–342 (1997)Google Scholar
  47. Weijters, B., Geuens, M., Schillewaert, N.: The individual consistency of acquiescence and extreme response style in self-report questionnaires. Appl. Psychol. Meas. 34(2), 105–121 (2010). doi: 10.1177/0146621609338593 CrossRefGoogle Scholar
  48. Weijters, B., Schillewaert, N., Geuens, M.: Assessing response styles across modes of data collection. J. Acad. Mark. Sci. 36(3), 409–422 (2008). doi: 10.1007/s11747-007-0077-6 CrossRefGoogle Scholar
  49. Welkenhuysen-Gybels, J., Billiet, J., Cambré, B.: Adjustment for acquiescence in the assessment of the construct equivalence of likert-type score items. J. Cross-Cult. Psychol. 34(6), 702–722 (2003). doi: 10.1177/0022022103257070 CrossRefGoogle Scholar
  50. Yang, Y., Harkness, J.A., Chin, T.-Y., Villar, A.: Response styles and culture. In: Survey Methods in Multinational, Multiregional, and Multicultural Contexts, 203–223 (2010)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Mingnan Liu
    • 1
    Email author
  • Frederick G. Conrad
    • 2
    • 3
  • Sunghee Lee
    • 4
  1. 1.SurveyMonkeyPalo AltoUSA
  2. 2.Program in Survey MethodologyUniversity of MichiganAnn ArborUSA
  3. 3.Joint Program in Survey MethodologyUniversity of MarylandCollege ParkUSA
  4. 4.Survey Research CenterUniversity of MichiganAnn ArborUSA

Personalised recommendations

Cite article

Buy options