Skip to main content
SpringerLink
Log in

Response quality and ideological dispositions: an integrative approach using geometric and classifying techniques

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

When analyzing survey data, response quality has consequential implications for substantial conclusions. Differences in response quality are usually explained by personality, or socio-demographic or cognitive characteristics. Little, however, is known about how respondents’ political attitudes, values, and opinions impact on quality aspects. This is a striking analytical omission, as potential associations between political values and various forms of response biases and artefacts call into question surveys’ ability to represent ‘public opinion’. In this contribution, response quality is traced back to respondents’ political and ideological dispositions. For this purpose, a relational understanding of response quality is applied that takes into account different aspects of response behaviors, as well as the interrelations between these indicators. Using data from the US General Social Survey (2010–2014), an empirical typology of response quality is created via finite mixture analysis. The resulting classes are then related to positions in the US field of ideological dispositions, constructed via multiple correspondence analysis. The analyses reveal that there are (1) different combinations of response patterns and thus different empirical response types, and (2) that these types of response quality systematically vary with regard to the respondents’ political and ideological (dis)positions. Implications of the findings for public opinion surveys are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. E.g., the request to select a certain answer category as proof that instructions were read (Revilla 2016; Meade and Craig 2012).

  2. Due to coding inconsistencies and the small number of cases with missing values in some variables, ‘don’t know’ and ‘no answer’ are treated as a single category in the analysis.

  3. Applying MCA to data from one ballot only yields a similar spatial structure to the one reported in our results.Regarding future analyses, the recent development of Specific MCA constitutes a promising and adequate alternative to the procedure chosen here.

  4. Cross signifies above average contribution to horizontal axis; filled dot signifies above average contribution to vertical axis; empty dot signifies above average contribution to both axes.

  5. Categories comprising less than one percent of the sample are not displayed. The response style classes ‘middle’ and ‘middle 7pt,’ as well as middle age groups, are located very close to the centroid and thus not displayed for reasons of clarity.

References

  • Atkeson, L.R., Rapoport, R.B.: The more things change the more they stay the same: examining gender differences in political attitude expression, 1952–2000. Public Opin. Quart. 67(4), 495–521 (2003)

    Article  Google Scholar 

  • Bachman, J.G., O’Malley, P.M.: Yea-saying, nay-saying, and going to extremes: black-white differences in response styles. Public Opin. Quart. 48(2), 491–509 (1984)

    Article  Google Scholar 

  • Baron-Epel, O., Kaplan, G., Weinstein, R., Green, M.S.: Extreme and acquiescence bias in a bi-ethnic population. Eur. J. Public Health 20(5), 543–548 (2010)

    Article  Google Scholar 

  • Barth, A.: The changing nature of attitude constructs: an application of multiple correspondence analysis on gender role attitudes. Qual. Quant. 50(4), 1507–1523 (2016)

    Article  Google Scholar 

  • Bates, N., Dahlhamer, J., Singer, E.: Privacy concerns, too busy, or just not interested: using doorstep concerns to predict survey nonresponse. J. Off. Stat. 24(4), 591–612 (2008)

    Google Scholar 

  • Baumgartner, H., Steenkamp, J.-B.E.: Response styles in marketing research: a cross-national investigation. J. Mark. Res. 38(2), 143–156 (2001)

    Article  Google Scholar 

  • Bergström, Y.: Educational and social dimensions of political participation: producing a political opinion. Praktiske Grunde. Nordisk tidsskrift for kultur-og samfundsvidenskab 4, 23–34 (2012)

  • Billiet, J.B., Davidov, E.: Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociol. Methods Res. 36(4), 542–562 (2008)

    Article  Google Scholar 

  • Blasius, J., Thiessen, V.: Assessing the quality of survey data. Sage, London (2012)

    Book  Google Scholar 

  • Blasius, J., Schmitz, A.: The empirical construction of Bourdieu's social space. In: Greenacre, M., Blasius, J. (eds.) The Visualization and Verbalization of Data, pp. 205–222. Chapman & Hall, London (2014)

  • Bourdieu, P.: Public opinion does not exist. In: Siegelaub, S., Mattelart, A. (eds.) Communication and Class Struggle 1, pp. 124–130. IG/IMMRC, New York/Bagnolet (1979)

  • Bourdieu, P.: Distinction: A social critique of the judgment of taste. Harvard University Press, Cambridge (1984)

  • Bourdieu, P.: Some properties of fields. In: Bourdieu, P. (ed.) Sociology in Question, pp. 72–77. Sage, London (1993)

    Google Scholar 

  • Carmines, E.G., Ensley, M.J., Wagner, M.W.: Political ideology in American politics: one, two, or none? Forum 10(3), 1–20 (2012a)

    Google Scholar 

  • Carmines, E.G., Ensley, M.J., Wagner, M.W.: Who fits the left-right divide? Partisan polarization in the American electorate. Am. Behav. Sci. 56(12), 1631–1653 (2012b)

    Article  Google Scholar 

  • Champagne, P.: Die öffentliche Meinung als neuer politischer Fetisch. Berliner Journal für Soziologie 1(4), 517–526 (1991)

    Google Scholar 

  • Claggett, W.J., Engle, P.J., Shafer, B.E.: The evolution of mass ideologies in modern American politics. Forum 12(2), 223–256 (2014)

    Google Scholar 

  • Cole, J.S., McCormick, A.C., Gonyea, R.M.: Respondent use of straight-lining as a response strategy in education survey research: prevalence and implications. In: Annual Meeting of the American Educational Research Association (AERA) (2012)

  • Colsher, P.L., Wallace, R.B.: Data quality and age: health and psychobehavioral correlates of item nonresponse and inconsistent responses. J. Gerontol. 44(2), 45–52 (1989)

    Article  Google Scholar 

  • Cronbach, L.J.: Response sets and test validity. Educ. Psychol. Meas. 6(4), 475–494 (1946)

    Article  Google Scholar 

  • De Jong, M.G., Steenkamp, J.B.E., Fox, J.P., Baumgartner, H.: Using item response theory to measure extreme response style in marketing research: a global investigation. J. Mark. Res. 45(1), 104–115 (2008)

    Article  Google Scholar 

  • de Leeuw, D., Hox, J., Huisman, M.: Prevention and treatment of item nonresponse. J. Off. Stat. 19(2), 153–176 (2003)

    Google Scholar 

  • Desrosières, A., Thévenot, L.: Les catégories socio-professionelles. Editions La Découverte, Paris (2002)

    Google Scholar 

  • Deutskens, E., de Ruyter, K., Wetzels, M., Oosterveld, P.: Response rate and response quality of internet-based surveys: an experimental study. Mark. Lett. 15(1), 21–36 (2004)

    Article  Google Scholar 

  • Esaiasson, P., Granberg, D.: Hidden negativism: evaluation of Swedish parties and their leaders under different survey methods. Int. J. Public Opin. Res. 5(3), 265–277 (1993)

    Article  Google Scholar 

  • Evans, G.: The continued significance of class voting. Annu. Rev. Polit. Sci. 3, 401–417 (2000)

    Article  Google Scholar 

  • Feldman, S., Johnston, C.: Understanding the determinants of political ideology: implications of structural complexity. Polit. Psychol. 35(3), 337–358 (2014)

    Article  Google Scholar 

  • Fourcade, M., Lande, B., Schofer, E.: Political space and the space of polities: doing politics across nations. Poetics 55, 1–18 (2016)

    Article  Google Scholar 

  • Galesic, M., Bosnjak, M.: Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opin. Quart. 73(2), 349–360 (2009)

    Article  Google Scholar 

  • Gerber, A.S., Huber, G.A., Doherty, D., Dowling, C.M., Ha, S.E.: Personality and political attitudes: relationships across issue domains and political contexts. Am. Polit. Sci. Rev. 104(1), 111–133 (2010)

    Article  Google Scholar 

  • Grauenhorst, T., Blohm, M., Koch, A.: Respondent incentives in a national face-to-face survey—do they affect response quality? Field Methods 28(3), 266–283 (2016)

    Article  Google Scholar 

  • Greenleaf, E.A.: Improving rating scale measures by detecting and correcting bias components in some response styles. J. Mark. Res. 29(2), 176–188 (1992a)

    Article  Google Scholar 

  • Greenleaf, E.A.: Measuring extreme response style. Public Opin. Quart. 56(3), 328–351 (1992b)

    Article  Google Scholar 

  • Harzing, A.W.: Response styles in cross-national survey research—a 26-country study. Int. J. Cross Cult. Manag. 6(2), 243–266 (2006)

    Article  Google Scholar 

  • He, J., Van de Vijver, F.J.: A general response style factor: evidence from a multi-ethnic study in the Netherlands. Pers. Indiv. Differ. 55(7), 794–800 (2013)

    Article  Google Scholar 

  • He, J., Van de Vijver, F.J.: Response styles in factual items: personal, contextual and cultural correlates. Int. J. Psychol. 1–8 (2016). doi:10.1002/ijop.12263

  • He, J., Van de Vijver, F.J., Espinosa, A.D., Mui, P.H.: Toward a unification of acquiescent, extreme, and midpoint response styles—a multilevel study. Int. J. Cross Cult. Manag. 14(3), 306–322 (2014)

    Article  Google Scholar 

  • Heerwegh, D., Loosveldt, G.: An evaluation of the effect of response formats on data quality in web surveys. Soc. Sci. Comput. Rev. 20(4), 471–484 (2002)

    Article  Google Scholar 

  • Holbrook, A., Cho, Y.I., Johnson, T.: The impact of question and respondent characteristics on comprehension and mapping difficulties. Public Opin. Quart. 70(4), 565–595 (2006)

    Article  Google Scholar 

  • Inglehart, R.: Modernization and Postmodernization: Cultural, Economic, and Political Change in 43 Societies, vol. 19. Princeton University Press, Princeton (1997)

    Google Scholar 

  • Inglehart, R., Norris, P.: The developmental theory of the gender gap: women’s and men’s voting behavior in global perspective. Int. Polit. Sci. Rev. 21(4), 441–463 (2000)

    Article  Google Scholar 

  • Kaminska, O., McCutcheon, A.L., Billiet, J.: Satisficing among reluctant respondents in a cross-national context. Public Opin. Quart. 74(5), 956–984 (2010)

    Article  Google Scholar 

  • Katz, J.: Who will be president? The New York Times. Retrieved from http://www.nytimes.com/interactive/2016/upshot/presidential-polls-forecast.html (2016). Accessed Nov 8 2016

  • Kieruj, N.D., Moors, G.: Response style behavior: question format dependent or personal style? Qual. Quant. 47(1), 193–211 (2013)

    Article  Google Scholar 

  • Kleiner, B., Lipps, O., Ferrez, E.: Language ability and motivation among foreigners in survey responding. J. Surv. Stat. Methodol. 3(3), 339–360 (2015)

    Article  Google Scholar 

  • Knäuper, B., Belli, R.F., Hill, D.H., Herzog, A.R.: Question difficulty and respondents’ cognitive ability: the effect on data quality. J. Off. Stat. 13(2), 181–199 (1997)

    Google Scholar 

  • Knowles, E.S., Nathan, K.T.: Acquiescent responding in self-reports: cognitive style or social concern? J. Res. Pers. 31(2), 293–301 (1997)

    Article  Google Scholar 

  • Krosnick, J.A.: Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl. Cognit. Psychol. 5(3), 213–236 (1991)

    Article  Google Scholar 

  • Krosnick, J.A.: The impact of cognitive sophistication and attitude importance on response-order and question-order effects. In: Schwarz, N., Sudman, S. (eds.) Context Effects in Social and Psychological Research, pp. 203–218. Springer, New York (1992)

    Chapter  Google Scholar 

  • Krosnick, J.A., Holbrook, A.L., Berent, M.K., Carson, R.T., Hanemann, W.M., Moody, W.R.: The impact of “no opinion” response options on data quality: non-attitude reduction or an invitation to satisfice? Public Opin. Quart. 66(3), 371–403 (2002)

  • Laurison, D.: The willingness to state an opinion: inequality, don’t know responses, and political participation. Sociol. Forum 30(4), 925–948 (2015)

    Article  Google Scholar 

  • Lebart, L., Morineau, A., Warwick, K.M.: Multivariate Descriptive Statistical Analysis: Correspondence Analysis and Related Techniques for Large Matrices. Wiley, New York (1984)

    Google Scholar 

  • Lechner, C.M., Rammstedt, B.: Cognitive ability, acquiescence, and the structure of personality in a sample of older adults. Psychol. Assess. 27(4), 1301–1311 (2015)

    Article  Google Scholar 

  • Le Roux, B., Rouanet, H.: Geometric data analysis: from correspondence analysis to structured data analysis. Kluwer, Dordrecht (2004)

    Google Scholar 

  • Light, C.S., Zax, M., Gardiner, D.H.: Relationship of age, sex, and intelligence level to extreme response style. J. Pers. Soc. Psychol. 2(6), 907–909 (1965)

    Article  Google Scholar 

  • McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, New York (2004)

    Google Scholar 

  • Meade, A.W., Craig, S.B.: Identifying careless responses in survey data. Psychol. Methods 17(3), 437–455 (2012)

    Article  Google Scholar 

  • Medway, R.L., Tourangeau, R.: Response quality in telephone surveys: do prepaid cash incentives make a difference? Public Opin. Quart. 79(2), 524–543 (2015)

    Article  Google Scholar 

  • Meisenberg, G., Williams, A.: Are acquiescent and extreme response styles related to low intelligence and education? Pers. Indiv. Differ. 44(7), 1539–1550 (2008)

    Article  Google Scholar 

  • Milbrath, L.W.: Latent origins of liberalism-conservatism and party identification: a research note. J. Polit. 24(4), 679–688 (1962)

    Article  Google Scholar 

  • Moors, G.: Exploring the effect of a middle response category on response style in attitude measurement. Qual. Quant. 42(6), 779–794 (2008)

    Article  Google Scholar 

  • Naemi, B.D., Beal, D.J., Payne, S.C.: Personality predictors of extreme response style. J. Pers. 77(1), 261–286 (2009)

    Article  Google Scholar 

  • Narayan, S., Krosnick, J.A.: Education moderates some response effects in attitude measurement. Public Opin. Quart. 60(1), 58–88 (1996)

    Article  Google Scholar 

  • Nylund, K.L., Asparouhov, T., Muthén, B.O.: Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Struct. Equ. Model. 14(4), 535–569 (2007)

    Article  Google Scholar 

  • Oskamp, S., Schultz, P.W.: Attitudes and Opinions. Psychology Press, New York (2005)

    Google Scholar 

  • Pfau-Effinger, B.: Culture and welfare state policies: reflections on a complex interrelation. J. Soc. Policy 34(1), 3–20 (2005)

    Article  Google Scholar 

  • Rahat, G., Hazan, R.Y., Bloom, P.B.N.: Stable Blocs and Multiple Identities: The 2015 Elections in Israel. Representation 52(1), 99–117 (2016)

    Article  Google Scholar 

  • Revilla, M.: Impact of raising awareness of respondents on the measurement quality in a web survey. Qual. Quant. 50(4), 1469–1486 (2016)

    Article  Google Scholar 

  • Rogelberg, S.G., Fisher, G.G., Maynard, D.C., Hakel, M.D., Horvath, M.: Attitudes toward surveys: development of a measure and its relationship to respondent behavior. Organ. Res. Methods 4(1), 3–25 (2001)

    Article  Google Scholar 

  • Rorer, L.G.: The great response-style myth. Psychol. Bull. 63(3), 129–156 (1965)

    Article  Google Scholar 

  • Ross, C.E., Mirowsky, J.: Socially-desirable response and acquiescence in a cross-cultural survey of mental health. J. Health Soc. Behav. 25(2), 189–197 (1984)

    Article  Google Scholar 

  • Schoon, I., Cheng, H., Gale, C.R., Batty, G.D., Deary, I.J.: Social status, cognitive ability, and educational attainment as predictors of liberal social attitudes and political trust. Intelligence 38(1), 144–150 (2010)

    Article  Google Scholar 

  • Schmitz, A., Witte, D., Gengnagel, V.: Pluralizing field analysis: Toward a relational understanding of the field of power. Social Science Information/Information sur les sciences sociales 56(1), 1–25 (2016)

  • Schuman, H., Presser, S.: Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. Academic Press, New York (1981)

    Google Scholar 

  • Shafer, B.E., Claggett, W.J.: The Two Majorities: The Issue Context of Modern American Politics. Johns Hopkins University Press, Baltimore (1995)

    Google Scholar 

  • Smith, P.B.: Acquiescent response bias as an aspect of cultural communication style. J. Cross Cult. Psychol. 35(1), 50–61 (2004)

    Article  Google Scholar 

  • Smith, T., Marsden, P.V., Hout, M., Son, J., Schapiro, B., Mai, N., Milliff, A.: General Social Surveys, 1972–2014 Cumulative Codebook. National Opinion Research Centre, Chicago (2014)

    Google Scholar 

  • Stähli, M.E., Joye, D.: Non-respondent surveys: pertinence and feasibility. Survey Stat. 68, 16–22 (2013)

    Google Scholar 

  • Stocké, V.: Attitudes toward surveys, attitude accessibility and the effect on respondents’ susceptibility to nonresponse. Qual. Quant. 40(2), 259–288 (2006)

    Article  Google Scholar 

  • Stubager, R.: The development of the education cleavage: Denmark as a critical case. West Eur. Polit. 33(3), 505–533 (2010)

    Article  Google Scholar 

  • Sturgis, P., Baker, N., Callegaro, M., Fisher, S., Green, J., et al.: Report of the Inquiry Into the 2015 British General Election Opinion Polls. Market Research Society and British Polling Council, London (2016)

  • Tabachnick, B.G., Fidell, L.S.: Using Multivariate Statistics, 5th edn. Pearson/Allyn & Bacon, Boston (2007)

    Google Scholar 

  • Thomas, T.D., Abts, K., Vander Weyden, P.: Response Styles and the Rural-Urban Divide. Educ. Psychol. Meas. 74(1), 97–115 (2014)

    Article  Google Scholar 

  • Thiessen, V., Blasius, J.: Mathematics achievement and mathematics learning strategies: cognitive competencies and construct differentiation. Int. J. Educ. Res. 47(6), 362–371 (2008)

    Article  Google Scholar 

  • Traugott, M., Bolger, G., Davis, D.W., Franklin, C., Groves, R.M., et al.: An evaluation of the methodology of the 2008 pre-election primary polls. AAPOR Ad Hoc Committee on the 2008 Presidential Primary Polling. AAPOR (2009)

  • Van de Vijver, F.J., Poortinga, Y.H.: Towards an integrated analysis of bias in cross-cultural assessment. Eur. J. Psychol. Ass. 13(1), 29–37 (1997)

    Article  Google Scholar 

  • Van Hiel, A., Onraet, E., De Pauw, S.: The relationship between social-cultural attitudes and behavioural measures of cognitive style: a meta-analytic integration of studies. J. Pers. 78(6), 1765–1800 (2010)

    Article  Google Scholar 

  • Van Rosmalen, J., Van Herk, H., Groenen, P.: Identifying response styles: a latent-class bilinear multinomial logit model. J. Market. Res. 47(1), 157–172 (2010)

    Article  Google Scholar 

  • Van Vaerenbergh, Y., Thomas, T.D.: Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int. J. Public Opin. Res. 25(2), 195–217 (2013)

    Article  Google Scholar 

  • Verba, S.: The citizen as respondent: sample surveys and American democracy presidential address. Am. Polit. Sci. Rev. 90(1), 1–7 (1996)

    Article  Google Scholar 

  • Weech-Maldonado, R., Elliott, M.N., Oluwole, A., Schiller, K.C., Hays, R.D.: Survey response style and differential use of CAHPS rating scales by Hispanics. Med. Care 46(9), 963–968 (2008)

    Article  Google Scholar 

  • Weijters, B., Geuens, M., Schillewaert, N.: The stability of individual response styles. Psychol. Methods 15(1), 96–110 (2010)

    Article  Google Scholar 

  • Winkler, J.D., Kanouse, D.E., Ware, J.E.: Controlling for acquiescence response set in scale development. J. Appl. Psychol. 67(5), 555–561 (1982)

    Article  Google Scholar 

  • Zhang, C., Conrad, F.: Speeding in web surveys: the tendency to answer very fast and its association with straightlining. Surv. Res. Methods 8(2), 127–135 (2014)

    Google Scholar 

  • Zhou, B., McClendon, M.J.: Cognitive ability and acquiescence. In: Proceedings of the Survey Research Methods Section, pp. 1003–1012 (1999)

  • Zillmann, D., Schmitz, A., Skopek, J., Blossfeld, H.P.: Survey topic and unit nonresponse. Evidence from an online survey on mating. Qual. Quant. 48(4), 2069–2088 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Sociology, University of Bonn, Lennéstr. 27, 53113, Bonn, Germany

    Alice Barth & Andreas Schmitz

Authors
  1. Alice Barth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Schmitz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alice Barth.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barth, A., Schmitz, A. Response quality and ideological dispositions: an integrative approach using geometric and classifying techniques. Qual Quant 52, 175–194 (2018). https://doi.org/10.1007/s11135-016-0458-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-016-0458-3

Keywords

Search

Navigation