Skip to main content
SpringerLink
Log in

Is extreme response style domain specific? Findings from two studies in four countries

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Extreme response style (ERS) may bias responses and hamper the validity of conclusions in substantive research. ERS can be controlled for by using an additional (random) sample of response style indicators (i.e., a separate, random sample of survey items). There are two options to draw response style indicators to control for ERS: from only one versus from multiple domains. In two studies (four samples in total), this paper examines the domain dependency of ERS across three domains: consumer behavior, interpersonal relationships and politics. We find in the four samples repeated evidence suggesting that ERS has a domain specific component. This finding calls into question the (often encountered) assumption that it does not matter from which domains ERS measures are drawn.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. In terms of the terminology used, a cautionary note is warranted. In the context of their study, Weijters et al. (2008) use the term “response style indicator” to refer to an individual-level summary statistic denoting the number of times a focal response style occurs in a randomly drawn set of survey items (in their study: drawn from a broad variety of domains). However, as mentioned below, in our study every single survey item included in the random set (in this study: drawn from a more narrowly defined domain) is referred to as a response style indicator. As a consequence, in Weijters et al. (2008), a response style indicator represents a count variable (in our study a count variable would qualify as a “summary statistic”), whereas in our study a response style indicator can only take on a value of zero (i.e., response style is not present) or a value of one (i.e., response style is present), depending on what answer the respondent provided to the survey item under consideration.

  2. For studies involving an assessment of response styles, Weijters et al. (2008) proposed a criterion of using 10–14 response style indicators to quantify summary statistics for response styles. Closer inspection of the empirical results from a sensitivity analysis on which this criterion has been proposed, clearly shows that this criterion applies primarily to response styles other than ERS (i.e., acquiescence, disacquiescence, and midpoint response style). The sensitivity analysis conducted by Weijters et al. (2008), showed that ERS is—by far—the most stable and reliable response style, and ERS measures tend to demonstrate strong construct reliability and reliable variance estimates with as few as five survey items used to compute an ERS summary statistic. Hence, our current use of eight survey items per summary statistic is consistent with response style measurement guidelines.

  3. We used the random number generator in MS Excel to obtain 400 different ways of splitting the 16 ERS indicators into two halves forming the two ERS summary statistics; e.g., in one dataset, the first CB ERS summary statistic might consist of CB survey items 1, 4, 5, 7, 10, 11, 12, 16 while the other ERS summary statistic consists of CB survey items 2, 3, 6, 8, 9, 13, 14, 15.

References

  • Baumgartner, H., Steenkamp, J.-B.E.M.: Response styles in marketing research: a cross-national investigation. J. Mark. Res. 38, 143–156 (2001). doi:10.1509/jmkr.38.2.143.18840

    Article  Google Scholar 

  • Bearden, W.O., Netemeyer, R.G.: Handbook of Marketing Scales: Multi-item Measures for Marketing and Consumer Behavior Research, 2nd edn. Sage, London (1999)

    Book  Google Scholar 

  • Bruner II, G.C., Hensel, P.J., James, K.E.: Marketing Scales Handbook: A Compilation of Multi-item Measures for Consumer Behavior and Advertising Research, vol. 4. American Marketing Association, Chicago (2005)

    Google Scholar 

  • Cheung, G., Rensvold, R.B.: Assessing extreme and acquiescent response sets in cross-cultural research using structural equations modeling. J. Cross Cult. Psychol. 31, 187–212 (2000). doi:10.1177/0022022100031002003

    Article  Google Scholar 

  • Davidov, E., De Beuckelaer, A.: Testing the equivalence of an instrument to assess Schwartz’s human values: how harmful are translations? Int. J. Public Opin. Res. 22, 485–510 (2010). doi:10.1093/ijpor/edq030

    Article  Google Scholar 

  • Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., Billiet, J.: Measurement equivalence in cross-national research. Annual Review of Sociology 40, 55–75 (2014). doi:10.1146/annurev-soc-071913-043137

    Article  Google Scholar 

  • Davis, C.M.: Handbook of Sexuality-Related Measures. Sage, Thousand Oaks (1998)

    Google Scholar 

  • De Beuckelaer, A., Weijters, B., Rutten, A.: Using ad hoc measures for response styles: a cautionary note. Qual. Quant. 44, 761–775 (2010). doi:10.1007/s11135-009-9225-z

    Article  Google Scholar 

  • De Jong, M.G., Steenkamp, J.-B.E.M., Fox, J.P., Baumgartner, H.: Using item response theory to measure extreme response style in marketing research: global investigation. J. Mark. Res. 45, 104–115 (2008). doi:10.1509/jmkr.45.1.104

    Article  Google Scholar 

  • De Langhe, B., Puntoni, S., Fernandes, D., Van Osselaer, S.M.: The anchor contraction effect in international marketing research. J. Mark. Res. 48(2), 366–380 (2011)

    Article  Google Scholar 

  • Diamantopoulos, A., Reynolds, N.L., Simintiras, A.C.: The impact of response styles on the stability of cross-national comparisons. J. Bus. Res. 59(8), 925–935 (2006).

    Article  Google Scholar 

  • Dolnicar, S., Grün, B.: Cross-cultural differences in survey response patterns. Int. Mark. Rev. 24, 127–143 (2007). doi:10.1108/02651330710741785

    Article  Google Scholar 

  • Eid, M., Lischetzke, T., Nussbeck, F. W.: Structural equation models for multitrait-multimethod data. In: Eid, M., Diener, E. (eds.) Handbook of Multimethod Measurement in Psychology, pp. 283–299. American Psychological Association, Washington, DC (2006). doi:10.1037/11383-020

  • Harzing, A.-W.: Response styles in cross-national survey research: a 26-country study. Int. J. Cross Cult. Manag. 6, 243–266 (2006). doi:10.1177/1470595806066332

    Article  Google Scholar 

  • Hirschfeld, L.A., Gelman, S.A.: Mapping the Mind: Domain Specificity in Cognition and Culture. Cambridge University Press, New York (1994)

    Book  Google Scholar 

  • Hui, C.H., Triandis, H.C.: Effects of culture and response format on extreme response style. J. Cross Cult. Psychol. 20, 296–309 (1989). doi:10.1177/0022022189203004

    Article  Google Scholar 

  • Kieruj, N.D., Moors, G.: Response style behavior: question format dependent or personal style? Qual. Quant. 47, 193–211 (2013). doi:10.1007/s11135-011-9511-4

    Article  Google Scholar 

  • Paulhus, D.L.: Measurement and control of response bias. In: Robinson, J.P., Shaver, P.R., Wrightsman, L.S. (eds.) Measures of Personality and Social Psychology Attitudes, vol. 1, 1st edn, pp. 17–59. Academic Press, San Diego (1991)

    Chapter  Google Scholar 

  • Perlmutter, B.F., Touliatos, J., Holden, G.W.: Handbook of Family Measurement Techniques. Sage, Thousand Oaks (2001)

    Google Scholar 

  • Podsakoff, P.M., MacKenzie, S.B., Lee, J.Y., Podsakoff, N.P.: Common method biases in behavioral research: a critical review of the literature and recommended remedies. J. Appl. Psych. 88(5), 879–903 (2003).

    Article  Google Scholar 

  • Robinson, J.P., Rusk, J.G., Head, K.B.: Measures of Political Attitudes. Survey Research Center, Institute for Social Research, Ann Arbor (1969)

    Google Scholar 

  • Smith, P.B.: Acquiescent response bias as an aspect of cultural communication style. J. Cross Cult. Psychol. 35, 50–61 (2004). doi:10.1177/0022022103260380

    Article  Google Scholar 

  • Steenkamp, J.B.E., Baumgartner, H.: Assessing measurement invariance in cross-national consumer research. J. Cons. Res. 25(1), 78–90 (1998)

    Article  Google Scholar 

  • Steenkamp, J.B.E., De Jong, M.G., Baumgartner, H.: Socially desirable response tendencies in survey research. J. Mark. Res. 47, 199–214 (2010). doi:10.1509/jmkr.47.2.199

    Article  Google Scholar 

  • Traub, R.E.: MMSS Reliability for the Social Sciences: Theory and Applications. Sage, Thousand Oaks (1994)

    Google Scholar 

  • Tzeng, O.C.S.: Measurement of Love and Intimate Relations: Theories, Scales, and Applications for Love Development, Maintenance, and Dissolution. Praeger, Westport (1993)

    Google Scholar 

  • Van Dijk, T.K., Datema, F., Piggen, A.L.J.H.F., Welten, S.C.M., Van De Vijver, F.J.R.: Acquiescence and extremity in cross-national surveys: domain dependence and country-level correlates. In: Gari, A., Mylonas, K. (eds.) Quod erat demonstrandum: From Herodotus’ ethnographic journeys to cross-cultural research, pp. 149–158. Pedio Books, Athens (2009)

    Google Scholar 

  • Van Herk, H., Poortinga, Y.H., Verhallen, T.M.M.: Response styles in rating scales: evidence of method bias in data from six EU countries. J. Cross Cult. Psychol. 35, 346–360 (2004). doi:10.1177/0022022104264126

    Article  Google Scholar 

  • Van Vaerenbergh, Y., Thomas, T.D.: Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int. J. Public Opin. Res. 25, 195–217 (2013). doi:10.1093/ijpor/eds021

    Article  Google Scholar 

  • Weathers, D., Sharma, S., Niedrich, R.W.: The impact of the number of scale points, dispositional factors, and the status quo decision heuristic on scale reliability and response accuracy. J. Bus. Res. 58, 1516–1524 (2005). doi:10.1016/j.jbusres.2004.08.002

    Article  Google Scholar 

  • Weijters, B., Cabooter, E., Schillewaert, N.: The effect of rating scale format on response styles: the number of response categories and response category labels. Int. J. Res. Mark. 27, 236–247 (2010). doi:10.1016/j.ijresmar.2010.02.004

    Article  Google Scholar 

  • Weijters, B., Geuens, M., Baumgartner, H.: The effect of familiarity with the response category labels on item response to Likert scales. J. Consum. Res. 40, 368–381 (2013). doi:10.1086/670394

    Article  Google Scholar 

  • Weijters, B., Schillewaert, N., Geuens, M.: Assessing response styles across modes of data collection. J. Acad. Mark. Sci. 36, 409–422 (2008). doi:10.1007/s11747-007-0077-6

    Article  Google Scholar 

  • Wright, P.H.: A bare-bones guide to the acquaintance description form—F2. Unpublished manuscript. Department of Psychology, University of North Dakota, Grand Forks (1997)

Download references

Acknowledgments

The authors thank Paulette Gadet, Myriam Van Der Vliet and Mirjam Van Dijk for collecting data for this study. In addition, Lisa Trierweiler deserves credit for the English proofreading of this paper. The fourth author would like to thank the URPP project on Social Networks at the University of Zürich for their financial support during work on this paper.

Author information

Authors and Affiliations

  1. IÉSEG School of Management, Lille, France

    Elke Cabooter

  2. Department of Personnel Management, Work and Organizational Psychology, Ghent University, Ghent, Belgium

    Bert Weijters

  3. Institute for Management Research, Radboud University, Thomas van Aquinostraat 1, P.O. Box 9108, 6500 HK, Nijmegen, The Netherlands

    Alain De Beuckelaer

  4. School of Sociology and Population Studies, Renmin University of China, Beijing, People’s Republic of China

    Alain De Beuckelaer

  5. Department of Sociology, University of Zürich, Zurich, Switzerland

    Eldad Davidov

  6. Institute of Sociology and Social Psychology, University of Cologne, Cologne, Germany

    Eldad Davidov

Authors
  1. Elke Cabooter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bert Weijters
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alain De Beuckelaer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eldad Davidov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elke Cabooter.

Appendix: Survey items used in this research

Appendix: Survey items used in this research

1.1 Consumer behavior

  1. # 01:

    I enjoy being surrounded by exciting new products.

  2. # 02:

    The games and contests that manufacturers sponsor to encourage people to buy their products are usually dishonest.

  3. # 03:

    Buying a high price brand makes me feel good about myself.

  4. # 04:

    When I see a new brand on the shelf, I am not afraid of giving it a try.

  5. # 05:

    If a salesperson has gone through a lot of trouble to find an item for me, I would be embarrassed not to buy it, even if it is not exactly right.

  6. # 06:

    I will make a scene at a store if necessary to get a complaint handled to my satisfaction.

  7. # 07:

    My standards and expectations for products I buy are very high.

  8. # 08:

    If a cashier is talking with friends while I am waiting to be waited on, it would not bother me to interrupt the conversation and ask for assistance.

  9. # 09:

    Advertising makes people buy unaffordable products just to show off.

  10. # 010:

    Advertising results in better products for the public.

  11. # 011:

    In general, advertising presents a true picture of the product being advertised.

  12. # 012:

    I do not like to talk to my friends about my purchases.

  13. # 013:

    Even though certain food products are available in a number of different flavors, I always tend to buy the same flavor.

  14. # 014:

    I often make unplanned purchases.

  15. # 015:

    When I use coupons, I feel that I am getting a good deal.

  16. # 016:

    The time it takes to find low prices is usually not worth the effort.

1.2 Interpersonal relationships

  1. # 01:

    The wife should trust and accept the husband’s judgments on important decisions.

  2. # 02:

    If there are young children, the wife should not work outside the home.

  3. # 03:

    I have a lot in common with the people around me.

  4. # 04:

    I find it relatively easy to get close to others.

  5. # 05:

    I sometimes get frustrated or angry because no one loves me the way I would like to be loved.

  6. # 06:

    My romantic partners have generally been trustworthy.

  7. # 07:

    I see myself combining work and marriage and not having children.

  8. # 08:

    I like to see a relationship in which the man and the woman have equal power.

  9. # 09:

    A woman having casual sex (that is, without being involved in a relationship) is just as acceptable to me as a man having casual sex.

  10. # 010:

    Most of my friends have a more exciting love life than I do.

  11. # 011:

    I am very happy with my friendships.

  12. # 012:

    I do not have enough friends.

  13. # 013:

    If I am in an embarrassing situation, I can count on my friends to do things that will make me feel as much at ease as possible.

  14. # 014:

    We (the family) work together to sort out our problems.

  15. # 015:

    I believe it is the parents’ right to spank their children if they think it is necessary.

  16. # 016:

    Families should plan for the possibility of providing care for an elderly disabled family member.

1.3 Political matters

  1. # 01:

    The government should leave things like electric power and housing for private businesses to handle.

  2. # 02:

    Students should be trained in times of peace to carry out military duties.

  3. # 03:

    Improving poor and underdeveloped areas is a waste of time.

  4. # 04:

    Nations should pay high honor to their military leaders.

  5. # 05:

    In the present state of capitalist societies, real social progress is impossible without the aid of revolution.

  6. # 06:

    The findings of science may someday show that many of our most deeply held beliefs are wrong.

  7. # 07:

    The government should help people get doctors and hospital care at low cost.

  8. # 08:

    If people do not care how an election comes out they should not vote for it.

  9. # 09:

    One can be democratic and believe in free speech, and still deny fascists the right to speak and hold meetings.

  10. # 010:

    Our country should refuse to cooperate in a total disarmament program even if some other nations would agree to cooperate.

  11. # 011:

    There will always be wars, for one thing, there will be groups who ruthlessly try to grab more than their share.

  12. # 012:

    The government should own and operate all public utilities (transportation, gas and electric, railroads, etc.).

  13. # 013:

    Government authorities should be allowed to ban books and movies which they consider harmful to the public interest.

  14. # 014:

    There will always be superior and inferior countries in the world and, in the interests of all concerned, it is best that the superior countries be in control of world affairs.

  15. # 015:

    It is not so important to vote when you know that your political party does not have a chance to win.

  16. # 016:

    Protesters and radicals are good for society, even though they may cause a change in normally accepted standards.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cabooter, E., Weijters, B., De Beuckelaer, A. et al. Is extreme response style domain specific? Findings from two studies in four countries. Qual Quant 51, 2605–2622 (2017). https://doi.org/10.1007/s11135-016-0411-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-016-0411-5

Keywords

Search

Navigation