Journal of the Academy of Marketing Science

, Volume 36, Issue 3, pp 409–422 | Cite as

Assessing response styles across modes of data collection

  • Bert WeijtersEmail author
  • Niels Schillewaert
  • Maggie Geuens


Cross-mode surveys are on the rise. The current study compares levels of response styles across three modes of data collection: paper-and-pencil questionnaires, telephone interviews, and online questionnaires. The authors make the comparison in terms of acquiescence, disacquiescence, and extreme and midpoint response styles. To do this, they propose a new method, namely, the representative indicators response style means and covariance structure (RIRSMACS) method. This method contributes to the literature in important ways. First, it offers a simultaneous operationalization of multiple response styles. The model accounts for dependencies among response style indicators due to their reliance on common item sets. Second, it accounts for random error in the response style measures. As a consequence, random error in response style measures is not passed on to corrected measures. The method can detect and correct cross-mode response style differences in cases where measurement invariance testing and multitrait multimethod designs are inadequate. The authors demonstrate and discuss the practical and theoretical advantages of the RIRSMACS approach over traditional methods.


Response styles Modes of data-collection Measurement invariance Survey research 



The authors would like to thank the Intercollegiate Center for Management Sciences (Belgium) and Insites for supporting the research reported in this paper. Further, the authors would like to thank the following people for their feedback on previous versions of the paper: Hans Baumgartner, Jaak Billiet, Marion Debruyne, Koen Dewettinck, Alain De Beuckelaer and Patrick Van Kenhove.

Supplementary material

11747_2007_77_MOESM1_ESM.pdf (312 kb)
Supplement 1 ESM 1 (PDF 319 341 kb)


  1. Ayidiya, S., & McClendon, M. J. (1990). Response effects in mail surveys. Public Opinion Quarterly, 54(2), 229–247.CrossRefGoogle Scholar
  2. Bagozzi, R. P., & Yi, Y. (1990). Assessing method variance in multitrait-multimethod matrices: The case of self-reported affect and perceptions at work. Journal of Applied Psychology, 75(5), 547–560.CrossRefGoogle Scholar
  3. Baumgartner, H., & Steenkamp, J.-B. E. M. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143–156, May.CrossRefGoogle Scholar
  4. Baumgartner, H., & Steenkamp, J.-B. E. M. (2006). An extended paradigm for measurement analysis of marketing constructs applicable to panel data. Journal of Marketing Research, 43, 431–442, (August).CrossRefGoogle Scholar
  5. Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling, 7(4), 608–628.CrossRefGoogle Scholar
  6. Bruner, G. C., James, K. E., & Hensel, P. J. (2001). Marketing scales handbook: A compilation of multi-item measures, vol. 3. Chicago: American Marketing Association.Google Scholar
  7. Cheung, G. W., & Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equation modeling. Journal of Cross-Cultural Psychology, 31(2), 187–212.CrossRefGoogle Scholar
  8. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indices for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.CrossRefGoogle Scholar
  9. Cox III., E. P. (1980). The optimal number of response alternatives for a scale: A review. Journal of Marketing Research, 17(4), 407–422.CrossRefGoogle Scholar
  10. De Jong, M. G., Steenkamp, J.-B. E .M., Fox, J.-P., & Baumgartner, H. (2007). Using item response theory to measure extreme response style in marketing research: A global investigation. Journal of Marketing Research (in press).Google Scholar
  11. De Leeuw, E. D. (2005). To mix or not to mix: data collection modes in surveys. Journal of Official Statistics, 21(2), 233–255.Google Scholar
  12. Deutskens, E. C., de Ruyter, K., & Wetzels, M. G. M. (2006). An assessment of equivalence between online and mail surveys in service research. Journal of Service Research, 8(4), 346–355.CrossRefGoogle Scholar
  13. Drolet, A., & Morrison, D. G. (2001). Do we really need multiple-item measures in service research? Journal of Service Research, 3(3), 196–204.CrossRefGoogle Scholar
  14. Ferrando, P. J., & Lorenzo-Seva, U. (2005). IRT-related factor analytic procedures for testing the equivalence of paper-and-pencil and internet-administered questionnaires. Psychological Methods, 10(2), 193–205.CrossRefGoogle Scholar
  15. Finney, S. J., & DiStefano, C. (2006). Nonnormal and categorical data in. In G. R. Hancock, & R. O. Mueller (Eds.) Structural equation modeling: A second course. Greenwich, CT: Information Age Publishing.Google Scholar
  16. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables. Journal of Marketing Research, 18(1), 39–50.CrossRefGoogle Scholar
  17. Graham, J. W., Taylor, B. J., Olchowski, A. E., & Cumsille, P. E. (2006). Planned missing data designs in psychological research. Psychological Methods, 11(4), 323–343.CrossRefGoogle Scholar
  18. Greenleaf, E. A. (1992a). Improving rating scale measures by detecting and correcting bias components in some response styles. Journal of Marketing Research, 29(2), 176–188.CrossRefGoogle Scholar
  19. Greenleaf, E. A. (1992b). Measuring extreme response style. Public Opinion Quarterly, 56(3), 328–350.CrossRefGoogle Scholar
  20. Harzing, A.-W. (2006). Response styles in cross-national survey research. International Journal of Cross-Cultural Management, 6(2), 243–266.CrossRefGoogle Scholar
  21. Jordan, L. A., Marcus, A. C., & Reeder, L. G. (1980). Response styles in telephone and household interviewing: A field experiment. Public Opinion Quarterly, 44(2), 210–222.CrossRefGoogle Scholar
  22. Kiesler, S., & Sproul, L. S. (1986). Response effects in the electronic survey. Public Opinion Quarterly, 50(3), 402–143.CrossRefGoogle Scholar
  23. Kumar, A., & Dillon, W. R. (1992). An integrative look at the use of additive and multiplicative covariance structure models in the analysis of MTMM data. Journal of Marketing Research, 39(1), 51–64.CrossRefGoogle Scholar
  24. Kwak, H., Jaju, A., & Larsen, T. (2006). Consumer ethnocentrism offline and online: the mediating role of marketing efforts and personality traits in the United States, South Korea, and India. Journal of the Academy of Marketing Science, 34(3), 367–385.CrossRefGoogle Scholar
  25. Little, T. D. (2000). On the comparability of constructs in cross-cultural research: A critique of Cheung and Rensvold. Journal of Cross-Cultural Psychology, 31(2), 213–219.CrossRefGoogle Scholar
  26. Marsh, H. W., & Bailey, M. (1991). Confirmatory factor analyses of multitrait-multimethod data: a comparison of alternative models. Applied Psychological Measurement, 15(1), 47–70.CrossRefGoogle Scholar
  27. Marsh, H. W., Bailey, M., Balla, J. R., & McDonald, R. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103(3), 391–410.CrossRefGoogle Scholar
  28. McClendon, M. J. (1991). Acquiescence and response-order effects in interview surveys. Sociological Methods and Research, 20(1), 60–103.CrossRefGoogle Scholar
  29. McGee, R. K. (1967). Response set in relation to personality: An orientation. In I. A. Berg (Ed.) Response set in personality assessment (pp. 1–31). Chicago: Aldine.Google Scholar
  30. Muthén, L. K., & Muthén, B. O. (2004). Mplus, statistical analysis with latent variables: user guide (3rd ed.). Los Angeles, CA: Muthén & Muthén.Google Scholar
  31. Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903.CrossRefGoogle Scholar
  32. Rorer, L. G. (1965). The great response-style myth. Psychological Bulletin, 63(3), 129–156.CrossRefGoogle Scholar
  33. Saris, W. E., & Aalberts, C. (2003). Different explanations for correlated disturbance terms in MTMM studies. Structural Equation Modeling, 10(2), 193–213.CrossRefGoogle Scholar
  34. Saris, W. E., Aalberts, C., Satorra, A., & Coenders, G. (2004). A new approach to evaluating the quality of measurement instruments: the split-ballot MTMM design. Sociological Methodology, 34, 311–347.CrossRefGoogle Scholar
  35. Sirdeshmukh, D., Singh, J., & Sabol, B. (2002). Consumer trust, value, and loyalty in relational exchanges. Journal of Marketing, 66(1), 15–37.CrossRefGoogle Scholar
  36. Steenkamp, J.-B. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–90.CrossRefGoogle Scholar
  37. Stening, B. W., & Everett, J. E. (1984). Response styles in a cross-cultural managerial study. Journal of Social Psychology, 122(22), 151–156.CrossRefGoogle Scholar
  38. Venkatesh, S., Smith, A. K., & Rangaswamy, A. (2003). Customer satisfaction and loyalty in online and offline environments. International Journal of Research in Marketing, 20(2), 153–175.Google Scholar

Copyright information

© Academy of Marketing Science 2007

Authors and Affiliations

  • Bert Weijters
    • 1
    Email author
  • Niels Schillewaert
    • 1
  • Maggie Geuens
    • 2
  1. 1.Vlerick Leuven Gent Management SchoolGhentBelgium
  2. 2.Ghent UniversityGhentBelgium

Personalised recommendations