Skip to main content
SpringerLink
Log in

Assessing response styles across modes of data collection

  • Published:
Journal of the Academy of Marketing Science Aims and scope Submit manuscript

Abstract

Cross-mode surveys are on the rise. The current study compares levels of response styles across three modes of data collection: paper-and-pencil questionnaires, telephone interviews, and online questionnaires. The authors make the comparison in terms of acquiescence, disacquiescence, and extreme and midpoint response styles. To do this, they propose a new method, namely, the representative indicators response style means and covariance structure (RIRSMACS) method. This method contributes to the literature in important ways. First, it offers a simultaneous operationalization of multiple response styles. The model accounts for dependencies among response style indicators due to their reliance on common item sets. Second, it accounts for random error in the response style measures. As a consequence, random error in response style measures is not passed on to corrected measures. The method can detect and correct cross-mode response style differences in cases where measurement invariance testing and multitrait multimethod designs are inadequate. The authors demonstrate and discuss the practical and theoretical advantages of the RIRSMACS approach over traditional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2

Similar content being viewed by others

Notes

  1. A further elaboration of a similar example is appears in Appendix A of ESM 1, which is available on request from the first author. For a full understanding of Appendix A, it is recommended to first read the empirical study in the body of the paper.

  2. Note that this requirement is independent of the relationship between measurement invariance and response styles that Cheung and Rensvold (2000) discussed. They stated that measurement noninvariance may be indicative of response styles. We posit that measures of response styles need to meet the condition of measurement invariance to be valid and useful for group comparisons of response style levels.

  3. An additional test on a P&P and an online sample appears in Appendix B of ESM 1, which is available from the first author on request.

  4. See also Appendix A in ESM 1, which is available from the first author on request.

  5. On the basis of a random sample (N = 40) of scales from the Marketing Scales Handbook by Bruner et al. (2001), it is estimated that marketing scales consist of 6.43 items on average (SD = 3.70).

References

  • Ayidiya, S., & McClendon, M. J. (1990). Response effects in mail surveys. Public Opinion Quarterly, 54(2), 229–247.

    Article  Google Scholar 

  • Bagozzi, R. P., & Yi, Y. (1990). Assessing method variance in multitrait-multimethod matrices: The case of self-reported affect and perceptions at work. Journal of Applied Psychology, 75(5), 547–560.

    Article  Google Scholar 

  • Baumgartner, H., & Steenkamp, J.-B. E. M. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143–156, May.

    Article  Google Scholar 

  • Baumgartner, H., & Steenkamp, J.-B. E. M. (2006). An extended paradigm for measurement analysis of marketing constructs applicable to panel data. Journal of Marketing Research, 43, 431–442, (August).

    Article  Google Scholar 

  • Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling, 7(4), 608–628.

    Article  Google Scholar 

  • Bruner, G. C., James, K. E., & Hensel, P. J. (2001). Marketing scales handbook: A compilation of multi-item measures, vol. 3. Chicago: American Marketing Association.

    Google Scholar 

  • Cheung, G. W., & Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equation modeling. Journal of Cross-Cultural Psychology, 31(2), 187–212.

    Article  Google Scholar 

  • Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indices for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.

    Article  Google Scholar 

  • Cox III., E. P. (1980). The optimal number of response alternatives for a scale: A review. Journal of Marketing Research, 17(4), 407–422.

    Article  Google Scholar 

  • De Jong, M. G., Steenkamp, J.-B. E .M., Fox, J.-P., & Baumgartner, H. (2007). Using item response theory to measure extreme response style in marketing research: A global investigation. Journal of Marketing Research (in press).

  • De Leeuw, E. D. (2005). To mix or not to mix: data collection modes in surveys. Journal of Official Statistics, 21(2), 233–255.

    Google Scholar 

  • Deutskens, E. C., de Ruyter, K., & Wetzels, M. G. M. (2006). An assessment of equivalence between online and mail surveys in service research. Journal of Service Research, 8(4), 346–355.

    Article  Google Scholar 

  • Drolet, A., & Morrison, D. G. (2001). Do we really need multiple-item measures in service research? Journal of Service Research, 3(3), 196–204.

    Article  Google Scholar 

  • Ferrando, P. J., & Lorenzo-Seva, U. (2005). IRT-related factor analytic procedures for testing the equivalence of paper-and-pencil and internet-administered questionnaires. Psychological Methods, 10(2), 193–205.

    Article  Google Scholar 

  • Finney, S. J., & DiStefano, C. (2006). Nonnormal and categorical data in. In G. R. Hancock, & R. O. Mueller (Eds.) Structural equation modeling: A second course. Greenwich, CT: Information Age Publishing.

    Google Scholar 

  • Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables. Journal of Marketing Research, 18(1), 39–50.

    Article  Google Scholar 

  • Graham, J. W., Taylor, B. J., Olchowski, A. E., & Cumsille, P. E. (2006). Planned missing data designs in psychological research. Psychological Methods, 11(4), 323–343.

    Article  Google Scholar 

  • Greenleaf, E. A. (1992a). Improving rating scale measures by detecting and correcting bias components in some response styles. Journal of Marketing Research, 29(2), 176–188.

    Article  Google Scholar 

  • Greenleaf, E. A. (1992b). Measuring extreme response style. Public Opinion Quarterly, 56(3), 328–350.

    Article  Google Scholar 

  • Harzing, A.-W. (2006). Response styles in cross-national survey research. International Journal of Cross-Cultural Management, 6(2), 243–266.

    Article  Google Scholar 

  • Jordan, L. A., Marcus, A. C., & Reeder, L. G. (1980). Response styles in telephone and household interviewing: A field experiment. Public Opinion Quarterly, 44(2), 210–222.

    Article  Google Scholar 

  • Kiesler, S., & Sproul, L. S. (1986). Response effects in the electronic survey. Public Opinion Quarterly, 50(3), 402–143.

    Article  Google Scholar 

  • Kumar, A., & Dillon, W. R. (1992). An integrative look at the use of additive and multiplicative covariance structure models in the analysis of MTMM data. Journal of Marketing Research, 39(1), 51–64.

    Article  Google Scholar 

  • Kwak, H., Jaju, A., & Larsen, T. (2006). Consumer ethnocentrism offline and online: the mediating role of marketing efforts and personality traits in the United States, South Korea, and India. Journal of the Academy of Marketing Science, 34(3), 367–385.

    Article  Google Scholar 

  • Little, T. D. (2000). On the comparability of constructs in cross-cultural research: A critique of Cheung and Rensvold. Journal of Cross-Cultural Psychology, 31(2), 213–219.

    Article  Google Scholar 

  • Marsh, H. W., & Bailey, M. (1991). Confirmatory factor analyses of multitrait-multimethod data: a comparison of alternative models. Applied Psychological Measurement, 15(1), 47–70.

    Article  Google Scholar 

  • Marsh, H. W., Bailey, M., Balla, J. R., & McDonald, R. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103(3), 391–410.

    Article  Google Scholar 

  • McClendon, M. J. (1991). Acquiescence and response-order effects in interview surveys. Sociological Methods and Research, 20(1), 60–103.

    Article  Google Scholar 

  • McGee, R. K. (1967). Response set in relation to personality: An orientation. In I. A. Berg (Ed.) Response set in personality assessment (pp. 1–31). Chicago: Aldine.

    Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (2004). Mplus, statistical analysis with latent variables: user guide (3rd ed.). Los Angeles, CA: Muthén & Muthén.

    Google Scholar 

  • Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903.

    Article  Google Scholar 

  • Rorer, L. G. (1965). The great response-style myth. Psychological Bulletin, 63(3), 129–156.

    Article  Google Scholar 

  • Saris, W. E., & Aalberts, C. (2003). Different explanations for correlated disturbance terms in MTMM studies. Structural Equation Modeling, 10(2), 193–213.

    Article  Google Scholar 

  • Saris, W. E., Aalberts, C., Satorra, A., & Coenders, G. (2004). A new approach to evaluating the quality of measurement instruments: the split-ballot MTMM design. Sociological Methodology, 34, 311–347.

    Article  Google Scholar 

  • Sirdeshmukh, D., Singh, J., & Sabol, B. (2002). Consumer trust, value, and loyalty in relational exchanges. Journal of Marketing, 66(1), 15–37.

    Article  Google Scholar 

  • Steenkamp, J.-B. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–90.

    Article  Google Scholar 

  • Stening, B. W., & Everett, J. E. (1984). Response styles in a cross-cultural managerial study. Journal of Social Psychology, 122(22), 151–156.

    Article  Google Scholar 

  • Venkatesh, S., Smith, A. K., & Rangaswamy, A. (2003). Customer satisfaction and loyalty in online and offline environments. International Journal of Research in Marketing, 20(2), 153–175.

    Google Scholar 

Download references

Acknowledgement

The authors would like to thank the Intercollegiate Center for Management Sciences (Belgium) and Insites for supporting the research reported in this paper. Further, the authors would like to thank the following people for their feedback on previous versions of the paper: Hans Baumgartner, Jaak Billiet, Marion Debruyne, Koen Dewettinck, Alain De Beuckelaer and Patrick Van Kenhove.

Author information

Authors and Affiliations

  1. Vlerick Leuven Gent Management School, Reep 1, 9000, Ghent, Belgium

    Bert Weijters & Niels Schillewaert

  2. Ghent University, Hoveniersberg 24, 9000, Ghent, Belgium

    Maggie Geuens

Authors
  1. Bert Weijters
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Niels Schillewaert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maggie Geuens
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bert Weijters.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplement 1

ESM 1 (PDF 319 341 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Weijters, B., Schillewaert, N. & Geuens, M. Assessing response styles across modes of data collection. J. of the Acad. Mark. Sci. 36, 409–422 (2008). https://doi.org/10.1007/s11747-007-0077-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11747-007-0077-6

Keywords

Search

Navigation