Advertisement

Journal of the Academy of Marketing Science

, Volume 44, Issue 1, pp 119–134 | Cite as

Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies

  • Clay M. Voorhees
  • Michael K. Brady
  • Roger Calantone
  • Edward Ramirez
Methodological Paper

Abstract

The results of this research suggest a new mandate for discriminant validity testing in marketing. Specifically, the authors demonstrate that the AVE-SV comparison (Fornell and Larcker 1981) and HTMT ratio (Henseler et al. 2015) with 0.85 cutoff provide the best assessment of discriminant validity and should be the standard for publication in marketing. These conclusions are based on a thorough assessment of the literature and the results of a Monte Carlo simulation. First, based on a content analysis of articles published in seven leading marketing journals from 1996 to 2012, the authors demonstrate that three tests—the constrained phi (Jöreskog 1971), AVE-SV (Fornell and Larcker 1981), and overlapping confidence intervals (Anderson and Gerbing 1988)—are by far most common. Further review reveals that (1) more than 20% of survey-based and over 80% of non-survey-based marketing studies fail to document tests for discriminant validity, (2) there is wide variance across journals and research streams in terms of whether discriminant validity tests are performed, (3) conclusions have already been drawn about the relative stringency of the three most common methods, and (4) the method that is generally perceived to be most generous is being consistently misapplied in a way that erodes its stringency. Second, a Monte Carlo simulation is conducted to assess the relative rigor of the three most common tests, as well as an emerging technique (HTMT). Results reveal that (1) on average, the four discriminant validity testing methods detect violations approximately 50% of the time, (2) the constrained phi and overlapping confidence interval approaches perform very poorly in detecting violations whereas the AVE-SV test and HTMT (with a ratio cutoff of 0.85) methods perform well, and (3) the HTMT.85 method offers the best balance between high detection and low arbitrary violation (i.e., false positive) rates.

Keywords

Discriminant validity Theory testing Monte Carlo simulation Measurement Structural equation modeling Survey research PLS PLS-SEM HTMT Heterotrait-monotrait 

Notes

Acknowledgments

The authors would like to thank Peter Bentler for his comments on an earlier version of the simulations used in this paper.

References

  1. Anderson, J. C., & Gerbing, D. W. (1988). Structural Equation Modeling in Practice: A Review and Recommended Two-Step Approach. Psychological Bulletin, 103(3), 411–423.Google Scholar
  2. Bacharach, S. B. (1989). Organizational theories: Some criteria for evaluation. Academy of Management Review, 14(4), 496–515.Google Scholar
  3. Baggozi, R. P., & Philips, L. W. (1982). Representing and Testing Organizational Theories: A Holistic Construal. Administrative Science Quarterly, 27(3), 459–489.Google Scholar
  4. Bagozzi, R. P. (1981). Evaluating Structural equation models with unobservable variables and measurement error: a comment. Journal of Marketing Research, 18(3), 375–381.Google Scholar
  5. Batra, R., & Sinha, I. (2000). Consumer-Level Factors Moderating the Success of Private Label Brands. Journal of Retailing, 76(2), 175–191.Google Scholar
  6. Burton, S., Liechenstein, D. R., Netemeyer, R. G., & Garreston, J. A. (1998). A Scale for Measuring Attitude Toward Private Label Products and Examination of Its Psychological and Behavioral Correlates. Journal of the Academy of Marketing Science, 26(4), 293–306.Google Scholar
  7. Campbell, D. T., & Fiske, D. W. (1959). Convergent and Discriminant Validity by the Multitrait-Multimethod Matrix. Psychological Bulletin, 56(2), 81–105.Google Scholar
  8. Cannon, J. P., & Homburg, C. (2001). Buyer–supplier Relationships and Firm Costs. Journal of Marketing, 65(1), 29–43.Google Scholar
  9. Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16(1), 64–73.CrossRefGoogle Scholar
  10. Farrell, A. M. (2010). Insufficient discriminant validity: A comment on Bove, Pervan, Beatty, and Shiu (2009). Journal of Business Research, 63(3), 324–327.Google Scholar
  11. Finn, J. D. (1974). A general model for multivariate analysis. New York: Holt, Rinehart, & Winston.Google Scholar
  12. Folger, R. (1989). Significance Tests and the Duplicity of Binary Decisions. Psychological Bulletin, 106(1), 155–160.Google Scholar
  13. Fornell, C., & Larcker, D. F. (1981). Evaluating Structural Equation Models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50.Google Scholar
  14. Frambach, R. T., Prabhu, J., & Verhallen, T. M. M. (2003). The Influence of Business Strategy on New Product Activity: The Role of Market Orientation. International Journal of Research in Marketing, 20(4), 377–397.Google Scholar
  15. Grewal, R., Cote, J. A., & Baumgartner, H. (2004). Multicollinearity and Measurement Error in Structural Equation Models: Implications for Theory Testing. Marketing Science, 23(4), 519–529.Google Scholar
  16. Harris, L. C., & Goode, M. M. H. (2004). The four levels of loyalty and the pivotal role of trust: a study of online service dynamics. Journal of Retailing, 80, 139–158.Google Scholar
  17. Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43, 115–135.CrossRefGoogle Scholar
  18. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453.Google Scholar
  19. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.Google Scholar
  20. Jap, S. D. (2001). “Pie Sharing” in Complex Collaboration Contexts. Journal of Marketing Research, 38(1), 86–99.Google Scholar
  21. Joreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36(4), 409–426.CrossRefGoogle Scholar
  22. Kohli, A. K., Shervani, T. A., & Challagalla, G. N. (1998). Learning and Performance Orientation of Salespeople: The Role of Supervisors. Journal of Marketing Research, 35(2), 263–274.CrossRefGoogle Scholar
  23. Lord, F. M. (1957), A signif'icance test for the hypothesis that two variables measure the same trait except for errors of'measurement, Psychometrika, 207–220.Google Scholar
  24. Low, G. S., & Mohr, J. J. (2001). Factors Affecting the Use of Information in the Evaluation of Marketing Communications Productivity. Journal of the Academy of Marketing Science, 29(1), 70–88.CrossRefGoogle Scholar
  25. Lytle, R. S., Hom, P. W., & Mokwa, M. P. (1998). SERV*OR: A Managerial Measure of Service Orientation. Journal of Retailing, 74(4), 455–589.Google Scholar
  26. Mathwick, C., & Rigdon, E. (2004). Play, Flow, and the Online Search Experience. Journal of Consumer Research, 31(2), 324–332.Google Scholar
  27. Mathwick, C., Malhotra, N., & Rigdon, E. (2001). Experiential Value: Conceptualization, Measurement, and Application in the Catalog and Internet Shopping Environment. Journal of Retailing, 77(1), 39–56.Google Scholar
  28. Maxham, J. G., & Netemeyer, R. G. (2002). A Longitudinal Study of Complaining Customers’ Evaluations of Multiple Service Failures and Recovery Efforts. Journal of Marketing, 66(4), 57–71.Google Scholar
  29. Peter, J. P. (1981). Construct Validity: A Review of Basic Issues and Marketing Practices. Journal of Marketing Research, 18(2), 133–145.Google Scholar
  30. Pollard, P. (2014), How Significant is ‘Significance’? in A Handbook for Data Analysis in the Behaviorial Sciences, Volume 1: Methodological Issues Volume 2: Statistical Issues, 449Google Scholar
  31. Rich, G. A. (1997). The Sales Manager as a Role Model: Effects on Trust, Job Satisfaction, and Performance of Salespeople. Journal of the Academy of Marketing Science, 25(4), 319–328.Google Scholar
  32. Spreng, R. A., MacKenzie, S. B., & Olshavsky, R. W. (1996). A Reexamination of the Determinants of Consumer Satisfaction. Journal of Marketing, 60(3), 15–32.Google Scholar
  33. Wang, G., & Netemeyer, R. G. (2002). The effects of job autonomy, customer demandingness, and trait competitiveness on salesperson learning, self-efficacy, and performance. Journal of the Academy of Marketing Science, 30(3), 217–228.CrossRefGoogle Scholar
  34. Yilmaz, C., & Hunt, S. D. (2001). Salesperson Cooperation: The Influence of Relational, Task, Organizational, and Personal Factors. Journal of the Academy of Marketing Science, 29(4), 335–357.Google Scholar

Copyright information

© Academy of Marketing Science 2015

Authors and Affiliations

  • Clay M. Voorhees
    • 1
  • Michael K. Brady
    • 2
  • Roger Calantone
    • 1
  • Edward Ramirez
    • 3
  1. 1.Department of Marketing, The Eli Broad Graduate School of ManagementMichigan State UniversityEast LansingUSA
  2. 2.Department of MarketingFlorida State UniversityTallahasseeUSA
  3. 3.Department of Marketing and Management, College of Business AdministrationUniversity of Texas at El PasoEl PasoUSA

Personalised recommendations