Journal of Business and Psychology

, Volume 26, Issue 3, pp 241–248 | Cite as

To Aggregate or Not to Aggregate: Steps for Developing and Validating Higher-Order Multidimensional Constructs

  • Russell E. JohnsonEmail author
  • Christopher C. Rosen
  • Chu-Hsiang Chang


The use of higher-order multidimensional constructs (i.e., latent constructs comprised of standalone variables) in the organizational psychology and behavior literatures is becoming commonplace. Despite their advantages (e.g., greater parsimony and bandwidth), the development and validation of such constructs often unfolds in an indiscriminant fashion. It is not surprising, then, that much debate has arisen regarding the viability of many higher-order constructs. In this article, we outline ten recommendations for improving the construct- and criterion-related validity of higher-order constructs. Chief among these recommendations include the need for researchers to specify precise theoretical and empirical inclusion criteria, to rule out alternative explanations for the emergence of a higher-order factor and to assess incremental and relative importance. To illustrate how these recommendations play out, we apply them to core self-evaluation as an example. We believe that higher-order constructs may offer unique insight into work-relevant phenomena, provided they are established via rigorous means.


Higher-order constructs Multidimensional constructs Validation Common method variance Incremental and relative importance Core self-evaluation 


  1. Ambrose, M. L., & Schminke, M. (2009). The role of overall justice judgments in organizational justice research: A test of mediation. Journal of Applied Psychology, 94, 491–500.PubMedCrossRefGoogle Scholar
  2. Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411–423.CrossRefGoogle Scholar
  3. Bollen, K. A., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110, 305–314.CrossRefGoogle Scholar
  4. Bollen, K. A., & Ting, K. -F. (2000). A tetrad test for causal indicators. Psychological Methods, 5, 3–22.PubMedCrossRefGoogle Scholar
  5. Chang, C. -H., Ferris, D. L., Johnson, R. E., Rosen, C. C., & Tan, J. A. (in press). Core self-evaluations: A review and evaluation of the literature. Journal of Management (2012 Review Issue).Google Scholar
  6. Chen, G. (in press). Evaluating the core: Critical assessment of core self-evaluations theory. Journal of Organizational Behavior.Google Scholar
  7. Conway, J. M., & Lance, C. E. (2010). What reviewers should expect from authors regarding common method bias in organizational research. Journal of Business and Psychology, 25, 325–334.CrossRefGoogle Scholar
  8. Crampton, S. M., & Wagner, J. A., I. I. I. (1994). Percept-percept inflation in microorganizational research: An investigation of prevalence and effect. Journal of Applied Psychology, 79, 67–76.CrossRefGoogle Scholar
  9. Diamantopoulos, A., & Siguaw, J. A. (2006). Formative versus reflective indicators in organizational measure development: A comparison and empirical illustration. British Journal of Management, 17, 263–282.CrossRefGoogle Scholar
  10. Digman, J. M. (1997). Higher order factors of the Big Five. Journal of Personality and Social Psychology, 73, 1246–1256.PubMedCrossRefGoogle Scholar
  11. Edwards, J. R. (2001). Multidimensional constructs in organizational behavior research: An integrative analytical framework. Organizational Research Methods, 4, 144–192.CrossRefGoogle Scholar
  12. Edwards, J. R. (2011). The fallacy of formative measurement. Organizational Research Methods, 14, 370–388.CrossRefGoogle Scholar
  13. Ferris, D. L., Rosen, C. C., Johnson, R. E., Brown, D. J., Risavy, S., & Heller, D. (2011). Approach or avoidance (or both?): Integrating core self-evaluations within an approach/avoidance framework. Personnel Psychology, 64, 137–161.CrossRefGoogle Scholar
  14. Franke, G. R., Preacher, K. J., & Rigdon, E. E. (2008). Proportional structural effects of formative indicators. Journal of Business Research, 61, 1229–1237.CrossRefGoogle Scholar
  15. Gerbing, D. W., Hamilton, J. G., & Freeman, E. B. (1994). A large-scale second-order structural equation model of the influence of management participation on organizational planning benefits. Journal of Management, 20, 859–885.Google Scholar
  16. Goldberg, L. R. (1992). The development of markers for the big-five factor structure. Journal of Personality and Social Psychology, 59, 1216–1229.CrossRefGoogle Scholar
  17. Harrison, D. A., McLaughlin, M. E., & Coalter, T. M. (1996). Context, cognition, and common method variance: Psychometric and verbal protocol evidence. Organizational Behavior and Human Decision Processes, 68, 246–261.CrossRefGoogle Scholar
  18. Harrison, D. A., Newman, D. A., & Roth, P. L. (2006). How important are job attitudes? Meta-analytic comparisons of integrative behavioral outcomes and time sequences. Academy of Management Journal, 49, 305–325.CrossRefGoogle Scholar
  19. Jenkins, M., & Griffith, R. (2004). Using personality constructs to predict performance: Narrow or broad bandwidth. Journal of Business and Psychology, 19, 255–269.CrossRefGoogle Scholar
  20. Johnson, J. W. (2000). A heuristic method for estimating the relative weight of predictor variables in multiple regression. Multivariate Behavioral Research, 35, 1–19.CrossRefGoogle Scholar
  21. Johnson, R. E., Rosen, C. C., & Levy, P. E. (2008). Getting to the core of core self-evaluations: A review and recommendations. Journal of Organizational Behavior, 29, 391–413.CrossRefGoogle Scholar
  22. Johnson, R. E., Rosen, C. C., Chang, C. -H., Djurdjevic, E., & Taing, M. U. (in press-a). Recommendations for improving the construct clarity of higher-order multidimensional constructs. Human Resource Management Review.Google Scholar
  23. Johnson, R. E., Rosen, C. C., & Djurdjevic, E. (in press-b). Assessing the impact of common method variance on higher-order multidimensional constructs. Journal of Applied Psychology.Google Scholar
  24. Judge, T. A., Locke, E. A., & Durham, C. C. (1997). The dispositional causes of job satisfaction: A core evaluations approach. Research in Organizational Behavior, 19, 151–188.Google Scholar
  25. Judge, T. A., Erez, A., Bono, J. E., & Thoresen, C. J. (2003). The core self-evaluations scale: Development of a measure. Personnel Psychology, 56, 303–331.CrossRefGoogle Scholar
  26. Kelley, T. L. (1927). Interpretation of educational measurements. Yonkers, New York: World Book Company.Google Scholar
  27. Kline, T. J. B., Sulsky, L. M., & Rever-Moriyama, S. D. (2002). Common method variance and specification errors: A practical approach to detection. Journal of Psychology, 134, 401–421.CrossRefGoogle Scholar
  28. Law, K. S., Wong, C. S., & Mobley, W. H. (1998). Toward a taxonomy of multidimensional constructs. Academy of Management Review, 23, 741–755.Google Scholar
  29. LeBreton, J. M., Hargis, M. B., Griepentrog, B., Oswald, F. L., & Ployhart, R. E. (2007). A multidimensional approach for evaluating variables in organizational research and practice. Personnel Psychology, 60, 475–498.CrossRefGoogle Scholar
  30. Macey, W. H., & Schneider, B. (2008). The meaning of employee engagement. Industrial and Organizational Psychology: Perspectives on Science and Practice, 1, 3–30.CrossRefGoogle Scholar
  31. MacKenzie, S. B., Podsakoff, P. M., & Jarvis, C. B. (2005). The problem of measurement model misspecification in behavioral and organizational research and some recommended solutions. Journal of Applied Psychology, 90, 710–730.PubMedCrossRefGoogle Scholar
  32. McCrae, R. R., & Costa, P. T., Jr. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52, 81–90.PubMedCrossRefGoogle Scholar
  33. Podsakoff, P. M., MacKenzie, S. B., Lee, J. -Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903.PubMedCrossRefGoogle Scholar
  34. Richardson, H. A., Simmering, M. J., & Sturman, M. C. (2009). A tale of three perspectives: Examining post hoc statistical techniques for detection and correction of common method variance. Organizational Research Methods, 12, 762–800.CrossRefGoogle Scholar
  35. Rosen, C. C., Chang, C. H., Johnson, R. E., & Levy, P. E. (2009). Perceptions of the organizational context and psychological contract breach: Assessing competing perspectives. Organizational Behavior and Human Decision Processes, 108, 202–217.CrossRefGoogle Scholar
  36. Rushton, J. P., & Irwing, P. (2008). A general factor of personality (GFP) from two meta-analyses of the Big Five: Digman (1997) and Mount, Barrick, Scullen, and Rounds (2005). Personality and Individual Differences, 45, 679–683.CrossRefGoogle Scholar
  37. Schmitt, N. (2004). Beyond the Big Five: Increases in understanding and practical utility. Human Performance, 17, 349–359.CrossRefGoogle Scholar
  38. Spearman, C. (1904). “General intelligence” objectively determined and measured. American Journal of Psychology, 15, 201–293.CrossRefGoogle Scholar
  39. Spector, P. E. (1992). Summated rating scales. Thousand Oaks, CA: Sage.Google Scholar
  40. Taing, M. U., Johnson, R. E., & Jackson, E. M. (2010, August). On the nature of core self-evaluation: A formative or reflective construct? Paper presented at the 70th Academy of Management Annual Meeting, Montreal, Quebec.Google Scholar
  41. Tonidandel, S., & LeBreton, J. M. (2011). Relative importance analysis: A useful supplement to regression analysis. Journal of Business and Psychology, 26, 1–9.CrossRefGoogle Scholar
  42. Tonidandel, S., LeBreton, J. M., & Johnson, J. W. (2009). Statistical significance tests for relative weights. Psychological Methods, 14, 387–399.PubMedCrossRefGoogle Scholar
  43. Williams, L. J., Cote, J. A., & Buckley, M. R. (1989). Lack of method variance in self-reported affect and perceptions at work: Reality or artifact? Journal of Applied Psychology, 74, 462–468.CrossRefGoogle Scholar
  44. Williams, L. J., Hartman, N., & Cavazotte, F. (2010). Method variance and marker variables: A review and comprehensive CFA marker technique. Organizational Research Methods, 13, 477–514.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Russell E. Johnson
    • 1
    Email author
  • Christopher C. Rosen
    • 2
  • Chu-Hsiang Chang
    • 3
  1. 1.Department of Management, Broad College of BusinessMichigan State UniversityEast LansingUSA
  2. 2.Department of Management, Sam M. Walton College of BusinessUniversity of ArkansasFayettevilleUSA
  3. 3.Department of PsychologyMichigan State UniversityEast LansingUSA

Personalised recommendations