Abstract
The composite direct product (CDP) model is a multiplicative model for multitrait-multimethod (MTMM) designs. It is extended to incomplete MTMM correlation matrices where some trait-method combinations are not available. Rules for omitting trait-method combinations without resulting in an indeterminate model are also suggested. Maximum likelihood estimation and the log absolute correlation procedure are used to fit the model, and are found to yield similar results. The balanced incomplete MTMM design tends to yield more accurate estimates than the randomly missing design.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arthur, W.A., Jr., Woehr, D.J., & Maldegen, R. (2000). Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical re-examination of the assessment center construct-related validity paradox. Journal of Management, 26, 813–815.
Bobrow, W., & Leonards, J.S. (1997). Development and validation of an assessment center during organizational change. Journal of Social Behavior and Personality, 12, 217–236.
Browne, M.W. (1982). Covariance structures. In D.M. Hawkins (Ed.), Topics in applied multivariate analysis (pp. 72–141). Cambridge, England: Cambridge University Press.
Browne, M.W. (1984). The decomposition of multitrait-multimethod matrices. British Journal of Mathematical and Statistical Psychology, 37, 1–21.
Browne, M.W. (2000). Cross-validation methods. Journal of Mathematical Psychology, 44, 108–132.
Browne, M.W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K.A. Bollen & J.S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.
Browne, M.W., & Strydom, H.F. (1997). Non-iterative fitting of the direct product model for multitrait-multimethod matrices. In M. Berkane (Ed.), Latent variable modeling with applications to causality (pp. 229–145). New York: Springer-Verlag.
Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.
Campbell, D.T., & O’Connell, E.J. (1967). Method factors in multitrait-multimethod matrices: Multiplicative rather than additive. Multivariate Behavioral Research, 2, 409–426.
Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317–327.
Dean, A., & Voss, D. (1999). Design and analysis of experiments. New York: Springer-Verlag.
Flamer, S. (1983). Assessment of the multitrait-multimethod matrix validity of Likert scales via confirmatory factor analysis. Multivariate Behavioral Research, 18, 275–308.
Harris, M.M., Becker, A.S., & Smith, D.E. (1993). Does the assessment center scoring method affect the cross-situational consistency of ratings? Journal of Applied Psychology, 78, 675–678.
Jennrich, R.I. (1970). An asymptotic chi-square test for the equality of two correlation matrices. Journal of the American Statistical Association, 65, 904–915.
Jöreskog, K.G. (1974). Analysing psychological data by structural analysis of covariance matrices. In R.C. Atkinson et al. (Eds.), Contemporary developments in mathematical psychology (pp. 1–16). San Francisco: Freeman.
Jöreskog, K.G., & Söbom, D. (1996). LISREL 8: User’s reference guide. Chicago: Scientific Software.
Joyce, L.W., Thayer, P.W., & Pond, S.B. (1994). Managerial functions: An alternative to traditional assessment center dimensions? Personnel Psychology, 47, 109–121.
Kudisch, J.D., Ladd, R.T., & Dobbins, G.H. (1997). New evidence on the construct validity of diagnostic assessment centers: The findings may not be so troubling after all. Journal of Social Behavior and Personality, 12, 129–144.
Lance, C.E., Foster, M.R., Gentry, W.A., & Thoresen, J.D. (2004). Assessor cognitive processes in an operational assessment center. Journal of Applied Psychology, 89, 22–35.
Lievens, P., & van Keer, E. (2001). The construct validity of a Belgian assessment center: A comparison of different models. Journal of Occupational and Organizational Psychology, 74, 373–378.
Marsh, H.W. (1990). Confirmatory factor analysis of multitrait-multimethod data: The construct validation of multidimensional self-concept responses. Journal of Personality, 58, 661–692.
Marsh, H.W., & Hocevar, D. (1984). The factorial invariance of students’ evaluations of college teaching. American Educational Research Journal, 21, 341–366.
Neale, M.C., & Miller, M.B. (1997). The use of likelihood-based confidence intervals in genetic models. Behavior Genetics, 27, 113–120.
Neale, M.C., Boker, S.M., Xie, G., & Maes, H.H. (2004). Mx: Statistical modeling (6th ed., last revised on April 15, 2004). Richmond, VA: Author (http://www.vcu.edu/mx)
Saris, W.E., Satorra, A., & Coenders, G. (2004). A new approach to evaluating the quality of measurement instruments: The split-ballot MTMM design. Sociological Methodology, 34, 311–347.
Seber, G.A.F. (1977). Linear regression analysis. New York: Wiley.
Steiger, J.H. (1989). EzPATH: A supplementary module for SYSTAT and SYGRAPH. Evanston, IL: SYSTAT.
Swain, A.J. (1975). Analysis of parametric structures for variance matrices. Unpublished doctoral dissertation, University of Adelaide.
Wothke, W. (1996). Models for multitrait-multimethod analysis. In G.A. Marcoulides & R.E. Schumacker (Eds.), Advanced structural equation modeling: Issues and techniques. Hillsdale, NJ: Erlbaum.
Wothke, W., & Browne, M.W. (1990). The direct product model for the MTMM matrix parametrized as a second-order factor analysis model. Psychometrika, 55, 255–262.
Zhang, G., & Browne, M.W. (2004). LINLOGM: User guide. Department of Psychology, The Ohio State University.
Author information
Authors and Affiliations
Corresponding author
Additional information
We thank Filip Lievens for bringing the problem of incomplete MTMM matrices to our attention and for providing us with his data, Herbert Marsh for providing us with information about MTMM data sets, and Kris Preacher for reading the manuscript and suggesting changes. We thank the Associate Editor and two anonymous reviewers for extensive and constructive suggestions. In particular, we are indebted to a reviewer for suggesting the use of Mx for maximum likelihood estimation and sending us Mx code (that we adapted to suit our formulation of the model).
Rights and permissions
About this article
Cite this article
Zhang, G., Browne, M.W. Design And Analysis Of Incomplete Multitrait-Multimethod Studies From A Multiplicative Perspective. Psychometrika 72, 361–375 (2007). https://doi.org/10.1007/s11336-004-1224-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-004-1224-3