Skip to main content
SpringerLink
Log in

A Multimethod Latent State-Trait Model for Structurally Different And Interchangeable Methods

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

A new multiple indicator multilevel latent state-trait (LST) model for the analysis of multitrait–multimethod–multioccasion (MTMM-MO) data is proposed. The LST-COM model combines current CFA-MTMM modeling approaches of interchangeable and structurally different methods and LST modeling approaches. The model enables researchers to specify construct and method factors on the level of time-stable (trait) as well as time-variable (occasion-specific) latent variables and analyze the convergent and discriminant validity among different rater groups across time. The statistical performance of the model is scrutinized by a simulation study and guidelines for empirical applications are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The complete files of this simulation study can be obtained on request by the first author via Email (tobias.koch@leuphana.de).

References

  • Bentler, P., & Chou, C.-P. (1987). Practical issues in structural modeling. Sociological Methods & Research, 17, 78–117. doi:10.1177/0049124187016001004.

    Article  Google Scholar 

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56(2), 81–105. doi:10.1037/h0046016.

    Article  PubMed  Google Scholar 

  • Ciesla, J. A., Cole, D. A., & Steiger, J. H. (2007). Extending the trait-state-occasion model: How important is within-wave measurement equivalence? Structural Equation Modeling, 14(1), 77–97. doi:10.1080/10705510709336737.

    Google Scholar 

  • Cole, D. A., Martin, N. M., & Steiger, J. H. (2005). Empirical and conceptual problems with longitudinal trait-state models: Introducing a trait-state-occasion model. Psychological Methods, 10, 3–20. doi:10.1037/1082-989X.10.1.3.

    Article  PubMed  Google Scholar 

  • Courvoisier, D. S. (2006). Unfolding the constituents of psychological scores: Development and application of mixture and multitrait–multimethod lst models. Doctoral dissertation, University of Geneva. Retrieved from https://archive-ouverte.unige.ch/unige:447.

  • Courvoisier, D. S., Eid, M., & Nussbeck, F. W. (2007). Mixture distribution latent state-trait analysis: Basic ideas and applications. Psychological Methods, 12(1), 80–104. doi:10.1037/1082-989X.12.1.80.

    Article  PubMed  Google Scholar 

  • Courvoisier, D. S., Nussbeck, F. W., Eid, M., Geiser, C., & Cole, D. A. (2008). Analyzing the convergent and discriminant validity of states and traits: Development and applications of multimethod latent state-trait models. Psychological Assessment, 20(3), 270–280. doi:10.1037/a0012812.

    Article  PubMed  Google Scholar 

  • Crayen, C. (2008). Eine Monte-Carlo Simulationsstudie zur Untersuchung der Anwendbarkeit von Strukturgleichungsmodellen für Multitrait-Multimethod-Multioccasion Daten. [A simulation study for investigating the applicability of structural equation models for multitrait-multimethod-multioccasion data]. Diploma Thesis, Freie Universität Berlin.

  • Dumenci, L. (2000). Multitrait–multimethod analysis. In S. D. Brown & H. E. Tinsley (Eds.), Handbook of applied multivariate statistics and mathematerial modeling (pp. 583–611). San Diego, CA: Academic Press.

    Chapter  Google Scholar 

  • Eid, M. (1995). Modelle der Messung von Personen in Situationen [Models for measuring persons in situations]. Weinheim: Psychologie Verlags Union.

  • Eid, M. (1996). Longitudinal confirmatory factor analysis for polytomous item responses: Model definition and model selection on the basis of stochastic measurement theory. Methods of Psychological Research Online, 1(4), 65–85. Retrieved from http://www.dgps.de/fachgruppen/methoden/mpr-online/issue1/art4/eid.pdf.

  • Eid, M. (2000). A multitrait–multimethod model with minimal assumptions. Psychometrika, 65(2), 241–261. doi:10.1007/BF02294377.

    Article  Google Scholar 

  • Eid, M., Courvoisier, D., & Lischetzke, T. (2012). Structural equation modeling of ambulatory assessment data. In M. R. Mehl & T. Connor (Eds.), Handbook of research methods for studying daily life (pp. 384–406). New York: Guilford.

    Google Scholar 

  • Eid, M., Geiser, C., & Koch, T. (2016). Measuring method effects: From traditional to design-oriented approaches. Current Direction in Psychological Science, 25(4), 275–280. doi:10.1177/0963721416649624.

  • Eid, M., & Hoffmann, L. (1998). Measuring variability and change with an item response model for polytomous variables. Journal of Educational and Behavioral Statistics, 23, 193–215. doi:10.3102/10769986023003193.

    Article  Google Scholar 

  • Eid, M., & Langeheine, R. (1999). The measurement of consistency and occasion specificity with latent class models: A new model and its application to the measurement of affect. Psychological Methods, 4(1), 100–116. doi:10.1037/1082-989X.4.1.100.

    Article  Google Scholar 

  • Eid, M., & Langeheine, R. (2007). Detecting population heterogeneity in stability and change of subjective well-being by mixture distribution models. In A. Ong & M. van Dulmen (Eds.), Handbook of methods in positive psychology (pp. 501–607). Oxford: Oxford University Press.

    Google Scholar 

  • Eid, M., Lischetzke, T., Nussbeck, F. W., & Trierweiler, L. I. (2003). Separating trait effects from trait-specific method effects in multitrait–multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8(1), 38–60. doi:10.1037/1082-989X.8.1.38.

    Article  PubMed  Google Scholar 

  • Eid, M., Nussbeck, F. W., Geiser, C., Cole, D. A., Gollwitzer, M., & Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13(3), 230–253. doi:10.1037/a0013219.

    Article  PubMed  Google Scholar 

  • Eid, M., Nussbeck, F. W., & Lischetzke, T. (2006). Multitrait–multimethod-analyse. In F. Petermann & M. Eid (Eds.), Handbuch der Psychologischen Diagnostik (pp. 332–345). Göttingen: Hogrefe.

    Google Scholar 

  • Eid, M., Schneider, C., & Schwenkmezger, P. (1999). Do you feel better or worse? The validity of perceived deviations of mood states from mood traits. European Journal of Personality, 13(4), 283–306. doi:10.1002/(SICI)1099-0984(199907/08)13:43.0.CO;2-0.

    Article  Google Scholar 

  • Fleenor, J. W., McCauley, C. D., & Brutus, S. (1997). Self-other rating agreement and leader effectiveness. The Leadership Quarterly, 7(4), 487–506. doi:10.1016/S1048-9843(96)90003-X.

    Article  Google Scholar 

  • Geiser, C. (2009). Multitrait–multimethod–multioccasion modeling. München: Akademischer Verlag München.

    Google Scholar 

  • Geiser, C., Eid, M., Nussbeck, F., Courvoisier, D. S., & Cole, D. A. (2010). Multitrait-multimethod change modelling. Advanced Statistical Analysis, 94, 185–201. doi:10.1007/s10182-010-0127-0.

    Article  Google Scholar 

  • Geiser, C., Eid, M., & Nussbeck, F. W. (2008). On the meaning of the latent variables in the CT-C(M-1) model: A comment on Maydeu-Olivares and Coffman (2006). Psychological Methods, 13(1), 49–57. doi:10.1037/1082-989X.13.1.49.

    Article  PubMed  Google Scholar 

  • Geiser, C., Eid, M., West, S. G., Lischetzke, T., & Nussbeck, F. W. (2012). A comparison of method effects in two confirmatory factor models for structurally different methods. Structural Equation Modeling, 19(3), 409–436. doi:10.1080/10705511.2012.687658.

    Article  Google Scholar 

  • Geiser, C., Keller, B., Lockhart, G., Eid, M., Cole, D., & Koch, T. (2015). Distinguishing state variability from trait change in longitudinal data: The role of measurement (non)invariance in latent state-trait analyses. Behavior Research Methods,. doi:10.3758/s13428-014-0457-z.

    PubMed  PubMed Central  Google Scholar 

  • Geiser, C., Koch, T., & Eid, M. (2014). Data-generating mechanisms versus constructively defined latent variables in multitrait–multimethod analysis: A comment on Castro-Schilo, Widaman, and Grimm (2013). Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 509–523. doi:10.1080/10705511.2014.919816.

    Article  Google Scholar 

  • Geiser, C., & Lockhart, G. (2012). A comparison of four approaches to account for method effects in latent state-trait analyses. Psychological Methods, 17(2), 255–283. doi:10.1037/a0026977.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ghorpade, J. (2000). Managing five paradoxes of 360-degree feedback. The Academy of Management Executive, 14(1), 140–150. doi:10.5465/AME.2000.2909846.

    Google Scholar 

  • Heck, R. H., Thomas, S. L., & Tabata, L. N. (2013). Multilevel and longitudinal modeling with IBM SPSS. New York: Routledge.

    Google Scholar 

  • Hertzog, C., & Nesselroade, J. R. (1987). Beyond autoregressive models: Some implications of the trait-state distinction for the structural modeling of developmental change. Child Development, 58(1), 93–109. doi:10.2307/1130294.

    Article  PubMed  Google Scholar 

  • Hox, J. J., & Maas, C. J. (2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling, 8(2), 157–174. doi:10.1207/S15328007SEM08021.

    Article  Google Scholar 

  • Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. doi:10.1080/10705519909540118.

    Article  Google Scholar 

  • Kenny, D. A. (1976). An empirical application of confirmatory factor analysis to the multitrait–multimethod matrix. Journal of Experimental Social Psychology, 12(3), 247–252. doi:10.1016/0022-1031(76)90055-X.

    Article  Google Scholar 

  • Kenny, D. A. (1995). The multitrait–multimethod matrix: Design, analysis, and conceptual issues. In P. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory: A festschrift honoring Donald W. Fiske (pp. 111–124). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Kenny, D. A., & Kashy, D. A. (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin, 112(1), 165–172.

    Article  Google Scholar 

  • Koch, T., Eid, M., & Lochner, K. (in press). Multitrait-multimethod-analysis: The psychometric foundation of CFA-MTMM models. In P. Irwing, T. Booth, & D. Hughes (Eds.), The Wiley handbook of psychometric testing. Wiley.

  • Koch, T., Ortner, T. M., Eid, M., Caspers, J., & Schmitt, M. (2014a). Evaluating the construct validity of objective personality tests using a multitrait–multimethod–multioccasion-(MTMM-MO)-approach. European Journal of Psychological Assessment, 30(3), 208–230. doi:10.1027/1015-5759/a000212.

  • Koch, T., Schultze, M., Burrus, J., Roberts, R. D., & Eid, M. (2015). A multilevel CFA-MTMM model for nested structurally different methods. Journal of Educational and Behavioral Statistics, 40(5), 477–510. doi:10.3102/1076998615606109.

    Article  Google Scholar 

  • Koch, T., Schultze, M., Eid, M., & Geiser, C. (2014b). A longitudinal multilevel CFA-MTMM model for interchangeable and structurally different methods. Frontiers in Quantative Psychology and Measurement, 5, 311. doi:10.3389/fpsyg.2014.00311.

  • Koch, T., Schultze, M., Jeon, M., Nussbeck, F. W., Praetorius, A.-K., & Eid, M. (2016). A cross-classified CFA-MTMM model for structurally different and nonindependent interchangeable methods. Multivariate Behavioral Research, 51(1), 67–85. doi:10.1080/00273171.2015.1101367. (PMID: 26881958).

    Article  PubMed  Google Scholar 

  • Little, T. D., Schnabel, K. U., & Baumert, J. (2000). Modeling longitudinal and multilevel data. Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13(3), 203. doi:10.1037/a0012869.

    Article  PubMed  Google Scholar 

  • Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology. European Journal of Research Methods for the Behavioral and Social Sciences, 1, 85–91. doi:10.1027/1614-1881.1.3.86.

    Article  Google Scholar 

  • Mahlke, J., Schultze, M., Koch, T., Eid, M., Eckert, R., & Brodbeck, F. C. (2016). A multilevel cfa-mtmm approach for multisource feedback instruments: Presentation and application of a new statistical model. Structural Equation Modeling: A Multidisciplinary Journal, 23(1), 91–110.

    Article  Google Scholar 

  • Marsh, H. W. (1989). Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13(4), 335–61. doi:10.1177/014662168901300402.

    Article  Google Scholar 

  • Marsh, H. W., & Grayson, D. (1995). Latent variable models of multitrait–multimethod data. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 177–198). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Marsh, H. W., & Hocevar, D. (1988). A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. Journal of Applied Psychology, 73(1), 107–117. doi:10.1037/0021-9010.73.1.107.

    Google Scholar 

  • Meuleman, B., & Billiet, J. (2009). A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM? Survey Research Methods, 3(1), 45–58.

    Google Scholar 

  • Pohl, S., & Steyer, R. (2010). Modeling common traits and method effects in multitrait-multimethod analysis. Multivariate Behavioral Research, 45(1), 45–72. doi:10.1080/00273170903504729.

    Article  PubMed  Google Scholar 

  • Pohl, S., Steyer, R., & Kraus, K. (2008). Modelling method effects as individual causal effects. Journal of the Royal Statistical Society: Series A (Statistics in Society), 171(1), 41–63. doi:10.1111/j.1467-985X.2007.00517.x.

    Google Scholar 

  • Rabe-Hesketh, S., & Skrondal, A. (2004). Generalized multilevel structural equation modeling. Psychometrica, 69, 167–190. doi:10.1007/BF02295939.

    Article  Google Scholar 

  • Ryu, E. (2014). Model fit evaluation in multilevel structural equation models. Frontiers in Psychology, 5(81). doi:10.3389/fpsyg.2014.00081.

  • Schermelleh-Engel, K., Keith, N., Moosbrugger, H., & Hodapp, V. (2004). Decomposing person and occasion-specific effects: An extension of latent state-trait (LST) theory to hierarchical LST models. Psychological Methods, 9(2), 198. doi:10.1007/s10802-011-9547-x.

    Article  PubMed  Google Scholar 

  • Scherpenzeel, A., & Saris, W. (2007). Multitrait–multimethod models for longitudinal research. In K. van Montfort, J. Oud, & A. Satorra (Eds.), Longitudinal models in the behavioral and related sciences (pp. 381–401). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Steele, F. (2008). Multilevel models for longitudinal data. Journal of the Royal Statistical Society: Series A (Statistics in Society), 171(1), 5–19. doi:10.1111/j.1467-985X.2007.00509.x.

    Google Scholar 

  • Steyer, R. (1988). Experiment, Regression und Kausalität. Die logische Struktur kausaler Regressionsmodelle [Experiment, regression, and causality. The logical structure of causal regression models]. Habilitation Thesis, Universität Trier, Germany.

  • Steyer, R., & Eid, M. (2001). Messen und Testen [Measurement and testing] (2nd ed.). Heidelberg: Springer.

  • Steyer, R., Ferring, D., & Schmitt, M. J. (1992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8(2), 79–98.

    Google Scholar 

  • Steyer, R., Majcen, A.-M., Schwenkmezger, P., & Buchner, A. (1989). A latent state-trait anxiety model and its application to determine consistency and specificity coefficients. Anxiety Research, 1(4), 281–299. doi:10.1080/08917778908248726.

    Article  Google Scholar 

  • Steyer, R., Mayer, A., Geiser, C., & Cole, D. A. (2015). A theory of states and traits—Revised. Annual Review of Clinical Psychology, 11(1), 71–98. doi:10.1146/annurev-clinpsy-032813-153719. (PMID: 25062476).

    Article  PubMed  Google Scholar 

  • Steyer, R., Schmitt, M., & Eid, M. (1999). Latent state-trait theory and research in personality and individual differences. European Journal of Personality, 13(5), 389–408. doi:10.1002/(SICI)1099-0984(199909/10)13:53.0.CO;2-A.

    Article  Google Scholar 

  • Steyer, R., & Schmitt, M. J. (1990). The effects of aggregation across and within occasions on consistency, specificity and reliability. Methodika, 4, 58–94.

    Google Scholar 

  • Steyer, R., & Schmitt, T. (1994). The theory of confounding and its application in causal modeling with latent variables. In A. Eye & C. C. Clogg (Eds.), Latent variable analysis: Applications for developmental research (pp. 36–67). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Vautier, S. (2004). A longitudinal sem approach to STAI data: Two comprehensive multitrait–multistate models. Journal of Personality Assessment, 83(2), 167–179.

    Article  PubMed  Google Scholar 

  • Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait–multimethod data. Applied Psychological Measurement, 9(1), 1–26. doi:10.1177/014662168500900101.

    Article  Google Scholar 

  • Wothke, W. (1995). Covariance components analysis of the multitrait–multimethod matrix. In P. E. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory: A festschrift honoring Donald W. Fiske (pp. 125–144). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Yammarino, F. J. (2003). Modern data analytic techniques for multisource feedback. Organizational Research Methods, 6(1), 6–14. doi:10.1177/1094428102239423.

    Article  Google Scholar 

  • Yammarino, F. J., & Atwater, L. E. (1997). Do managers see themselves as others see them? implications of self-other rating agreement for human resources management. Organizational Dynamics, 25(4), 35–44. doi:10.1016/S0090-2616(97)90035-8.

    Article  Google Scholar 

  • Zimmerman, D. W. (1975). Probability spaces, hilbert spaces, and the axioms of test theory. Psychometrika, 40, 395–412. doi:10.1007/BF02291765.

    Article  Google Scholar 

Download references

Acknowledgments

This research was funded by the German Research Foundation (Deutsche Forschungsgesellschaft, DFG, Grant # EI 379/6-1). Christian Geiser’s work was funded by a grant from the National Institutes on Drug Abuse (NIH-NIDA), Grant #1 R01 DA034770-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Author information

Authors and Affiliations

  1. Leuphana Universität Lüneburg, Lüneburg, Germany

    Tobias Koch

  2. Freie Universität Berlin, Berlin, Germany

    Martin Schultze, Jana Holtmann & Michael Eid

  3. Utah State University, Logan, USA

    Christian Geiser

Authors
  1. Tobias Koch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Schultze
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jana Holtmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christian Geiser
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Eid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tobias Koch.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 153 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koch, T., Schultze, M., Holtmann, J. et al. A Multimethod Latent State-Trait Model for Structurally Different And Interchangeable Methods. Psychometrika 82, 17–47 (2017). https://doi.org/10.1007/s11336-016-9541-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-016-9541-x

Keywords

Search

Navigation