Skip to main content
SpringerLink
Log in

Comparison of latent variable and psychological network models in PROMIS data: output metrics and factor structure

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Much research is still needed to compare traditional latent variable models such as confirmatory factor analysis (CFA) to emerging psychometric models such as the Gaussian graphical model (GGM). Previous comparisons of GGM centrality indices with factor loadings from CFA have discovered redundancies, and investigations into how well a GGM-based alternative to exploratory factor analysis (i.e., exploratory graph analysis, or EGA) is able to recover the hypothesized factor structure show mixed results. Importantly, such comparisons have not typically been examined in real mental and physical health symptom data, despite such data being an excellent candidate for the GGM. Our goal was to extend previous work by comparing the GGM and CFA using data from Wave 1 of the Patient Reported Outcomes Measurement Information System (PROMIS).

Methods

Models were fit to PROMIS data based on 16 test forms designed to measure 9 mental and physical health domains. Our analyses borrowed a two-stage approach for handling missing data from the structural equation modeling literature.

Results

We found weaker correspondence between centrality indices and factor loadings than found by previous research, but in a similar pattern of correspondence. EGA recommended a factor structure discrepant with PROMIS domains in most cases yet may be taken to provide substantive insight into the dimensionality of PROMIS domains.

Conclusion

In real mental and physical health data, the GGM and EGA may provide complementary information to traditional CFA metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. These symptoms correspond to items EDDEP06, EDDEP07, and FATEXP7, respectively, on PROMIS form H. The data and test forms will be described later.

  2. For details on data collection and sample selection criteria, see Cella et al. [21].

  3. If some items are never observed together at the same time, the GGM may not be identified.

  4. Code for all analyses is available at: https://osf.io/79ys4/.

References

  1. Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003). The theoretical status of latent variables. Psychological Review, 110(2), 203–219.

    Article  PubMed  Google Scholar 

  2. Borsboom, D., & Cramer, A. O. J. (2013). Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9(1), 91–121.

    Article  PubMed  Google Scholar 

  3. McNally, R. J. (2016). Can network analysis transform psychopathology? Behaviour Research and Therapy, 86, 95–104.

    Article  PubMed  Google Scholar 

  4. Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., Waldorp, L. J., et al. (2018). An introduction to network psychometrics: Relating ising network models to item response theory models. Multivariate Behavioral Research, 53(1), 15–35.

    Article  CAS  PubMed  Google Scholar 

  5. van Bork, R., Rhemtulla, M., Waldorp, L. J., Kruis, J., Rezvanifar, S., & Borsboom, D. (2019). Latent variable models and networks: Statistical equivalence and testability. Multivariate Behavioral Research, 56, 175–198.

    Article  PubMed  Google Scholar 

  6. Epskamp, S., Waldorp, L. J., Mõttus, R., & Borsboom, D. (2018). The Gaussian graphical model in cross-sectional and time-series data. Multivariate Behavioral Research, 53(4), 453–480.

    Article  PubMed  Google Scholar 

  7. Molenaar, P. C. (2010). Latent variable models are network models. The Behavioral and Brain Sciences, 33(2–3), 166.

    Article  PubMed  Google Scholar 

  8. Christensen, A. P., & Golino, H. (2021). On the equivalency of factor and network loadings. Behavior Research Methods. https://doi.org/10.3758/s13428-020-01500-6

    Article  PubMed  Google Scholar 

  9. Epskamp, S., Rhemtulla, M., & Borsboom, D. (2017). Generalized network psychometrics: Combining network and latent variable models. Psychometrika, 82(4), 904–927.

    Article  PubMed  Google Scholar 

  10. McNally, R. J., Robinaugh, D. J., Wu, G. W., Wang, L., Deserno, M. K., & Borsboom, D. (2015). Mental disorders as causal systems: A network approach to posttraumatic stress disorder. Clinical Psychological Science: A Journal of the Association for Psychological Science, 3(6), 836–849.

    Article  Google Scholar 

  11. Ruzzano, L., Borsboom, D., & Geurts, H. M. (2015). Repetitive behaviors in autism and obsessive-compulsive disorder: New perspectives from a network analysis. Journal of Autism and Developmental Disorders, 45(1), 192–202.

    Article  PubMed  Google Scholar 

  12. Opsahl, T., Agneessens, F., & Skvoretz, J. (2010). Node centrality in weighted networks: Generalizing degree and shortest paths. Social Networks, 32(3), 245–251.

    Article  Google Scholar 

  13. Robinaugh, D. J., Millner, A. J., & McNally, R. J. (2016). Identifying highly influential nodes in the complicated grief network. Journal of Abnormal Psychology, 125(6), 747–757.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Hallquist, M. N., Wright, A. G. C., & Molenaar, P. C. M. (2019). Problems with centrality measures in psychopathology symptom networks: Why network psychometrics cannot escape psychometric theory. Multivariate Behavioral Research., 56, 199–223.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PloS one, 12(6), e0174035.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Burger, J., Isvoranu, A.-M., Lunansky, G., Haslbeck, J., Epskamp, S., Hoekstra, R. H., et al. (2020). Reporting standards for psychological network analyses in cross-sectional data. PsyArXiv. https://doi.org/10.31234/osf.io/4y9nz

    Article  Google Scholar 

  17. Pons, P., & Latapy, M. (2005). Computing communities in large networks using random walks. In P. Yolum, T. Güngör, F. Gürgen, & C. Özturan (Eds.), Computer and Information Sciences ISCIS 2005 (pp. 284–93). Heidelberg: Springer.

    Chapter  Google Scholar 

  18. Christensen AP, Garrido LE, Golino H. 2020 Comparing community detection algorithms in psychological data: A Monte Carlo simulation. PsyArXiv. https://doi.org/10.31234/osf.io/hz89e

  19. Golino, H., Shi, D., Christensen, A. P., Garrido, L. E., Nieto, M. D., Sadana, R., et al. (2020). Investigating the performance of exploratory graph analysis and traditional techniques to identify the number of latent factors: A simulation and tutorial. Psychological Methods, 25(3), 292–320.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Golino, H. F., & Demetriou, A. (2017). Estimating the dimensionality of intelligence like data using Exploratory Graph Analysis. Intelligence, 62, 54–70.

    Article  Google Scholar 

  21. Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45(5 Suppl 1), S22-31.

    Article  PubMed  Google Scholar 

  23. DeWalt, D. A., Rothrock, N., Yount, S., Stone, A. A., PROMIS Cooperative Group. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(5 Suppl 1), S12-21.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Liu, H. H., Cella, D., Gershon, R., Shen, J., Morales, L. S., Riley, W., et al. (2010). Representativeness of the PROMIS Internet panel. Journal of Clinical Epidemiology, 63(11), 1169–1178.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Cella D. PROMIS 1 Wave 1. Harvard Dataverse; 2015. Available from: 10.7910/DVN/0NGAKG

  26. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2021. Available from: https://www.R-project.org/

  27. Lavaan, R. Y. (2012). An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.

    Google Scholar 

  28. Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195–212.

    Article  PubMed  Google Scholar 

  29. Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software., 48(4), 1–18. https://doi.org/10.18637/jss.v048.i04

    Article  Google Scholar 

  30. Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological Methods, 23(4), 617–634.

    Article  PubMed  Google Scholar 

  31. Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3), 432–441.

    Article  PubMed  Google Scholar 

  32. Chen, J., & Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika, 95(3), 759–771.

    Article  Google Scholar 

  33. Csárdi, G., & Nepusz, T. (2006). The igraph software package for complex network research. InterJournal Complex Systems, 1695(5), 1–9.

    Google Scholar 

  34. Newman, M. E. J., & Girvan, M. (2004). Finding and evaluating community structure in networks. Physical Review E: Statistical, Nonlinear, and Soft Matter Physics, 69(2 Pt 2), 026113.

    Article  CAS  PubMed  Google Scholar 

  35. Borsboom, D. (2017). A network theory of mental disorders. World Psychiatry, 16(1), 5–13.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Bringmann, L. F., Elmer, T., Epskamp, S., Krause, R. W., Schoch, D., Wichers, M., Wigman, J. T., & Snippe, E. (2019). What do centrality measures measure in psychological networks? Journal of Abnormal Psychology, 128(8), 892–903.

    Article  PubMed  Google Scholar 

  37. Rose, M., Bjorner, J. B., Gandek, B., Bruce, B., Fries, J. F., & Ware, J. E., Jr. (2014). The PROMIS physical function item bank was calibrated to a standardized metric and shown to improve measurement efficiency. Journal of Clinical Epidemiology, 67(5), 516–526.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Kessler, R. C., Chiu, W. T., Demler, O., Merikangas, K. R., & Walters, E. E. (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry, 62(6), 617–627.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Pasquini, M., Picardi, A., Biondi, M., Gaetano, P., & Morosini, P. (2004). Relevance of anger and irritability in outpatients with major depressive disorder. Psychopathology, 37(4), 155–160.

    Article  CAS  PubMed  Google Scholar 

  40. Judd, L. L., Schettler, P. J., Coryell, W., Akiskal, H. S., & Fiedorowicz, J. G. (2013). Overt irritability/anger in unipolar major depressive episodes: Past and current characteristics and implications for long-term course. JAMA Psychiatry., 70(11), 1171–1180.

    Article  PubMed  Google Scholar 

  41. Genovese, T., Dalrymple, K., Chelminski, I., & Zimmerman, M. (2017). Subjective anger and overt aggression in psychiatric outpatients. Comprehensive Psychiatry, 73, 23–30.

    Article  PubMed  Google Scholar 

  42. Jones, P. J., Ma, R., & McNally, R. J. (2019). Bridge centrality: A network approach to understanding comorbidity. Multivariate Behavioral Research, 56, 353–367.

    Article  PubMed  Google Scholar 

  43. Fried, E. I., & Cramer, A. O. J. (2017). Moving forward: Challenges and directions for psychopathological network theory and methodology. Perspectives on Psychological Science, 12(6), 999–1020.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Mijke Rhemtulla for her helpful comments on a previous version of this manuscript and Felix Fischer for discussions regarding PROMIS.

Funding

The authors acknowledge the support of the Fonds de la recherche en Santé du Québec (FRQS), doctoral training award reference number BF2-2022–302883; the Natural Science and Engineering Research Council of Canada (NSERC), funding reference number DGECR-2018–00083; and the Fonds de recherche du Québec – Nature et technologies (FRQNT), funding reference number 2019-NC-255344.

Author information

Authors and Affiliations

  1. Department of Psychology, McGill University, 2001 McGill College Ave., 7th Floor, Montreal, QC, H3A 1G1, Canada

    Joshua Starr & Carl F. Falk

Authors
  1. Joshua Starr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carl F. Falk
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JS and CFF both authors contributed to study conception and design, co-wrote code for conducting analyses and generating results. JS wrote the first draft of the manuscript, and both authors contributed to editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Carl F. Falk.

Ethics declarations

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

As the manuscript involves re-analysis of publicly available data, it did not require ethics approval.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 6979 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Starr, J., Falk, C.F. Comparison of latent variable and psychological network models in PROMIS data: output metrics and factor structure. Qual Life Res 32, 3247–3255 (2023). https://doi.org/10.1007/s11136-023-03471-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-023-03471-5

Keywords

Search

Navigation