Skip to main content
SpringerLink
Log in

Multilevel Confirmatory Factor Analysis of the Systems of Care Implementation Survey (SOCIS)

  • Published:
The Journal of Behavioral Health Services & Research Aims and scope Submit manuscript

Abstract

A major impediment to obtaining national information on systems of care implementation has been the lack of a psychometrically sound large-scale survey instrument. The present study provided information on the factorial and concurrent validity of the Systems of Care Implementation Survey scales. Multilevel confirmatory factor analysis and multilevel regression analysis were used to test these indicators of internal and external validity. Two hundred twenty-five counties were randomly selected and stratified by population size and poverty level. Nine hundred ten informants responded to the survey questionnaire, M = 4.04 informants per county (SD = 3.17). Results indicated that all models had at least adequate fit to the data, with nine of the 14 factor models having excellent fit. Overall, 11 of the 14 factors had some indication that receiving federal funding to create systems of care was associated with higher scores on the factors. Implications for future research were discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

Notes

  1. A freely estimated correlated error between the two dichotomous items (6A and 6B) was included to achieve model convergence for the regression model on the implementation plan factor.

References

  1. Stroul BA, Friedman RM. A System of Care for Severely Emotionally Disturbed Children and Youth. Washington, DC: Georgetown University, 1986.

    Google Scholar 

  2. Boothroyd RA, Greenbaum PE, Wang W, et al. Development of a measure to assess the implementation of children’s systems of care: The System of Care Implementation Survey (SOCIS). Journal of Behavioral Health Services and Research, 2011; 38(3).

  3. Procter EK, Landsverk J, Aarons G, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health, 2009; 36:24–34.

    Article  Google Scholar 

  4. Raudenbush SW, Rowan B, Kang, SJ. A multilevel, multivariate model for studying school climate with estimation via the EM algorithm and application to U.S. high-school data. Journal of Educational Statistics, 1991; 16:295–330.

    Article  Google Scholar 

  5. Muthén BO. Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 1990; 28:338–354.

    Article  Google Scholar 

  6. Muthén BO. Multilevel covariance structure analysis. Sociological Methods and Research, 1994; 22:376–398.

    Article  Google Scholar 

  7. Berman E, Wang X. Performance measurement in U. S. counties: capacity for reform. Public Administration Review. 2000: 60, 409–420.

    Article  Google Scholar 

  8. Health Resources and Services Administration. Community health status report. Washington, D.C.; Health Resources and Services Administration: 2000.

  9. Andrews FM. Construct validity and error components of survey measures: a structural modeling approach. Public Opinion Quarterly, 1984; 48:409–442.

    Article  Google Scholar 

  10. Zypher MJ, Kaplan SA, & Christian MS. Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: problems and solutions. Group Dynamics: Theory, Research, and Practice, 2008; 12:127–140.

    Article  Google Scholar 

  11. Muthén LK, Muthén BO. Mplus User’s Guide. Fifth Edition. Los Angeles, CA: Muthén & Muthén, 1998–2007.

  12. Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychological Methods, 2001; 7:147–177.

    Article  Google Scholar 

  13. Bentler PM. On the fit of models to covariances and methodology to the Bulletin. Psychological Bulletin, 1992; 112:400-404.

    Article  PubMed  CAS  Google Scholar 

  14. Tucker LR, Lewis C. The reliability coefficient for maximum likelihood factor analysis. Psychometrika, 1973; 38:1–10.

    Article  Google Scholar 

  15. Steiger JH, Lind JC. Statistically-based Tests for the Number of Common Factors. Paper presented at the annual meeting of the Psychonomic Society, Iowa City, IA, May 1980.

  16. McDonald RP, Ho M. Principles and practice in reporting structural equation analyses. Psychological Methods. 2002; 7: 64–82.

    Article  PubMed  Google Scholar 

  17. Cicchetti DV, Sparrow SS. Developing criteria for establishing the interrater reliability of specific items in a given inventory. American Journal of Mental Deficiencies. 1981; 86:127–137.

    CAS  Google Scholar 

  18. Hox J. Multilevel Analysis: Techniques and Applications. Mahwah, NJ: Erlbaum, 2002.

    Google Scholar 

  19. Kamata A, Bauer DJ, Miyazaki Y. Multilevel measurement modeling. In AA O’Connell and DB McCoach (eds). Multilevel Modeling of Educational Data. Charlotte, NC: IAP, 2008: 345–388.

    Google Scholar 

  20. Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2 nd Edition. Thousand Oaks, CA: Sage, 2002.

    Google Scholar 

  21. Fleiss JL. Statistical Methods for Rates and Proportions. 2 nd Edition. New York: Wiley, 1981.

    Google Scholar 

  22. Kutash K, Greenbaum P, Wang W, et al. Levels of system of care implementation: A national benchmarking study. Journal of Behavioral Health Services and Research, 2011; 38(3).

  23. Lunn LM, Heflinger CA, Wang W, et al. Community characteristics and implementation factors associated with effective systems of care. Journal of Behavioral Health Services and Research, 2011; 38(3).

Download references

Acknowledgments

This research was partially supported by Grant H133B90004 from the Center for Mental Health Services, Substance Abuse and Mental Health Administration and the National Institute for Disability and Rehabilitation Research. The opinions contained in this manuscript are those of the authors and do not necessarily reflect those of either the US Department of Education or the Center for Mental Health Services, SAMSHA. We gratefully acknowledge helpful comments on the statistical analyses by members of the Prevention Science and Methodology Group (PI: C. Hendricks Brown) supported by the National Institute of Mental Health and the National Institute on Drug Abuse through Grant 5R01MH40859.

Author information

Authors and Affiliations

  1. Department of Child and Family Studies, University of South Florida, Tampa, FL, USA

    Paul E. Greenbaum PhD, Wei Wang PhD, Roger Boothroyd PhD, Krista Kutash PhD & Robert M. Friedman PhD

Authors
  1. Paul E. Greenbaum PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Wang PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roger Boothroyd PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Krista Kutash PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert M. Friedman PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul E. Greenbaum PhD.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Greenbaum, P.E., Wang, W., Boothroyd, R. et al. Multilevel Confirmatory Factor Analysis of the Systems of Care Implementation Survey (SOCIS). J Behav Health Serv Res 38, 303–326 (2011). https://doi.org/10.1007/s11414-011-9240-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11414-011-9240-4

Keywords

Search

Navigation