Journal of Child and Family Studies

, Volume 28, Issue 1, pp 17–29 | Cite as

How Children and Adolescents Evaluate their Families: Psychometric Properties and Factor Structure of the Evaluation of Social Systems (EVOS) Scale

  • Dennis Grevenstein
  • Jochen Schweitzer
  • Corina Aguilar-RaabEmail author
Original Paper


The Evaluation of Social Systems (EVOS) scale was developed as a measure of the quality of social relationships vis-à-vis context-unspecific aspects of relationship quality. Although constructed for adolescents above the age of 11, it has to date only been validated in adult samples. The current study investigated if EVOS can be applied to adolescents through a cross-sectional study. We investigated factorial validity and construct validity of EVOS in a sample of adolescents with a mean age of 14.74. Additionally, we examined measurement invariance between youth and adult samples. The two-factor model that was developed for adults also held in the present study. EVOS showed good reliability and exhibited strict measurement invariance. Construct validity was supported by correlations with measures of family functioning and psychological distress. We conclude that EVOS is an economic, reliable, and valid measure of the quality of family relations in adolescents. Scores are directly comparable between youths and adults.


Quality of relationships Family therapy and counseling Measurement invariance Construct validity reliability 


Author Contributions

D.G. and C.A.R. drafted the manuscript. C.A.R. and D.G. developed the study, both performed the statistical analysis. J.S. supervised this research project. All authors approved the manuscript.


This research was supported by a grant from Deutsche Gesellschaft für Systemische Therapie, Beratung und Familientherapie (DGSF).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study. The study was approved by the ethics committee of the University Hospital Heidelberg (S-508/2012).

Supplementary material

10826_2018_1254_MOESM1_ESM.doc (522 kb)
Supplementary Information


  1. Achenbach, T. M. (1991). Manual for child behavior checklist. Burlington: University of Vermont.Google Scholar
  2. Achenbach, T. M., & Rescorla, L. A. (2004). The Achenbach System of Empirically Based Assessment (ASEBA) for ages 1.5 to 18 years. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (Vol. 2, 3rd edn.). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  3. Ackard, D. M., Neumark-Sztainer, D., Story, M., & Perry, C. (2006). Parent-child connectedness and behavioral and emotional health among adolescents. American Journal of Preventive Medicine, 30, 59–66. Scholar
  4. Adelman, C. B., Baucom, D. H., Belus, J. M., Fischer, M. S., & Paprocki, C. (2014). Couple‐based interventions for psychopathology: a renewed direction for the field. Family Process, 53, 445–461. Scholar
  5. Aguilar-Raab, C., Grevenstein, D., Gotthardt, L., Jarczok, M. N., Hunger, C., Ditzen, B., & Schweitzer, J. (2018). Changing me, changing us: relationship quality and collective efficacy as major outcomes in systemic couple therapy. Family Process, 57, 342–358. Scholar
  6. Aguilar-Raab, C., Grevenstein, D., & Schweitzer, J. (2015). Measuring social relationships in different social systems: The construction and validation of the Evaluation of Social Systems (EVOS) Scale. PLoS ONE, 10, e0133442Google Scholar
  7. Allen, J. P., Kilmer, S. L., Kuperminc, G. P., Land, D. J., McElhaney, K. B., Moore, C. W., & O’Beirne–Kelly, H. (2003). A secure base in adolescence: markers of attachment security in the mother–adolescent relationship. Child Development, 74, 292–307. Scholar
  8. Anderson, N., & West, M. A. (1996). The Team Climate Inventory: Development of the TCI and its applications in teambuilding for innovativeness. European Journal of Work and Organizational Psychology, 5, 53–66. Scholar
  9. Aspland, H., & Gardner, F. (2003). Observational measures of parent-child interaction: an introductory review. Child and Adolescent Mental Health, 8, 136–143. Scholar
  10. Atkin, Q. G., Burlingame, G. M., Gawain Wells, M., Lambert, M. J., Latkowski, M., Mosier, J. I., & Whoolery, M. (2001). Tracking the influence of mental health treatment: the development of the youth outcome questionnaire. Clinical Psychology & Psychotherapy, 8, 361–379. Scholar
  11. Baldwin, L. M., Bishop, D. S., & Epstein, N. B. (1983). The McMaster family assessment device. Journal of Marital and Family Therapy, 9, 171–180. Scholar
  12. Bandura, A., Caprara, G. V., Barbaranelli, C., Regalia, C., & Scabini, E. (2011). Impact of family efficacy beliefs on quality of family functioning and satisfaction with family life. Applied Psychology, 60, 421–448. Scholar
  13. Beavers, W. R. (1981). A systems model of family for family therapists. Journal of Marital and Family Therapy, 7, 299–307. Scholar
  14. Beavers, W. R., & Hampson, R. B. (2000). The beavers systems model of family functioning. Journal of Family Therapy, 22, 128–143. Scholar
  15. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. Scholar
  16. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588 Scholar
  17. Bishop, D. S., Epstein, N. B., & Levin, S. (1978). The McMaster model of family functioning. Journal of Marital and Family Therapy, 4, 19–31. Scholar
  18. Brooks-Gunn, J., Browning, C. R., & Maimon, D. (2010). Collective efficacy, family attachment and urban adolescent suicide attempts. Journal of Health and Social Behavior, 51, 307–324. Scholar
  19. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Beverly Hills, CA: Sage.Google Scholar
  20. Burlingame, G. M., Cox, J. C., Ferre, R., Lambert, M. J., Latkowski, M., & Wells, M. G. (2005). Administration and scoring manual for the Y-OQ (Youth Outcome Questionnaire). Stevenson, MD: American Professional Credentialing Services.Google Scholar
  21. Burlingame, G. M., Cox, J. C., Lambert, M. J., & Wells, M. G. (2004). Youth Outcome Questionnaire (Y-OQ) (Vol. 2, 3rd edn.). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  22. Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin, 105, 456 Scholar
  23. Calheiros, M. M., & Patrício, J. N. (2014). Assessment of needs in residential care: perspectives of youth and professionals. Journal of Child and Family Studies, 23, 461–474. Scholar
  24. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. Scholar
  25. Carr, A. (2014a). The evidence base for couple therapy, family therapy and systemic interventions for adult-focused problems. Journal of Family Therapy, 36, 158–194. Scholar
  26. Carr, A. (2014b). The evidence base for family therapy and systemic interventions for child-focused problems. Journal of Family Therapy, 36, 107–157. Scholar
  27. Carr, A., & Hamilton, E. (2016). Systematic review of self-report family assessment measures. Family Process, 55, 16–30. Scholar
  28. Carr, A., & Stratton, P. (2017). The score family assessment questionnaire: a decade of progress. Family Process, 56, 285–301. Scholar
  29. Celinska, K., Cheng, C.-C., & Virgil, N. J. (2015). Youth and parental perspectives on the functional family therapy programme. Journal of Family Therapy, 37, 450–470. Scholar
  30. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464–504. Scholar
  31. Chen, F. F. (2008). What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. Journal of Personality and Social Psychology, 95, 1005–1018. Scholar
  32. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233–255. Scholar
  33. Chou, C.-P., Bentler, P. M., & Satorra, A. (1991). Scaled test statistics and robust standard errors for non-normal data in covariance structure analysis: A Monte Carlo study. British Journal of Mathematical and Statistical Psychology, 44, 347–357. Scholar
  34. Cleveland, K., Garrett, S. B., Hair, E. C., Ling, T., & Moore, K. A. (2008). The continued importance of quality parent–adolescent relationships during late adolescence. Journal of Research on Adolescence, 18, 187–200. Scholar
  35. Conger, R. D., Conger, K. J., & Martin, M. J. (2010). Socioeconomic status, family processes, and individual development. Journal of Marriage and the Family, 72, 685–704. Scholar
  36. Cook, T. D., Herman, M. R., Phillips, M., & Settersten, J. R. A. (2002). Some ways in which neighborhoods, nuclear families, friendship groups, and schools jointly affect changes in early adolescent development. Child Development, 73, 1283–1309. Scholar
  37. Demaray, M. K., & Malecki, C. K. (2002). The relationship between perceived social support and maladjustment for students at risk. Psychology in the Schools, 39, 305–316. Scholar
  38. Erel, O., & Kissil, K. (2003). The linkage between multiple perspectives of the marital relationship and preschoolers’ adjustment. Journal of Child and Family Studies, 12, 411–423. Scholar
  39. Essau, C. A., Olaya, B., Anastassiou‐Hadjicharalambous, X., Pauli, G., Gilvarry, C., Bray, D., & Ollendick, T. H. (2012). Psychometric properties of the strength and difficulties questionnaire from five European countries. International Journal of Methods in Psychiatric Research, 21, 232–245. Scholar
  40. Farley, J. P., & Kim-Spoon, J. (2014). The development of adolescent self-regulation: reviewing the role of parent, peer, friend, and romantic relationships. Journal of Adolescence, 37, 433–440. Scholar
  41. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G* Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. Scholar
  42. Faulstich, P. (1998). Strategien betrieblicher Weiterbildung. Kompetenz und Organisation. München, Germany: Vahlen.Google Scholar
  43. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50. Scholar
  44. Fruzzetti, A. E., Shenk, C., & Hoffman, P. D. (2005). Family interaction and the development of borderline personality disorder: a transactional model. Development and Psychopathology, 17, 1007–1030. Scholar
  45. Goodman, R. (1997). The Strengths and difficulties questionnaire: a research note. Child Psychology & Psychiatry & Allied Disciplines, 38, 581–586. Scholar
  46. Grevenstein, D., Bluemke, M., Schweitzer, J., & Aguilar-Raab, C. (2018). Better family relationships - higher well-being: The connection between relationship quality and health related ressources. Manuscript submitted for publication.Google Scholar
  47. Hamilton, E., Carr, A., Cahill, P., Cassells, C., & Hartnett, D. (2015). Psychometric properties and responsiveness to change of 15‐and 28‐Item versions of the SCORE: A family assessment questionnaire. Family Process, 54, 454–463. Scholar
  48. Heatherington, L., Diamond, G. M., Escudero, V., Friedlander, M. L., & Pinsof, W. M. (2015). 25 Years of systemic therapies research: Progress and promise. Psychotherapy Research, 25, 348–364. Scholar
  49. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. Scholar
  50. Johnson, H. D., Lavoie, J. C., & Mahoney, M. (2001). Interparental conflict and family cohesion. Journal of Adolescent Research, 16, 304–318. Scholar
  51. Jose, P. E., Ryan, N., & Pryor, J. (2012). Does social connectedness promote a greater sense of well-being in adolescence over time? Journal of Research on Adolescence, 22, 235–251. Scholar
  52. Kauffeld, S. (2001). Teamdiagnose. Göttingen, Germany: Hogrefe.Google Scholar
  53. Kenny, D. A. (2015). Measuring model fit. Retrieved from
  54. Laursen, B., & Colling, W. A. (2004). Chapter 16: Parent-child communication during adolescence. In A. L. Vangelisti (Ed.), Handbook of family communication. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  55. Lee, I. A., & Preacher, K. J. (2013). Calculation for the test of the difference between two dependent correlations with one variable in common. Retrieved from
  56. Lohbeck, A., Schultheiß, J., Petermann, F., & Petermann, U. (2015). Die deutsche selbstbeurteilungsversion des strengths and difficulties questionnaire (SDQ-Deu-S). Diagnostica, 61, 222–235. Scholar
  57. Lubke, G. H., & Dolan, C. V. (2003). Can unequal residual variances across groups mask differences in residual means in the common factor model? Structural Equation Modeling, 10, 175–192. Scholar
  58. Lucia, V. C., & Breslau, N. (2006). Family cohesion and children’s behavior problems: a longitudinal investigation. Psychiatry Research, 141, 141–149. Scholar
  59. Mansfield, A. K., Keitner, G. I., & Dealy, J. (2015). The family assessment device: an update. Family Process, 54, 82–93. Scholar
  60. McClendon, D. T., Warren, J. S., M. Green, K., Burlingame, G. M., Eggett, D. L., & McClendon, R. J. (2011). Sensitivity to change of youth treatment outcome measures: a comparison of the CBCL, BASC-2, and Y-OQ. Journal of Clinical Psychology, 67, 111–125. Scholar
  61. McGrath, J. E., Arrow, H., & Berdahl, J. L. (2000). The study of groups: past, present, and future. Personality and Social Psychology Review, 4, 95–105. Scholar
  62. Meltzer, H., Ford, T., Goodman, R., & Vostanis, P. (2011). The burden of caring for children with emotional or conduct disorders. International Journal of Family Medicine, 2011.
  63. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525–543. Scholar
  64. Merikangas, K. R., He, J.-P., Burstein, M., Swanson, S. A., Avenevoli, S., Cui, L., & Swendsen, J. (2010). Lifetime prevalence of mental disorders in U.S. adolescents: Results from the National Comorbidity Survey Replication - Adolescent Supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 49, 980–989. Scholar
  65. Merikangas, K. R., Nakamura, E. F., & Kessler, R. C. (2009). Epidemiology of mental disorders in children and adolescents. Dialogues in Clinical Neuroscience, 11, 7–20. Scholar
  66. Miller, G., Chen, E., & Cole, S. W. (2009). Health psychology: developing biologically plausible models linking the social world and physical health. Annual Review of Psychology, 60, 501–524. Scholar
  67. Moore, K. A., Guzman, L., Hair, E. C., Lippman, L., & Garrett, S. B. (2004). Parent–teen relationship and interactions: Far more positive than not. Washington, DC: Child Trends.Google Scholar
  68. Moore, L., & Seu, I. B. (2011). Giving children a voice: children’s positioning in family therapy. Journal of Family Therapy, 33, 279–301. Scholar
  69. Muthén, L. K., & Muthén, B. O. (1998). Mplus. Los Angeles, CA: Muthén & Muthén. 2012.Google Scholar
  70. Noller, P., Feeney, J., & Petersen, C. (2001). Personal relationships across the lifespan. London, UK: Psychology Press.Google Scholar
  71. O’Brien, M. P., Gordon, J. L., Bearden, C. E., Lopez, S. R., Kopelowicz, A., & Cannon, T. D. (2006). Positive family environment predicts improvement in symptoms and social functioning among adolescents at imminent risk for onset of psychosis. Schizophrenia Research, 81, 269–275. Scholar
  72. Olson, D. H. (1985). FACES III - Family Adaptability and Cohesion Evaluation Scales. St. Paul, MN: University of Minnesota.Google Scholar
  73. Olson, D. H. (2011). FACES IV and the circumplex model: validation study. Journal of Marital and Family Therapy, 37, 64–80. Scholar
  74. Olson, D. H., Sprenkle, D. H., & Russell, C. S. (1979). Circumplex model of marital and family systems: I. cohesion and adaptability dimensions, family types, and clinical applications. Family Process, 18, 3–28. Scholar
  75. Paradisopoulos, D., Pote, H., Fox, S., & Kaur, P. (2015). Developing a model of sustained change following multisystemic therapy: young people’s perspectives. Journal of Family Therapy, 37, 471–491. Scholar
  76. Patterson, J. M. (2002). Understanding family resilience. Journal of Clinical Psychology, 58, 233–246. Scholar
  77. Phillips-Salimi, C. R., Robb, S. L., Monahan, P. O., Dossey, A., & Haase, J. E. (2014). Perceptions of communication, family adaptability and cohesion: a comparison of adolescents newly diagnosed with cancer and their parents. International Journal of Adolescent Medicine & Health, 26, 19–26. Scholar
  78. Pinquart, M., Oslejsek, B., & Teubert, D. (2016). Efficacy of systemic therapy on adults with mental disorders: a meta-analysis. Psychotherapy Research, 26, 241–257. Scholar
  79. Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163. Scholar
  80. Rayner, M., & Montague, M. (2000). Resilient children and young people. Melbourne: Deakin University.Google Scholar
  81. Resnick, M. D., Harris, L. J., & Blum, R. W. (1993). The impact of caring and connectedness on adolescent health and well-being. Journal of Paediatrics and Child Health, 29, S3–S9. Scholar
  82. Retzlaff, R., von Sydow, K., Beher, S., Haun, M. W., & Schweitzer, J. (2013). The efficacy of systemic therapy for internalizing and other disorders of childhood and adolescence: a systematic review of 38 randomized trials. Family Process, 52, 619–652. Scholar
  83. Reynolds, C. R., & Kamphaus, R. W. (2004). Behavioral assessment system for children: Second edition (BASC-2). Circle Pines, MN: AGS Publishing.Google Scholar
  84. Rusbult, C. E., & Van Lange, P. A. M. (2003). Interdependence, interaction, and relationships. Annual Review of Psychology, 54, 351–375. Scholar
  85. Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In R. D. H. Heijmans, D. S. G. Pollock, & A. Satorra (Eds.), Innovations in multivariate statistical analysis. A Festschrift for Heinz Neudecker (pp. 233–247). London, UK: Kluwer Academic Publishers.CrossRefGoogle Scholar
  86. Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507–514. Scholar
  87. Schmitt, N., & Kuljanin, G. (2008). Measurement invariance: review of practice and implications. Human Resource Management Review, 18, 210–222. Scholar
  88. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464. Scholar
  89. Sexton, T. L., Datchi, C., Evans, L., LaFollette, J., & Wright, L. (2013). The effectiveness of couple and family-based clinical interventions. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change. (6th edn., pp. 587–639). Hoboken, NJ: John Wiley & Sons.Google Scholar
  90. Skinner, H. A., Steinhauer, P. D., & Santa-Barbara, J. (1983). The family assessment measure. Canadian Journal of Community Mental Health, 2, 91–103. Scholar
  91. Skinner, H. A., Steinhauer, P. D., & Sitarenios, G. (2000). Family assessment measure (FAM) and process model of family functioning. Journal of Family Therapy, 22, 190–210. Scholar
  92. Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245–251. Scholar
  93. Stratton, P., Bland, J., Janes, E., & Lask, J. (2010). Developing an indicator of family function and a practicable outcome measure for systemic family and couple therapy: the SCORE. Journal of Family Therapy, 32, 232–258. Scholar
  94. Stratton, P., Lask, J., Bland, J., Nowotny, E., Evans, C., Singh, R., & Peppiatt, A. (2014). Detecting therapeutic improvement early in therapy: validation of the SCORE‐15 index of family functioning and change. Journal of Family Therapy, 36, 3–19. Scholar
  95. Van De Schoot, R., Schmidt, P., De Beuckelaer, A., Lek, K., & Zondervan-Zwijnenburg, M. (2015). Editorial: Measurement Invariance. Frontiers in Psychology.
  96. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70. Scholar
  97. von Sydow, K., Retzlaff, R., Beher, S., Haun, M. W., & Schweitzer, J. (2013). The efficacy of systemic therapy for childhood and adolescent externalizing disorders: a systematic review of 47 RCT. Family Process, 52, 576–618. Scholar
  98. Walsh, F. (2003). Family resilience: a framework for clinical practice. Family Process, 42, 1–18. Scholar
  99. Wells, M. G., Burlingame, G. M., & Rose, P. M. (2003). Administration and scoring manual for the Y-OQ SR-2.0 (Youth Outcome Questionnaire-Self Report). Wilmington, DE: American Professional Credentialing Services.Google Scholar
  100. Wells, M. S., Widmer, M. A., & McCoy, J. K. (2004). Grubs and grasshoppers: challenge-based recreation and the collective efficacy of families with at-risk youth. Family Relations, 53, 326–333. Scholar
  101. Yuan, K.-H., & Bentler, P. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological Methodology, 30, 165–200. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Psychological InstituteUniversity of HeidelbergHeidelbergGermany
  2. 2.Institute of Medical Psychology, Center for Psychosocial MedicineUniversity HospitalHeidelbergGermany

Personalised recommendations