Measurement Invariance and Informant Discrepancies of the KIDSCREEN-27 in Children with Mental Disorder

Abstract

Study objectives were to examine measurement invariance of the KIDSCREEN-27 between children with mental disorder and their parents; compare health-related quality of life (HRQL) scores to identify potential domain discrepancies; and, identify mental disorders associated with informant discrepancies. Participants consisted of parents with children who were currently receiving mental health services and screened positive for mental disorder according to the Mini International Neuropsychiatric Interview (n = 92). Measurement invariance was investigated using multiple-group confirmatory factor analysis using maximum likelihood estimation with robust standard errors. Multiple regression was used to identify child mental disorders associated with informant discrepancies, adjusting for relevant child and parent characteristics. Partial invariance was obtained for all domains on the KIDSCREEN-27 with the exception of Physical Well-being for which full invariance was obtained. Parents reported lower KIDSCREEN-27 scores compared to children with significant differences for Psychological Well-being [d = 1.98 (1.62, 2.33)] and Social Support and Peers [d = 0.41 (0.11, 0.70)]. For children with oppositional defiant or conduct disorder who report low HRQL, parents underestimate Psychological Well-being scores (B = 1.29, p < 0.001; B = 1.00, p < 0.01) and for children with major depressive episode, parents underestimate Social Support and Peers scores (B = 0.57, p < 0.05). Findings suggest that the KIDSCREEN-27 is a valid measure of HRQL in children with mental disorder. Externalizing behaviors exhibited by children with oppositional defiant and conduct disorder may prevent parents from recognizing compromises in their psychological well-being. Health professionals should be cognizant that these clinical profiles may predispose informant discrepancies when assessing HRQL in children.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

References

  1. Ackard, D. M., Neumark-Sztainer, D., Story, M., & Perry, C. (2006). Parent-child connectedness and behavioral and emotional health among adolescents. American Journal of Preventive Medicine, 30(1), 59–66.

    Google Scholar 

  2. Albrecht, G. L., & Devlieger, P. J. (1999). The disability paradox: high quality of life against all odds. Social Science and Medicine, 48, 977–988.

    Google Scholar 

  3. Bagheri, Z., Jafari, P., Tashakor, E., Kouhpayeh, A., & Riazi, H. (2014). Assessing whether measurement invariance of the KIDSCREEN-27 across child-parent dyad depends on the child gender: a multiple group confirmatory factor analysis. Global Journal of Health Science, 6(5), 142–153.

    Google Scholar 

  4. Baumgartner, H., & Steenkamp, J.-B. E. M. (1998). Multi-group latent variable models for varying numbers of items and factors with cross-national and longitudinal applications. Marketing Letters, 9(1), 21–35.

    Google Scholar 

  5. Berman, A. H., Liu, B., Ullman, S., Jadback, I., & Engstrom, K. (2016). Children’s quality of life based on the KIDSCREEN-27: child self-report, parent ratings and child-parent agreement in a Swedish random population sample. PLoS One, 11(3), e0150545.

    Google Scholar 

  6. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

    Google Scholar 

  7. Boyle, M. H., Duncan, L., Georgiades, K., Bennett, K., Gonzalez, A., Van Lieshout, R. J., et al. (2017). Classifying child and adolescent psychiatric disorder by problem checklists and standardized interviews. International Journal of Methods in Psychiatric Research, 26, e1544.

    Google Scholar 

  8. Briggs, H. E., Cox, W., Sharkey, C. N., Corley, N., Briggs, A. C., & Black, M. (2016). The role of behavioral theory in model development research with single parent families. Child and Adolescent Social Work Journal, 33(4), 349–363.

    Google Scholar 

  9. Brown, T. A. (2006). Confirmatory factor analysis for applied research (first edit. ed.). New York: The Guilford Press.

    Google Scholar 

  10. Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York: Taylor and Francis Group, LLC.

    Google Scholar 

  11. Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466.

    Google Scholar 

  12. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504.

    Google Scholar 

  13. Chiariello, M. A., & Orvaschel, H. (1995). Patterns of parent-child communication: relationship to depression. Clinical Psychology Review, 15(5), 395–407.

    Google Scholar 

  14. Chiorri, C., Day, T., & Malmberg, L.-E. (2014). An approximate measurement invariance approach to within-couple relationship quality. Frontiers in Psychology, 5(983), 1–10.

    Google Scholar 

  15. Choudhury, M. S., Pimentel, S. S., & Kendall, P. C. (2003). Childhood anxiety disorders: parent-child (dis)agreement using a structured interview for the DSM-IV. Journal of the American Academy of Child and Adolescent Psychiatry, 42(8), 957–964.

    Google Scholar 

  16. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (second Edi. ed.). New York: Lawrence Erlbaum Associates.

    Google Scholar 

  17. Cole, D. A., & Rehm, L. P. (1986). Family interaction patterns and childhood depression. Journal of Abnormal Child Psychology, 14(2), 297–314.

    Google Scholar 

  18. Cook, J. A., Lefley, H. P., Pickett, S. A., & Cohler, B. J. (1994). Age and family burden among parents of offspring with severe mental illness. American Journal of Orthopsychiatry, 64(3), 435–447.

    Google Scholar 

  19. Dadds, M. R., Sanders, M. R., Morrison, M., & Rebgetz, M. (1992). Childhood depression and conduct disorder: II. An analysis of family interaction patterns in the home. Journal of Abnormal Psychology, 101(3), 505–513.

    Google Scholar 

  20. De Civita, M., Regier, D., Alamgir, A. H., Anis, A. H., FitzGerald, M. J., & Marra, C. A. (2005). Evaluating health-related quality-of-life studies in paediatric populations: some conceptual, methodological and developmental considerations and recent applications. PharmacoEconomics, 23(7), 659–685.

    Google Scholar 

  21. Dey, M., Landolt, M. A., & Mohler-Kuo, M. (2013). Assessing parent-child agreement in health-related quality of life among three health status groups. Social Psychiatry and Psychiatric Epidemiology, 48(3), 503–511.

    Google Scholar 

  22. Dodge, K. A. (1993). Social-cognitive mechanisms in the development of conduct disorder and depression. Annual Review of Psychology, 44(1), 559–584.

    Google Scholar 

  23. Dolgin, M. J., Phipps, S., Harow, E., & Zeltzer, L. K. (1990). Parental management of fear in chronically ill and healthy children. Journal of Pediatric Psychology, 15(6), 733–744.

    Google Scholar 

  24. Duncan, L., Georgiades, K., Wang, L., Van Lieshout, R. J., Macmillan, H. L., Ferro, M. A., et al. (2017). Psychometric evaluation of the Mini international neuropsychiatric interview for children and adolescents (MINI-KID). Psychological Assessment, 1–13.

  25. Eiser, C., & Varni, J. W. (2013). Health-related quality of life and symptom reporting: Similarities and differences between children and their parents. European Journal of Pediatrics, 172(10), 1299–1304.

    Google Scholar 

  26. Erhart, M., Ellert, U., Kurth, B. M., & Ravens-Sieberer, U. (2009). Measuring adolescents’ HRQoL via self reports and parent proxy reports: an evaluation of the psychometric properties of both versions of the KINDL-R instrument. Health and Quality of Life Outcomes, 7, 77.

    Google Scholar 

  27. Ferro, M. A., & Boyle, M. H. (2013). Brief report: Testing measurement invariance and differences in self-concept between adolescents with and without physical illness or developmental disability. Journal of Adolescence, 36(5), 947–951.

    Google Scholar 

  28. Ferro, M. A., & Speechley, K. N. (2013). Factor structure and longitudinal invariance of the Center for Epidemiological Studies Depression Scale (CES-D) in adult women: application in a population-based sample of mothers of children with epilepsy. Archives of Women’s Mental Health, 16(2), 159–166.

    Google Scholar 

  29. Ferro, M. A., Avison, W. R., Campbell, M. K., & Speechley, K. N. (2010). Do depressive symptoms affect mothers’ reports of child outcomes in children with new-onset epilepsy? Quality of Life Research, 19(7), 955–964.

    Google Scholar 

  30. Ferro, M. A., Boyle, M. H., Scott, J. G., & Dingle, K. (2014). The child behavior checklist and youth self-report in adolescents with epilepsy: testing measurement invariance of the attention and thought problems subscales. Epilepsy and Behavior, 31, 34–42.

    Google Scholar 

  31. Ferro, M. A., Lipman, E. L., Van Lieshout, R. J., Boyle, M. H., Gorter, J. W., MacMillan, H. L., et al. (2019). Mental–physical multimorbidity in youth: associations with individual, family, and health service use outcomes. Child Psychiatry and Human Development, 50(3), 400–410.

  32. Ford, T., Goodman, R., & Meltzer, H. (2003). The British child and adolescent mental health survey 1999: the prevalence of DSM-IV disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 42(10), 1203–1211.

    Google Scholar 

  33. Gilbody, S. M., House, A. O., & Sheldon, T. (2002). Routine administration of health related quality of life (HRQoL) and needs assessment instruments to improve psychological outcome - a systemic review. Psychological Medicine, 32(8), 1345–1356.

    Google Scholar 

  34. Greenhalgh, J., & Meadows, K. (1999). The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. Journal of Evaluation in Clinical Practice, 5(4), 401–416.

    Google Scholar 

  35. Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44(11 Suppl 3), S78–S94.

    Google Scholar 

  36. Helseth, S., Haraldstad, K., & Christophersen, K. A. (2015). A cross-sectional study of health related quality of life and body mass index in a Norwegian school sample (8-18 years): a comparison of child and parent perspectives. Health and Quality of Life Outcomes, 13, 47.

    Google Scholar 

  37. Hood, K. K. (2009). The influence of caregiver depressive symptoms on proxy report of youth depressive symptoms: a test of the depression-distortion hypothesis in pediatric type 1 diabetes. Journal of Pediatric Psychology, 34(3), 294–303.

    Google Scholar 

  38. Jafari, P., Bagheri, Z., Hashemi, S. Z., & Shalileh, K. (2013). Assessing whether parents and children perceive the meaning of the items in the PedsQLTM 4.0 quality of life instrument consistently: a differential item functioning analysis. Global Journal of Health Science, 5(5), 80–88.

    Google Scholar 

  39. Jafari, P., Sharafi, Z., Bagheri, Z., & Shalileh, S. (2014). Measurement equivalence of the KINDL questionnaire across child self-reports and parent proxy-reports: a comparison between item response theory and ordinal logistic regression. Child Psychiatry and Human Development, 45(3), 369–376.

    Google Scholar 

  40. Jensen, P. S. (2003). Comorbidity and child psychopathology: recommendations for the next decade. Journal of Abnormal Child Psychology, 31(3), 293–300.

    Google Scholar 

  41. Jensen, P. S., Rubio-Stipec, M., Canino, G., Bird, H. R., Dulcan, M. K., Schwab-Stone, M. E., & Lahey, B. B. (1999). Parent and child contributions to diagnosis of mental disorder: are both informants always necessary? Journal of the American Academy of Child and Adolescent Psychiatry, 38(12), 1569–1579.

    Google Scholar 

  42. Johnston, C., & Jassy, J. S. (2007). Attention-deficit/hyperactivity disorder and oppositional/conduct problems: links to parent-child interactions. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 16(2), 74–79.

    Google Scholar 

  43. Johnston, C., & Mash, E. J. (2001). Families of children with attention-deficit/hyperactivity disorder: review and recommendations for future research. Clinical Child and Family Psychology Review, 4(3), 183–207.

    Google Scholar 

  44. Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods and Research, 44(3), 486–507.

    Google Scholar 

  45. Kessler, R. C., Ormel, J., Petukhova, M., McLaughlin, K. A., Green, J. G., Russo, L. J., et al. (2011). Development of lifetime comorbidity in the World Health Organization world mental health surveys. Archives of General Psychiatry, 68(1), 90–100.

    Google Scholar 

  46. Klassen, A. F., Miller, A., & Fine, S. (2004). Health-related quality of life in children and adolescents who have a diagnosis of attention-deficit/hyperactivity disorder. Pediatrics, 114(5), e541–e547.

    Google Scholar 

  47. Kraemer, H. C., Kupfer, D. J., Clarke, D. E., Narrow, W. E., & Regier, D. A. (2012). DSM-5: how reliable is reliable enough? American Journal of Psychiatry, 169(1), 13–15.

    Google Scholar 

  48. Lin, C.-Y., Luh, W.-M., Cheng, C.-P., Yang, A.-L., Su, C.-T., & Ma, H.-I. (2013). Measurement equivalence across child self-reports and parent-proxy reports in the Chinese version of the pediatric quality of life inventory version 4.0. Child Psychiatry and Human Development, 44(5), 583–590.

    Google Scholar 

  49. Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P., & Van Os, J. (2009). Experience sampling research in psychopathology: opening the black box of daily life. Psychological Medicine, 39(9), 1533–1547.

    Google Scholar 

  50. Okun, A., Stein, R. E., Bauman, L. J., & Silver, E. J. (1996). Content validity of the psychiatric symptom index, CES-depression scale, and state-trait anxiety inventory from the perspective of DSM-IV. Psychological Reports, 79, 1059–1069.

    Google Scholar 

  51. Owens, J. S., Goldfine, M. E., Evangelista, N. M., Hoza, B., & Kaiser, N. M. (2007). A critical review of self-perceptions and the positive illusory bias in children with ADHD. Clinical Child and Family Psychology Review, 10(4), 335–351.

    Google Scholar 

  52. Patel, V., Flisher, A. J., Hetrick, S., & McGorry, P. (2007). Mental health of young people: a global public-health challenge. Lancet, 369, 1302–1313.

    Google Scholar 

  53. Polanczyk, G. V., Salum, G. A., Sugaya, L. S., Caye, A., & Rohde, L. A. (2015). Annual research review: a meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. Journal of Child Psychology and Psychiatry, 56(3), 345–365.

    Google Scholar 

  54. Qadeer, R. A., & Ferro, M. A. (2017). Child–parent agreement on health-related quality of life in children with newly diagnosed chronic health conditions: a longitudinal study. International Journal of Adolescence and Youth, 23(1), 99–108.

    Google Scholar 

  55. Radloff, L. S. (1977). The CES-D: a self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401.

    Google Scholar 

  56. Ravelli, A., Viola, S., Migliavacca, D., Pistorio, A., Ruperto, N., & Martini, A. (2001). Discordance between proxy-reported and observed assessment of functional ability of children with juvenile idiopathic arthritis. Rheumatology, 40(8), 914–919.

    Google Scholar 

  57. Ravens-Sieberer, U., Gosch, A., Erhart, M., von Rueden, E., Nickel, J., Kurth, B., et al. (2006). The KIDSCREEN questionnaires: Quality of life questionnaires for children and adolescents. Germany: Lengerich.

    Google Scholar 

  58. Ravens-Sieberer, U., Auquier, P., Erhart, M., Gosch, A., Rajmil, L., Bruil, J., Power, M., Duer, W., Cloetta, B., Czemy, L., Mazur, J., Czimbalmos, A., Tountas, Y., Hagquist, C., Kilroe, J., & European KIDSCREEN Group. (2007). The KIDSCREEN-27 quality of life measures for children and adolescents: psychometric results from a cross-cultural survey in 13 European countries. Quality of Life Research, 16, 1347–1356.

    Google Scholar 

  59. Ravens-Sieberer, U., Wille, N., Erhart, M., Bettge, S., Wittchen, H.-U., Rothenberger, A., et al. (2008). Prevalence of mental health problems among children and adolescents in Germany: results of the BELLA study within the national health interview and examination survey. European Child and Adolescent Psychiatry, 17(S1), 22–33.

    Google Scholar 

  60. Ravens-Sieberer, U., Herdman, M., Devine, J., Otto, C., Bullinger, M., Rose, M., & Klasen, F. (2014). The European KIDSCREEN approach to measure quality of life and well-being in children: development, current application, and future advances. Quality of Life Research, 23(3), 791–803.

    Google Scholar 

  61. Robitail, S., Ravens-Sieberer, U., Simeoni, M. C., Rajmil, L., Bruil, J., Power, M., Duer, W., Cloetta, B., Czemy, L., Mazur, J., Czimbalmos, A., Tountas, Y., Hagquist, C., Kilroe, J., Auquier, P., & KIDSCREEN Group. (2007a). Testing the structural and cross-cultural validity of the KIDSCREEN-27 quality of life questionnaire. Quality of Life Research, 16, 1335–1345.

    Google Scholar 

  62. Robitail, S., Simeoni, M. C., Ravens-Sieberer, U., Bruil, J., & Auquier, P. (2007b). Children proxies’ quality-of-life agreement depended on the country using the European KIDSCREEN-52 questionnaire. Journal of Clinical Epidemiology, 60(5), 469–478.

    Google Scholar 

  63. Rothman, K. J. (1990). No adjustments are needed for multiple comparisons. Epidemiology, 1(1), 43–46.

    Google Scholar 

  64. Sattoe, J. N., van Staa, A., & Moll, H. A. (2012). The proxy problem anatomized: child-parent disagreement in health related quality of life reports of chronically ill adolescents. Health and Quality of Life Outcomes, 10(1), 10.

    Google Scholar 

  65. Sawyer, M. G., Whaites, L., Rey, J. M., Hazell, P. L., Graetz, B. W., & Baghurst, P. (2002). Health-related quality of life of children and adolescents with mental disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 41(5), 530–537.

    Google Scholar 

  66. Shaw, K. L., Southwood, T. R., & McDonagh, J. E. (2006). Growing up and moving on in rheumatology: parents as proxies of adolescents with juvenile idiopathic arthritis. Arthritis Care and Research, 55(2), 189–198.

    Google Scholar 

  67. Sheehan, D. V., Sheehan, K. H., Shytle, R. D., Janavs, J., Bannon, Y., Rogers, J. E., et al. (2010). Reliability and validity of the mini international neuropsychiatric interview for children and adolescents (MINI-KID). Journal of Clinical Psychiatry, 71(3), 313–326.

    Google Scholar 

  68. Spielberger, C. D. (1983). State-trait anxiety inventory for adults. Menlo Park: Mind Garden Inc..

    Google Scholar 

  69. Steinmetz, H. (2013). Analyzing observed composite differences across groups: is partial measurement invariance enough? Methodology, 9(1), 1–12.

    Google Scholar 

  70. Steinmetz, H., Schmidt, P., Tina-Booh, A., Wieczorek, S., & Schwartz, S. H. (2009). Testing measurement invariance using multigroup CFA: differences between educational groups in human values measurement. Quality and Quantity, 43(4), 599–616.

    Google Scholar 

  71. Streiner, D. L. (2015). Best (but oft-forgotten) practices: the multiple problems of multiplicity-whether and how to correct for many statistical tests. American Journal of Clinical Nutrition, 102(4), 721–728.

    Google Scholar 

  72. Tracey, S. A., Chorpita, B. F., Douban, J., & Barlow, D. H. (1997). Empirical evaluation of DSM-IV geralized anxiety disorder criteria in children and adolescents. Journal of Clinical Child Psychology, 26(4), 404–414.

    Google Scholar 

  73. Upton, P., Lawford, J., & Eiser, C. (2008). Parent-child agreement across child health-related quality of life instruments: a review of the literature. Quality of Life Research, 17(6), 895–913.

    Google Scholar 

  74. van de Schoot, R., Lugtig, P., & Hox, J. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492.

    Google Scholar 

  75. Vance, Y. H., Morse, R. C., Jenney, M. E., & Eiser, C. (2001). Issues in measuring quality of life in childhood cancer: measures, proxies, and parental mental health. Journal of Child Psychology and Psychiatry, 42(5), 661–667.

    Google Scholar 

  76. Varni, J. W., Burwinkle, T. M., & Lane, M. M. (2005). Health-related quality of life measurement in pediatric clinical practice: an appraisal and precept for future research and application. Health and Quality of Life Outcomes, 3(1), 34.

    Google Scholar 

  77. Varni, J. W., Limbers, C. A., & Burwinkle, T. M. (2007). Parent proxy-report of their children’s health-related quality of life: an analysis of 13,878 parents’ reliability and validity across age subgroups using the PedsQL 4.0 generic Core scales. Health and Quality of Life Outcomes, 5(1), 2.

    Google Scholar 

  78. Wagner, A. K., & Vickrey, B. G. (1995). The routine use of health-related quality of life measures in the care of patients with epilepsy: rationale and research agenda. Quality of Life Research, 4(2), 169–177.

    Google Scholar 

  79. Wittchen, H. U., Nelson, C. B., & Lachner, G. (1998). Prevalence of mental disorders and psychosocial impairments in adolescents and young adults. Psychological Medicine, 28(1), 109–126.

    Google Scholar 

  80. Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913–934.

    Google Scholar 

  81. Wymbs, B. T., Wymbs, F. A., & Dawson, A. E. (2015). Child ADHD and ODD behavior interacts with parent ADHD symptoms to worsen parenting and Interparental communication. Journal of Abnormal Child Psychology, 43(1), 107–119.

    Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the children, parents and health professionals and their staff without whose participation this study would not have been possible. We especially Jessica Zelman for coordinating the study and Felice Bontempo, who, through many discussions, provided inspiration for this manuscript. This research was supported by a grant from Hamilton Health Sciences (NIF-14363). Mr. Tompke is supported by funds from the Early Researcher Award from the Ministry of Research, Innovation and Science awarded to Dr. Ferro. Dr. Ferro holds the Canada Research Chair in Youth Mental Health.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mark A. Ferro.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tompke, B.K., Ferro, M.A. Measurement Invariance and Informant Discrepancies of the KIDSCREEN-27 in Children with Mental Disorder. Applied Research Quality Life 16, 891–910 (2021). https://doi.org/10.1007/s11482-019-09801-5

Download citation

Keywords

  • Adolescent
  • Health-related quality of life
  • Measurement
  • Validity