Continuity of Care in Children’s Mental Health: Development of a Measure

  • Juliana I. Tobon
  • Graham J. Reid
  • Richard D. Goffin
Original Article

Abstract

A new measure, Continuity of Care in Children’s Mental Health (C3MH), is presented. The study involved item generation, pre-testing, pilot testing, and validation. The C3MH was administered to 364 parents recruited from 13 children’s mental health agencies in Ontario, Canada. The measure includes five scales supported by confirmatory factor analysis. Scale validity was supported through analyses of relationships with established measures of satisfaction, problem severity, and therapeutic alliance, as well as through known-group differences. The results of a pilot youth-report version (N = 57) are presented. The C3MH will be a useful tool for assessing improvements in system integration.

Keywords

Continuity of care Children’s mental health Youth Health services research 

Supplementary material

10488_2013_518_MOESM1_ESM.docx (18 kb)
Supplementary material 1 (DOCX 18 kb)

References

  1. Adair, C. E., Wild, C. T., Joyce, A., McDougall, G., Gordon, A., Costigan, N., et al. (2004). Continuity of mental health services study of Alberta: A research program on continuity of mental health care. Ottawa, ON: Canadian Health Services Research Foundation.Google Scholar
  2. Adamson, J., Gooberman-Hill, R., Woolhead, G., & Donovan, J. (2004). ‘Questerviews’: Using questionnaires in qualitative interviews as a method of integrating qualitative and quantitative health services research. Journal of Health Services Research and Policy, 9, 139–145. doi:10.1258/1355819041403268.PubMedCrossRefGoogle Scholar
  3. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B (Methodological), 289–300. Retrieved from http://www.stat.purdue.edu/doerge/BIOINFORM.D/FALL06/BenjaminiandYFDR.pdf .
  4. Benjamini, Y., & Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. Journal of Educational and Behavioral Statistics, 25, 60–83. Retrieved from http://www.unt.edu/rss/class/mike/5030/articles/benjaminihochberg.pdf.Google Scholar
  5. Bentler, P. M. (1992). On the fit of models to covariances and methodology to the Bulletin. Psychological Bulletin, 112, 400–404. Retrieved from http://psycnet.apa.org/psycinfo/1993-12081-001.
  6. Bentler, P. M., & Wu, E. J. C. (2003). EQS 6.1 for Windows [Computer software]. Retrieved from http://www.mvsoft.com/pub/EQS61QuickStart.pdf.
  7. Bickman, L. (1996). A continuum of care: More is not always better. American Psychologist, 51, 689–701. doi:10.1037/0003-066X.51.7.689.PubMedCrossRefGoogle Scholar
  8. Bollen, K. A. (1989). A new incremental fit index for general structural equation models. Sociological Methods and Research, 17, 303–316. Retrieved from http://smr.sagepub.com/content/17/3/303.short.Google Scholar
  9. Boydell, K. M., Pong, R., Volpe, T., Tilleczek, K., Wilson, E., & Lemieux, S. (2006). Family perspectives on pathways to mental health care for children and youth in rural communities. Journal of Rural Health, 22, 182–188. doi:10.1111/j.1748-0361.2006.00029.x.PubMedCrossRefGoogle Scholar
  10. Boyle, M. H., Cunningham, C. E., Georgiades, K., Cullen, J., Racine, Y., & Pettingill, P. (2009). The Brief Child and Family Phone Interview (BCFPI): 2. Usefulness in screening for child and adolescent psychopathology. Journal of Child Psychology and Psychiatry, 50, 424–431. doi:10.1111/j.1469-7610.2008.01971.x.PubMedCrossRefGoogle Scholar
  11. Brannan, A. M., Sonnichsen, S. E., & Heflinger, C. A. (1996). Measuring satisfaction with children’s mental health services: Validity and reliability of the satisfaction scales. Evaluation and Program Planning, 19, 131–141. doi:10.1016/0149-7189(96)00004-3.CrossRefGoogle Scholar
  12. Briggs, S. R., & Cheek, J. M. (1986). The role of factor analysis in the development and evaluation of personality scales. Journal of Personality, 54, 106–148. doi:10.1111/j.1467-6494.1986.tb00391.x.CrossRefGoogle Scholar
  13. Byalin, K. (1993). Assessing parental satisfaction with children’s mental health services: A pilot study. Evaluation and Program Planning, 16, 69–72. Retrieved from http://www.sciencedirect.com/science/article/pii/0149718993900184.
  14. Byrne, B. (2006). Structural equation modeling with EQS: Basic concepts, applications, and programming. Mahwah, NJ: Erlbaum.Google Scholar
  15. Chao, J. (1988). Continuity of care: Incorporating patient perceptions. Family Medicine, 20, 333–337. Retrieved from PM:3266158.Google Scholar
  16. Cheung, A. H., & Dewa, C. S. (2007). Mental health service use among adolescents and young adults with major depressive disorder and suicidality. Canadian Journal of Psychiatry, 52, 228–232. Retrieved from http://cat.inist.fr/?aModele=afficheN&cpsidt=18725886.Google Scholar
  17. Christakis, D. A., Wright, J. A., Zimmerman, F. J., Bassett, A. L., & Connell, F. A. (2003). Continuity of care is associated with well-coordinated care. Ambulatory Pediatrics, 3, 82–86. doi:10.1367/1539-4409(2003)003<0082:COCIAW>2.0.CO;2.PubMedCrossRefGoogle Scholar
  18. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. Retrieved from http://psycnet.apa.org/journals/bul/112/1/155/.
  19. Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24, 385–396. doi:10.2307/2136404.PubMedCrossRefGoogle Scholar
  20. Comrey, A. L. (1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56, 754–761. Retrieved from http://psycnet.apa.org/journals/ccp/56/5/754/.Google Scholar
  21. Costa, P. T., Jr., & McRae, R. R. (1992). The Revised NEO Personality Inventory manual. Odessa, FL: Psychological Assessment Resources.Google Scholar
  22. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Retrieved from http://www.springerlink.com/index/n435u12541475367.pdf.
  23. Cunningham, C. E., Boyle, M. H., Hong, S., Pettingill, P., & Bohaychuk, D. (2009). The Brief Child and Family Phone Interview (BCFPI): 1. Rationale, development, and description of a computerized children’s mental health intake and outcome assessment tool. Journal of Child Psychology and Psychiatry, 50, 416–423. doi:10.1111/j.1469-7610.2008.01970.x.PubMedCrossRefGoogle Scholar
  24. Davidson, S., & Manion, I. G. (1996). Facing the challenge: Mental health and illness in Canadian youth. Psychology, Health and Medicine, 1, 41–56. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/13548509608400005.
  25. DeVellis, R. F. (2011). Scale development: Theory and applications. Thousand Oaks, CA: Sage Publications, Inc.Google Scholar
  26. Digiuseppe, R., Linscott, J., & Jilton, R. (1996). Developing the therapeutic alliance in child—Adolescent psychotherapy. Applied and Preventive Psychology, 5, 85–100. Retrieved from http://www.sciencedirect.com/science/article/pii/S0962184996800023.
  27. Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. doi:10.3758/BF03193146.PubMedCrossRefGoogle Scholar
  28. Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.Google Scholar
  29. Flocke, S. A. (1997). Measuring attributes of primary care: Development of a new instrument. Journal of Family Practice, 45, 64–74. Retrieved from ISI:A1997XK68300010.Google Scholar
  30. Gati, I. (1981). Properties of the item efficiency index for minimum redundancy item analysis. Educational and Psychological Measurement, 41, 973–978. Retrieved from http://epm.sagepub.com/content/41/4/973.short.
  31. Glasgow, R. E., Wagner, E. H., Schaefer, J., Mahoney, L. D., Reid, R. J., & Greene, S. M. (2005). Development and validation of the patient assessment of chronic illness care (PACIC). Medical Care, 43, 436. Retrieved from http://journals.lww.com/lww-medicalcare/Abstract/2005/05000/Development_and_Validation_of_the_Patient.3.aspx.
  32. Goffin, R. D., & Olson, J. M. (2011). Is it all relative? Comparative judgments and the possible improvement of self-ratings and ratings of others. Perspectives on Psychological Science, 6, 48–60. Retrieved from http://pps.sagepub.com/content/6/1/48.short.
  33. Government of Ontario. (2011). Open minds, healthy minds: Ontarios comprehensive mental health and addictions strategy. Retrieved from http://www.health.gov.on.ca/en/common/ministry/publications/reports/mental_health2011/mentalhealth.aspx.
  34. Guadagnoli, E., & Velicer, W. F. (1988). Relation to sample size to the stability of component patterns. Psychological Bulletin, 103, 265–275. Retrieved from http://psycnet.apa.org/psycinfo/1988-15800-001.
  35. Haggerty, J. L., Reid, R. J., Freeman, G. K., Starfield, B. H., Adair, C. E., & McKendry, R. (2003). Continuity of care: A multidisciplinary review. British Medical Journal, 327, 1219–1221. doi:10.1136/bmj.327.7425.1219.PubMedCentralPubMedCrossRefGoogle Scholar
  36. Hawley, K. M., & Garland, A. F. (2008). Working alliance in adolescent outpatient therapy: Youth, parent and therapist reports and associations with therapy outcomes. Child and Youth Care Forum, 37, 59–74. doi:10.1007/s10566-008-9050-x.CrossRefGoogle Scholar
  37. Heflinger, C. A. (1996). Measuring service system coordination in managed mental health care for children and youth. Evaluation and Program Planning, 19, 155–163. doi:10.1016/0149-7189(96)00006-7.CrossRefGoogle Scholar
  38. Heflinger, C. A., Sonnichsen, S. E., & Brannan, A. M. (1996). Parent satisfaction with children’s mental health services in a children’s mental health managed care demonstration. Journal of Mental Health Administration, 23, 69–79. doi:10.1007/BF02518644.PubMedCrossRefGoogle Scholar
  39. Henry, J. D., & Crawford, J. R. (2005). The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 44, 227–239. Retrieved from WOS:000229910200007.Google Scholar
  40. Holmbeck, G. N., & Devine, K. A. (2009). Editorial: An author’s checklist for measure development and validation manuscripts. Journal of Pediatric Psychology, 34, 691. Retrieved from http://jpepsy.oxfordjournals.org/content/34/7/691.short.
  41. Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36, 223–233. doi:10.1037/0022-0167.36.2.223.CrossRefGoogle Scholar
  42. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424. Retrieved from http://people.cehd.tamu.edu/okwok/epsy651R/Articles/HuBentler1998.pdf.
  43. Hu, L.T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.Google Scholar
  44. Jackson, D. N. (1970). A sequential system for personality scale development. In C. D. Spielberger (Ed.), Current topics in clinical and community psychology (pp. 61–96). New York: Academic Press.Google Scholar
  45. Jackson, D. N. (1974). Personality research form manual. Port Huron, MI: Research Psychologists Press.Google Scholar
  46. Kazdin, A. E. (1996). Dropping out of child psychotherapy: Issues for research and implications for practice. Clinical Child Psychology and Psychiatry, 1, 133–156. doi:10.1177/1359104596011012.CrossRefGoogle Scholar
  47. Kazdin, A. E., Holland, L., & Crowley, M. (1997). Family experience of barriers to treatment and premature termination from child therapy. Journal of Consulting and Clinical Psychology, 65, 453–463. Retrieved from ISI:A1997XA22300011.Google Scholar
  48. Kazdin, A. E., & Mazurick, J. L. (1994). Dropping out of child psychotherapy: Distinguishing early and late dropouts over the course of treatment. Journal of Consulting and Clinical Psychology, 62, 1069–1074.PubMedCrossRefGoogle Scholar
  49. Kincaid, J. P., Fishburne Jr., R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and Flesch reading ease formula) for navy enlisted personnel. Retrieved from http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA006655.
  50. King, S., King, G., & Rosenbaum, P. (2004). Evaluating health service delivery to children with chronic conditions and their families: Development of a refined Measure of Processes of Care (MPOC-20). Children’s Health Care, 33, 35–57. doi:10.1207/s15326888chc3301_3.CrossRefGoogle Scholar
  51. King, S., Rosenbaum, P., & King, G. (1995). The Measure of Processes of Care: A means to assess family-centred behaviors of health providers. Hamilton, ON: CanChild Centre for Childhood Disability Research.Google Scholar
  52. Kleinbaum, D. G., Kupper, L. L., Muller, K. E., & Nizam, A. (1978). Applied regression analysis and other multivariable methods. North Scituate, MA: Duxbury Press.Google Scholar
  53. Koren, P. E., Paulson, R. I., Kinney, R. F., Yatchmenoff, D. K., Gordon, L. J., & DeChillo, N. (1997). Service coordination in children’s mental health: An empirical study from the caregiver’s perspective. Journal of Emotional and Behavioral Disorders, 5, 162–172. doi:10.1177/106342669700500304.CrossRefGoogle Scholar
  54. Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., & Nguyen, T. D. (1979). Assessment of client/patient satisfaction: Development of a general scale., 2, 197–207. doi:10.1016/0149-7189(79)90094-6.Google Scholar
  55. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149. Retrieved from http://psycnet.apa.org/journals/met/1/2/130/.
  56. Mazor, K. M., Clauser, B. E., Field, T., Yood, R. A., & Gurwitz, J. H. (2002). A demonstration of the impact of response bias on the results of patient satisfaction surveys. Health Services Research, 37, 1403–1417. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/1475-6773.11194/full.
  57. Measelle, J. R., Weinstein, R. S., & Martinez, M. (1998). Parent satisfaction with case managed systems of care for children and youth with severe emotional disturbance. Journal of Child and Family Studies, 7, 451–467. Retrieved from http://www.springerlink.com/index/l2qlj1816787205g.pdf.Google Scholar
  58. Minore, B., Boone, M., Arthur, A., & O’Sullivan, J. (2005). Managing continuity of care for children with special needs in rural and remote parts of northern Ontario. Ottawa, ON: Canadian Health Services Research Foundation. Retrieved from www.coespecialneeds.ca/PDF/continuityofcare.pdf.
  59. Morrissey, J. P. (1992). An interorganizational network approach to evaluating children’s mental health service systems. In L. Bickman & D. Rog (Eds.), Children’s mental health services: Methodological issues (pp. 85–98). San Francisco: Jossey Bass.Google Scholar
  60. Neill, J. A., & Jackson, D. N. (1976). Minimum redundancy item analysis. Educational and Psychological Measurement, 36, 123–134. Retrieved from http://epm.sagepub.com/content/36/1/123.short.
  61. Nunnally, J. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.Google Scholar
  62. Offord, D. R., Boyle, M. H., Fleming, J. E., & Blum, H. M. (1989). Summary of selected results. The Canadian Journal of Psychiatry/La Revue canadienne de psychiatrie, 34, 483–491. Retrieved from http://psycnet.apa.org/psycinfo/1990-04596-001.
  63. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). San Diego, CA: Academic Press. Retrieved from http://doi.apa.org/psycinfo/1991-97206-001.
  64. Paulhus, D. L., & Reid, D. B. (1991). Enhancement and denial in socially desirable responding. Journal of Personality and Social Psychology, 60, 307. doi:10.1037/0022-3514.60.2.307.CrossRefGoogle Scholar
  65. Perneger, T. V., Chamot, E., & Bovier, P. A. (2005). Nonresponse bias in a survey of patient perceptions of hospital care. Medical Care, 43, 374. Retrieved from http://journals.lww.com/lww-medicalcare/Abstract/2005/04000/Nonresponse_Bias_in_a_Survey_of_Patient.8.aspx.Google Scholar
  66. Reid, R., Haggerty, J., & McKendry, R. (2002). Defusing the confusion: Concepts and measures of continuity of healthcare. Ottawa, ON: Canadian Health Services Research Foundation. Retrieved from http://www.chsrf.ca/Migrated/PDF/ResearchReports/CommissionedResearch/cr_contcare_e.pdf.
  67. Rey, J. M., Plapp, J. M., & Simpson, P. L. (1999). Parental satisfaction and outcome: A 4-year study in a child and adolescent mental health service. Australian and New Zealand Journal of Psychiatry, 33, 22–28. Retrieved from http://informahealthcare.com/doi/abs/10.1046/j.1440-1614.1999.00516.x.
  68. Riley, S. E., Stromberg, A. J., & Clark, J. (2005). Assessing parental satisfaction with children’s mental health services with the youth services survey for families. Journal of Child and Family Studies, 14, 87–99. Retrieved from http://www.springerlink.com/index/G27237722440136R.pdf.Google Scholar
  69. Rosenthal, T. C. (2008). The medical home: Growing evidence to support a new approach to primary care. The Journal of the American Board of Family Medicine, 21, 427–440. Retrieved from http://www.jabfp.com/content/21/5/427.full.Google Scholar
  70. Satorra, A., & Bentler, P. M. (2001). A scaled difference Chi square test statistic for moment structure analysis. Psychometrika, 66, 507–514. Retrieved from http://link.springer.com/article/10.1007/BF02296192.
  71. Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71, 452. Retrieved from http://users.phhp.ufl.edu/rbauer/EBPP/shirk_carver.pdf.
  72. Shirk, S. R., & Saiz, C. C. (1992). Clinical, empirical, and developmental perspectives on the therapeutic relationship in child psychotherapy. Development and Psychopathology, 4, 713–728. Retrieved from http://journals.cambridge.org/production/action/cjoGetFulltext?fulltextid=2541760.
  73. Spector, P. E. (1992). Summated rating scale construction: An Introduction. Newbury Park, CA: Sage Publications, Inc.CrossRefGoogle Scholar
  74. Stroul, B. A. (1996). Children’s mental health: Creating systems of care in a changing society. Baltimore: Paul H. Brookes Publishing Co.Google Scholar
  75. Stroul, B. A., & Friedman, R. (1986). A system of care for severely emotionally disturbed youth. Washington, DC: Georgetown University, Child and Adolescent Service System Program Technical Assistance Center.Google Scholar
  76. The Provincial Centre of Excellence for Child and Youth Mental Health at CHEO. (2006). Indicators of quality for child and youth mental healthcare in Ontario. Ottawa, ON: The Provincial Centre of Excellence for Child and Youth Mental Health at CHEO.Google Scholar
  77. Thornicroft, G., & Tansella, M. (1999). Translating ethical principles into outcome measures for mental health service research. Psychological Medicine, 29, 761–767. doi:10.1017/S0033291798008034.PubMedCrossRefGoogle Scholar
  78. Tobon, J. I., Reid, G. J., & Brown, J. B. (2013). Continuity of care in children’s mental health: Parent, youth, and professional perspectives. (submitted).Google Scholar
  79. Tracey, T. J., & Kokotovic, A. M. (1989). Factor structure of the Working Alliance Inventory. Psychological Assessment, 1, 207–210.CrossRefGoogle Scholar
  80. Traugott, M. W. (1987). The importance of persistence in respondent selection for preelection surveys. Public Opinion Quarterly, 51, R-48. doi:10.1086/269013.
  81. University of South Florida. (2007). System of care practice review-revised: Finding your way in a local system of care. Retrieved from http://logicmodel.fmhi.usf.edu/resources/PDF/SOCPR-Protocol.pdf.
  82. Ware, N. C., Dickey, B., Tugenberg, T., & McHorney, C. A. (2003). CONNECT: A measure of continuity of care in mental health services. Mental Health Services Research, 5, 209–221. Retrieved from PM:14672500.Google Scholar
  83. Wintersteen, M. B., Mensinger, J. L., & Diamond, G. S. (2005). Do gender and racial differences between patient and therapist affect therapeutic alliance and treatment retention in adolescents? Professional Psychology: Research and Practice, 36, 400–408. doi:10.1037/0735-7028.36.4.400.CrossRefGoogle Scholar
  84. Young, S. C., Nicholson, J., & Davis, M. (1995). An overview of issues in research on consumer satisfaction with child and adolescent mental health services. Journal of Child and Family Studies, 4, 219–238. doi:10.1007/BF02234097.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Juliana I. Tobon
    • 1
  • Graham J. Reid
    • 1
    • 2
    • 3
  • Richard D. Goffin
    • 1
  1. 1.Department of PsychologyThe University of Western OntarioLondonCanada
  2. 2.Department of Family Medicine, Schulich School of Medicine and DentistryThe University of Western OntarioLondonCanada
  3. 3.Department of Paediatrics, Schulich School of Medicine and DentistryThe University of Western OntarioLondonCanada

Personalised recommendations