Continuity of Care in Children’s Mental Health: Development of a Measure

  • Juliana I. Tobon
  • Graham J. Reid
  • Richard D. Goffin
Original Article


A new measure, Continuity of Care in Children’s Mental Health (C3MH), is presented. The study involved item generation, pre-testing, pilot testing, and validation. The C3MH was administered to 364 parents recruited from 13 children’s mental health agencies in Ontario, Canada. The measure includes five scales supported by confirmatory factor analysis. Scale validity was supported through analyses of relationships with established measures of satisfaction, problem severity, and therapeutic alliance, as well as through known-group differences. The results of a pilot youth-report version (N = 57) are presented. The C3MH will be a useful tool for assessing improvements in system integration.


Continuity of care Children’s mental health Youth Health services research 

Supplementary material

10488_2013_518_MOESM1_ESM.docx (18 kb)
Supplementary material 1 (DOCX 18 kb)


  1. Adair, C. E., Wild, C. T., Joyce, A., McDougall, G., Gordon, A., Costigan, N., et al. (2004). Continuity of mental health services study of Alberta: A research program on continuity of mental health care. Ottawa, ON: Canadian Health Services Research Foundation.Google Scholar
  2. Adamson, J., Gooberman-Hill, R., Woolhead, G., & Donovan, J. (2004). ‘Questerviews’: Using questionnaires in qualitative interviews as a method of integrating qualitative and quantitative health services research. Journal of Health Services Research and Policy, 9, 139–145. doi:10.1258/1355819041403268.PubMedCrossRefGoogle Scholar
  3. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B (Methodological), 289–300. Retrieved from .
  4. Benjamini, Y., & Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. Journal of Educational and Behavioral Statistics, 25, 60–83. Retrieved from Scholar
  5. Bentler, P. M. (1992). On the fit of models to covariances and methodology to the Bulletin. Psychological Bulletin, 112, 400–404. Retrieved from
  6. Bentler, P. M., & Wu, E. J. C. (2003). EQS 6.1 for Windows [Computer software]. Retrieved from
  7. Bickman, L. (1996). A continuum of care: More is not always better. American Psychologist, 51, 689–701. doi:10.1037/0003-066X.51.7.689.PubMedCrossRefGoogle Scholar
  8. Bollen, K. A. (1989). A new incremental fit index for general structural equation models. Sociological Methods and Research, 17, 303–316. Retrieved from Scholar
  9. Boydell, K. M., Pong, R., Volpe, T., Tilleczek, K., Wilson, E., & Lemieux, S. (2006). Family perspectives on pathways to mental health care for children and youth in rural communities. Journal of Rural Health, 22, 182–188. doi:10.1111/j.1748-0361.2006.00029.x.PubMedCrossRefGoogle Scholar
  10. Boyle, M. H., Cunningham, C. E., Georgiades, K., Cullen, J., Racine, Y., & Pettingill, P. (2009). The Brief Child and Family Phone Interview (BCFPI): 2. Usefulness in screening for child and adolescent psychopathology. Journal of Child Psychology and Psychiatry, 50, 424–431. doi:10.1111/j.1469-7610.2008.01971.x.PubMedCrossRefGoogle Scholar
  11. Brannan, A. M., Sonnichsen, S. E., & Heflinger, C. A. (1996). Measuring satisfaction with children’s mental health services: Validity and reliability of the satisfaction scales. Evaluation and Program Planning, 19, 131–141. doi:10.1016/0149-7189(96)00004-3.CrossRefGoogle Scholar
  12. Briggs, S. R., & Cheek, J. M. (1986). The role of factor analysis in the development and evaluation of personality scales. Journal of Personality, 54, 106–148. doi:10.1111/j.1467-6494.1986.tb00391.x.CrossRefGoogle Scholar
  13. Byalin, K. (1993). Assessing parental satisfaction with children’s mental health services: A pilot study. Evaluation and Program Planning, 16, 69–72. Retrieved from
  14. Byrne, B. (2006). Structural equation modeling with EQS: Basic concepts, applications, and programming. Mahwah, NJ: Erlbaum.Google Scholar
  15. Chao, J. (1988). Continuity of care: Incorporating patient perceptions. Family Medicine, 20, 333–337. Retrieved from PM:3266158.Google Scholar
  16. Cheung, A. H., & Dewa, C. S. (2007). Mental health service use among adolescents and young adults with major depressive disorder and suicidality. Canadian Journal of Psychiatry, 52, 228–232. Retrieved from Scholar
  17. Christakis, D. A., Wright, J. A., Zimmerman, F. J., Bassett, A. L., & Connell, F. A. (2003). Continuity of care is associated with well-coordinated care. Ambulatory Pediatrics, 3, 82–86. doi:10.1367/1539-4409(2003)003<0082:COCIAW>2.0.CO;2.PubMedCrossRefGoogle Scholar
  18. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. Retrieved from
  19. Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24, 385–396. doi:10.2307/2136404.PubMedCrossRefGoogle Scholar
  20. Comrey, A. L. (1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56, 754–761. Retrieved from Scholar
  21. Costa, P. T., Jr., & McRae, R. R. (1992). The Revised NEO Personality Inventory manual. Odessa, FL: Psychological Assessment Resources.Google Scholar
  22. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Retrieved from
  23. Cunningham, C. E., Boyle, M. H., Hong, S., Pettingill, P., & Bohaychuk, D. (2009). The Brief Child and Family Phone Interview (BCFPI): 1. Rationale, development, and description of a computerized children’s mental health intake and outcome assessment tool. Journal of Child Psychology and Psychiatry, 50, 416–423. doi:10.1111/j.1469-7610.2008.01970.x.PubMedCrossRefGoogle Scholar
  24. Davidson, S., & Manion, I. G. (1996). Facing the challenge: Mental health and illness in Canadian youth. Psychology, Health and Medicine, 1, 41–56. Retrieved from
  25. DeVellis, R. F. (2011). Scale development: Theory and applications. Thousand Oaks, CA: Sage Publications, Inc.Google Scholar
  26. Digiuseppe, R., Linscott, J., & Jilton, R. (1996). Developing the therapeutic alliance in child—Adolescent psychotherapy. Applied and Preventive Psychology, 5, 85–100. Retrieved from
  27. Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. doi:10.3758/BF03193146.PubMedCrossRefGoogle Scholar
  28. Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.Google Scholar
  29. Flocke, S. A. (1997). Measuring attributes of primary care: Development of a new instrument. Journal of Family Practice, 45, 64–74. Retrieved from ISI:A1997XK68300010.Google Scholar
  30. Gati, I. (1981). Properties of the item efficiency index for minimum redundancy item analysis. Educational and Psychological Measurement, 41, 973–978. Retrieved from
  31. Glasgow, R. E., Wagner, E. H., Schaefer, J., Mahoney, L. D., Reid, R. J., & Greene, S. M. (2005). Development and validation of the patient assessment of chronic illness care (PACIC). Medical Care, 43, 436. Retrieved from
  32. Goffin, R. D., & Olson, J. M. (2011). Is it all relative? Comparative judgments and the possible improvement of self-ratings and ratings of others. Perspectives on Psychological Science, 6, 48–60. Retrieved from
  33. Government of Ontario. (2011). Open minds, healthy minds: Ontarios comprehensive mental health and addictions strategy. Retrieved from
  34. Guadagnoli, E., & Velicer, W. F. (1988). Relation to sample size to the stability of component patterns. Psychological Bulletin, 103, 265–275. Retrieved from
  35. Haggerty, J. L., Reid, R. J., Freeman, G. K., Starfield, B. H., Adair, C. E., & McKendry, R. (2003). Continuity of care: A multidisciplinary review. British Medical Journal, 327, 1219–1221. doi:10.1136/bmj.327.7425.1219.PubMedCentralPubMedCrossRefGoogle Scholar
  36. Hawley, K. M., & Garland, A. F. (2008). Working alliance in adolescent outpatient therapy: Youth, parent and therapist reports and associations with therapy outcomes. Child and Youth Care Forum, 37, 59–74. doi:10.1007/s10566-008-9050-x.CrossRefGoogle Scholar
  37. Heflinger, C. A. (1996). Measuring service system coordination in managed mental health care for children and youth. Evaluation and Program Planning, 19, 155–163. doi:10.1016/0149-7189(96)00006-7.CrossRefGoogle Scholar
  38. Heflinger, C. A., Sonnichsen, S. E., & Brannan, A. M. (1996). Parent satisfaction with children’s mental health services in a children’s mental health managed care demonstration. Journal of Mental Health Administration, 23, 69–79. doi:10.1007/BF02518644.PubMedCrossRefGoogle Scholar
  39. Henry, J. D., & Crawford, J. R. (2005). The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 44, 227–239. Retrieved from WOS:000229910200007.Google Scholar
  40. Holmbeck, G. N., & Devine, K. A. (2009). Editorial: An author’s checklist for measure development and validation manuscripts. Journal of Pediatric Psychology, 34, 691. Retrieved from
  41. Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36, 223–233. doi:10.1037/0022-0167.36.2.223.CrossRefGoogle Scholar
  42. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424. Retrieved from
  43. Hu, L.T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.Google Scholar
  44. Jackson, D. N. (1970). A sequential system for personality scale development. In C. D. Spielberger (Ed.), Current topics in clinical and community psychology (pp. 61–96). New York: Academic Press.Google Scholar
  45. Jackson, D. N. (1974). Personality research form manual. Port Huron, MI: Research Psychologists Press.Google Scholar
  46. Kazdin, A. E. (1996). Dropping out of child psychotherapy: Issues for research and implications for practice. Clinical Child Psychology and Psychiatry, 1, 133–156. doi:10.1177/1359104596011012.CrossRefGoogle Scholar
  47. Kazdin, A. E., Holland, L., & Crowley, M. (1997). Family experience of barriers to treatment and premature termination from child therapy. Journal of Consulting and Clinical Psychology, 65, 453–463. Retrieved from ISI:A1997XA22300011.Google Scholar
  48. Kazdin, A. E., & Mazurick, J. L. (1994). Dropping out of child psychotherapy: Distinguishing early and late dropouts over the course of treatment. Journal of Consulting and Clinical Psychology, 62, 1069–1074.PubMedCrossRefGoogle Scholar
  49. Kincaid, J. P., Fishburne Jr., R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and Flesch reading ease formula) for navy enlisted personnel. Retrieved from
  50. King, S., King, G., & Rosenbaum, P. (2004). Evaluating health service delivery to children with chronic conditions and their families: Development of a refined Measure of Processes of Care (MPOC-20). Children’s Health Care, 33, 35–57. doi:10.1207/s15326888chc3301_3.CrossRefGoogle Scholar
  51. King, S., Rosenbaum, P., & King, G. (1995). The Measure of Processes of Care: A means to assess family-centred behaviors of health providers. Hamilton, ON: CanChild Centre for Childhood Disability Research.Google Scholar
  52. Kleinbaum, D. G., Kupper, L. L., Muller, K. E., & Nizam, A. (1978). Applied regression analysis and other multivariable methods. North Scituate, MA: Duxbury Press.Google Scholar
  53. Koren, P. E., Paulson, R. I., Kinney, R. F., Yatchmenoff, D. K., Gordon, L. J., & DeChillo, N. (1997). Service coordination in children’s mental health: An empirical study from the caregiver’s perspective. Journal of Emotional and Behavioral Disorders, 5, 162–172. doi:10.1177/106342669700500304.CrossRefGoogle Scholar
  54. Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., & Nguyen, T. D. (1979). Assessment of client/patient satisfaction: Development of a general scale., 2, 197–207. doi:10.1016/0149-7189(79)90094-6.Google Scholar
  55. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149. Retrieved from
  56. Mazor, K. M., Clauser, B. E., Field, T., Yood, R. A., & Gurwitz, J. H. (2002). A demonstration of the impact of response bias on the results of patient satisfaction surveys. Health Services Research, 37, 1403–1417. Retrieved from
  57. Measelle, J. R., Weinstein, R. S., & Martinez, M. (1998). Parent satisfaction with case managed systems of care for children and youth with severe emotional disturbance. Journal of Child and Family Studies, 7, 451–467. Retrieved from Scholar
  58. Minore, B., Boone, M., Arthur, A., & O’Sullivan, J. (2005). Managing continuity of care for children with special needs in rural and remote parts of northern Ontario. Ottawa, ON: Canadian Health Services Research Foundation. Retrieved from
  59. Morrissey, J. P. (1992). An interorganizational network approach to evaluating children’s mental health service systems. In L. Bickman & D. Rog (Eds.), Children’s mental health services: Methodological issues (pp. 85–98). San Francisco: Jossey Bass.Google Scholar
  60. Neill, J. A., & Jackson, D. N. (1976). Minimum redundancy item analysis. Educational and Psychological Measurement, 36, 123–134. Retrieved from
  61. Nunnally, J. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.Google Scholar
  62. Offord, D. R., Boyle, M. H., Fleming, J. E., & Blum, H. M. (1989). Summary of selected results. The Canadian Journal of Psychiatry/La Revue canadienne de psychiatrie, 34, 483–491. Retrieved from
  63. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). San Diego, CA: Academic Press. Retrieved from
  64. Paulhus, D. L., & Reid, D. B. (1991). Enhancement and denial in socially desirable responding. Journal of Personality and Social Psychology, 60, 307. doi:10.1037/0022-3514.60.2.307.CrossRefGoogle Scholar
  65. Perneger, T. V., Chamot, E., & Bovier, P. A. (2005). Nonresponse bias in a survey of patient perceptions of hospital care. Medical Care, 43, 374. Retrieved from Scholar
  66. Reid, R., Haggerty, J., & McKendry, R. (2002). Defusing the confusion: Concepts and measures of continuity of healthcare. Ottawa, ON: Canadian Health Services Research Foundation. Retrieved from
  67. Rey, J. M., Plapp, J. M., & Simpson, P. L. (1999). Parental satisfaction and outcome: A 4-year study in a child and adolescent mental health service. Australian and New Zealand Journal of Psychiatry, 33, 22–28. Retrieved from
  68. Riley, S. E., Stromberg, A. J., & Clark, J. (2005). Assessing parental satisfaction with children’s mental health services with the youth services survey for families. Journal of Child and Family Studies, 14, 87–99. Retrieved from Scholar
  69. Rosenthal, T. C. (2008). The medical home: Growing evidence to support a new approach to primary care. The Journal of the American Board of Family Medicine, 21, 427–440. Retrieved from Scholar
  70. Satorra, A., & Bentler, P. M. (2001). A scaled difference Chi square test statistic for moment structure analysis. Psychometrika, 66, 507–514. Retrieved from
  71. Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71, 452. Retrieved from
  72. Shirk, S. R., & Saiz, C. C. (1992). Clinical, empirical, and developmental perspectives on the therapeutic relationship in child psychotherapy. Development and Psychopathology, 4, 713–728. Retrieved from
  73. Spector, P. E. (1992). Summated rating scale construction: An Introduction. Newbury Park, CA: Sage Publications, Inc.CrossRefGoogle Scholar
  74. Stroul, B. A. (1996). Children’s mental health: Creating systems of care in a changing society. Baltimore: Paul H. Brookes Publishing Co.Google Scholar
  75. Stroul, B. A., & Friedman, R. (1986). A system of care for severely emotionally disturbed youth. Washington, DC: Georgetown University, Child and Adolescent Service System Program Technical Assistance Center.Google Scholar
  76. The Provincial Centre of Excellence for Child and Youth Mental Health at CHEO. (2006). Indicators of quality for child and youth mental healthcare in Ontario. Ottawa, ON: The Provincial Centre of Excellence for Child and Youth Mental Health at CHEO.Google Scholar
  77. Thornicroft, G., & Tansella, M. (1999). Translating ethical principles into outcome measures for mental health service research. Psychological Medicine, 29, 761–767. doi:10.1017/S0033291798008034.PubMedCrossRefGoogle Scholar
  78. Tobon, J. I., Reid, G. J., & Brown, J. B. (2013). Continuity of care in children’s mental health: Parent, youth, and professional perspectives. (submitted).Google Scholar
  79. Tracey, T. J., & Kokotovic, A. M. (1989). Factor structure of the Working Alliance Inventory. Psychological Assessment, 1, 207–210.CrossRefGoogle Scholar
  80. Traugott, M. W. (1987). The importance of persistence in respondent selection for preelection surveys. Public Opinion Quarterly, 51, R-48. doi:10.1086/269013.
  81. University of South Florida. (2007). System of care practice review-revised: Finding your way in a local system of care. Retrieved from
  82. Ware, N. C., Dickey, B., Tugenberg, T., & McHorney, C. A. (2003). CONNECT: A measure of continuity of care in mental health services. Mental Health Services Research, 5, 209–221. Retrieved from PM:14672500.Google Scholar
  83. Wintersteen, M. B., Mensinger, J. L., & Diamond, G. S. (2005). Do gender and racial differences between patient and therapist affect therapeutic alliance and treatment retention in adolescents? Professional Psychology: Research and Practice, 36, 400–408. doi:10.1037/0735-7028.36.4.400.CrossRefGoogle Scholar
  84. Young, S. C., Nicholson, J., & Davis, M. (1995). An overview of issues in research on consumer satisfaction with child and adolescent mental health services. Journal of Child and Family Studies, 4, 219–238. doi:10.1007/BF02234097.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Juliana I. Tobon
    • 1
  • Graham J. Reid
    • 1
    • 2
    • 3
  • Richard D. Goffin
    • 1
  1. 1.Department of PsychologyThe University of Western OntarioLondonCanada
  2. 2.Department of Family Medicine, Schulich School of Medicine and DentistryThe University of Western OntarioLondonCanada
  3. 3.Department of Paediatrics, Schulich School of Medicine and DentistryThe University of Western OntarioLondonCanada

Personalised recommendations