Advertisement

Satisfaction in Child and Adolescent Mental Health Services: Translating Users’ Feedback into Measurement

  • Anna BrownEmail author
  • Tamsin Ford
  • Jessica Deighton
  • Miranda Wolpert
Original Article

Abstract

The present research addressed gaps in our current understanding of validity and quality of measurement provided by patient reported experience measures. We established the psychometric properties of a freely available experience of service questionnaire (ESQ), based on responses from 7,067 families of patients across 41 UK providers of child and adolescent mental health services, using the two-level latent trait modeling. Responses to the ESQ were subject to strong ‘halo’ effects, which were thought to represent the overall positive or negative affect towards one’s treatment. Two strongly related constructs measured by the ESQ were interpreted as specific aspects of global satisfaction, namely satisfaction with care, and with environment. The Care construct was sensitive to differences between less satisfied patients, facilitating individual and service-level problem evaluation. The effects of nesting within service providers were strong, with parental reports being the most reliable source of data for the between-provider comparisons. We provide a scoring protocol for converting the hand-scored ESQ to the model-based population-referenced scores with supplied standard errors, which can be used for benchmarking services as well as individual evaluations.

Keywords

Patient satisfaction PREM Halo effects Affective overtones Approximated IRT scores 

Notes

Acknowledgments

We are grateful to all members of the CORC collaboration for providing the data, and to the CORC central team researchers Jenna Bradley and Halina Flannery for preparing the data for analyses. Anna Brown’s work on this article was supported by Grant RG63087 from the Isaac Newton Trust, University of Cambridge. Jessica Deighton’s work on this article was provided as part of the Department of Health Child Policy Research Unit (Reference number: 109/0001).

Conflict of interest

The authors have no competing interests. CORC is not-for-profit Company limited by guarantee. Tamsin Ford is an unpaid director of CORC (other than travel expenses). Miranda Wolpert is a paid director of CORC (3 days a week).

References

  1. Anderson, J. A., Rivera, V. R., & Kutash, K. (1998). Measuring consumer satisfaction with children’s mental health. In M. H. Epstein, K. Kutash, & A. J. Duchnowski (Eds.), Outcomes for children with emotional and behavioral disorders and their families: Program and evaluation best practices (pp. 455–482). Austin: Pro-Ed.Google Scholar
  2. Attkisson, C., & Greenfield, T. (1996). The client satisfaction questionnaire (CSQ) scales and the service satisfaction scale-30 (SSS-30). In L. Sederer & B. Dickey (Eds.), Outcomes assessment in clinical practice. Baltimore: Williams & Wilkins.Google Scholar
  3. Attride-Stirling, J. (2002). Development of methods to capture users’ views of CAMHS in clinical governance reviews. http://www.corc.uk.net/resources/downloads/.
  4. Ayton, A., Mooney, M., Sillifant, K., Powls, J., & Rasool, H. (2007). The development of the child and adolescent versions of the verona service satisfaction scale (CAMHSSS). Social Psychiatry and Psychiatric Epidemiology, 42, 892–901. doi: 10.1007/s00127-007-0241-9.PubMedCrossRefGoogle Scholar
  5. Barber, A. J., Tischler, V. A., & Healy, E. (2006). Consumer satisfaction and child behaviour problems in child and adolescent mental health services. Journal of Child Health Care, 10(1), 9–21. doi: 10.1177/1367493506060200.PubMedCrossRefGoogle Scholar
  6. Barnes, D., Devanney, C., Uglebjerg, A., Wistow, R. & Hartley, C. (2010). A profile of children’s health services, child and adolescent mental health services and maternity services in England 2008/9. Durham: Durham University. http://dro.dur.ac.uk/8220/1/8220.pdf.
  7. Biering, P., & Jensen, V. (2011). The concept of patient satisfaction in adolescent psychiatric care: A qualitative study. Journal of Child and Adolescent Psychiatric Nursing, 24, 3–10. doi: 10.1111/j.1744-6171.2010.00261.x.PubMedCrossRefGoogle Scholar
  8. Brannan, A., Sonnichesen, E., & Heflinger, C. (1996). Measuring satisfaction with children’s mental health services: Validity and reliability of the satisfaction scales. Evaluation and Program Planning, 19, 131–141. doi: 10.1016/0149-7189(96)00004-3.CrossRefGoogle Scholar
  9. Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150. doi: 10.1207/S15327906MBR3601_05.CrossRefGoogle Scholar
  10. Day, C., Michelson, D., & Hassan, I. (2011). Child and adolescent service experience (ChASE): Measuring service quality and therapeutic process. British Journal of Clinical Psychology, 50, 452–464. doi: 10.1111/j.2044-8260.2011.02008.x.PubMedCrossRefGoogle Scholar
  11. Department of Health. (2009). Understanding what matters: A guide to using patient feedback to transform services. London: Department of Health. Retrieved from http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_099780.
  12. Edwards, A., & Elwyn, G. (2001). Developing professional ability to involve patients in their care: Pull or push? Quality in Health Care, 10, 129–134. doi: 10.1136/qhc.0100129.PubMedCentralPubMedCrossRefGoogle Scholar
  13. Garland, A., Saltzman, M., & Aarons, G. (2000). Adolescent satisfaction with mental health services: Development of a multidimensional scale. Evaluation and Program Planning, 23, 165–175. doi: 10.1016/S0149-7189(00)00009-4.CrossRefGoogle Scholar
  14. Goodman, R. (2001). Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ). Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345. doi: 10.1097/00004583-200111000-00015.
  15. Hart, A., Saunders, A., & Thomas, H. (2005). Attuned practice: a service user study of specialist child and adolescent mental health, UK. Epidemiology and Psychiatric Sciences, 14, 22–31. doi: 10.1017/S1121189X00001895.CrossRefGoogle Scholar
  16. Holbrook, M. B. (1983). Using a structural model of halo effect to assess perceptual distortion due to affective overtones. Journal of Consumer Research, 10, 247–252. http://www.jstor.org/stable/2488928.
  17. Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York: Routledge.Google Scholar
  18. Kazdin, A., & Mazurick, J. (1994). Dropping out of child psychotherapy: Distinguishing early and late dropouts over the course of treatment. Journal of Consulting and Clinical Psychology, 62, 1069–1074. doi: 10.1177/1359104596011012.PubMedCrossRefGoogle Scholar
  19. Kennedy, I. (2010). Getting it right for children and young people: Overcoming cultural barriers in the NHS so as to meet their needs. London: Department of Health. http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_119445.
  20. McNaughton, D. (1994). Measuring parent satisfaction with early childhood intervention programs. Topics in Early Childhood Special Education, 14, 26–48. doi: 10.1177/027112149401400106.CrossRefGoogle Scholar
  21. Murphy, K. R., Jako, R. A., & Anhalt, R. L. (1993). Nature and consequences of halo error: A critical analysis. Journal of Applied Psychology, 78, 218–225. doi: 10.1037/0021-9010.78.2.218.CrossRefGoogle Scholar
  22. Muthén, L. K. & Muthén, B. O. (1998-2010). Mplus user’s guide. 6th Ed. Los Angeles, CA: Muthén & Muthén. www.stamodel.com.
  23. Muthén, B. & Satorra, A. (1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267–316. http://www.jstor.org/stable/271070.
  24. Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized multilevel structural equation modeling. Psychometrika, 69, 167–190. doi: 10.1007/BF02295939.CrossRefGoogle Scholar
  25. Reise, S., & Haviland, M. (2005). Item response theory and the measurement of clinical change. Journal of Personality Assessment, 84, 228–238. doi: 10.1207/s15327752jpa8403_02.PubMedCrossRefGoogle Scholar
  26. Reise, S., & Waller, G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 25–46. doi: 10.1146/annurev.clinpsy.032408.153553.CrossRefGoogle Scholar
  27. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society.Google Scholar
  28. SDQ: Normative SDQ data from Britain. (2001). http://www.sdqinfo.com/UKNorm.html.
  29. Shapiro, J., Welker, C., & Jacobson, B. (1997). The youth client satisfaction questionnaire: Development, construct validation and factor structure. Journal of Child Clinical Psychology, 26, 87–98. doi: 10.1207/s15374424jccp2601_9.CrossRefGoogle Scholar
  30. Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107–120. doi: 10.1007/s11336-008-9101-0.PubMedCentralPubMedCrossRefGoogle Scholar
  31. Stallard, P. (1995). Parental satisfaction with intervention: differences between respondents and non-respondents to postal questionnaire. British Journal of Clinical Psychology, 34, 397–405. doi: 10.1111/j.2044-8260.1995.tb01474.x.PubMedCrossRefGoogle Scholar
  32. Thissen, D., & Orlando, M. (2001). Item response theory for items scored in two categories. In D. Thissen & H. Wainer (Eds.), Test Scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  33. Williams, B., Coyle, J., & Healy, D. (1998). The meaning of patient satisfaction: An explanation of high reported levels. Social Science and Medicine, 47, 1351–1359. doi: 10.1016/S0277-9536(98)00213-5.PubMedCrossRefGoogle Scholar
  34. Young, S., Nicholson, J., & Davis, M. (1995). An overview of issues in research on consumer satisfaction with child and adolescent mental health services. Journal of Child and Family Studies, 4, 219–238. doi: 10.1007/BF02234097.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Anna Brown
    • 1
    • 4
    Email author
  • Tamsin Ford
    • 2
  • Jessica Deighton
    • 3
  • Miranda Wolpert
    • 3
  1. 1.Department of PsychiatryUniversity of CambridgeCambridgeUK
  2. 2.Peninsula Medical SchoolUniversity of ExeterExeterUK
  3. 3.CAMHS EBPUUniversity College London and the Anna Freud CentreLondonUK
  4. 4.School of PsychologyUniversity of KentCanterbury, KentUK

Personalised recommendations