Satisfaction in Child and Adolescent Mental Health Services: Translating Users’ Feedback into Measurement
- 1.2k Downloads
The present research addressed gaps in our current understanding of validity and quality of measurement provided by patient reported experience measures. We established the psychometric properties of a freely available experience of service questionnaire (ESQ), based on responses from 7,067 families of patients across 41 UK providers of child and adolescent mental health services, using the two-level latent trait modeling. Responses to the ESQ were subject to strong ‘halo’ effects, which were thought to represent the overall positive or negative affect towards one’s treatment. Two strongly related constructs measured by the ESQ were interpreted as specific aspects of global satisfaction, namely satisfaction with care, and with environment. The Care construct was sensitive to differences between less satisfied patients, facilitating individual and service-level problem evaluation. The effects of nesting within service providers were strong, with parental reports being the most reliable source of data for the between-provider comparisons. We provide a scoring protocol for converting the hand-scored ESQ to the model-based population-referenced scores with supplied standard errors, which can be used for benchmarking services as well as individual evaluations.
KeywordsPatient satisfaction PREM Halo effects Affective overtones Approximated IRT scores
We are grateful to all members of the CORC collaboration for providing the data, and to the CORC central team researchers Jenna Bradley and Halina Flannery for preparing the data for analyses. Anna Brown’s work on this article was supported by Grant RG63087 from the Isaac Newton Trust, University of Cambridge. Jessica Deighton’s work on this article was provided as part of the Department of Health Child Policy Research Unit (Reference number: 109/0001).
Conflict of interest
The authors have no competing interests. CORC is not-for-profit Company limited by guarantee. Tamsin Ford is an unpaid director of CORC (other than travel expenses). Miranda Wolpert is a paid director of CORC (3 days a week).
- Anderson, J. A., Rivera, V. R., & Kutash, K. (1998). Measuring consumer satisfaction with children’s mental health. In M. H. Epstein, K. Kutash, & A. J. Duchnowski (Eds.), Outcomes for children with emotional and behavioral disorders and their families: Program and evaluation best practices (pp. 455–482). Austin: Pro-Ed.Google Scholar
- Attkisson, C., & Greenfield, T. (1996). The client satisfaction questionnaire (CSQ) scales and the service satisfaction scale-30 (SSS-30). In L. Sederer & B. Dickey (Eds.), Outcomes assessment in clinical practice. Baltimore: Williams & Wilkins.Google Scholar
- Attride-Stirling, J. (2002). Development of methods to capture users’ views of CAMHS in clinical governance reviews. http://www.corc.uk.net/resources/downloads/.
- Barnes, D., Devanney, C., Uglebjerg, A., Wistow, R. & Hartley, C. (2010). A profile of children’s health services, child and adolescent mental health services and maternity services in England 2008/9. Durham: Durham University. http://dro.dur.ac.uk/8220/1/8220.pdf.
- Department of Health. (2009). Understanding what matters: A guide to using patient feedback to transform services. London: Department of Health. Retrieved from http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_099780.
- Goodman, R. (2001). Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ). Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345. doi: 10.1097/00004583-200111000-00015.
- Holbrook, M. B. (1983). Using a structural model of halo effect to assess perceptual distortion due to affective overtones. Journal of Consumer Research, 10, 247–252. http://www.jstor.org/stable/2488928.
- Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York: Routledge.Google Scholar
- Kennedy, I. (2010). Getting it right for children and young people: Overcoming cultural barriers in the NHS so as to meet their needs. London: Department of Health. http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_119445.
- Muthén, L. K. & Muthén, B. O. (1998-2010). Mplus user’s guide. 6th Ed. Los Angeles, CA: Muthén & Muthén. www.stamodel.com.
- Muthén, B. & Satorra, A. (1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267–316. http://www.jstor.org/stable/271070.
- Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society.Google Scholar
- SDQ: Normative SDQ data from Britain. (2001). http://www.sdqinfo.com/UKNorm.html.
- Thissen, D., & Orlando, M. (2001). Item response theory for items scored in two categories. In D. Thissen & H. Wainer (Eds.), Test Scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
- CORC Measures Website http://www.corc.uk.net/index.php?contentkey=81.