Abstract
Background
Measurement of what happens in mental health services is needed to describe services, identify variation in care provision and understand service outcomes. However, there is no consensus about appropriate methods or measures for content of care. Previous research has primarily used a single information source and prioritised staff over patient perspectives on content of care. This study aims to enhance understanding of how to measure content of care by developing and evaluating four instruments, each using a different measurement method.
Methods
Development is described by four instruments—CaSPAR, CaRICE, CCCQ-S and CCCQ-P—which use researcher observation, staff-report and patient-report to measure the intensity and nature of care at services. Inter-rater reliability of CaRICE and CCCQ-S was investigated. Concordance between staff and patient perspectives was explored through assessing inter-rater agreement of CCCQ-S and CCCQ-P questionnaires. The convergence of data from the measures collected in an inpatient multi-site study was investigated.
Results
CaRICE demonstrated good inter-rater reliability (κ = 0.71). CCCQ-S inter-rater reliability was poor. Concordance between staff and patient reports was low: there was a trend for patients to report less care received than staff reported had been provided. Results from CaSPAR, CaRICE and CCCQ-P exhibited divergence, indicating possible differences in patient, staff and researcher perspectives.
Discussion
Information about content of care should be sought as close as possible to the point of delivery. There may be differing, valid perspectives about care provided by services. Further development of methods to measure content of care is required. Meanwhile, a multi-methods approach should be adopted, which allows inclusion of different perspectives (specifically including the patient’s perspective) and triangulation of results from different measurement methods. CaSPAR, CaRICE and CCCQ-P can provide multi-perspective content of care measurement in inpatient services.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altman D (1991) Practical statistics for medical research. Chapman and Hall, London
Altschul A (1972) Patient–nurse interaction: a study of interaction patterns in acute psychiatric wards. Churchill Livingstone, Edinburgh
Bowie P, Mountain G (1993) Using direct observation to record the behaviour of long-stay patients with dementia. Int J Geriatr Psychiatry 8:857–864
Brekke J (1987) The model-guided method for monitoring program implementation. Eval Rev 11(3):281–299
Brugha TS, Lindsay F (1996) Quality of mental health service care: the forgotten pathway from process to outcome. Soc Psychiatry Psychiatr Epidemiol 31(2):89–98
Burns T, Priebe S (1996) Mental health care systems and their characteristics: a proposal. Acta Psychiatr Scand 94(6):381–385
Cormack D (1976) Psychiatric nursing observed. Royal College of Nursing, London
Craig P, Dieppe P, Macintyre S, Mitchie S, Nazareth I, Petticrew M (2008) Developing and evaluating complex interventions: the new Medical Research Council guidance. Br Med J 337:979–983
Dean R, Proudfoot R (1993) The quality of interactions schedule (QUIS): development, reliability and use in the evaluation of two domus units. Int J Geriatr Psychiatry 8:819–826
Department of Health (2005) Mental Health Policy Implementation Guide: Adult Acute Inpatient Care Provision. Department of Health
Donabedian A (1980) Explorations in quality assessment and monitoring. The definition of quality and approaches to its assessment, vol 1. Health Administration Press
Fiander M, Burns T (2000) A Delphi approach to describing service models of community mental health practice. Psychiatr Serv 51(5):656–658
Glick I, Burti L, Suzuki K, Sacks M (1991) Effectiveness in psychiatric care: 1. A cross-national study of the process of treatment and outcomes of major depressive disorder. J Nerv Mental Dis 179(2):55–63
Higgins R, Hurst K, Wistow G (1999) Nursing acute psychiatric patients: a quantitative and qualitative study. J Adv Nurs 29(1):52–63
Johnson S, Salvador-Carulla L (1998) Description and classification of mental health services: a European perspective. Eur Psychiatry 13:333–341
Johnson S, Billings J, Bingham C, Pilling S, Bebbington P, Morant N, Dalton J, McNicholas S (2001) An evaluation of Drayton Park Women’s Crisis Service, a community-based alternative to the psychiatric ward. Unpublished report
Johnson S, Gilburt H, Lloyd-Evans B, Slade M (2007) Acute inpatient psychiatry: residential alternatives to hospital admission. Psychiatr Bull 31:262–264
Johnson S, Gilburt H, Lloyd-Evans B, Boardman H, Leese M, Osborn D, Shepherd G, Thornicroft G, Slade M (2009) Alternatives to hospital admission: emerging models and populations served. Br J Psychiatry 194(5):456–463
Lloyd-Evans B, Slade M, Johnson S (2007) Assessing the content of mental health services: a review of measures. Soc Psychiatry Psychiatr Epidemiol 42(8):673–682
McCabe R, Roeder-Wanner U, Hoffman K, Priebe S (1999) Therapeutic relationships and quality of life: associations of two subjective constructs in schizophrenia patients. Int J Soc Psychiatry 45:276–283
Mechanic D (1996) Emerging issues in international mental health services research. Psychiatr Serv 47(4):371–375
Moos R (1996) Ward atmosphere scale manual, 3rd edn. Mind Garden
Paul G (1987) Rational operations in residential treatment settings through ongoing assessment of staff and client functioning. In: Peterson D, Fishman D (eds) Assessment for Decision. Rutgers University Press, pp 145–203
Quirk A, Lelliot P (2001) What do we know about life on acute psychiatric wards in the UK? A review of the research evidence. Soc Sci Med 53(12):1565–1574
Sandford D, Elzinger R, Iverson R (1990) A quantitative study of nursing staff interactions in psychiatric wards. Acta Psychiatr Scand 81:46–51
Sanson-Fisher R, Poole A, Thompson V (1979) Behavioural patterns within a general hospital psychiatric unit: an observational study. Behav Res Ther 17:317–332
Shepherd G, Richardson R (1979) Organization and interaction in psychiatric day centres. Psychol Med 9:573–579
Slade M, Phelan M, Thornicroft G (1996) The Camberwell Assessment of Need (CAN): comparison of assessments by staff and patients of the needs of the mentally ill. Soc Psychiatry Psychiatr Epidemiol 31:109–113
Tugwell P (1979) A methodological perspective on process measures of the quality of medical care. Clini Investig Med 2:113–121
Tyson G, Lambert G, Beattie L (1995) The quality of psychiatric nurses’ interactions with patients: an observational study. Int J Nurs Stud 32(1):49–58
Acknowledgments
We are very grateful to the people using, working in and managing the participating services who gave their time generously, and for advice and input from the steering group for The Alternatives Study. This report presents independent research commissioned by the National Institute for Health Research Service Delivery and Organisation R&D Programme. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. The authors have no competing interests to declare.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Lloyd-Evans, B., Slade, M., Osborn, D.P. et al. Developing and comparing methods for measuring the content of care in mental health services. Soc Psychiatry Psychiatr Epidemiol 46, 219–229 (2011). https://doi.org/10.1007/s00127-010-0192-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00127-010-0192-4