Social Psychiatry and Psychiatric Epidemiology

, Volume 46, Issue 3, pp 219–229

Developing and comparing methods for measuring the content of care in mental health services

  • B. Lloyd-Evans
  • M. Slade
  • D. P. Osborn
  • R. Skinner
  • S. Johnson
Original Paper



Measurement of what happens in mental health services is needed to describe services, identify variation in care provision and understand service outcomes. However, there is no consensus about appropriate methods or measures for content of care. Previous research has primarily used a single information source and prioritised staff over patient perspectives on content of care. This study aims to enhance understanding of how to measure content of care by developing and evaluating four instruments, each using a different measurement method.


Development is described by four instruments—CaSPAR, CaRICE, CCCQ-S and CCCQ-P—which use researcher observation, staff-report and patient-report to measure the intensity and nature of care at services. Inter-rater reliability of CaRICE and CCCQ-S was investigated. Concordance between staff and patient perspectives was explored through assessing inter-rater agreement of CCCQ-S and CCCQ-P questionnaires. The convergence of data from the measures collected in an inpatient multi-site study was investigated.


CaRICE demonstrated good inter-rater reliability (κ = 0.71). CCCQ-S inter-rater reliability was poor. Concordance between staff and patient reports was low: there was a trend for patients to report less care received than staff reported had been provided. Results from CaSPAR, CaRICE and CCCQ-P exhibited divergence, indicating possible differences in patient, staff and researcher perspectives.


Information about content of care should be sought as close as possible to the point of delivery. There may be differing, valid perspectives about care provided by services. Further development of methods to measure content of care is required. Meanwhile, a multi-methods approach should be adopted, which allows inclusion of different perspectives (specifically including the patient’s perspective) and triangulation of results from different measurement methods. CaSPAR, CaRICE and CCCQ-P can provide multi-perspective content of care measurement in inpatient services.


Process measurement Content of care Inpatient Mental health services 

Supplementary material

127_2010_192_MOESM1_ESM.doc (36 kb)
Supplementary material 1 (DOC 36 kb)


  1. 1.
    Altman D (1991) Practical statistics for medical research. Chapman and Hall, LondonGoogle Scholar
  2. 2.
    Altschul A (1972) Patient–nurse interaction: a study of interaction patterns in acute psychiatric wards. Churchill Livingstone, EdinburghGoogle Scholar
  3. 3.
    Bowie P, Mountain G (1993) Using direct observation to record the behaviour of long-stay patients with dementia. Int J Geriatr Psychiatry 8:857–864CrossRefGoogle Scholar
  4. 4.
    Brekke J (1987) The model-guided method for monitoring program implementation. Eval Rev 11(3):281–299Google Scholar
  5. 5.
    Brugha TS, Lindsay F (1996) Quality of mental health service care: the forgotten pathway from process to outcome. Soc Psychiatry Psychiatr Epidemiol 31(2):89–98CrossRefPubMedGoogle Scholar
  6. 6.
    Burns T, Priebe S (1996) Mental health care systems and their characteristics: a proposal. Acta Psychiatr Scand 94(6):381–385CrossRefPubMedGoogle Scholar
  7. 7.
    Cormack D (1976) Psychiatric nursing observed. Royal College of Nursing, LondonGoogle Scholar
  8. 8.
    Craig P, Dieppe P, Macintyre S, Mitchie S, Nazareth I, Petticrew M (2008) Developing and evaluating complex interventions: the new Medical Research Council guidance. Br Med J 337:979–983CrossRefGoogle Scholar
  9. 9.
    Dean R, Proudfoot R (1993) The quality of interactions schedule (QUIS): development, reliability and use in the evaluation of two domus units. Int J Geriatr Psychiatry 8:819–826CrossRefGoogle Scholar
  10. 10.
    Department of Health (2005) Mental Health Policy Implementation Guide: Adult Acute Inpatient Care Provision. Department of HealthGoogle Scholar
  11. 11.
    Donabedian A (1980) Explorations in quality assessment and monitoring. The definition of quality and approaches to its assessment, vol 1. Health Administration PressGoogle Scholar
  12. 12.
    Fiander M, Burns T (2000) A Delphi approach to describing service models of community mental health practice. Psychiatr Serv 51(5):656–658CrossRefPubMedGoogle Scholar
  13. 13.
    Glick I, Burti L, Suzuki K, Sacks M (1991) Effectiveness in psychiatric care: 1. A cross-national study of the process of treatment and outcomes of major depressive disorder. J Nerv Mental Dis 179(2):55–63CrossRefGoogle Scholar
  14. 14.
    Higgins R, Hurst K, Wistow G (1999) Nursing acute psychiatric patients: a quantitative and qualitative study. J Adv Nurs 29(1):52–63CrossRefPubMedGoogle Scholar
  15. 15.
    Johnson S, Salvador-Carulla L (1998) Description and classification of mental health services: a European perspective. Eur Psychiatry 13:333–341CrossRefPubMedGoogle Scholar
  16. 16.
    Johnson S, Billings J, Bingham C, Pilling S, Bebbington P, Morant N, Dalton J, McNicholas S (2001) An evaluation of Drayton Park Women’s Crisis Service, a community-based alternative to the psychiatric ward. Unpublished reportGoogle Scholar
  17. 17.
    Johnson S, Gilburt H, Lloyd-Evans B, Slade M (2007) Acute inpatient psychiatry: residential alternatives to hospital admission. Psychiatr Bull 31:262–264CrossRefGoogle Scholar
  18. 18.
    Johnson S, Gilburt H, Lloyd-Evans B, Boardman H, Leese M, Osborn D, Shepherd G, Thornicroft G, Slade M (2009) Alternatives to hospital admission: emerging models and populations served. Br J Psychiatry 194(5):456–463CrossRefPubMedGoogle Scholar
  19. 19.
    Lloyd-Evans B, Slade M, Johnson S (2007) Assessing the content of mental health services: a review of measures. Soc Psychiatry Psychiatr Epidemiol 42(8):673–682CrossRefPubMedGoogle Scholar
  20. 20.
    McCabe R, Roeder-Wanner U, Hoffman K, Priebe S (1999) Therapeutic relationships and quality of life: associations of two subjective constructs in schizophrenia patients. Int J Soc Psychiatry 45:276–283CrossRefPubMedGoogle Scholar
  21. 21.
    Mechanic D (1996) Emerging issues in international mental health services research. Psychiatr Serv 47(4):371–375PubMedGoogle Scholar
  22. 22.
    Moos R (1996) Ward atmosphere scale manual, 3rd edn. Mind GardenGoogle Scholar
  23. 23.
    Paul G (1987) Rational operations in residential treatment settings through ongoing assessment of staff and client functioning. In: Peterson D, Fishman D (eds) Assessment for Decision. Rutgers University Press, pp 145–203Google Scholar
  24. 24.
    Quirk A, Lelliot P (2001) What do we know about life on acute psychiatric wards in the UK? A review of the research evidence. Soc Sci Med 53(12):1565–1574CrossRefPubMedGoogle Scholar
  25. 25.
    Sandford D, Elzinger R, Iverson R (1990) A quantitative study of nursing staff interactions in psychiatric wards. Acta Psychiatr Scand 81:46–51CrossRefPubMedGoogle Scholar
  26. 26.
    Sanson-Fisher R, Poole A, Thompson V (1979) Behavioural patterns within a general hospital psychiatric unit: an observational study. Behav Res Ther 17:317–332CrossRefPubMedGoogle Scholar
  27. 27.
    Shepherd G, Richardson R (1979) Organization and interaction in psychiatric day centres. Psychol Med 9:573–579CrossRefPubMedGoogle Scholar
  28. 28.
    Slade M, Phelan M, Thornicroft G (1996) The Camberwell Assessment of Need (CAN): comparison of assessments by staff and patients of the needs of the mentally ill. Soc Psychiatry Psychiatr Epidemiol 31:109–113CrossRefPubMedGoogle Scholar
  29. 29.
    Tugwell P (1979) A methodological perspective on process measures of the quality of medical care. Clini Investig Med 2:113–121Google Scholar
  30. 30.
    Tyson G, Lambert G, Beattie L (1995) The quality of psychiatric nurses’ interactions with patients: an observational study. Int J Nurs Stud 32(1):49–58CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • B. Lloyd-Evans
    • 1
    • 2
  • M. Slade
    • 3
  • D. P. Osborn
    • 4
  • R. Skinner
    • 3
  • S. Johnson
    • 1
    • 2
  1. 1.Department of Mental Health SciencesUniversity College London (Bloomsbury Campus)LondonUK
  2. 2.Camden and Islington NHS Foundation TrustEarly Intervention ServiceLondonUK
  3. 3.Health Service and Population Research Department, Institute of PsychiatryKings College LondonLondonUK
  4. 4.Department of Mental Health Sciences, Royal Free HospitalUniversity College London (Hampstead Campus)LondonUK

Personalised recommendations