Skip to main content

Advertisement

SpringerLink
Log in

Measuring Quality of Care in Community Mental Health: Validation of Concordant Clinician and Client Quality-of-Care Scales

  • Published:
The Journal of Behavioral Health Services & Research Aims and scope Submit manuscript

Abstract

Measuring quality of care can transform care, but few tools exist to measure quality from the client’s perspective. The aim of this study was to create concordant clinician and client self-report quality-of-care scales in a sample of community mental health clinicians (n = 189) and clients (n = 469). The client scale had three distinct factors (Person-Centered Care, Negative Staff Interactions, and Inattentive Care), while the clinician scale had two: Person-Centered Care and Discordant Care. Both versions demonstrated adequate internal consistency and validity with measures related to satisfaction and the therapeutic relationship. These measures are promising, brief quality assessment tools.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Demyttenaere K, Bruffaerts R, Posada-Villa J, et al. Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization world mental health surveys. JAMA. 2004;291(21):2581–2590.

    Article  PubMed  Google Scholar 

  2. Whiteford HA, Degenhardt L, Rehm J, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. The Lancet. 2013;382(9904):1575–1586.

    Article  Google Scholar 

  3. Kessler RC, Heeringa S, Lakoma MD, et al. Individual and societal effects of mental disorders on earnings in the United States: results from the national comorbidity survey replication. The American Journal of Psychiatry. 2008;165(6):703–711.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Dixon LB, Dickerson F, Bellack AS, et al. The 2009 Schizophrenia PORT psychosocial treatment recommendations and summary statements. Schizophrenia Bulletin. 2010;36(1):48–70.

    Article  PubMed  Google Scholar 

  5. Institute of Medicine: Improving the quality of health care for mental and substance-use conditions: quality chasm series. Washington DC: National Academy Press, 2006.

    Google Scholar 

  6. Lora A. Call for information, call for quality in mental health care. Epidemiology and Psychiatric Sciences. 2013;22(1):9–13.

    Article  CAS  PubMed  Google Scholar 

  7. Rebello TJ, Marques A, Gureje O, et al. Innovative strategies for closing the mental health treatment gap globally. Current Opinion in Psychiatry. 2014;27(4):308–314.

    Article  PubMed  Google Scholar 

  8. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001.

    Google Scholar 

  9. Chassin MR, Loeb JM, Schmaltz SP, et al. Accountability measures: using measurement to promote quality improvement. New England Journal of Medicine. 2010;363(7):683–688.

    Article  CAS  PubMed  Google Scholar 

  10. Pincus HA, Spaeth-Rublee B, Watkins, KE. The case for measuring quality in mental health and substance abuse care. Health Affairs. 2011;30(4):730–736.

    Article  PubMed  Google Scholar 

  11. Fisher ES, Staiger DO, Bynum JP, et al. Creating accountable care organizations: the extended hospital medical staff. Health Affairs. 2007;26(1):w44-w57.

    PubMed  Google Scholar 

  12. Kilbourne AM, Keyser D, Pincus HA. Challenges and opportunities in measuring the quality of mental health care. The Canadian Journal of Psychiatry. 2010;55(9):549–557.

    Article  PubMed  Google Scholar 

  13. Campbell SM, Roland MO, Buetow SA. Defining quality of care. Social Science & Medicine. 2000;51(11):1611–1625.

    Article  CAS  Google Scholar 

  14. Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly. 1966;44(3 suppl):166–206.

    Article  Google Scholar 

  15. Donabedian A, Wheeler JR, Wyszewianski L. Quality, cost, and health: an integrative model. Medical Care. 1982;975–992.

  16. Hermann RC, Leff HS, Palmer RH, et al. Quality Measures for mental health care: results from a national inventory. Medical Care Research and Review. 2000;57(3 suppl):136–154.

    Article  PubMed  Google Scholar 

  17. Moran V, O'Connor S, Borowitz, M. International approaches to measuring the quality of mental health care. Epidemiology and Psychiatric Sciences. 2013;22(1):3–7.

    Article  CAS  PubMed  Google Scholar 

  18. Parameswaran SG, Spaeth-Rublee B, Pincus HA. Measuring the quality of mental health care: consensus perspectives from selected industrialized countries. Administration and Policy in Mental Health and Mental Health Services Research. 2015;42(3):288–295.

    Article  PubMed  Google Scholar 

  19. Patel MM, Brown JD, Croake S, et al. The current state of behavioral health quality measures: where are the gaps? Psychiatric Services. 2015;66(8):865–871.

    Article  PubMed  Google Scholar 

  20. Lyu H, Wick EC, Housman M, et al. Patient satisfaction as a possible indicator of quality surgical care. Journal of American Medical Association Surgery. 2013;148(4):362–367.

    Google Scholar 

  21. Chang JT, Hays RD, Shekelle PG, et al. Patients’ global ratings of their health care are not associated with the technical quality of their care. Annals of Internal Medicine. 2006;144(9):665–672.

    Article  PubMed  Google Scholar 

  22. Borrelli, B. The assessment, monitoring, and enhancement of treatment fidelity in public heath trials. Journal of Public Health Dentistry. 2011;71(s1):S52–S63.

    Article  PubMed  PubMed Central  Google Scholar 

  23. McHugh MD, Stimpfel AW. Nurse reported quality of care: a measure of hospital quality. Research in Nursing & Health. 2012;35(6):566–575.

    Article  Google Scholar 

  24. You L-M, Aiken LH, Sloane DM, et al. Hospital nursing, care quality, and patient satisfaction: cross-sectional surveys of nurses and patients in hospitals in China and Europe. International Journal of Nursing Studies. 2013;50(2):154–161.

    Article  PubMed  Google Scholar 

  25. Lucero RJ, Lake ET, Aiken LH. Nursing care quality and adverse events in U.S. hospitals. Journal of Clinical Nursing. 2010;19(15–16):2185–2195.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Ball JE, Murrells T, Rafferty AM, et al. ‘Care left undone’ during nursing shifts: associations with workload and perceived quality of care. British Medical Journal Quality & Safety. 2013;23(2):116–125.

    Article  Google Scholar 

  27. Shanafelt TD, Bradley KA, Wipf JE, et al. Burnout and self-reported patient care in an internal medicine residency program. Annals of Internal Medicine. 2002;136(5):358–367.

    Article  PubMed  Google Scholar 

  28. Salyers MP, Fukui S, Rollins AL, et al. Burnout and self-reported quality of care in community mental health. Administration and Policy in Mental Health and Mental Health Services Research. 2015;42(1):61–69.

    Article  PubMed  Google Scholar 

  29. Lee Y-Y, Lin JL. Do patient autonomy preferences matter? Linking patient-centered care to patient–physician relationships and health outcomes. Social Science & Medicine. 2010;71(10):1811–1818.

    Article  Google Scholar 

  30. Howell SJ. Empathy and its role in quality of care. Osteopathic Family Physician, 2016; 8(4):20–23.

    Google Scholar 

  31. Rollins AL, Kukla M, Morse G, et al. Comparative effectiveness of a burnout reduction intervention for behavioral health providers. Psychiatric Services. 2016;67(8):920–923.

    Article  PubMed  Google Scholar 

  32. Salyers MP, Hudson C, Morse G, et al. BREATHE: a pilot study of a one-day retreat to reduce burnout among mental health professionals. Psychiatric Services. 2011;62(2):214–217.

    Article  PubMed  Google Scholar 

  33. Miller WR, Rollnick S. Motivational interviewing: preparing people for change, second edition. New York, NY: Guilford Press, 2002.

    Google Scholar 

  34. Luther L, Miller AP, Hedrick HM, et al. Client and provider perspectives on the impact of burnout on quality of care and client outcomes in community mental health. Journal of Behavioral and Social Sciences. 2016;3(2):73–85.

    Google Scholar 

  35. Horvath AO, Greenberg LS. Development and validation of the working alliance inventory. Journal of Counseling Psychology. 1989;36(2):223–233.

    Article  Google Scholar 

  36. Tracey TJ, Kokotovic AM. Factor structure of the working alliance inventory. Psychological Assessment: a Journal Of Consulting and Clinical Psychology. 1989;1(3):207–210.

    Article  Google Scholar 

  37. Hanson WE, Curry KT, Bandalos DL. Reliability generalization of working alliance inventory scale scores. Educational and Psychological Measurement. 2002;62(4):659–673.

    Article  Google Scholar 

  38. Larsen DL, Attkisson CC, Hargreaves WA, et al. Assessment of client/patient satisfaction: development of a general scale. Evaluation and Program Planning. 1979;2(3):197–207.

    Article  CAS  PubMed  Google Scholar 

  39. Williams GC, Grow VM, Freedman ZR, et al. Motivational predictors of weight loss and weight-loss maintenance. Psychological Assessment: a Journal Of Consulting and Clinical Psychology. 1996;70(1):115–126.

    CAS  Google Scholar 

  40. Williams GC, McGregor HA, King D. Variation in perceived competence, glycemic control, and patient satisfaction: relationship to autonomy support from physicians. Patient Education Counseling. 2005;57(1):39–45.

    Article  PubMed  Google Scholar 

  41. Nagy MS. Using a Single-Item Approach to Measure Facet Job Satisfaction. Journal of Occupational and Organizational Psychology. 2002;75(1):77–87.

    Article  Google Scholar 

  42. Wanous JP, Reichers AE, Hudy MJ. Overall job satisfaction: how good are single-item measures? Journal of Applied Psychology. 1997;82(2):247–252.

  43. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: a Multidisciplinary Journal. 1999;6(1):1–55.

    Article  Google Scholar 

  44. Epstein RM, Fiscella K, Lesser CS, et al. Why the nation needs a policy push on patient-centered health care. Health Affairs. 2010;29(8):1489–1495.

    Article  PubMed  Google Scholar 

  45. Clark PA. Medical practices’ sensitivity to patients’ needs: opportunities and practices for improvement. Journal of Ambulatory Care Management. 2003;26(2):110–123.

    Article  PubMed  Google Scholar 

  46. Wanzer MB, Booth-Butterfield M, Gruber K. Perceptions of health care providers’ communication: relationships between patient-centered communication and satisfaction. Health Communication. 2004;16(3):363–384.

    Article  PubMed  Google Scholar 

  47. Pincus HA, Scholle SH, Spaeth-Rublee B, et al. Quality measures for mental health and substance use: gaps, opportunities, and challenges. Health Affairs. 2016;35(6):1000–1008.

    Article  PubMed  Google Scholar 

  48. Boulding W, Glickman SW, Manary MP, et al. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. The American Journal of Managed Care. 2011;17(1):41–48.

    PubMed  Google Scholar 

  49. Glickman SW, Boulding W, Manary M, et al. Patient satisfaction and its relationship with clinical quality and inpatient mortality in acute myocardial infarction. Circulation: Cardiovascular Quality and Outcomes. 2010;3(2):188–195.

    Google Scholar 

  50. Isaac T, Zaslavsky AM, Cleary PD, et al. The relationship between patients' perception of care and measures of hospital quality and safety. Health Services Research. 2010;45(4):1024–1040.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Fenton JJ, Jerant AF, Bertakis KD, et al. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Archives of Internal Medicine. 2012;172(5):405–411.

    Article  PubMed  Google Scholar 

  52. Tsai TC, Orav EJ, Jha AK. Patient satisfaction and quality of surgical care in U.S. hospitals. Annals of Surgery. 2015;261(1):2–8.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Hojat M, Louis DZ, Maio V, et al. Editorial: empathy and health care quality. American Journal of Medical Quality. 2013;28(1):6–7.

    Article  PubMed  Google Scholar 

  54. Rathert C, Wyrwich MD, Boren SA. Patient-centered care and outcomes: a systematic review of the literature. Medical Care Research and Review. 2012;70(4):351–379.

    Article  PubMed  Google Scholar 

  55. Wolf DM, Lehman L, Quinlin R, et al. Effect of patient-centered care on patient satisfaction and quality of care. Journal of Nursing Care Quality. 2008;23(4):316–321.

    Article  PubMed  Google Scholar 

  56. Epstein RM, Street RL. The values and value of patient-centered care. The Annals of Family Medicine. 2011;9(2):100–103.

    Article  PubMed  Google Scholar 

  57. Chang WY, Ma JC, Chiu HT, et al. Job satisfaction and perceptions of quality of patient care, collaboration, and teamwork in acute care hospitals. Journal of Advanced Nursing. 2009;65(9):1946–1955.

    Article  PubMed  Google Scholar 

  58. Grembowski D, Paschane D, Diehr P, et al. Managed care, physician job satisfaction, and the quality of primary care. Journal of General Internal Medicine. 2005;20(3):271–277.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Spence Laschinger HK, Fida R. Linking nurses’ perceptions of patient care quality to job satisfaction. Journal of Nursing Administration. 2015;45(5):276–283.

    Article  PubMed  Google Scholar 

  60. Fortney L, Luchterhand C, Zakletskaia L, et al. Abbreviated mindfulness intervention for job satisfaction, quality of life, and compassion in primary care clinicians: a pilot study. The Annals of Family Medicine. 2013;11(5):412–420.

    Article  PubMed  Google Scholar 

  61. Spector, PE, Van Katwyk, PT, Brannick, MT, et al. When two factors don't reflect two constructs: how item characteristics can produce artifactual factors. Journal of Management. 1997; 23(5):659–677.

    Article  Google Scholar 

  62. Drake RE, Latimer E. Lessons learned in developing community mental health care in North America. World Psychiatry. 2012;11(1):47–51.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Koch JR, Breland AB, Nash M, et al. Assessing the utility of consumer surveys for improving the quality of behavioral health care services. Journal of Behavioral Health Services & Research. 2011;38(2):234–248.

    Article  Google Scholar 

Download references

Disclaimer

The statements in this publication are solely the responsibility of the authors and do not necessarily represent the views of the Indiana Clinical and Translational Sciences Institute, the National Institutes of Health, the Patient-Centered Outcomes Research Institute (PCORI), PCORI’s Board of Governors or Methodology Committee, or the Department of Veterans Affairs or the United States government.

Funding

Research reported in this publication was partially funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (No. IH-1304-6597) and from an Award (No. UL1TR001108) from the Indiana Clinical and Translational Sciences Institute, which is funded in part by a National Institutes of Health, National Center for Advancing Translational Sciences, Clinical and Translational Sciences Award.

Author information

Authors and Affiliations

  1. Department of Psychology, Indiana University-Purdue University Indianapolis (IUPUI), 402 North Blackford Street, LD 124, Indianapolis, Indiana, 46220, USA

    Lauren Luther MS, Jennifer M. Garabrant BSW, Nancy Henry BA, Dawn Shimp & Michelle P. Salyers PhD

  2. ACT Center of Indiana, 402 North Blackford Street, LD 124, Indianapolis, IN, 46220, USA

    Lauren Luther MS, Jennifer M. Garabrant BSW, Angela L. Rollins PhD, Nancy Henry BA, Dawn Shimp & Michelle P. Salyers PhD

  3. Center for Mental Health Research and Innovation, University of Kansas School of Social Welfare, 1545 Lilac Lane, Twente Hall, Lawrence, KS, 66045, USA

    Sadaaki Fukui PhD

  4. Center for Health Information and Communication, Richard L. Roudebush Veterans Affairs Medical Center, 1481 W. 10th Street, 11-H Indianapolis, Indianapolis, Indiana, 46202, USA

    Angela L. Rollins PhD

  5. Places for People, Inc., 4130 Lindell Boulevard, St. Louis, MO, 63108, USA

    Gary Morse PhD

  6. Department of Psychological Sciences, University of Missouri-St. Louis, One University Boulevard, 325 Stadler Hall, St. Louis, MO, 63121, USA

    Gary Morse PhD

  7. Four County Counseling Center, 1015 Michigan Avenue, Logansport, IN, 46947, USA

    Timothy Gearhart MSW, LCSW

Authors
  1. Lauren Luther MS
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sadaaki Fukui PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jennifer M. Garabrant BSW
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Angela L. Rollins PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gary Morse PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nancy Henry BA
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dawn Shimp
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Timothy Gearhart MSW, LCSW
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Michelle P. Salyers PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lauren Luther MS.

Ethics declarations

Conflict of Interest Statement

The authors declare that they have no conflict of interest.

Appendix

Appendix

Table 4 Factor loadings, means, and standard deviations for clinician QOC items (n = 189)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luther, L., Fukui, S., Garabrant, J.M. et al. Measuring Quality of Care in Community Mental Health: Validation of Concordant Clinician and Client Quality-of-Care Scales. J Behav Health Serv Res 46, 64–79 (2019). https://doi.org/10.1007/s11414-018-9601-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11414-018-9601-3

Keywords

Search

Navigation