Skip to main content
SpringerLink
Log in

Psychometric Evaluation of an Inpatient Consumer Survey Measuring Satisfaction with Psychiatric Care

  • Original Research Article
  • Published:
The Patient - Patient-Centered Outcomes Research Aims and scope Submit manuscript

Abstract

Background

Measurement of consumers’ satisfaction in psychiatric settings is important because it has been correlated with improved clinical outcomes and administrative measures of high-quality care. These consumer satisfaction measurements are actively used as performance measures required by the accreditation process and for quality improvement activities.

Objectives

Our objectives were (i) to re-evaluate, through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), the structure of an instrument intended to measure consumers’ satisfaction with care in psychiatric settings and (ii) to examine and publish the psychometric characteristics, validity and reliability, of the Inpatient Consumer Survey (ICS).

Methods

To psychometrically test the structure of the ICS, 34878 survey results, submitted by 90 psychiatric hospitals in 2008, were extracted from the Behavioral Healthcare Performance Measurement System (BHPMS). Basic descriptive item-response and correlation analyses were performed for total surveys. Two datasets were randomly created for analysis. A random sample of 8229 survey results was used for EFA. Another random sample of 8261 consumer survey results was used for CFA. This same sample was used to perform validity and reliability analyses.

Results

The item-response analysis showed that the mean range for a disagree/agree five-point scale was 3.10–3.94. Correlation analysis showed a strong relationship between items. Six domains (dignity, rights, environment, empowerment, participation, and outcome) with internal reliabilities between good to moderate (0.87–0.73) were shown to be related to overall care satisfaction. Overall reliability for the instrument was excellent (0.94). Results from CFA provided support for the domains structure of the ICS proposed through EFA.

Conclusion

The overall findings from this study provide evidence that the ICS is a reliable measure of consumer satisfaction in psychiatric inpatient settings. The analysis has shown the ICS to provide valid and reliable results and to focus on the specific concerns of consumers of psychiatric inpatient care. Scores by item indicate that opportunity for improvement exists across healthcare organizations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Table I
Table II
Fig. 1
Table III
Table IV
Table V

Similar content being viewed by others

References

  1. Barker DA, Orrell MW. The Psychiatric Care Satisfaction Questionnaire: a reliability and validity study. Soc Psychiatry Psychiatr Epidemiol 1999; 34: 111–6.

    Article  PubMed  CAS  Google Scholar 

  2. Davies AR, Ware JE. Involving consumers in quality of care assessment. Health Aff 1988; 7: 33–48.

    Article  CAS  Google Scholar 

  3. Edgman-Levitan S, Clearly PD. What information do consumers want and need? Health Aff 1996; 15(4): 42–56.

    Article  CAS  Google Scholar 

  4. Pellegrin KL, Stuart GW, Maree B, et al. A brief scale for assessing patients’ satisfaction with care in outpatient psychiatric services. Psychiatr Serv 2001 Jun; 52(6): 816–9.

    Article  PubMed  CAS  Google Scholar 

  5. Woodring S, Polomano RC, Haagen BF, et al. Development and testing of a patient satisfaction measure for inpatient psychiatric care. J Nurs Care Qual 2003 Dec 8; 19(2): 137–48.

    Article  Google Scholar 

  6. Eisen S, Shaul J A, Clarridge B, et al. Development of a consumer survey for behavioral health services. Psychiatr Serv 1999 June; 50(6): 793–8.

    PubMed  CAS  Google Scholar 

  7. The Joint Commission. About The Joint Comission [online]. Available from URL: http://www.jointcommission.org/about_us/about_the_joint_commission_main.aspx [Accessed 2011 Nov 2].

  8. The Joint Commission on Accreditation of Healthcare Organizations. ORYX® outcomes: the next evolution in accreditation. Oakbrook Terrace (IL): The Joint Commission, 1997.

  9. Friedman MM. ORYX®: the next evolution in accreditation. Home Healthc Nurse 1998 Apr; 16(4): 236–9.

    Article  PubMed  CAS  Google Scholar 

  10. Schacht L. NRI/MHSIP Inpatient Consumer Survey: results of pilot implementation. Alexandria (VA): National Association of State Mental Health Program Directors Research Institute, Inc., 2001.

  11. Kolb SJ, Race KEH, Seibert JH. Psychometric evaluation of an inpatient psychiatric care consumer satisfaction survey. J Behav Health Serv Res 2000 Feb; 27(1): 75–86.

    Article  PubMed  CAS  Google Scholar 

  12. SPSS Inc. SPSS® Statistics [computer program]. Version 17. Chicago (IL): SPSS Inc., 2008.

    Google Scholar 

  13. IBM. Amos [computer program]. Version 18. Chicago (IL): IBM, 2009.

    Google Scholar 

  14. National Quality Forum. NQF endorses mental health outcome measures [media release]. 2011 Jan 26 [online]. Available from URL: http://www.qualityforum.org/News_And_Resources/Press_Releases/2011/NQF_Endorses_Mental_Health_Outcome_Measures.aspx [Accessed 2011 Jan 26].

  15. Munro BH. Statistical methods for health care research. 5th rev. ed. Philadelphia (PA): Lippincott Williams & Wilkins, 2005.

    Google Scholar 

  16. Fabrigar LR, Wegener DT, MacCallum RC, et al. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods 1999; 4(3): 272–99.

    Article  Google Scholar 

  17. Hayton JC, Allen DG, Scarpello V. Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organ Res Methods 2004 Apr; 7(2): 191–205.

    Article  Google Scholar 

  18. Kieffer KM. Orthogonal versus oblique factor rotation: a review of the literature regarding the pros and cons. Annual meeting of the Mid-South Educational Research Association; 1998 Nov 4; New Orleans (LA).

  19. Hurley AE, Scandura TA, Schriesheim CA, et al. Exploratory and confirmatory factor analysis: guidelines, issues, and alternatives. J Organiz Behav 1997 Feb 18; 18: 667–83.

    Article  Google Scholar 

  20. Blunch NJ. Introduction to structural equation modelling using SPSS and Amos. London: SAGE Publications Ltd, 2008.

    Google Scholar 

  21. Hu L, Bentler PM. Cutoff criteria for fit indexes in co-variance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling 1999; 6(1): 1–55.

    Article  Google Scholar 

  22. Nunnally JC. Psychometric theory. 2nd rev. ed. New York: McGraw-Hill, 1978.

    Google Scholar 

  23. Ang RP, Huan VS. Academic expectations stress inventory: development, factor analysis, reliability, and validity. Educ Psychol Meas 2006 Jun; 66(3): 522–39.

    Article  Google Scholar 

  24. Boyer L, Baumstarck-Barrau K, Cano N, et al. Assessment of psychiatric inpatient satisfaction: a systematic review of self-reported instruments. Eur Psychiat 2009; 24: 540–9.

    Article  CAS  Google Scholar 

  25. Meehan T, Bergen H, Stedman T. Monitoring consumer satisfaction with inpatient service delivery: the Inpatient Evaluation of Service Questionnaire. Aust NZ J Psychiatry 2002; 36: 807–11.

    Article  Google Scholar 

  26. Eisen SV, Wilcox M, Idiculla T, et al. Assessing consumer perception of inpatient psychiatric treatment. Jt Comm J Qual Improv 2002 Sep; 28(9): 510–26.

    PubMed  Google Scholar 

  27. Holcomb WR, Adams NA, Ponder HM, et al. The development and construct validation of a consumer satisfaction questionnaire for psychiatric inpatients. Eval Program Plann 1989; 12: 189–94.

    Article  PubMed  CAS  Google Scholar 

  28. Lebow J. Consumer satisfaction with mental health treatment. Psychol Bull 1982; 91(2): 244–59.

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

The study design, collection, analysis, and interpretation of the data, and the writing, review, and approval of the manuscript were fully funded by the BHPMS. The BHPMS is funded by state and private psychiatric hospitals.

Glorimar Ortiz was responsible for the study protocol and design and the statistical analyses, and drafted the manuscript. Glorimar Ortiz also acts as the guarantor for the overall content of this paper. Lucille Schacht contributed to the Discussion and Conclusion sections, and revision of the final manuscript.

The authors declare no potential conflicts of interest.

Author information

Authors and Affiliations

  1. National Association of State Mental Health Program Directors Research Institute (NRI), Inc., 3141 Fairview Park Dr., Suite 650, Falls Church, Virginia, VA 22042, USA

    Glorimar Ortiz & Lucille Schacht

Authors
  1. Glorimar Ortiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lucille Schacht
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Glorimar Ortiz.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ortiz, G., Schacht, L. Psychometric Evaluation of an Inpatient Consumer Survey Measuring Satisfaction with Psychiatric Care. Patient 5, 163–173 (2012). https://doi.org/10.1007/BF03262489

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03262489

Keywords

Search

Navigation