The Patient - Patient-Centered Outcomes Research

, Volume 5, Issue 3, pp 163–173 | Cite as

Psychometric Evaluation of an Inpatient Consumer Survey Measuring Satisfaction with Psychiatric Care

  • Glorimar OrtizEmail author
  • Lucille Schacht
Original Research Article



Measurement of consumers’ satisfaction in psychiatric settings is important because it has been correlated with improved clinical outcomes and administrative measures of high-quality care. These consumer satisfaction measurements are actively used as performance measures required by the accreditation process and for quality improvement activities.


Our objectives were (i) to re-evaluate, through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), the structure of an instrument intended to measure consumers’ satisfaction with care in psychiatric settings and (ii) to examine and publish the psychometric characteristics, validity and reliability, of the Inpatient Consumer Survey (ICS).


To psychometrically test the structure of the ICS, 34878 survey results, submitted by 90 psychiatric hospitals in 2008, were extracted from the Behavioral Healthcare Performance Measurement System (BHPMS). Basic descriptive item-response and correlation analyses were performed for total surveys. Two datasets were randomly created for analysis. A random sample of 8229 survey results was used for EFA. Another random sample of 8261 consumer survey results was used for CFA. This same sample was used to perform validity and reliability analyses.


The item-response analysis showed that the mean range for a disagree/agree five-point scale was 3.10–3.94. Correlation analysis showed a strong relationship between items. Six domains (dignity, rights, environment, empowerment, participation, and outcome) with internal reliabilities between good to moderate (0.87–0.73) were shown to be related to overall care satisfaction. Overall reliability for the instrument was excellent (0.94). Results from CFA provided support for the domains structure of the ICS proposed through EFA.


The overall findings from this study provide evidence that the ICS is a reliable measure of consumer satisfaction in psychiatric inpatient settings. The analysis has shown the ICS to provide valid and reliable results and to focus on the specific concerns of consumers of psychiatric inpatient care. Scores by item indicate that opportunity for improvement exists across healthcare organizations.


Confirmatory Factor Analysis Exploratory Factor Analysis Psychiatric Hospital Healthcare Organization Psychiatric Inpatient 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The study design, collection, analysis, and interpretation of the data, and the writing, review, and approval of the manuscript were fully funded by the BHPMS. The BHPMS is funded by state and private psychiatric hospitals.

Glorimar Ortiz was responsible for the study protocol and design and the statistical analyses, and drafted the manuscript. Glorimar Ortiz also acts as the guarantor for the overall content of this paper. Lucille Schacht contributed to the Discussion and Conclusion sections, and revision of the final manuscript.

The authors declare no potential conflicts of interest.


  1. 1.
    Barker DA, Orrell MW. The Psychiatric Care Satisfaction Questionnaire: a reliability and validity study. Soc Psychiatry Psychiatr Epidemiol 1999; 34: 111–6.PubMedCrossRefGoogle Scholar
  2. 2.
    Davies AR, Ware JE. Involving consumers in quality of care assessment. Health Aff 1988; 7: 33–48.CrossRefGoogle Scholar
  3. 3.
    Edgman-Levitan S, Clearly PD. What information do consumers want and need? Health Aff 1996; 15(4): 42–56.CrossRefGoogle Scholar
  4. 4.
    Pellegrin KL, Stuart GW, Maree B, et al. A brief scale for assessing patients’ satisfaction with care in outpatient psychiatric services. Psychiatr Serv 2001 Jun; 52(6): 816–9.PubMedCrossRefGoogle Scholar
  5. 5.
    Woodring S, Polomano RC, Haagen BF, et al. Development and testing of a patient satisfaction measure for inpatient psychiatric care. J Nurs Care Qual 2003 Dec 8; 19(2): 137–48.CrossRefGoogle Scholar
  6. 6.
    Eisen S, Shaul J A, Clarridge B, et al. Development of a consumer survey for behavioral health services. Psychiatr Serv 1999 June; 50(6): 793–8.PubMedGoogle Scholar
  7. 7.
    The Joint Commission. About The Joint Comission [online]. Available from URL: [Accessed 2011 Nov 2].
  8. 8.
    The Joint Commission on Accreditation of Healthcare Organizations. ORYX® outcomes: the next evolution in accreditation. Oakbrook Terrace (IL): The Joint Commission, 1997.Google Scholar
  9. 9.
    Friedman MM. ORYX®: the next evolution in accreditation. Home Healthc Nurse 1998 Apr; 16(4): 236–9.PubMedCrossRefGoogle Scholar
  10. 10.
    Schacht L. NRI/MHSIP Inpatient Consumer Survey: results of pilot implementation. Alexandria (VA): National Association of State Mental Health Program Directors Research Institute, Inc., 2001.Google Scholar
  11. 11.
    Kolb SJ, Race KEH, Seibert JH. Psychometric evaluation of an inpatient psychiatric care consumer satisfaction survey. J Behav Health Serv Res 2000 Feb; 27(1): 75–86.PubMedCrossRefGoogle Scholar
  12. 12.
    SPSS Inc. SPSS® Statistics [computer program]. Version 17. Chicago (IL): SPSS Inc., 2008.Google Scholar
  13. 13.
    IBM. Amos [computer program]. Version 18. Chicago (IL): IBM, 2009.Google Scholar
  14. 14.
    National Quality Forum. NQF endorses mental health outcome measures [media release]. 2011 Jan 26 [online]. Available from URL: [Accessed 2011 Jan 26].
  15. 15.
    Munro BH. Statistical methods for health care research. 5th rev. ed. Philadelphia (PA): Lippincott Williams & Wilkins, 2005.Google Scholar
  16. 16.
    Fabrigar LR, Wegener DT, MacCallum RC, et al. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods 1999; 4(3): 272–99.CrossRefGoogle Scholar
  17. 17.
    Hayton JC, Allen DG, Scarpello V. Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organ Res Methods 2004 Apr; 7(2): 191–205.CrossRefGoogle Scholar
  18. 18.
    Kieffer KM. Orthogonal versus oblique factor rotation: a review of the literature regarding the pros and cons. Annual meeting of the Mid-South Educational Research Association; 1998 Nov 4; New Orleans (LA).Google Scholar
  19. 19.
    Hurley AE, Scandura TA, Schriesheim CA, et al. Exploratory and confirmatory factor analysis: guidelines, issues, and alternatives. J Organiz Behav 1997 Feb 18; 18: 667–83.CrossRefGoogle Scholar
  20. 20.
    Blunch NJ. Introduction to structural equation modelling using SPSS and Amos. London: SAGE Publications Ltd, 2008.Google Scholar
  21. 21.
    Hu L, Bentler PM. Cutoff criteria for fit indexes in co-variance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling 1999; 6(1): 1–55.CrossRefGoogle Scholar
  22. 22.
    Nunnally JC. Psychometric theory. 2nd rev. ed. New York: McGraw-Hill, 1978.Google Scholar
  23. 23.
    Ang RP, Huan VS. Academic expectations stress inventory: development, factor analysis, reliability, and validity. Educ Psychol Meas 2006 Jun; 66(3): 522–39.CrossRefGoogle Scholar
  24. 24.
    Boyer L, Baumstarck-Barrau K, Cano N, et al. Assessment of psychiatric inpatient satisfaction: a systematic review of self-reported instruments. Eur Psychiat 2009; 24: 540–9.CrossRefGoogle Scholar
  25. 25.
    Meehan T, Bergen H, Stedman T. Monitoring consumer satisfaction with inpatient service delivery: the Inpatient Evaluation of Service Questionnaire. Aust NZ J Psychiatry 2002; 36: 807–11.CrossRefGoogle Scholar
  26. 26.
    Eisen SV, Wilcox M, Idiculla T, et al. Assessing consumer perception of inpatient psychiatric treatment. Jt Comm J Qual Improv 2002 Sep; 28(9): 510–26.PubMedGoogle Scholar
  27. 27.
    Holcomb WR, Adams NA, Ponder HM, et al. The development and construct validation of a consumer satisfaction questionnaire for psychiatric inpatients. Eval Program Plann 1989; 12: 189–94.PubMedCrossRefGoogle Scholar
  28. 28.
    Lebow J. Consumer satisfaction with mental health treatment. Psychol Bull 1982; 91(2): 244–59.PubMedCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2012

Authors and Affiliations

  1. 1.National Association of State Mental Health Program Directors Research Institute (NRI), Inc.Falls ChurchUSA

Personalised recommendations