Psychometric Evaluation of an Inpatient Consumer Survey Measuring Satisfaction with Psychiatric Care
Measurement of consumers’ satisfaction in psychiatric settings is important because it has been correlated with improved clinical outcomes and administrative measures of high-quality care. These consumer satisfaction measurements are actively used as performance measures required by the accreditation process and for quality improvement activities.
Our objectives were (i) to re-evaluate, through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), the structure of an instrument intended to measure consumers’ satisfaction with care in psychiatric settings and (ii) to examine and publish the psychometric characteristics, validity and reliability, of the Inpatient Consumer Survey (ICS).
To psychometrically test the structure of the ICS, 34878 survey results, submitted by 90 psychiatric hospitals in 2008, were extracted from the Behavioral Healthcare Performance Measurement System (BHPMS). Basic descriptive item-response and correlation analyses were performed for total surveys. Two datasets were randomly created for analysis. A random sample of 8229 survey results was used for EFA. Another random sample of 8261 consumer survey results was used for CFA. This same sample was used to perform validity and reliability analyses.
The item-response analysis showed that the mean range for a disagree/agree five-point scale was 3.10–3.94. Correlation analysis showed a strong relationship between items. Six domains (dignity, rights, environment, empowerment, participation, and outcome) with internal reliabilities between good to moderate (0.87–0.73) were shown to be related to overall care satisfaction. Overall reliability for the instrument was excellent (0.94). Results from CFA provided support for the domains structure of the ICS proposed through EFA.
The overall findings from this study provide evidence that the ICS is a reliable measure of consumer satisfaction in psychiatric inpatient settings. The analysis has shown the ICS to provide valid and reliable results and to focus on the specific concerns of consumers of psychiatric inpatient care. Scores by item indicate that opportunity for improvement exists across healthcare organizations.
- 7.The Joint Commission. About The Joint Comission [online]. Available from URL: http://www.jointcommission.org/about_us/about_the_joint_commission_main.aspx [Accessed 2011 Nov 2].
- 8.The Joint Commission on Accreditation of Healthcare Organizations. ORYX® outcomes: the next evolution in accreditation. Oakbrook Terrace (IL): The Joint Commission, 1997.Google Scholar
- 10.Schacht L. NRI/MHSIP Inpatient Consumer Survey: results of pilot implementation. Alexandria (VA): National Association of State Mental Health Program Directors Research Institute, Inc., 2001.Google Scholar
- 12.SPSS Inc. SPSS® Statistics [computer program]. Version 17. Chicago (IL): SPSS Inc., 2008.Google Scholar
- 13.IBM. Amos [computer program]. Version 18. Chicago (IL): IBM, 2009.Google Scholar
- 14.National Quality Forum. NQF endorses mental health outcome measures [media release]. 2011 Jan 26 [online]. Available from URL: http://www.qualityforum.org/News_And_Resources/Press_Releases/2011/NQF_Endorses_Mental_Health_Outcome_Measures.aspx [Accessed 2011 Jan 26].
- 15.Munro BH. Statistical methods for health care research. 5th rev. ed. Philadelphia (PA): Lippincott Williams & Wilkins, 2005.Google Scholar
- 18.Kieffer KM. Orthogonal versus oblique factor rotation: a review of the literature regarding the pros and cons. Annual meeting of the Mid-South Educational Research Association; 1998 Nov 4; New Orleans (LA).Google Scholar
- 20.Blunch NJ. Introduction to structural equation modelling using SPSS and Amos. London: SAGE Publications Ltd, 2008.Google Scholar
- 22.Nunnally JC. Psychometric theory. 2nd rev. ed. New York: McGraw-Hill, 1978.Google Scholar