Skip to main content
SpringerLink
Log in

Measuring quality of health care from the user’s perspective in 41 countries: psychometric properties of WHO’s questions on health systems responsiveness

  • Original Paper
  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Objective

To evaluate, for different populations, psychometric properties of questions on “health systems responsiveness”, a concept developed by World Health Organization (WHO) to describe non-clinical and non-financial aspects of quality of health care.

Data sources/study setting/data collection

The 2000–2002 WHO Multi-Country Study comprised 70 general population surveys. Forty-one surveys were interviewer-administered, from which we extracted respondent records indicating ambulatory and inpatient health services use (excluding long-term institutions) in the previous 12 months (50,876 ambulatory and 7,964 hospital interviews).

Study design

We evaluated feasibility, reliability, and construct validity using 33 items with polytomous response options, comparing responses from populations identified by countries, sex, age, education, health and income.

Principal findings

Average item missing rates ranged from 0 to 16%. Domain-specific alpha coefficients exceeded 0.7 in 7 (of 9) cases. Average intertemporal reliability was acceptable in 6 (of 10) sites, where Kappas ranged from 0.54 to 0.79, but low in 4 sites (K < 0.5). Kappa statistics were higher for male, educated and healthier populations than for female, less educated and less healthy populations. Factor solutions confirmed the domain structure of 7 domains (only 7 were operationalized for ambulatory settings). As in other studies, higher incomes and age was associated with more positive responsiveness reports and ratings.

Conclusions

Quality issues addressed by WHO’s questions are understood and reported adequately across diverse populations. More research is needed to interpret user-assessed quality of care comparisons across population groups within and between countries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. High human development - Argentina, Bahrain, Belgium, Canada (telephone), Costa Rica, Croatia, Czech Republic, Estonia, Finland, France, Germany, Iceland, Ireland, Italy, Latvia, Luxembourg, Malta, Mexico (long survey - 9 modules), Portugal, Slovakia (long), Spain, Sweden, The Netherlands, United Arab Emirates. Low human development—Bulgaria, China (long), Colombia (long), Egypt (long), Georgia (long), India (long), Indonesia (long), Iran (long), Jordan, Morocco, Nigeria (long), Oman, Romania, Russian Federation, Syria (long), Turkey (long), Venezuela.

Abbreviations

AHRQ:

United States Agency for Healthcare Research and Quality

CAHPS:

Consumer Assessment of Health Plans Survey

K:

Kappa

MCS Study:

Multi-Country Survey Study on Health and Health Systems Responsiveness

ML:

Maximum Likelihood (factor analysis)

QUOTE:

Quality of care through patients’ eyes

r:

Correlation Coefficient (Pearson or Spearman, as specified)

UNDP:

United Nations Development Programme

UNESCO:

United Nations Educational, Scientific and Cultural Organization

WHO:

World Health Organization

References

  1. Kelley, E., & Hurst, J. (2006). Health care quality indicators project conceptual framework paper. OECD Health Working Papers No. 23. Retrieved 30 May at: http://www.oecd.or..

  2. Donabedian, A. (1980). Explorations in quality assessment and monitoring: the definition of quality and approaches to assessment. Ann Arbor, Michigan: Health Administration Press

    Google Scholar 

  3. Murray, C. J. L., & Frenk, J. (2000). A framework for assessing the performance of health systems. Bulletin of the World Health Organisation, 78(6), 717–731

    CAS  Google Scholar 

  4. Thompson, A. G. H., & Sunol, R. (1995). Expectations as determinants of patient satisfaction. Health Expectation, 2, 93–104.

    Google Scholar 

  5. Sitzia, J., & Wood, N. (1997). Patient satisfaction: a review of issues and concepts. Social Science and Medicine, 45(12), 1829–1843

    PubMed  CAS  Google Scholar 

  6. Wensing, M., Jung, H. P., Mainz J., Olesen, F., & Grol, R. (1998). A systematic review of the literature on patient priorities for general practice care. Part 1: Description of the research domain. Social Science and Medicine, 47(10), 1573–1588.

    Google Scholar 

  7. Ware, J. E., & Hays, R. D. (1988). Methods for measuring patient satisfaction with specific medical encounters. Medical Care, 26(4), 393–402.

    Article  PubMed  Google Scholar 

  8. Penchansky, R., & Thomas, J. W. (1981). The concept of access. Medical Care, 19(2), 127–140

    Article  PubMed  CAS  Google Scholar 

  9. De Silva A. A framework for measuring responsiveness. (Discussion Paper 32) 2000. Retrieved December 1, 2005, from http://www.who.int/responsiveness/papers/e.

  10. Zastowny, T. R., Stratmann, W. C., Adams, E. J., & Fox, M. L. (1995). Patient satisfaction and experience with health services and quality of care. Quality Management in Health Care, 3, 50–61.

    PubMed  CAS  Google Scholar 

  11. Murray, C. J. L., Kawabata, K., & Valentine, N. (2001). People’s experience versus people’s expectations. Health Affairs, 20(3), 41–43

    Article  Google Scholar 

  12. Hargraves, J. K., Hays, R. D., & Cleary, P. D. (2003) Psychometric properties of the consumer assessment of health plans study 2.0 adult core survey—CAHSP®. Health Services Research, 38(6p1), 1509–1528

    Article  PubMed  Google Scholar 

  13. Ustun, B. S. et al. WHO multi-country survey study on health and responsiveness 2000–2001 (Discussion Paper 37) 2003. Retrieved December 1, 2005, from http://www.who.int/responsiveness/papers/e..

  14. Schafer, J. L. (2001). NORM: multiple imputation of incomplete multivariate data under a normal model [statistical software]. University Park: Penn. State University. Department of Statistics

    Google Scholar 

  15. Elliott, M. N., Edwards, C., Hambarsoomians, K., Angeles, J., & Hays, R. D. (2005). Patterns of unit and item nonresponse in the CAHPS® Hospital Survey. Health Services Research, 40(6p2), 2096–2119

    Article  PubMed  Google Scholar 

  16. DeVellis, R. F. (1991). Scale Development. Theory and applications. (Applied Social Research Methods Series Volume 26.) London: Sage Publications

    Google Scholar 

  17. UNDP (2001). Human development report. New York: UNDP.

    Google Scholar 

  18. Gandek, B. et al. (1998). Tests of data quality, scaling assumptions, and reliability of the SF-36 in eleven countries: Results from the IQOLA project. Journal of Clinical Epidemiology, 51(11), 1149–1158 (spec iss)

    Article  PubMed  CAS  Google Scholar 

  19. Cramer, D., & Howitt, D. (2004). The SAGE dictionary of statistics. London: Sage Publications

    Google Scholar 

  20. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. (3rd ed.). New York: McGraw-Hill

    Google Scholar 

  21. Kim, J., & Mueller, C. W. (1978). Factor analysis statistical methods and practical issues. In E. M. Uslaner (Ed.), Quantitative Applications in the Social Sciences. London: Sage

    Google Scholar 

  22. Hall, J. A., & Dornan, M. C. (1990). Patient sociodemographic characteristics as predictors of satisfaction with medical care: a meta-analysis. Social Science and Medicine, 30, 811–818

    Article  PubMed  CAS  Google Scholar 

  23. UNESCO. Educational Statistic, 2002. Retrieved May 30, 2006, from http://www.uis.unesco.org..

  24. Jenkinson, C. A., Coulter, A., & Bruster, S. (2002). The Picker patient experience questionnaire: development and validation using data from in-patient surveys in five countries. International Journal for Quality in Health Care, 14(5), 353–358

    Article  PubMed  Google Scholar 

  25. Sitzia, J. (1999). How valid and reliable are patient satisfaction data? An analysis of 195 studies. International Journal for Quality in Health Care, 11(4), 319–328

    Article  PubMed  CAS  Google Scholar 

  26. Sofaer, S., Crofton, E., Goldstein, C., Hoy, E., & Crabb, J. (2005) What do consumers want to know about the quality of care in hospitals? Health Services Research, 40(6p2), 2018–2037

    Article  PubMed  Google Scholar 

  27. Ware, J. (1976). Scales for measuring general health perceptions. Health Services Research, (Winter), 396–415.

  28. Hudson, S., Weisman, C., Anderson, R., & Camacho, F. (2004). The development and validation of the primary care satisfaction survey for women. Women Health Issues, 14, 35–50

    Google Scholar 

  29. Coulter, A., & Jenkinson, C. (2005). European patients’ views on the responsiveness of health systems and healthcare providers. European Journal for Public Health, 15(4), 355–360

    Article  Google Scholar 

  30. Grogan, S., Conner, M., Norman, P., & Willits, D. (2000) Porter Validation of a questionnaire measuring patient satisfaction with general practitioner services. Quality and Safety in Health, Care 9, 210–215

    CAS  Google Scholar 

  31. Stewart, M. (2005). Reflections on the doctor–patient relationship from evidence and experience. British Journal of General Practice, 55(519), 793–801.

    PubMed  Google Scholar 

  32. Campbell, A., Converse, P. E., & Rodgers, W. L. (1976). The quality of American life; perceptions evaluations and satisfactions. New York: Russell Sage Foundation

    Google Scholar 

  33. Edwards, C., Staniszwesda, S., & Crichton, N. (2004). Investigation of the ways in which patients’ reports of their satisfaction with healthcare are constructed. Sociology of Health and Illness, 26(2), 159–183

    Article  PubMed  Google Scholar 

  34. Sitzia, J., & Wood, N. (1998). Response rate in patient satisfaction research: an analysis of 210 published studies. International Journal for Quality in Health Care, 10(4), 311–317

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. WHO, 20, Avenue Appia, CH1211, Geneva 27, Switzerland

    N. B. Valentine

  2. Erasmus Medical Center, Institute of Health Policy and Management (iBMG), Rotterdam, The Netherlands

    N. B. Valentine & G. J. Bonsel

  3. Harvard University, Cambridge, USA

    C. J. L. Murray

Authors
  1. N. B. Valentine
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. J. Bonsel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. J. L. Murray
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to N. B. Valentine.

Additional information

The views expressed by the authors do not necessarily represent the stated policy of the World Health Organization.

Appendices

Appendices

Appendix 1 Performance questions* from the responsiveness module in the WHO MCS Study
Full size table
Appendix 2 Sample descriptive statistics for the 41 surveys in the WHO MCS Study
Full size table
Appendix 3 Likert-scaled responsiveness inpatient item wording and item properties
Full size table
Appendix 4(a) Internal consistency reliability in high human development countries (n = 24)
Full size table
Appendix 4(b) Internal consistency reliability in low human development countries (n = 17)
Full size table
Appendix 5 Temporal reliability measured by Kappa statistics
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Valentine, N.B., Bonsel, G.J. & Murray, C.J.L. Measuring quality of health care from the user’s perspective in 41 countries: psychometric properties of WHO’s questions on health systems responsiveness. Qual Life Res 16, 1107–1125 (2007). https://doi.org/10.1007/s11136-007-9189-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-007-9189-1

Keywords

Search

Navigation