Advertisement

Assessment of the safety climate in outpatient diagnostic services: Development and psychometric evaluation of a questionnaire

  • Marianne Jossen
  • Fabio Valeri
  • Christina Heilmaier
  • David Schwappach
Health Economy
  • 59 Downloads

Abstract

Introduction

Safe practice and safety culture are important issues in outpatient diagnostic imaging services. As questionnaires assessing safety culture through the measurement of safety climate in this setting are not yet available, the present study aimed to develop and validate such an instrument.

Materials and methods

After adaptation of an existing questionnaire and qualitative pretesting, the instrument was tested by collaborators from three outpatient imaging services in Switzerland. Results were first assessed using descriptive statistics. Scores of individual services were compared using a Wilcoxon test assessing differences between rank distributions. The final instrument was tested for validity using inter-rater agreement measures, such as reliability within groups (rWG), and an intraclass correlation coefficient measure (ICC(1)). These measures allowed the assessment of validity of aggregation into a total score (rWG(j)) and validated the instrument for its capacity to distinguish various safety climates of different groups by comparing inter-rater agreement in the overall sample to inter-rater agreement of individual services (rWG) and by measuring group effects (ICC(1)). Furthermore, the final instrument was tested for internal consistency and reliability using Cronbach’s Alpha.

Results

Safety climate scores vary significantly between services. Inter-rater agreement measures show that item aggregation is justified and that the instrument distinguishes various patterns of safety climate. The final instrument proves to be valid, consistent and reliable.

Conclusions

The final instrument presents a valid, consistent and reliable option to measure safety climate in outpatient diagnostic imaging services. Results can be used as a basis for quality improvement.

Key Points

• An adapted questionnaire that assesses safety climate in outpatient diagnostic imaging services was developed and tested in Switzerland.

• Psychometric evaluation showed the questionnaire to be a valid, consistent and reliable instrument.

• Results are of interest for imaging services as well as for stakeholders interested more globally in monitoring and quality improvement.

Keywords

Outpatient service Safety Psychometrics Organisational culture Surveys and questionnaires 

Abbreviations

aQUA

Institut Institut für angewandte Qualitätsförderung

CSS

Culture of Safety Survey

EQUAM

Externe Qualitätssicherungin der ambulanten Medizin

HSOPS

Hospital Survey on Patient Safety

ICC(1)

Intraclass correlation coefficient

IRA

Inter-rater agreement

rwg

Reliability within groups

rwg(j)

Reliability within groups for multiple judgments

SAQ

Safety Attitude Questionnaire

Notes

Funding

The authors state that this work has not received any funding.

Compliance with ethical standards

Guarantor

The scientific guarantor of this publication is Marianne Jossen

Conflict of interest

The authors of this manuscript declare relationships with the following companies:

Marianne Jossen works for the EQUAM foundation, which uses the developed questionnaires as products.

Statistics and biometry

One of the authors, Fabio Valeri, has significant statistical expertise.

Informed consent

Written informed consent was not required for this study because data collection was anonymous, and participation was voluntary and did not include medical data.

Ethical approval

Institutional Review Board approval was not required because data collection was anonymous, and participation was voluntary and did not include medical data.

Methodology

• Cross-sectional study

• Multicentre study

Supplementary material

330_2018_5646_MOESM1_ESM.docx (22 kb)
ESM 1 (DOCX 21 kb)

References

  1. 1.
    Foederatio Medicorum Helveticorum FMH (2017) Ärztestatistik 2016. Foederatio Medicorum Helveticorum, Bern. Available at http://aerztestatistik.myfmh2.fmh.ch. Accessed 13 march 2018
  2. 2.
    Bundesamt für Gesundheit BAG (2017) [Change of the Ordinance on the Determination and Adaptation of Structures of the Reimbursement System of the Health Insurance]. Bundesamt für Gesundheit, BernGoogle Scholar
  3. 3.
    George R, Lau L, Ng K (2014) Team approach to optimize radiology techniques. In: Lau L, Ng K (eds) Radiological safety and quality. Paradigms in leadership and innovation. Springer, DordrechtGoogle Scholar
  4. 4.
    Gehring K, Schwappach D, Battaglia M et al (2012) Frequency of harm associated with primary care safety incidents. Am J Manag Care 19:323–337Google Scholar
  5. 5.
    Weaver SJ, Lubomksi LH, Wilson RF et al (2013) Promoting a culture of safety as a patient safety strategy. A systematic review. Ann Intern Med 158:369–374CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Zohar D, Livine Y, Tenne-Gazit O, Admi H, Donchin Y (2007) Healthcare climate: A framework for measuring and improving patient safety. Crit Care Med 35:1312–1317CrossRefPubMedGoogle Scholar
  7. 7.
    DiCuccio MH (2015) The relationship between patient safety culture and patient outcomes: A systematic review. J Patient Saf 11:135–142CrossRefPubMedGoogle Scholar
  8. 8.
    Azevedo KB, da Silva CA, Abrantes A (2017) Assessing the hospital survey of patient safety culture. A review of the assumptions, methods and data analysis protocol. Desenvolvimento e Sociedade 3:7-31Google Scholar
  9. 9.
    Colla JB, Bracken AC, Kinney LM, Weeks WB (2005) Measuring patient safety climate: A review of surveys. Qual Saf Health Care 14:364–366Google Scholar
  10. 10.
    Manser T, Brösterhaus M, Hammer A (2014) You can't improve what you don't measure: Safety climate measures available in the German-speaking countries to support safety culture development in healthcare. Z Evid Fortbild Qual Gesundhwes 114:58–71CrossRefGoogle Scholar
  11. 11.
    Sorra JS, Dyert N (2010) Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture. BMC Health Serv Res 10:199CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Sexton JB, Helmreich RL, Neilands TB et al (2006) The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 6:44CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Weingart SN, Farbstein K, Davis RB, Phillips RS (2004) Using a multihospital survey to examine the safety culture. Jt Comm J Qual Saf 30:125–132Google Scholar
  14. 14.
    Pfeiffer Y, Manser T (2010) Development of the german version of the hospital survey on patient safety culture: Dimensionality and psychometric properties. Saf Sci 48:1452–1462CrossRefGoogle Scholar
  15. 15.
    Zimmermann N, Küng K, Sereika SM, Engberg S, Sexton B, Schwendimann R (2013) Assessing the safety attitudes questionnaire (SAQ), German language version in Swiss university hospitals - a validation study. BMC Health Serv Res 13:347CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Gehring K, Mascherek A, Bezzola P, Schwappach D (2015) Safety climate in Swiss hospital units: Swiss version of the safety climate survey. J Eval Clin Pract 21:332–338Google Scholar
  17. 17.
    Schutz A, Counte MA, Meurer S (2007) Development of a patient safety culture measurement tool for ambulatory health care settings: Analysis of content validity. Health Care Manag Sci 10:139–149CrossRefPubMedGoogle Scholar
  18. 18.
    Hoffmann B, Domanska OM, Müller V, Gerlach FM (2009) Development of a questionnaire on the safety climate in family doctor’s services (FraSiK): Transcultural adaptation – report on methods. Z Evid Fortbild Qual Gesundhwes 103:521–529CrossRefPubMedGoogle Scholar
  19. 19.
    Modak I, Helmreich RL, Thomas EJ, Sexton BJ, Lux TR (2007) Measuring safety culture in the ambulatory setting: The safety attitudes questionnaire - ambulatory version. J Gen Intern Med 22:1–5CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Ferris N, Kavnoudias H, Thiel C, Stuckey S (2007) The 2005 Australian MRI safety survey. AJR Am J Roentgenol 188:1388–1394CrossRefPubMedGoogle Scholar
  21. 21.
    De Wet C, Spence W, Mash R, Johnson P, Bowie P (2010) The development and psychometric evaluation of a safety climate measure for primary care. Qual Saf Health Care 19:578–854PubMedGoogle Scholar
  22. 22.
    Häder M (2006) Empiric social research – An introduction. Verlag für Sozialforschung, WiesbadenGoogle Scholar
  23. 23.
    Giai J, Boussat B, Occelli P et al (2017) Hospital survey on patient safety culture (HSOPS): variability of scoring strategies. Int J Qual Healthcare 29:685–692CrossRefGoogle Scholar
  24. 24.
    Sorra J, Gray L, Streagle S et al (2016) AHRQ Hospital Survey on Patient Safety Culture: User’s Guide. AHRQ Publication No. 15-0049-EF. Rockville, MD: Agency for Healthcare Research and QualityGoogle Scholar
  25. 25.
    Bauer DF (1972) Constructing confidence sets using rank statistics. J Am Stat Assoc 67:687–690CrossRefGoogle Scholar
  26. 26.
    Hollander M, Douglas AW (1973) Nonparametric statistical methods. Wiley, New YorkGoogle Scholar
  27. 27.
    LeBreton JM, Senter JL (2008) Answers to 20 questions about inter-rater reliability and inter-rater agreement. Organizational Research Methods 11:815–852Google Scholar
  28. 28.
    James LR, Demaree RG, Wolf G (1984) Estimating within-group inter-rater reliability with and without response bias. J Appl Psychol 69:85–98CrossRefGoogle Scholar
  29. 29.
    James LR, Demaree RG, Wolf G (1993) rwg: An assessment of within-group inter-rater agreement. J Appl Psychol 78:306–309CrossRefGoogle Scholar
  30. 30.
    Ginsburg L, Oore GD (2016) Patient safety climate strength: a concept that requires more attention. BMJ Qual Saf. 25:680–687CrossRefPubMedGoogle Scholar
  31. 31.
    Smith-Crowe K, Burke MJ, Cohen A, Doveh E (2014) Statistical significance criteria for the rWG and average deviation inter-rater agreement indices. J Appl Psychol 99:239–261CrossRefPubMedGoogle Scholar
  32. 32.
    Mascherek AC, Schwappach DLB (2017) Patient safety climate profiles across time: Strength and level of safety climate associated with a quality improvement program in Switzerland - A cross-sectional survey study. PLoS One 12:e0181410.  https://doi.org/10.1371/journal.pone.0181410 CrossRefPubMedPubMedCentralGoogle Scholar
  33. 33.
    Cronbach LJ (1951) Coefficient alpha and the internal strucutre of tests. Psychometrika 16:297–334CrossRefGoogle Scholar
  34. 34.
    R DevelopmentCore Team (2008) R: A language and environment for statistical computing. R Foundation for Statistical Computing, ViennaGoogle Scholar

Copyright information

© European Society of Radiology 2018

Authors and Affiliations

  1. 1.EQUAM FoundationBernSwitzerland
  2. 2.Institute of Primary CareUniversity of Zurich and University Hospital of ZurichZurichSwitzerland
  3. 3.Department of Diagnostic and Interventional RadiologyUniversity Hospital ZurichZurichSwitzerland
  4. 4.Swiss Patient Safety FoundationZurichSwitzerland
  5. 5.Institute for Social and Preventive Medicine (ISPM)University BernBernSwitzerland

Personalised recommendations