Background

Patient safety indicates the absence of adverse events (AE), which has received increased interest as it is endangered by the increasing complexity of healthcare, the simultaneously increasing economic pressure and the shortages in the healthcare work force [1]. An AE is defined as an unintended injury resulting from a medical intervention, regardless of the patient’s underlying medical condition [2, 3]. Prevention of AEs has been the focus of many improvement initiatives and studies across the world [4].

The occurrence of AEs varies between 3 and 17% of all hospital stays [1, 5, 6]. In Germany, AEs are expected to affect 5–10% of all patients in the inpatient sector, implying 950.000–1.900.000 AEs per year [3]. Up to 60% of these AEs are considered to be preventable, resulting in approximately 19.000 avoidable deaths per year in Germany [3, 6, 7].

To understand structures and procedures in health care that might lead to an AE, it is crucial to analyse them objectively assuming AEs being reported at first [8]. Hospital risk management, which is obligatory in German hospitals according to prevailing regulations of the Joint Federal Committee (gemeinsamer Bundesausschuss) since 2014, contains instruments to reporting of AEs and critical incidents in general [3, 9]. The critical incident reporting system (CIRS) is one of these instruments and comprises a platform where hospital employees can anonymously report AEs as well as other critical incidents that occur in the context of health care and increase the risk for AEs. These reports are then analysed by experts who give recommendations for the future avoidance of events of this kind [10].

However, there is a range of barriers that impede AE reporting. As possible reasons for underreporting of AEs, a lack of support from authorities or the person ‘causing’ the error, fear of consequences, lack of knowledge and high workload were named [11, 12]. There is strong evidence, that the number of unreported or unpublished AEs is much higher than the ones being reported and included in AE estimations leaving approximately up to 91% unreported [3, 13, 14].

Beneficial safety culture is necessary to achieve a supporting environment and thus encourage staff to report AEs [8]. In Germany, within the last 15 years the ways of dealing with errors and AEs has changed. From avoiding this topic up to then, to approaches that aim to analyse factors that contribute to the occurrence of errors and also include an organisational perspective [3]. However, still more than half of German hospitals are estimated to not ensure an optimally beneficial safety culture that includes the ability to learn from reported incidents [15]. To promote a supportive safety culture in general, it is necessary to understand attitudes and values of employees towards adverse event reporting [4].

Yet, there is no German-language questionnaire assessing health professional’s attitudes to AE within German health services.

The Reporting of Clinical Adverse Events Scale (RoCAES) was developed and validated in the UK by Wilson, Bekker and Fylan in 2008 and assesses five-domains of staff attitudes to AEs: perceived blame (six items); perceived criteria for identifying events that should be reported (six items); perceptions of colleagues’ expectations (six items); perceived benefits of reporting (five items); perceived clarity of reporting procedures (two items) [16]. It was translated and validated for use with health professionals in China [17], and may be a robust measure for use within a German healthcare system. The aim of this study was to translate this questionnaire into German and assess its reliability and validity for use to assess attitudes of health professionals to AEs in German hospitals.

Methods

The questionnaire was translated into German language according to international translation guidelines. Psychometric properties of this translated version were then examined in a cross-sectional study in German hospitals.

Translation process

The translation of the RoCAES is based on international recommendations [18,19,20], following the five steps described below:

  1. a)

    Forward translation: The instrument was translated from English into German by two independent translators whose mother tongue is German and that are fluent in English.

  2. b)

    Comparison of the two translated versions: Within step a), two translated versions of the questionnaire were produced. A consensus meeting was held to discuss discrepancies and ambiguities of these two versions and to generate the preliminary initial translated version.

  3. c)

    Blind back-translation: Two bilingual translators, whose mother tongue is English, were asked to independently translate the preliminary initial translated version back into the source language. For this step, translators with no initial knowledge of the instrument were chosen.

  4. d)

    Comparison of the two back-translated versions: Within a second consensus meeting, discrepancies in sentence structures, meaning or other uncertainties have been discussed and an English consensus version has been merged. In addition to this step, the back-translated version has been sent to the original author of the RoCAES who compared this version with her initial intentions of the original questionnaire to ensure conceptional equivalence.

  5. e)

    Pilot testing of the pre-final version: several independent experts were asked to review the German pre-final version of the questionnaire. Furthermore, five experts matching the inclusion criteria for the subsequent survey were involved in a cognitive debriefing evaluating instructions, items and response format clarity.

Study population and data collection

For data collection, the RoCAES-D was transferred into an electronic version. The link to the online survey was sent to approximately 1300 staff members in 18 different institutions (14 hospitals, a medical school, a nursing school, a school for therapists and a germanwide network of professionals interested in incident reporting) step by step in spring 2019. Study population included all clinical staff members and students that work in the inpatient care sector and have direct patient contact (e.g. nurses, physicians, therapists, …). We covered this interprofessional perspective in the German version contrarily to the eligibility criteria of Wilson et al. and Sun et al. [16, 17], who addressed doctors and nurses only, as interprofessional teamwork plays an important role in patient safety and prevention of clinical incidents [21].

The hospitals cover various clinical departments such as neurology, orthopedy, rehabilitation clinics, geriatrics or internal medicine. Within the hospitals, the survey was sent to the management, quality management and staff councils via e-mail. They were asked to pass on the mail to their employees. In addition, we offered to send paper-based questionnaires to reach staff without access to their e-mail accounts. Because management staff is not included in the target group addressed by the questionnaire, an initial filter question excluding unsuitable test persons was added. Furthermore, the clinical staff was also asked to send the link to their colleagues, former fellow students or friends matching the eligibility criteria (snowball system).

The targeted sample size was estimated to be within the range of 100 to 250 in order to facilitate the planned confirmatory factor analysis (CFA) [19, 22, 23].

Measures

The RoCAES consists of 25 items rated with a 4-point-likert scale from ‘1’ (strongly agree) to ‘4’ (strongly disagree). 16 items required reversed scoring. As agreeing to item 4 (‘reporting adverse events protects patients’) results in a positive attitude and item 3 (‘It is not my responsibility to report adverse events involving colleagues’) is formulated negatively, we assumed that the scoring guideline for these two items was switched in the original study. Contrary to the recommendation, we therefore reverse scored item 3 instead of item 4 for our analysis.

Since the questionnaire was sent in an online version and included mandatory fields, the answer option ‘not specified’ had to be inserted.

An introductory definition of AEs and three additional items introduce the main scale focusing on whether the participant has ever witnessed or been involved in an AE, if he ever reported an AE and how likely the participant is to report an AE in the future. Closing the questionnaire, demographic data was collected including profession, gender and previous duration of employment. The student employee status was not assessed separately. The full German questionnaire can be found in the Additional file 1.

Data analysis

To assess reliability and validity of the RoCAES-D, only fully completed questionnaires were analysed. For further analysis, answers that chose ‘not specified’ were treated as missing data and have been imputed using the mean value of each item across participants [24]. Internal consistency was determined by calculating coefficient McDonald’s omega which can be used for multidimensional tests as a more suitable alternative to Cronbach’s alpha [25]. To find omega, a factor analysis is done by rotating the factors obliquely and performing a Schmid Leiman transformation. Cronbach’s alpha was assessed as well to provide better comparability and transparency. Alpha was calculated for the overall scale following the original authors and the Chinese validation paper [16, 17].

A CFA was selected for testing construct validity. For this, the factor model was based on data of original research by Wilson et al. and Xiao Sun et al. [16, 17] and was used to test its model fit for the empiric research data. Data analysis was done using R Version 3.5.1 and SPSS Version 25. In R we used the procedures ‘omega’ within the psych package and ‘cfa’ within the lavaan package.

The CFA was done using the original factor structure of the RoCAES. A one-factor model was used to compare model fit showing no relevant benefit for the existing five-factor model. For calculating reliability, the used coefficient McDonald’s omega estimates the general factor saturation of a questionnaire by doing an exploratory factor analysis (EFA) first. This exploratory approach was used, as well as residuals and covariances calculated within the CFA, to modify the existing factor model.

More precisely, items with a low factor loading (less than 0.25) and a low number of explained variance (R2 less than 0.1) were reallocated using the highest observed factor loadings in the exploratory analysis that was done to estimate omega.

Results

The translation process did not reveal any noteworthy discrepancies or ambiguities. The pilot testing via cognitive debriefing has led to an adaption of two sentences, that had to be changed marginally due to incongruities in sentence structure. Observed associations triggered by the phrasing of the items agreed with each other and with the intention of the questionnaire. Together with the standardized introduction, and the written instructions, objectivity of application can be assumed as given for this scale.

Furthermore, data analysis is clearly defined by referring to a fixed scoring for each answer option. The objectivity of the evaluation is therefore also seen as given.

The psychometric testing of the pre-final German RoCAES-D was conducted in spring of 2019. A total of n = 120 health professionals in 26 hospitals completed the survey. Because the recruitment was conducted as a snowball system, response rate cannot be estimated accurately. The sample included heterogeneous professions and work experience. Sociodemographic characteristics of the sample are shown in Table 1.

Table 1 sociodemographic data (n = 120)
Full size table

Confirmatory factor analysis

For the CFA, as the answer options ‘not specified’ were treated as missing values, two items showed no missings. In all other items, missing values range from 0.8 to 16%. The two items with the highest missing rate include Item 11 (‘My colleagues expect me to report adverse events’, 16% missing values) and Item 13 (‘The procedures in this hospital are clear on what sort of adverse events should be reported’, 12.5% missing values). Other items showed missing rates lower than 7.5%.

Results of the initial CFA showed a poor model fit (Table 2) for the original structure in the German context. Using the one-factor model to compare model fit also showed no relevant benefit for the existing five-factor model. Therefore, the original model was modified as described previously. This modified model resulted in the lowest RMSEA*, SRMR*, and AIC* (Table 2). Hence, it is considered to be the one with the best model fit [19, 22].

Table 2 model fit of the existing model, a one-factor model and the modified model
Full size table

Classifying the measures, the modified model showed a poor model fit for the German target group. Modifications in this model conclude the deletion of item 9 (‘I am not doing my job properly unless I report adverse events’) because of low factor loadings (0.024) and a low R2 (0.001). Item 1 (‘Reporting adverse events helps identify staff who need additional training’) and Item 24 (‘Reporting adverse events makes people accountable for their actions’) originally loaded on factor 4 (‘Perceived benefits of reporting’) and have now been relocated to factor 1 (‘Perceived blame’). Both items showed low factor loadings within the initial CFA and a correlation with other items within the EFA. Additionally, a content-related alignment lead to the described reallocation. Item 4 (‘Reporting adverse events protects patients’) also required a reallocation. However, a new assignment was not reasonable due to a bad model fit and a content-related match with the original factor 4 (perceived benefits). This resulting 24-item questionnaire can be seen in Table 3.

Table 3 factor loadings of the modified 24-item RoCAES-D
Full size table

Reliability

For the translated pre-final version, omega was 0.85, implying a high factor saturation. In order to ensure better comparability, Cronbach’s alpha was also calculated and was 0.79. Both coefficients were again estimated for the modified 24-item RoCAES-D. Omega increased to 0.87, alpha also increased to 0.81 (Factor 1: 0.8; Factor 2: 0.53; Factor 3: 0.45; Factor 4: 0.53; Factor 5: 0.68).

Discussion

The resulting RoCAES-D shows high reliability and a poor validity for the German target population. Cronbach’s alpha was somewhat lower than but still comparable with the English RoCAES (0.83) and the Chinese C-RoCAES (0.85) [16, 17]. However, coefficient omega showed a high reliability (0.87). The even poorer model fit of the one factor model (Table 2) in the RoCAES-D underlines the multidimensional structure of the scale [25].

The original factor structure did not have a good fit with the data. Reallocating items 1 and 24 may result from cultural differences between German and English health professions. The two items were originally associated with benefits of reporting and now correlate with perceived blame. As risk management and handling adverse events is not an official part of the professional education in German health professions [26, 27], staff is not inevitably habituated with reporting processes, implying a perceived blame when actually practiced. In contrast, being aware of risks of medical interventions and being familiar with reporting procedures is part of the English competence model of nurses [28], resulting in a more neutral and professional point of view perceiving benefits instead of blame and implying a higher rate of incident reporting [4]. The observed negative loadings of those two items (Table 3) might also support the cultural differences that have been identified throughout the reallocation. Furthermore, resulting from this different understanding of professional roles, German health care workers might not primarily associate their professional duties in health care with the responsibility of reporting adverse events, which can be seen in the CFA: item 9 (‘I am not doing my job properly unless I report adverse events’) showed particularly low factor loadings on any of the five factors and an average R2 of 0.001 and was therefore deleted for the RoCAES-D.

Due to the imputation of missing values, an impact on variance and therefore on the estimated psychometric properties leading to a miscalculation needed to be considered. Yet, the proportion of imputed values stayed relatively low with only two items higher than 7.5% (16% and 12,5%). In this order, no considerable impact is expected [29].

Besides the acceptable psychometric properties of the RoCAES-D, the usage of the scale in a multiprofessional target group has proven to be practical. This population differs from the original survey in England, which only included nurses and doctors. The German scale also addressed other health professions that are directly involved in daily patient care. In addition to actual evidence showing the necessity of involving an interprofessional team to prevent errors and adverse events, Wilson et al. also recommended an application to a more diverse clinical staff [16, 21]. The RoCAES-D can be carried out in German health care institutions without further restriction and hence, involve a larger number of clinical staff. Interventions that can be derived from performing these surveys will now be tailored more specifically and are therefore expected to be more effective as they address the whole multiprofessional team [30].

Limitations to our research concern the study population and the relatively low response rate as well as the model fit. In this survey, the response rate has been extremely low. However, the main purpose of this data collection was to gain a high variance and only use the data for the translation and validation process. For this, the sample size of n = 120, population structure and model fit are seen as appropriate [19, 22, 23]. To get an actual insight in the attitudes of health professionals, we recommend using the scale on institutional level. For this it is important to monitor the population structure and to achieve a higher response rate. On this basis, subgroup analysis, derivation of action plans or identification of needs for further investigation, e.g. employee interviews or process analyses, can be done properly. We also recommend a content-related revision of the scale after several years of use in different institutions. Possible additional cultural differences between the English, Chinese and German reporting culture have not been identified yet. The implementation of the scale for institutional use can help uncover cultural aspects and items that might need to be added for the German context. Even if the RoCAES-D showed a high reliability, the lower internal consistency on factor-level is reflected by the poor model fit of the modified model. This might support these cultural differences that require further investigation. Institutional usage with a data analysis on item-level, as well as additional psychometric testing to increase model fit is indicated in this regard.

The fact that a CFA had to be carried out instead of calculating divergent or convergent validity using a related survey underlines the existing research gaps. Nevertheless, translating and initially validating this scale might be a good starting point for further research and a progression for risk management in German health care institutions.

With 25 Items, the scale showed to be comprehensive. This comprehensiveness might explain the relatively low factor loadings. Yet, the purpose of this scale is to be an easy applicable screening device that identifies diverse starting points for further investigations or potential for further actions within a health care facility. As the scale is also short and concrete, we believe it can easily be embedded in a more extensive survey within institutions. It can for example be combined with a questionnaire addressing safety culture in general, or any other.

Conclusions

In conclusion, the successful translation and initial validation of the RoCAES-D might be a good starting point for further research whereby data can be collected for further training or further investigations regarding an internal AE management system in clinics. Based on these results, a cultural adaptation of the scale needs to be done to initiate a large-scale usage of the questionnaire and to gain a useful instrument. Usage of the scale can contribute to improve patient safety on a long-term basis and area-wide through tailored interventions, such as education of employees or promoting a supporting reporting culture.