Implementation of anesthesia quality indicators in Germany

Background In 2016 the German Society of Anesthesiology and Intensive Care Medicine (DGAI) and the Association of German Anesthetists (BDA) published 10 quality indicators (QI) to compare and improve the quality of anesthesia care in Germany. So far, there is no evidence for the feasibility of implementation of these QI in hospitals. Objective This study tested the hypothesis that the implementation of the 10 QI is feasible in German hospitals. Material and methods This prospective three-phase national multicenter quality improvement study was conducted in 15 German hospitals and 1 outpatient anesthesia center from March 2017 to February 2018. The trial consisted of an initial evaluation of pre-existing structures and processes by the heads of the participating anesthesia departments, followed by a 6-month implementation phase of the QI as well as a final re-evaluation phase. The implementation procedure was supported by web-based implementation aids (www.qi-an.org) and internal quality management programs. The primary endpoint was the difference in the number of implemented QI per center before and after implementation. Secondary endpoints were the number of newly implemented QI per center, the overall number of successful implementations of each QI, the identification of problems during the implementation as well as the kind of impediments preventing the QI implementation. Results The average number of implemented QI increased from 5.8 to 6.8 (mean of the differences 1.1 ± 1.3; P < 0.01). Most frequently the QI perioperative morbidity and mortality report (5 centers) and the QI temperature management (4 centers) could be implemented. After the implementation phase, the QI incidence management and patient blood management were implemented in all 16 centers. Implementation of other quality indicators failed mainly due to a lack of time and lack of structural resources. Conclusion In this study the implementation of QI was proven to be mostly feasible in the participating German hospitals. Although several QI could be implemented with minor effort, more time, financial and structural resources would be required for some QI, such as the QI postoperative visit.


Introduction
The quality of anesthesia is affected by an array of factors. These include patientcentered variables, such as patient safety Preliminary data of this study were presented at the Deutsche Anästhesiecongress (DAC), [9][10][11] May 2019, Leipzig, Germany.
The german full-text version of this article will be available at SpringerLink http://link.springer. com/article/10.1007/s00101-020-00775-w and satisfaction, mortality and functional outcome as well as economical aspects, optimized procedural organization and structured ambient conditions along with qualified and motivated medical personnel. These factors can be subsumed into three categories: first, the quality of the institution's structure, second, the quality of the anesthesia process itself and third, the quality of the outcome and its measurability [5]. Particularly in anesthesia, the discrimination between surgeryrelated and anesthesia-related criteria is of crucial importance since many outcome-related parameters depend on the success of both disciplines [6]. To compare and improve the quality of anesthesia care a measurable catalogue of relevant criteria is important. The development of quality indicators (QI), initially used in manufacturing processes is a feasible way to condense required information in order to achieve reliable and comparable data, also in healthcare [8].
Therefore, the executive committees of the German Society of Anaesthesiology and Intensive Care Medicine (DGAI) and the Association of German Anaesthetists (BDA) founded a task force to develop and evaluate QI for anesthesia in Germany. A decision-making process, consisting of an initial literature search followed by three consecutive surveys using the Delphi method among the task force members, took place to elaborate potential QI. Finally, a list of 10 QI was consented by the DGAI and BDA and published in 2016 [5]. Although improvement of patient outcome by enhanced QI adherence is still lacking evidence in any medical subject area, several of the QI are linked to direct impact on patient safety and outcome [3]. This refers, among others, to patient blood management, temperature management and the usage of the WHO (World Health Organization) safe surgery checklist [7,9,12]. In this prospective, national, multicenter quality improvement trial, the feasibility of implementation of the 10 QI of the DGAI in 16 German centers after a 6month implementation phase was investigated. The primary endpoint was the difference in number of implemented QI per center after the intervention compared with the initial assessment.

Material and methods
A total of 20 German centers registered for participation in this prospective, multicenter, quasi-experimental trial. Of the centers 16 successfully completed the entire three phases of the trial, while 4 retracted their offer to participate. Phase I comprised an initial self-evaluation of potentially pre-existing adherence to the QI prior to the QI implementation phase (. Fig. 1). Therefore, the participants were asked to report the existence of every single item required for one of the QI. In addition, the center's structural hospital data regarding anesthesia were acquired. For this purpose, each center was able to access an electronic case report form (eCRF) from March to April 2017, which was set up with an OpenClinica database (OpenClinica, LLC, Waltham, MA, USA). In phase II, the implementation of the 10 QI took place in all participating centers during a 6-month period between May and October 2017 (. Fig. 1). For support, the website www.qi-an.org, hosted by Docs in Clouds GmbH, Aachen, Germany, was launched. It provided detailed information regarding each QI, supporting literature as well as helpful tools for their implementation, e.g. presentation slides for employee training. Furthermore, direct contact with the coordinating investigator was possible at any time. A supplemental support by on-site lecture or training was proposed to each center by the coordinating investigator. The particular measures to implement the QI (e.g. teaching, structural improvements) were left to the discretion of each center's responsibility. For example, the following measures were taken to implement the QI in one of the centers: all missing standard operation procedures (SOP) were elaborated by selected consultants of the department of anesthe-siology and implemented after internal revision to meet the criteria of QI I. Implementation of QI II failed, since the anesthesia protocol used by the center is standardized for participation in the Perioperative Medicine Network (PoMNet) which would interfere with the DGAI recommendations. The implementation is likely to be feasible when the centre launches an electronic anesthesia record. Incident management (QI III) was already implemented and did not require further measures. Patient blood management (QI IV) was also already implemented, but the center introduced a system for electronic documentation of indications for transfusion of blood products. Regarding QI V, the centre elaborated a SOP for temperature management. An internal audit of 50 consecutive anesthesia protocols and patient protocols was conducted by a consultant. The aim was to determine whether intraoperative body temperature was measured and the corresponding temperature value at the end of the surgery as well as completion rates required by QI VI (safe surgery checklist) and VII (morbidity and mortality report). As a result, the implementation of QI V and VI failed due to an insufficient documentation rate in the analyzed sample. Items regarding QI VII were completed sufficiently, but not all items could be queried due to PoMNet participation as mentioned. Implementation of QI VIII (handover and discharge protocols) was not possible because of an interfering research project on this topic. Postoperative visits and physician staffing as recommended by DGAI, were already established. Therefore, the implementation of both QI IX as well as QI X was met without additional efforts in this center.
In phase III, following the implementation period, the participating center's adherence to the QI was re-evaluated by a second self-assessment via eCRF using the same questions as in phase I (. Fig. 1). In addition, two supplemental questions were asked of each QI: First, regarding difficulties during the implementation process that might be solved on site. Second, about the causes of implementation failure. Furthermore, the centers were asked to provide their appraisal of whether all necessary data for the QI were registered and accessible in their clinical routines. This phase III assessment took place in February 2018.
Each QI consisted of a different number of items, which were predefined by the QI task force [5]. For each QI comprising more than one item, an individual threshold was defined inordertodiscriminate whether adherence to a sufficient number of items for the QI could be classified as implemented. For example, for fulfilment of the criteria of QI VI, it was necessary to have both the filed WHO safe surgery checklist in the patient record as a routine procedure as well as a completion rate of this checklist of at least 95%. For a detailed description of all QI including the benchmark criteria, please refer to . Table 1. Successful implementation was rated as one point. A maximum sum of ten points was achievable indicating complete adherence to all QI. Fulfilment of routine data from anesthesia records was checked in each center either from a sample of 50 consecutive paper-based records from the preceding month or from the center's patient data management systems (PDMS). Procedures were left to the discretion of the centers as part of their routine quality assurance. The adherence to the QI in phase I and III was calculated by the investigators according to the participant's entries in the database. The accuracy of these entries has not been validated.
The primary endpoint was the difference in the number of implemented QI per center after the implementation phase compared with the initial assessment (phase I). Secondary endpoints were the number of newly implemented QI per center, the overall number of successful implementations of each QI, the identification of problems during the implementation and the kind of impediments preventing the QI implementation, as well as the availability of the required information data from the hospitals' routine databases. A problem during implementation was defined as a difficulty that could be solved resulting in QI adherence. It could take, for example, 3 months and several meetings before an additional item querying postoperative nausea and vomiting (PONV) was added to a center's PDMS. In con-

Implementation of anesthesia quality indicators in Germany. A prospective, national, multicenter quality improvement study Abstract
Background. In 2016 the German Society of Anesthesiology and Intensive Care Medicine (DGAI) and the Association of German Anesthetists (BDA) published 10 quality indicators (QI) to compare and improve the quality of anesthesia care in Germany. So far, there is no evidence for the feasibility of implementation of these QI in hospitals.
Objective. This study tested the hypothesis that the implementation of the 10 QI is feasible in German hospitals. Material and methods. This prospective three-phase national multicenter quality improvement study was conducted in 15 German hospitals and 1 outpatient anesthesia center from March 2017 to February 2018. The trial consisted of an initial evaluation of pre-existing structures and processes by the heads of the participating anesthesia departments, followed by a 6-month implementation phase of the QI as well as a final re-evaluation phase. The implementation procedure was supported by web-based implementation aids (www. qi-an.org) and internal quality management programs. The primary endpoint was the difference in the number of implemented QI per center before and after implementation. Secondary endpoints were the number of newly implemented QI per center, the overall number of successful implementations of each QI, the identification of problems during the implementation as well as the kind of impediments preventing the QI implementation.
Results. The average number of implemented QI increased from 5.8 to 6.8 (mean of the differences 1.1 ± 1.3; P < 0.01). Most frequently the QI perioperative morbidity and mortality report (5 centers) and the QI temperature management (4 centers) could be implemented. After the implementation phase, the QI incidence management and patient blood management were implemented in all 16 centers. Implementation of other quality indicators failed mainly due to a lack of time and lack of structural resources. Conclusion. In this study the implementation of QI was proven to be mostly feasible in the participating German hospitals. Although several QI could be implemented with minor effort, more time, financial and structural resources would be required for some QI, such as the QI postoperative visit.

Keywords
Anaesthesia · Quality indicators · Quality of Healthcare · Benchmarking · Quality assurance

Schlüsselwörter
Anästhesie · Qualitätsindikatoren · Qualität der Gesundheitsversorgung · Benchmarking · Qualitätssicherung Table 1 Detailed overview of the 10 quality indicators (QI) consented and published by the German Society of Anesthesiology and Intensive Care Medicine (DGAI) and the Association of German Anesthetists (BDA) [5]. Translated

Baseline hospital data
Among the 16 centers there were 7 university hospitals, 6 primary hospitals, 1 secondary and 1 tertiary hospital as well as 1 outpatient anesthesia centre. The capacity of the hospitals (excluding the outpatient centre) ranged from 244 to 1610 beds (mean 912). The number of anesthesia procedures per year ranged from 6041 to 33,649 (mean 17,803). The annual average case mix index ranged from 0.9 to 1.9 (mean 1.2). The number of anesthesia physicians ranged from 14 to 146 (mean 65.2), among them were 3-72 residents (mean 31.8) and 6-74 specialists (mean 33.8).

QI implementation
The average number of implemented QI increased from 5.8 to 6.8 in total (mean of differences 1.1 ± 1.3; 95% confidence interval, CI 0.3-1.8; P < 0.01) (. Fig. 2). The range of implemented QI was 3-8 before and 5-9 after the study intervention phase. A maximum of three new QI could be implemented per center (. Fig. 3). The most frequently newly implemented QI were QI VII-perioperative morbidity and mortality report (five centers) and QI V-temperature management (four centers). Conversely, the criteria for QI VI-WHO safe surgery checklist and QI X-physician staffing according to DGAI recommendations were less frequently fulfilled after study completion (. Fig. 4) Fig. 4).

Difficulties during implementation
Each of the 16 centers could theoretically adhere to a maximum of 10 points (10 QI), resulting in a maximum of 160 implementable QI in this study. At the end of the trial 109 QI were reported as implemented. For all of these successfully implemented QI, the centers were asked to report potential difficulties during the implementation phase that could be solved on site. The analysis revealed a total of 64 problems among the implemented QI. The leading problem was poor employee compliance (n = 22), followed by lack of time (n = 19), structural problems (n = 11) and lack of financial resources (n = 8) (. Fig. 5a). Technical problems, such as issues concerning the PDMS, were reported four times. Only seven (43.8%) centers reported that the complete information required was available through routine data collection.
Of the 160 theoretically implementable QI 51 failed to be implemented. For those QI requiring multiple measures for implementation, the reasons for failure of implementation are listed in . Table 2. Here, the centers were asked to explain reasons for non-implementation. Lack of time was reported 32 times and was the main reason impeding QI implementation during this study (. Fig. 5b). Structural reasons inhibited implementation 19 times, followed by technical reasons (n = 15), lack of financial resources (n = 11), poor employee compliance (n = 11) and other reasons (n = 4).
Consistently, the most frequently stated reasons inhibiting implementation were reported for the QI VII-perioperative morbidity and mortality report (n = 27) and the QI IX-postoperative visit (n = 20), which were the least implemented.

Discussion
In this trial the implementation of 10 different QI was proven to be mostly feasible. Each QI could either be implemented in at least one new center or was already implemented in the majority of the centers before. With respect to the limited resources provided within the framework of this trial, this indicates no insurmountable implementation barriers for any QI. The problems reported by the participating centers showed the need for further educational initiatives, research evidence, change in culture and financial and personnel resources for implementation of some QI.
Measuring quality of anesthesia is a challenging task. Many approaches have been made targeting mainly patient outcomes [8]. Recent work has focused on data acquired by healthcare authorities or insurances [14]. In other medical disciplines, such as intensive care medicine, national expert associations have already published comparable QI bundles. The German Interdisciplinary Association of Intensive Care and Emergency Medicine (DIVI) published a set of 10 treatment-focused QI that are already available in the third edition [11]. To the best of our knowledge, the present set of 10 QI, comprising structure, process and outcome indicators on a national level is unique in anesthesia [5].
The centers succeeded to implement new QI, even though the provided support for implementation was limited and all efforts had to be accomplished with the centers' own resources. Nevertheless, only 2 of 10 QI (QI III and QI IV) were implemented in all 16 centres at the end of the study, indicating relevant potentials. Less than half of the participating centers adhered to the criteria of the outcome indicators QI VII-perioperative morbidity and mortality reportand QI IX-postoperative visit. This carries the risk that the majority of the centers are not aware of the detailed postoperative outcome of their patients. This is an alarming result with respect to the Lancet commission on global surgery that stressed the postoperative mortality as a core indicator to improve surgical and anesthesia care that should be tracked in all hospitals worldwide by 2030 [13].
Obviously, some QI are easier to implement than others: The elaboration of a standard operation procedure (SOP, QI I), for instance, is relatively easy to accomplish in a half-year period. Other QI require specific items on the anesthesia records (QI II&VII). The adaptation of anesthesia records already established in clinical routine makes the adherence to these QI far more complex. The implementation of certain QI requires larger financial resources, which could not be provided within the framework of this trial. The QI IX-postoperative visit, for instance, was implemented only in five centers at the end of the study. This may be caused by the fact that for postoperative visits the attending anesthetist either needs to have a protected time after the responsibilities in the operating room or the center must provide additional personnel for this purpose. In times of increasing workload and the need of optimal capacity utilization in operating theaters, additional time for postoperative visits can hardly be allocated without additional resources. This complies with the reported reasons for the hindered implementation in the centers. Usage of the WHO safe surgery checklist, represented in QI VI, is known to reduce mortality [9]. A recent retrospective analysis showed a global completion rate of 79.1% [4]. This study showed that all 16 centers used the checklist in clinical routine and 12 of them achieved self-reported completion rates >95%, which was regarded as complete adherence to QI VI. Surprisingly, 13 centers had already fulfilled the criteria for QI VI in 2017 but 2 centers lost adherence in 2018 due to a new onset of insufficient completion rate, while only 1 center could newly implement the QI. Reasons for this can be manifold, but improvement of the employee compliance and awareness of the importance of complete safe surgery checklists might easily solve this problem. The analysis of problems during the implementation was intended to assess the need for enhanced implementation aids or other support in future. The reasons for non-implementation were questioned in order to detect potential QI, which are inappropriate due to immense implementation barriers. It must be pointed out that the differentiation between both questions was subjective and depended on the commitment of the participants. This is underlined by the fact, that 7 of the 16 centers did not implement any new QI and 1 centre even lost adherence to QI VIII in the course of the study, mostly due to lack of time. Meanwhile, the same QI could be implemented by the other participants. The QI are usually not designed to measure a defined quality only once but to evaluate the development of quality dur-ing several reassessments. The complete adherence to all QI of a predefined set is considered to be the optimum, which can only be achieved under ideal circumstances. Therefore, it is not a deficit of the present QI if some are less implemented than others, but would allow future improvements. The German Institute for Quality Assurance and Transparency in Healthcare (IQTiG), for example, defines the potential for future improvements as a key quality criterion for the development of new QI. A QI that is implemented among all participants is not suitable for differentiation between poor and good quality. The IQTiG even recommends replacing a QI from a predefined set, if it is fully implemented during multiple re-evaluations, even if it is targeting an important issue. It is then considered to be a standard [10].
For future widespread implementation of the QI, a simple accessibility of the required data is important, e.g. data concerning perioperative morbidity and mortality should be preferably available via electronic PDMS. Only 7 of the 16 centers reported routinely recording all necessary information. The other nine centers had to acquire parts of the data manually for the purpose of this study. Thus, the adaptation of anesthesia records and PDMS systems will be an important task for hospitals willing to gain complete QI adherence in clinical routine. Table 2 Reasons for failed implementation of QI consisting of multiple criteria.Overview of the factors, leading to non-implementation of those QI, consisting of more than one criterion. Note: QI III-incidence management consisted of two criteria, which were implemented in all centers and is hence not shown in the table It has been pointed out that feeding the gained data back to the care providers is paramount for the improvement of care [1]. Feedback reports have been shown to be an effective strategy for successful implementation of QI in studies mostly targeting the sector of internal medicine [3]. Comparative performance reports could be a next step for hospitals to estimate their individual performance, as shown by the example of performance reports among British anesthetists [2]. Subsequent to this work, it is planned to promote the broad usage of the QI among German hospitals in clinical routine. As a first step, it is planned to inaugurate a website providing benchmarking tools for hospitals willing to analyze and improve their adherence to DGAI's QI. This medium could also be used to provide comparative feedback data to participating hospitals.
Future investigations should address whether implemented QI of anesthesia care effectively help improving patientcentered outcomes. This proof would facilitate the acceptance and dissemination of QI among anesthetic care providers.

Limitations
This study was designed to evaluate the ability to implement the 10 anesthesia QI with limited resources. The support provided within the study framework was limited to direct implementation aids. The progress of the implementation itself depended solely on the commitment of the involved investigators. Financial or technical support could not be provided during this study to solve structural or technical barriers. An on-site monitoring was not feasible due to the limited resources. In order to disseminate the QI in further hospitals, the authors would recommend using a peer review process. Furthermore, this trial was not designed to evaluate the effect of QI implementation on the outcome of patients.

Conclusion
Implementation of the 10 QI is mostly feasible. The implementation of the QI is a suitable tool to establish a new quality target for German anesthesia departments. Comparative data from the centers are needed for benchmarking between hospitals.