From Postpartum Haemorrhage Guideline to Local Protocol: A Study of Protocol Quality

Objective Postpartum hemorrhage (PPH) has a continuously rising incidence worldwide, suggesting suboptimal care. An important step in optimizing care is the translation of evidence-based guidelines into comprehensive hospital protocols. However, knowledge about the quality of these protocols is lacking. The objective of this study was to evaluate the quality of PPH-protocols on structure and content in the Netherlands. Methods We performed an observational multicenter study. Eighteen PPH-protocols from 3 University Hospitals (UH), 8 Teaching Hospitals (TH) and 7 Non-Teaching hospitals (NTH) throughout the Netherlands were acquired. The structure of the PPH-protocols was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE-II) Instrument. The content was appraised using previously developed quality indicators, based on international guidelines and Advance-Trauma-Life-Support (ATLS)-based course instructions. Results The quality of the protocols for postpartum hemorrhage for both structure and content varied widely between different hospitals, but all of them showed room for improvement. The protocols scored mainly below average on the different items of the AGREE-II instrument (8 of the 10 items scored <4 on a 1–7 scale). Regarding the content, adoption of guideline recommendations in protocols was 46 %. In addition, a timely indication of ‘when to perform’ a recommendation was lacking in three-fourths of the items. Conclusion This study shows that the quality of the PPH-protocols for both structure and content in the Netherlands is suboptimal. This makes adherence to the guideline and ATLS-based course instructions difficult.


Significance
Postpartum hemorrhage (PPH) remains a major problem in high resource countries, regardless of the development and dissemination of evidence-based clinical guidelines including the instructions on Advance Trauma Life Support (ATLS)-based courses for obstetric emergencies. Putting evidence-based PPH recommendations into practice begins with the translation of evidence-based guidelines into high quality local protocols. For many care providers these protocols often are the only guide in the prevention and management of PPH in the actual care. However a recent study (Bialit et al. AJOG 2015) showed that merely the presence of PPH-protocols does not indicate a better outcome. Variation in the quality of these protocols could be a possible explanation. This quality and its variation both regarding structure and content is yet unknown. This manuscript gives insight in an underex-

Introduction
Postpartum Hemorrhage (PPH) is the number one cause of worldwide maternal death [21]. It does not only have its origin in low resource countries, but developed countries also contribute [13,9,22,32]. A high proportion (72-90 %) of the morbidity of obstetric hemorrhage is considered to be preventable if adequately managed through early recognition, adequate interventions in early stages and proper choices of therapies [8,12,5]. Actually, PPH-care consists of a prevention phase and a treatment phase, where different actions must be taken by different professionals, consecutively or simultaneously, in a limited time-frame, for PPH can develop into an urgent lifethreatening situation that requires an immediate response [20].
Evidence-based guidelines can assist professionals in standardizing adequate management and support the clinical evidence-based decision making [17]. Advanced Trauma Life Support (ATLS) courses educate the professionals in using a highly structured multidisciplinary approach of obstetric emergencies such as PPH [20]. Streamlining day-to-day PPH-care for every professional on the basis of evidence-based PPH-guidelines and ATLSbased course instructions is a challenge [17]. Several national societies of maternal-fetal-medicine [18,15] strongly recommend the use of protocols as a way to streamline PPH-care, because compliance of guidelines improves if a protocol is present [11,24,26]. In fact, for the majority of the professionals, such as nurses, midwives and residents, these protocols are the main guide in the prevention and management of PPH. However, a recent study showed that merely the presence of PPH-protocols does not mean a better outcome [4]. Variation in the quality of these protocols could be a possible explanation. This quality and its variation, both regarding structure and content, is yet unknown. Therefore, we aimed to evaluate the quality of PPH-protocols, both on structure and content, in the Netherlands.

Design, Setting and Study Population
We performed an observational multicenter study. The study was established within the Dutch Consortium for Healthcare Evaluation in Obstetrics and Gynecology. This Consortium aims at extending evidence-based medicine in obstetrics and improving the quality of the Dutch obstetric care. Nowadays all ten Dutch Perinatology Centers participate in this Consortium, together with 70 Dutch general hospitals. A viable selection of 1:5 of the Dutch hospitals was made and a total of eighteen (23 %) PPH-protocols from these Dutch hospitals that provide acute obstetric care were collected from February 2011 through February 2012. The selection of hospitals was based on the different types of hospitals [University Hospitals (UH), Teaching Hospitals (TH) and Non-Teaching Hospitals (NTH)], with a similar distribution by type across the country (3 UH, 8 TH, 7 NTH). The obstetrician of these hospitals was contacted through e-mail or telephone with the question to send us a copy of their most recent local PPH protocol, and all the hospitals willingly provided us with a copy.

Assessment of Protocol Quality
To evaluate the quality of the included protocols on structure and form, we used the Appraisal of Guidelines for Research & Evaluation (AGREE-II) instrument [2]. This instrument offers a systematic framework for assessing the most important aspects of quality of guidelines. We selected the following 10 from 23 scoring items for assessing form and structure of the protocols: objective, title with health questions and patient population (domain Scope and Purpose), publication date, revision date, externally reviewed yes/no and references (domain Rigor of Development), authors and target group (domain Stakeholder Involvement) and use of appendices/tools (domain Applicability). The remaining 13 items of the AGREE-II instrument were rated as unsuitable for assessing the protocols because they particularly relate to the process used to gather and synthesize evidence, cost implications and editorial independence. The AGREE-IIitems were scored on a 7-point scale from ''totally disagree'' to ''totally agree'' (score 1-7).
To evaluate the quality of the included protocols on content according to PPH-guidelines and the ATLS-based course, we used guideline-based quality indicators for prevention (n = 2), management (n = 15) and organization (n = 5) of PPH (Table 1) [30]. These indicators were previously developed according to the RAND-modified Delphi method to measure guideline adherence in the actual care, and are based on different international PPHguidelines, including the guidelines from the World Health Organization (WHO), international literature and ATLSbased courses [20,15,3,28,31]. The indicators for management of PPH were classified into three subsequent stages of seriousness of PPH, in terms of the amount of blood loss and/or signs of shock, namely: 1. [500 mL, 2.
[1000 mL or [500 mL with signs of shock, and 3.
[2000 mL. This set can be used to measure the actual performances and whether the performances are carried out in the right stages of blood loss.
Twenty (from 22) indicators were relevant to assess the content of protocols and we transformed them into 92 measurable items. All protocols were scored on the presence or absence of these items. In addition, items regarding 'actions in the management of PPH' were evaluated on whether they were accompanied by a description of 'when' (in terms of the amount of blood loss or vital signs) these actions would have to be taken. For example; it is recommended to place a second drip in the event of more than 1000 mL blood loss.
Two independent researchers performed all measurements.

Statistical Analysis
With regard to the structure of the protocols we calculated median scores per AGREE-II domain, for all the hospitals together and per type of hospital. The results regarding content were analyzed descriptively. At first a total score was calculated, meaning the sum of all present items (Y) in the 18 hospital protocols divided by the maximum amount of items (Y/92 X 18). Subsequently, frequencies per item per type of hospital were assessed. Cohen's kappa was calculated to measure conformity between the two assessors (HvV and FM) and totaled 0.9 for both structure and content measurements. All measurements were analyzed using SPSS 20,0.

Results
The quality of the analyzed protocols differed substantially for both structure and content.
Regarding the structure of the protocols the length of the total protocol varied, for example from half a page to five pages, and the presence of headlines and paragraphs varied, as well as the presence or absence of a flowchart. The presence of appendices/tools in the domain ''Applicability'' had a median score of 3 [ranging from 2 (TH and NTH) to 3 (UH)] ( Table 2).
With respect to the domain ''Scope and Purpose'' a clear title with health question was found in all protocols (median score of 7), unlike the item 'patient population', which was predominantly absent (median score of 2). Items in the domains ''Stakeholder Involvement'' and ''Rigor of Development'' did not score well in almost all protocols, in particular those from the TH. From all these items, the item 'publication date' scored best [median score of 5, ranging from 3 (TH) to 7 (NT)].
Overall, the scores on the different items on the AGREE-II instrument were mostly below average, e.g. eight of the total of ten items scored below four on a scale of one to seven.
Regarding the content of protocols about half (46 %) of the total number of 92 items could be found in 18 protocols ranging from 20 % in a NTH to 68 % in a UH (Table 3). Below we present the main results for the different stages of PPH-care.

Prevention of PPH
Recommendations concerning identification of high-risk patients during labor were found in 33 % [ranging from 29 % (NTH) to 38 % (TH)] of the protocols. Active management of the third stage recommendation was included in 22 % of the protocols [ranging from 0 % (TH) to 43 % (NTH)].

Management of PPH
Recommendations for continuous monitoring the vital parameters, e.g. pulse, O2-saturation and blood pressure, were included respectively in 6 % [ranging from 0 % (UH and TH) to14 % (NTH)], 11 % [ranging from 0 % (UH) to 14 % (NTH)] and 28 % [ranging from 13 % (TH) to 43 % (NTH)] of the total protocols. In all protocols was stipulated that cross-match blood has to be taken and in almost all protocols, except for one TH, that packed cells should be ordered. However, 11 % of the protocols mentioned to in a serious situation give O-negative blood in the absence of cross-match blood, with a range of 0 % in the UH to In an emergency situation to temporarily stop bleeding and catch up resuscitation, organize the next intervention or transport patient to a tertiary centre: -perform: bimanual compression of the uterus, aorta compression and place Bakri balloon or uterine tamponade through packing (also therapeutically) Organizational indicators for PPH: In every hospital system…

Protocols and agreements
The following local protocols and agreement should be available Protocol PPH according to the national guideline Local mass transfusion protocol Protocol for women refusing blood products A written agreement between the related disciplines (anesthesia, hematology, radiology) for a multidisciplinary approach in the treatment of PPH

Time Factor
Of 92 items, 61 indicated at what stage (expressed in the amount of blood loss or shock signs) action should be taken. Of the items that should be performed at the stage of 500-1000 mL blood loss, only 24 % gave an indication of when or under which circumstances these had to be undertaken (Fig. 1). Unfortunately, 76 % of these items were either mentioned with an incorrect time indication (too much blood loss) or without any time indication, or were not mentioned in protocols at all. This also counts for the next stages

Discussion
This study shows a large variation between hospitals in the quality of protocols for postpartum hemorrhage as regards both structure and content. The protocols scored mostly below average on the different items of the AGREE-II domains [8 out of 10 items scored below 4 (1-7 scale)]; protocols of the TH in particular scored lower (9 out of 10  items scored below average) compared with the UH and NTH (6 out of 10 items scored below average). Regarding the content, less than half (46 %) of the total number of 92 items were found in the 18 PPHprotocols. The content of the protocols of the NTH was the least in accordance with the guideline and ATLSbased course instructions (33 % NTH, 48 % TH, 55 % UH). Furthermore, as regards items that needed a time indication on 'when to perform', about three-fourths of these items were mentioned with either an incorrect time indication (too much blood loss), no time indication at all, or were simply not present in the protocols. So, the overall quality of protocols showed much room for improvement. A limitation of this study is, that we did not relate the quality of the protocols to the compliance with the guidelines in the actual care. It is possible that the current care is more in accordance with the guidelines than we now assume based on the protocols. It is possible that the variation in quality found could explain the findings of Bailit et al. [4] that the presence of protocols does not improve care as a rule. Therefore, to measure the current care will be the next step. A second limitation is that we used the Agree-II instrument which is meant to be used for Guidelines. However, a quality instrument for local protocols does not exist, and because local protocols are based on guidelines, the Agree-II instrument is the best instrument which met this purpose.
The strength of our study, however, is that we investigated the quality of local PPH-protocols, including both structure and content. Until now, the few studies that were performed regarding protocols only concerned the presence or absence of protocols, not the quality thereof (see below).
Specific omission of highly relevant clinical items could lead to a delayed recognition and treatment of PPH by the immediate care providers using the protocols. In our study, highly relevant clinical items in the prevention of PPH, such as identifying a high-risk patient, active management of the third stage and monitoring blood loss in high-risk patients, were only present in respectively 11, 22 and 67 % of the protocols. Furthermore, in the management of PPH, to monitor vital signs in case of a PPH was only found in less than one-third of the protocols. Delay and denial are key contributors to poor outcome in PPH while prevention and early recognition of PPH provide better results [8,12,5,18,6,25]. A risk assessment of the outpatientclinic patients, which helps identify high-risk patients, will increase vigilance of the staff and the taking of extra precautions when necessary. An active management of the third stage, as is strongly supported by evidence, diminishes blood loss [6,27]. Moreover, proper management of PPH includes analyzing maternal status for early recognition through accurate estimation of blood loss, vital signs monitoring and prompt intervention in the early stages using a rapid and adequate multifaceted approach [12,18]. Different international guidelines highlight the evaluation of vital signs and recommend more accurate management for PPH if blood loss causes changes in vital signs [15,28,7]. Omission of these items in protocols may be a factor for improper management of PPH; in our study only one out of 18 protocols suggested to monitor the pulse rate continuously if a PPH occurs. Certainly, it is arguable that these factors are a part of common knowledge and practice, but, the direct care providers in the PPH-care and therefore the ones who are responsible for the prevention and early recognition of PPH are usually the ones with the least experience, especially in the TH. Besides, midwives and nurses who are the professionals primarily responsible for ensuring patient safety, work mainly protocol based and use these protocols as their written source of knowledge and guidance in the daily care [26]. Therefore, it has to be clear for the direct care providers dealing with PPH, which acts must be performed at what amount of blood loss and at what condition the patient in. Unfortunately, only one-third of the items with a time indication were correctly described in the protocols.
In literature little is found about the incorporation of guideline-recommendations in protocols. Cromwell et al. [14] shows that only 20 % of all protocols in Great-Britain took over all recommendations of the national guideline ''Group B Streptococcus''. In our study, we see a similar trend. Lack of familiarity with a guideline's content, with the relevant research literature, disagreement with the guideline's interpretation of the literature, but also the  ways in which recommendations are formulated are reported factors for not adopting guideline-recommendations [17,16]. Cameron et al. [10] investigated current Australian practice in the development of a local policy with regard to prevention, early recognition and management of PPH. They found time and staffing issues to be significant barriers to local policy development from guidelines, especially the deficiency of skills and experience needed to develop written protocols. It is known that not only the content is important for delivering proper care, but the protocol must be feasible and have a clear structure for the direct caregivers as well [19]. Improved compliance with protocols is found if there are comprehensive protocols, especially if nurses are involved in the development of these protocols [1]. In our study the structure of the studied protocols differed greatly whereas the TH scored lower than the other two types of hospitals.
In order to improve adoption of guideline-recommendations and not to 'keep reinventing the wheel', guidelines should come up with a template or model protocol with a clear format, better structuring and with all the important guideline-recommendations that can easily be adapted to the local situation [10,19]. This template could be accompanied by additional materials such as a summary document, flowcharts educational tools, patient leaflets, or computer support for improving compliance with the protocols and therefore the guideline. It is known that the WHO has presented the recommendation as a list to be followed in case of a PPH and the FIGO has prepared a prevention and management protocol for PPH [29,23]. Despite the fact that these two guidelines mainly focus on low resource countries, they could be adopted by other countries as well.

Conclusion
This study shows that the quality of the local PPH-protocols for both structure and content is suboptimal, especially the adoption of guideline-recommendations in protocols. This makes adherence to the guideline and ATLS-based course instructions difficult. It is possible, however, that the current care is more in accordance with the guideline than we now assume based on the protocols. Therefore, to measure the current care will be the next step. In the future more attention and assistance is needed to ensure the quality of protocols, for example by adding a standard protocol template, flowcharts and checklists to PPHguidelines.