Introduction

In 2020, we presented a 13-year single-center observational cohort study including patients undergoing cranioplasty after decompressive craniectomy [1]. Similar to many published studies of comparably sized cohorts, post-operative complications occurred in 23% of patients, which makes this high rate of complications a serious point of attention [1,2,3,4,5,6,7].

As with many surgeries, the surgical method of cranioplasty is not standardized and surgeons perform their procedures in different ways. This makes it difficult to assess whether the way in which the surgery is performed affects the complication rate. Performing the surgery in a standardized way can help identify what parts of surgery improve or worsen the outcome and can serve as a benchmark for further changes. Uniform, standardized working procedures alone have been shown to result in less complications [8,9,10]. Creating uniformity in surgical method can be done by implementing a Standard operating procedure (SOP), in which specific parts of a surgical procedure are standardized.

Reaching consensus on surgical methods can however prove to be very difficult, especially if the level of evidence about different (peri)operative strategies is limited. In this Update section, we describe how consensus may be reached within a group of experts/neurosurgeons and how to create (or update) a SOP, inspired by the Delphi method.

Delphi method

Neurosurgery is a medical specialty with ancient roots. For example, the medical application of trephination started in the Greek era where the neurosurgical oracle called Hippocrates described the procedure in detail [11]. Like many (neuro)surgical procedures, a rich history of passing on knowledge from oracle to student forms the base of nowadays common practice (expert opinion). Randomized controlled trials are often lacking, i.e. due to a low volume of cases within the neurosurgical field, making it difficult to reach adequate statistical power. With the passage of time more oracles arise, sometimes even spreading their different prophecies within the same institution. Considering this variety, how does one reach consensus and create uniformity?

To address this issue, the answer is, again, to be found in ancient Greece. In the 1950s, Norman Dalkey and Olaf Helmer developed a method to find consensus within a group of experts on health issues and topics where minimal information or agreement prevails [12, 13]. They called it the Delphi method, named after the ancient Greek Oracle of Delphi: using (typically) three rounds of questionnaires, with a moderator collecting and processing the output of multiple experts, until final consensus is reached within the group. Anonymity in combination with the ability of experts to respond to and revise their answer in view of the previous responses from other panel members, makes this method a successful tool to create consensus [14, 15]. However, in its original form, the method is a time-consuming procedure, lowering the adherence to questionnaire rounds and thus the chance on successful outcomes useful in healthcare. Fortunately, new technological tools can help overcome these problems.

Live audience engagement platforms: a modified Digital Delphi method

Through ‘live audience engagement platforms’, an online meeting can be arranged in which the moderator presents a questionnaire to a group of experts. Each expert registers his/her presence at the start of the meeting using a smartphone or computer, after which they can anonymously fill in their answers and comments to the questionnaire. Once everyone has responded, the group results can directly be presented, for example in graphical form, and discussed with the group during the online session. This process can be repeated until consensus is reached. This platform type optimizes time management, while the anonymous and interactive characteristics of the Delphi Method are maintained. Examples of platforms are AhaSlides (AhaSlides PTE. LTD., Singapore), Glisser (Glisser LTD, London, United Kingdom), and Menti (Mentimeter, Stockholm, Sweden).

In the next subheadings, we give an overall, step-by-step explanation how to apply this modified Digital Delphi method, from collecting information on surgical methods to the actual implementation of the consent-based protocol. Table 1 shows the key phases and responsible persons for each step within this method. A real-life example is presented in the box below.

Table 1 Flow-diagram showing key phases involved in modified Digital Delphi method to create a consensus-based Standard Operating Procedure (SOP)
Full size table

Phase 1: preparatory steps

1.1 State scope of SOP, appointment of moderator

First, the scope and extent of the SOP should be stated beforehand. Second, a moderator should be appointed. This person does not have to be an expert on the topic but preferably someone who has some experience with the surgical procedure. For example, this can be a resident in training, since a resident typically experiences and observes the variety in surgical methods between attending (neuro)surgeons.

1.2 Problem identification: determine current surgical methods, evaluate literature

The moderator creates an overview of the steps of the surgical procedure by assessing (the differences in) practice among the experts, and current literature. In case an SOP already exists, the SOP should be evaluated by the moderator and compared routinely with the newest insights as described in literature, for example every year.

1.3 Select group of experts

It is important to clearly define characteristics of those who will be part of the expert group. Since a heterogenic group is more likely to cover all aspects and opinions related to a certain topic, it is advised to include all stakeholders/experts who have knowledge and affinity with the topic. Afterward, or during the session, subgroup analysis can be performed, based on certain characteristics, such as the years of (surgical) expertise or function (i.e., type of physician, (scrub)nurse).

1.4 Develop questionnaire

Subsequently, based on the previous steps, a questionnaire is developed by the moderator. In this questionnaire, points of attention resulting from evaluation of current practice, the current protocol (if applicable) and literature are grouped in overarching themes, and translated into questions to investigate how to deal with them. This questionnaire and literature (if applicable) should be sent to the group of experts before starting the consensus meeting, so the experts will have time to form their opinion on the subject. Furthermore, they will be able to revise the questions or answer possibilities (if needed) to make them clinically more relevant. (Fig. 1, modified Digital Delphi round 1).

Fig. 1
figure 1

Flow chart describing a modified Digital Delphi method to generate and implement a consensus-based SOP

Full size image

Phase 2: answer questionnaire and discuss topic during an interactive session

2.1 Defining consensus

After the questionnaire is sent to the group of experts, the next phase is to try to reach consensus by collecting and discussing the experts’ opinions during an interactive session. Before take-off, consensus criteria should be defined, for example an agreement of ≥ 80% within the group of experts [16, 17]. In case no consensus can be reached during the meeting, answer options can be modified or made less detailed to explore whether consensus could be reached on a more general level. In addition, one may choose to repeat the topic (with adjustments made) in a next meeting. Ultimately, one may choose to retain a certain point from the current SOP if no consensus can be reached.

2.2 Live audience engagement meeting

An ideal method to gather audience responses quickly and to subsequently discuss the answers within one meeting, is the use of live audience engagement platforms. Using this platform type, each expert can give her/his answer to a question, for example using their smartphone. After the answers are given, the (anonymous) results can be shown directly, for example in graphical form. Written commentary can be added to the answers anonymously to aid the discussion. The following discussion can be recorded, and experts are able to explain and motivate their answer. After discussion, the same question is repeated until consensus is reached. (Fig. 1, modified Digital Delphi round 2) Questions can be answered and discussed one-by-one, or, if questions have a more consecutive order, one may choose to first answer all (or a subgroup of) questions before revealing the results. In addition, questionnaires and responses can be exported to datasheets for further analysis if needed.

Phase 3: presentation and implementation of the consensus-based protocol

3.1 Creating an SOP

After the live audience engagement meeting, results obtained from the questionnaire and discussions can be analyzed and agreements incorporated into a new SOP. The draft SOP is sent to the same group of experts and discussed in an online meeting to ensure expert group’s agreement. If agreement still exist, the surgeons pledge to adhere to the (updated) SOP, after which the SOP can be used for clinical routine. (Fig. 1, modified Digital Delphi round 3).

3.2 Evaluation of the new SOP

Evaluating the newly implemented SOP is important (to assess effectiveness) and can be done at several levels. For example, patient data should be recorded prospectively to evaluate whether the applied changes result in less complications. In addition, stakeholders can be interviewed to assess whether the newly implemented SOP is executable. At the next periodic SOP revision, these data should be analyzed by the moderator in the preparatory phase.

Discussion

In this update, we explain how to reach consensus within a group of experts on healthcare-related problems where minimal evidence exists, using live audience engagement platforms. With this modified Digital Delphi method, one is able to create and implement SOP in clinical practice within a short period of time.

Standardization of surgical procedures is an important tool to increase efficacy and safety and has gained increased interest in recent years [18,19,20]. For example, implementation of SOP for cerebrospinal fluid shunt surgery reduced the rate of shunt infections significantly [8, 10]. The importance of creating and implementing SOPs has been explained in detail by Buis et al. [9]. They conclude that SOPs may lead to improved outcomes after surgery, at the price of some loss of individuality. Although we still agree with this statement, we think that this modified Digital Delphi technique retains the feeling of ownership, since the specialist’s opinion forms an integral part in the process, probably increasing protocol compliance. Moreover, feeling of ownership improves performance and satisfaction [21, 22].

There are some points of discussion on the methods we have applied: first, how does one define consensus? In our approach, we choose ≥ 80%, but this is arbitrary [16, 17, 23, 24]. Ideally, one may perform a sensitivity analysis investigating the influence of the consensus threshold on medical decisions. Second, to increase the compliance and usefulness of the newly formed SOP, we think it is important not to make too many changes at once, and keep the text short and simple, as addressed by Buis et al. [9], and seen in other aspects of quality improvement in healthcare [25]. This may be challenging, since we think it is also important to form a complete instruction guide to make it interpretable for (less experienced) trainees as well. To solve this problem, one can choose to divide the SOP into a few short and straightforward agreements upon which consensus was reached, and add in a separate section a more detailed (perioperative) guideline. Third, we modified the original Delphi technique since we think this a fast and pragmatic method to reach consensus, supported by readily available software. However, there are more described modifications to the original method, such as the ‘real-time Delphi’ or ‘policy Delphi’ [16, 26].

This method has not been described, studied or validated yet and some limitations should be addressed since discussions on the answers to the questionnaires are held directly, discussions could be dominated by verbally strong people or people higher in professional rank. This could influence other stakeholders to a greater extent compared to the original Delphi. This drawback is diminished by first writing arguments anonymously in the online environment, or repeating questions before discussion. Another drawback is that all stakeholders should be online at the same moment for the online discussions. Fourth, it is of great importance to register whether the implemented agreements are executable and lead to the desired outcomes, since only then the usefulness can be determined, and future improvements be made.

In the forthcoming years, with this method, we hope to contribute toward effective and safe consensus-based healthcare, despite lack of high-level evidence.

Conclusion

Standardized, evidence-based care results in less complications. However, it can be challenging to reach consensus on topics where no or minimal evidences exist. In this update, we describe a method how to reach consensus within a group of experts through live audience engagement platforms, consensus can be reached in a short time, while the anonymous and interactive characteristics of the Delphi method are largely maintained.