FormalPara Key Points for Decision Makers

It is feasible to develop a brief patient decision aid prototype to be used to facilitate shared decision making in the consultation and also meets the International Patient Decision Aid Standards qualifying and certifying criteria.

Several key characteristics of Cochrane Reviews make it easier and some circumstances make it more relevant for creating consult decision aids.

Developers need to balance providing adequate information on the options, benefits and harms while keeping the consult decision aid brief enough that it can be used in clinical practice.

1 Background

Health policy in several countries calls for using patient decision aids to facilitate shared decision making, provide more patient-centered care, enhance health literacy, and improve patient safety [1–4]. This demand for patient decision aids is part of a larger paradigm shift from doctor-centric decision making to shared decisions with patients which is achievable if effective implementation interventions such as patient decision aids are used [5]. Patient decision aids make explicit the decision, provide high-quality evidence on the outcomes of options including watchful waiting, and help patients clarify what is important to them based on their personal circumstances [6]. Longer formats have been developed and used by patients in preparation for the consultation with briefer formats used between patients and clinicians during the consultation. Compared with usual practice, patients exposed to decision aids have higher decision quality, increased participation in decision making, and lower personal uncertainty concerning the decision to be made (i.e., decisional conflict) [6]. Those decision aids designed for use in the consultation are more likely to facilitate shared decision making discussions between patients and their clinicians [6, 7]. Despite strong evidence on the effectiveness of patient decision aids, there are only about 300 up-to-date ones publicly available for a limited number of health conditions and most are restricted to English [8].

Evidence from Cochrane systematic reviews could be used to develop decision aids to inform patient-level decisions and reduce waste in health care by creating tools that promote the integration of research into everyday clinical practice [9]. There are over 6000 Cochrane Reviews that provide high quality evidence, free from industry interests, on the benefits and harms of options [10, 11]. Most reviews provide probabilities that can be used to communicate the chances of both benefits and harms for options; this is known to improve patients’ realistic expectations [6]. Another advantage to using evidence from Cochrane Reviews is the commitment to updates every 2 years which can ensure patient decision aids are consistent with the latest evidence [12]. As well, plain language summaries, available in multiple languages, provide a simpler description of the health condition and evidence from the Cochrane Review [6, 11]. However, current Cochrane Reviews do not present the evidence in an interactive format that meets the definition of a patient decision aid and are not in a user-friendly format to help patients with understanding the complex probabilistic nature of the evidence [10]. Current formats contain the full Cochrane Review, scientific abstract, and consumer summary. We argue that translation of Cochrane Review evidence into clinical practice would be greatly enhanced if it was embedded in patient decision aids.

2 Aim

Our overall aim was to translate evidence from Cochrane Reviews into a format that can be used to facilitate shared decision making during the consultation, namely patient decision aids. The specific objectives were to (a) develop and evaluate a prototype for a consult decision aid that is acceptable and usable; (b) apply the prototype to develop a series of 12 consult decision aids; and (c) identify considerations for embedding consult decision aids in Cochrane Reviews.

3 Methods

3.1 Study Design

We used a systematic development process that was guided by the Ottawa Decision Support Framework and the International Patient Decision Aid Standards (IPDAS) [11, 13, 14]. The process involved (a) establishing an expert committee of stakeholders; (b) developing the prototype for a consult decision aid; (c) conducting alpha testing with potential users; (d) applying the prototype to 12 Cochrane Reviews; and (e) reaching consensus on considerations for including consult decision aids in a Cochrane Review. Ethics approval was obtained from the Ottawa Hospital Research Ethics Boards (20130926-01H).

3.2 Theoretical Frameworks

We used IPDAS and the Ottawa Decision Support Framework. The IPDAS Collaboration established an evidence-informed framework for enhancing the quality and effectiveness of patient decision aids [15]. Over 100 stakeholders (e.g., researchers, policy makers, patients, clinicians, patient decision aid developers) from 14 countries reached consensus on 74 items in the IPDAS checklist [10]. The IPDAS checklist was transformed into a 47-item instrument [16]. More recently, IPDAS criteria were triaged into proposed qualifying (6 items), certifying (6 items plus 4 items for screening decisions), and quality criteria (28 items) [11]. For example, criteria for qualifying as a patient decision aid are explicitly stating the decision, providing information on the health condition and options, describing the positive and negative features of options, and helping patients clarify their values.

The Ottawa Decision Support Framework is an evidence-based mid-range theory for supporting patients making health or social decisions [14, 17, 18]. It was informed by theoretical concepts from psychology, decision analysis, decisional conflict, social support, and economics. This framework is most commonly used for developing and evaluating patient decision aids [18]. It has also been used to develop clinician decision support resources and tools to evaluate the quality and outcomes of providing patient decision aids [18, 19].

3.3 Stakeholder Committee

Our committee had a range of stakeholders including a consumer with regular exposure to a variety of consumers from different settings, clinicians (e.g., nurse, primary care physician, and rheumatologist), information technology specialists, and health services researchers with expertise in patient decision aid development and evaluation, knowledge translation to patients, shared decision making, systematic reviews, and web-based applications.

3.4 Consult Decision Aid Prototype

We developed a prototype for a patient decision aid that could be informed by evidence taken directly from Cochrane systematic reviews. Cochrane Reviews were chosen because the Cochrane Library has the largest international set of systematic reviews providing a robust source of regularly updated evidence free from industry interests. The patient decision aid was titled a consult decision aid given a brief 1-page format designed for facilitating discussion in the consultation. The prototype includes (a) a title that specifies the decision; (b) information on the health condition and the eligible users; (c) a summary of the options being considered; (d) evidence on the benefits and harms of each option with probabilities (evidence quality rated using GRADE); (e) an explicit values clarification exercise; (f) a question soliciting the preferred option; and (g) the SURE Test to screen for decisional conflict and thus identify decision making needs (Fig. 1). The prototype has several hyperlinks to additional information including definitions, graphic representation of probabilities, and references. The consult decision aid meets all IPDAS qualifying and certifying criteria except the criterion for using an implicit values-clarification method requiring detailed descriptions of the options and outcomes [11]. Instead, given there is “no single ‘best’ practice” for values clarification [20], an explicit values-clarification method was included using a rating scale. Probabilities are displayed using words, numbers and diagrams: formats known to be easier for patients to understand [21, 22].

Fig. 1
figure 1figure 1

Consult decision aid prototype indicating IPDAS criteria

Full size image

3.5 Alpha Testing with Potential Users

The consult decision aid prototype was populated with evidence from a Cochrane Review focused on alendronate as a treatment option for osteoporosis (Appendix, see electronic supplementary material). Alpha testing of this consult decision aid on alendronate was conducted with members of the Cochrane Consumer Network (potential patients) and staff from the Cochrane Editorial Unit to determine acceptability and usability (Table 1) [13]. Cochrane Consumer Network members and the Editorial Unit staff are familiar with Cochrane protocols and reviews but unlikely to have had previous exposure to values clarification exercises and tools to support decision making. Eligible participants needed to be able to read English and have computer access. Participants were interviewed in person or via Skype. During the audio-recorded interviews, the research assistant guided the participant through the consult decision aid in a manner similar to which it could be presented in a consultation and then asked questions using a structured interview guide. Prior to the interview, participants received a copy of the consult decision aid and interview guide. They were told not to review it before the interview and that it would be used during the interview.

Table 1 Demographic characteristics of participants (n = 14)
Full size table

The interview guide had seven questions from the Preparation for Decision Making Scale [23], ten questions from the acceptability questionnaire that has been extensively used for evaluating patient decision aids during development [24], and four common questions for usability testing [25] (Table 2). The Preparation for Decision Making Scale, previously validated, was shown to have high internal consistency with alpha coefficients of 0.92–0.96, and discriminated between patients who reported the patient decision aid was or was not helpful [23].

Table 2 Consult decision aid acceptability and usability
Full size table

Participant quantitative responses were entered into an Excel database and analyzed descriptively using frequency distribution. Audio recordings of interviews were transcribed verbatim and qualitative comments were analyzed using content analysis. More specifically, the transcript was read line by line to identify common themes that participants identified as issues affecting acceptability and/or usability of the patient decision aid. The research team reviewed findings and revised the prototype.

3.6 Applying the Prototype to Cochrane Reviews

The revised prototype was used to create a series of 12 consult decision aids populated with data from 12 different Cochrane Reviews. Eligible reviews had a summary of findings table and included two or more reasonable options (that may have included no treatment/screening). A clinician from the stakeholder committee with expertise in patient education reviewed the description of the health condition and options and verified accuracy of the probabilities presented. As well, the clinician verified use of plain language. Time to create the consult decision aids was monitored.

3.7 Considerations for Embedding a Consult Decision Aid in a Cochrane Review

Using a Delphi process [10], the stakeholder committee was tasked with reaching consensus on the considerations necessary for selecting which Cochrane Reviews should have an accompanying consult decision aid. This Delphi process involved three iterations: (1) in the first round, we gathered suggestions based on our experiences developing patient decision aids and creating the series of 12 consult decision aids; (2) in the second round, committee members individually rated each consideration on a scale from 1 to 9 in an Excel spread sheet (1 not important, 9 extremely important) [26]; and (3) in the third round, committee members discussed ratings having median differences of three or more points and reached consensus on those that were rated as important (e.g. rated as 7–9) [27].

4 Results

4.1 Participant Characteristics

Eleven members of the Cochrane Consumer Network and three staff from the Cochrane Editorial Unit consented to participate (Table 1). Two were male and 12 were female; ages 30–60 years old with a range of education levels (e.g., high school only to university graduate degrees). Participants were employed (e.g., health professionals, managers), retired, or unemployed with several receiving disability insurance. Although not required, ten had previous experience (personal, professional or family member) with the health condition in the consult decision aid (i.e., osteoporosis).

4.2 Acceptability and Usability

Most participants rated the details provided in each section of the consult decision aid as good or very good (Table 2). Ten of 14 rated the presentation of information as balanced and four rated it as slightly slanted toward taking treatment. Twelve participants thought the words made sense and icons were understood. Regarding information content, seven indicated that there was the right amount, six wanted more information, and one indicated there was too much. On the Preparation for Decision Making scale, most participants indicated it was helpful for preparing to make decisions with the clinician (Table 3). Thirteen participants indicated that it was suitable for use in consultations with their clinician and were willing to use it and/or tell someone else about it (Table 2). One consumer participant consistently rated the questions about acceptability and usability as low (Table 2); describing inadequate time with the clinician in the consultation to be able to use the consult decision aid as the main reason influencing his responses.

Table 3 Participants’ ratings on the Preparation for Decision Making Scale [23]
Full size table

4.3 Qualitative Feedback with Actions Taken (Table 4)

The suggestion to use more plain language (e.g., decline, contraindication, bisphosphonate, harm–benefit ratio) was addressed by changing to simpler words. Where words could not be changed, a link to the definition was added (e.g., best estimate, GRADE, osteoporosis). Topics/questions for patients to ask their clinician were added including safety with other drugs, long-term use of drugs, costs, changes in side effects over time, how to manage side effects, and allergies. Although the font was described as too small, an electronic version would allow the font size to be scalable. The font size in the prototype was chosen to ensure a one-page format when printed. More white space was added. The X icon used for the option of declining the active option was described as too strong and was changed to a size more consistent with other icons. Suggestions for using the consult decision aid in the consultation were developed (see Box 1).

Table 4 Summary of qualitative findings on the consult decision aid about alendronate as a treatment option for osteoporosis
Full size table

Some suggestions in qualitative comments were not addressed. For example, in step 2 we did not add a full list of options because the options were limited to those described in the Cochrane Review. If Cochrane Reviews provided a full list of options, this information could consistently be added into the consult decision aid. Visual aspects and images would need to be improved by a graphic designer during the publishing process. Confidence intervals were not added as they make the figures too busy and are already acknowledged within the GRADE rating.

4.4 Applying the Prototype to a Range of Reviews

In March 2014, there were 975 Cochrane Reviews with summary of findings tables. Twelve Cochrane Reviews were purposefully chosen to represent a range of decisions about surgery, screening, or medications. The average time for creating a consult decision aid was 3 h by a research assistant plus 1 h for verification by a clinician on the stakeholder committee.

4.5 Considerations for Embedding Consult Decision Aids in Cochrane Reviews

The stakeholder committee reached consensus on three characteristics of Cochrane Reviews that make it easier to develop a consult decision aid and seven circumstances indicating when it is relevant to include a consult decision aid. It is easier to create a consult decision aid when a Cochrane Review provides a summary of findings table, describes the health condition and interventions in the plain language summary, and is focused on a topic involving two reasonable options (including no treatment/screening). It is relevant to include a consult decision aid when the Cochrane Review (a) provides evidence on benefits and harms; (b) is focused on an important topic to patients and clinicians; (c) is about a topic that occurs commonly in clinical practice; (d) involves trade-offs between benefits and harms; (e) is a controversial topic (e.g., strong views for and against the options); (f) involves a decision that is irreversible (e.g., surgery, genetic testing); and/or (g) has recently been updated (e.g., <5 years old).

5 Discussion

Using a systematic process, we developed a consult decision aid prototype that was found to be acceptable for facilitating shared decision making during the consultation and feasible to use for translating evidence from Cochrane Reviews. We showed that the Ottawa Decision Support Framework and IPDAS approaches for developing patient decision aids were flexible enough to develop this prototype [11]. Although the participants rated the consult decision aid as acceptable and usable, most wanted more information. In addition, the prototype was easily applied to create a series of 12 consult decision aids from 12 different Cochrane Reviews with the process taking on average 4 h per consult decision aid. Finally, the stakeholder committee determined three characteristics of Cochrane Reviews making it easier to create consult decision aids and seven circumstances indicating when it is relevant to create one. These results lead us to four main points of discussion.

First, our findings demonstrate that the Ottawa Decision Support Framework provides a clear approach for producing these briefer patient decision aids for use in consultations [17, 18, 28]. In previous trials of patient decision aids developed using the Ottawa Framework, patients had, for example, improved knowledge, improved realistic expectations, decreased decisional conflict, and rated the decision aid as acceptable [17, 28–34]. A toolkit for developing patient decision aids and consult decision aids using the Ottawa Framework is available in a free-of-charge online training tutorial [35]. Similar to other newer approaches to building decision aids for use in consultations (e.g., decision box, option grids), further research is required to determine their effectiveness [6].

Second, briefer one-page patient decision aids with more limited information on options shift the focus from providing detailed information on options to being more oriented to the process of decision making. The ultimate aim of the consult decision aid is to facilitate discussion of the evidence on the options and on the patients’ values during a shared decision making process. Of eight randomized controlled trials evaluating previously developed patient decision aids designed for use within the consultation, seven reported improved patient–clinician communication and one trial found no difference when compared with usual care [6]. Hence, the new consult decision aid should stimulate patient–clinician discussion within the consultation on the options, prompt them to consider other information tailored to the patient’s personal situation, provide the patient opportunities to ask other questions; for example, how does this new medication fit with their other medications and health conditions (e.g., allergies), plus discuss possible side effects and ways they could be managed. A systematic review has indicated that question prompt lists are effective interventions for improving communication and cognitive outcomes [36].

Third, expanding the information in the consult decision aid to respond to participants’ feedback must be considered cautiously. The consult decision aid needs to provide enough information to improve patients’ knowledge while avoiding too much information that can interfere with its usability within clinical consultations. When more detailed decision aids were compared with briefer decision aids in trials, patients had slightly higher knowledge but the differences were small [37]. A unique feature of this consult decision aid is its layered presentation of information with one main page and links to additional information. The main page provides brief information on options, helps clarify patients’ values for outcomes, and screens for decisional conflict to indicate unmet decisional needs. Current links provide information on definitions but could be expanded to provide links to more detailed sources of information. Providing layers of information allows patients after the consultation to obtain the amount of information they desire. Using internet-based delivery of the consult decision aids would enable easier access to other relevant information resources [38].

Fourth, efficient or automated processes could be developed to transfer information from Cochrane Reviews into the consult decision aid prototype and facilitate updates. After applying the prototype to 12 Cochrane Reviews, we established a set of considerations for selecting Cochrane Reviews suitable for creating consult decision aids. Next, these considerations need to be applied by people less familiar with patient decision aid development and they could be applied to non-Cochrane systematic reviews. Given current initiatives to automate systematic reviews [39], it may also be possible to automate the creation of these consult decision aids. The Cochrane Library is in the process of establishing automation for some processes. For example, Evidence Pipeline was initiated to redirect primary studies automatically to their respective review groups, in order to facilitate and streamline Cochrane Review updates [40]. As well, the PICOtron is being used to extract information from reviews to produce summaries of the evidence [41]. Given that Cochrane evidence is currently reported in 12 languages, there is the potential to create consult decision aids in multiple languages which would better address health policies mandating their use in clinical practice [42].

There are a few limitations and strengths to consider. The consult decision aids developed from Cochrane Reviews are limited by the amount of information presented in the reviews. For example, many Cochrane Reviews identified a limited number of options but this could easily be changed in reviews by, at a minimum, including a fuller list of options. Second, the evaluation from potential users (who happened to be familiar with the topic) was simulated and conducted as part of the development process. Hence, further evaluation is required to determine their effectiveness in clinical consultations and how it compares to a more detailed decision aid. Third, participants evaluated mocked-up word documents and ideally the consult decision aids should be hosted online to provide easy access to hyperlinked information and scaling of fonts. Fourth, the 12 consult decision aids were not reviewed by a panel of topic experts. Given that Cochrane Review teams include topic experts, it is possible to incorporate a topic panel review into the development process. A subsequent study could evaluate the added value of topic expert review by determining the accuracy of information presented in the consult decision aid with and without expert review. These 12 consult decision aids require further evaluation by potential users, ideally including users with a broader range of characteristics. Finally, the acceptability evaluation was based on the consult decision aid prototype applied to one Cochrane Review and some participants’ comments were specific to this clinical decision. Strengths were the various characteristics of participants including several with secondary school education only and the relevance of most of their feedback across decisions.

6 Conclusions

The consult decision aid prototype uses a one-page brief format for presenting evidence from Cochrane Reviews to facilitate shared decision making in the consultation. This prototype meets the proposed minimal qualifying and certifying criteria from the IPDAS Collaboration. Health consumers and Cochrane editorial staff found it was acceptable and rated it positively for usability. Several suggestions were used to strengthen the amount and display of information. Application of the prototype to a series of Cochrane Reviews demonstrated the ability to apply it to a range of different health topics and efficiencies in creating consult decision aids. Issues to consider are the primary aim of the decision aid (e.g., information plus support versus process-oriented), the amount of information to provide and/or link to, and ways to automate the process to further improve efficiencies in creating patient decision aids. Consult decision aids were easier to create when there was a summary of findings table, a plain language summary, and a treatment/screening decision with two or more reasonable options (including no treatment/screening). Seven circumstances were identified for determining Cochrane Reviews relevant for including consult decision aids. These considerations for embedding consult decision aids in Cochrane Reviews are likely transferable across other types of high quality systematic reviews. Future studies will evaluate implementation of consult decision aids.