Background

Patients whose advanced chronic kidney disease (CKD) is approaching end-stage renal disease (ESRD), or complete kidney failure, face complex medical decision-making regarding the type of medical therapy they wish to pursue. Choices for treating ESRD are numerous and include renal replacement therapies (RRTs) such as hemodialysis delivered in a dialysis center three times per week, home care dialysis (including peritoneal dialysis and home hemodialysis, which patients administer to themselves daily), and living or deceased kidney transplantation. Patients may also choose to forgo RRT and instead opt for conservative medical therapy (i.e., ongoing medical care with their nephrologists to optimize health without dialysis or kidney transplantation). Each treatment option has different advantages, limitations, and implications for survival, quality of life, financial stability, general health status, and how patients experience their daily lives[1–3]. For instance, home dialysis therapies require patients’ substantial self-care and may afford patients more control over their daily schedules, while in-center hemodialysis requires less self-management but more frequent interface with medical professionals and may offer patients less autonomy[4]. Patients’ families are also impacted by RRT decisions, as they often play substantial caregiver roles[5].

Despite patients’ critical need to understand the characteristics of various RRTs to inform their treatment decisions, many patients report having little to no general awareness of what types of treatment options exist for ESRD and many have little knowledge of treatment risks and benefits prior to their initiation of RRT[2, 5–9]. Patients’ poor access to adequate education about treatments has been linked to their abrupt initiation of dialysis and their suboptimal access to other forms of RRT,[10, 11] such as kidney transplantation, which is associated with improved clinical outcomes[6, 12–14]. A substantial proportion of patients with ESRD are elderly and may have low educational attainment, poor health literacy and numeracy, and cognitive decline associated with their advancing kidney disease, heightening the challenges of educating patients about their RRT choices[15–18].

Comprehensive decision aids are needed to help patients with kidney disease and their families make informed RRT selection decisions aligned with their personal values. This paper describes our development of a patient-and-family centered decision aid to inform RRT selection decisions.

Methods

Overarching goals, decision Aid content, and approach to decision Aid design

Goals

We sought to develop a patient-and-family centered decision aid (handbook and video) to help patients and families choose among numerous RRT options. We sought to ensure that the decision aid could be understood by patients with a range of health literacy, health numeracy, and cognitive needs. We used the 2006 International Patient Decision Aids Standards (IPDAS) to guide our development process[19]. All aspects of the decision aid design, including the collection of primary patient and family data to inform development, were approved by the Johns Hopkins Medicine Institutional Review Board.

Decision aid content

We intended for the decision aid to address both positive and negative features of treatment options that patients (who had previously experienced treatments) and their family members felt others making RRT decisions should know about. We identified content to include via: (1) our own conduct of 20 focus groups of patients and families identifying information they felt should be included in decision aids guiding others’ RRT selections (described elsewhere)[5] as well as our selective review of qualitative studies published by other investigators,[2, 3, 20] and (2) our conduct of rigorous systematic literature reviews describing findings of scientific studies comparing outcomes among patients selecting different RRTs (described elsewhere)[21–24]. Our decision aid focused on seven key content areas or “concerns” about treatments, including how treatments might affect patients’: (1) morbidity and mortality (“health problems”), (2) autonomy (“doing things”), (3) treatment delivery (“how treatment works”), (4) symptoms (“symptoms”), (5) relationships (“relationships”), (6) psychological well being (“feelings”), and (7) finances (“money matters”) (Table1).

Table 1 Topics addressed in the decision aid
Full size table

Approach to designing the decision aid

We intended for the video and handbook to present complementary subjective (in video) and evidence-based (in handbook) information to a diverse English-speaking audience. We developed the video and handbook simultaneously in three “stages” (Figure1). We engaged a multidisciplinary group of experts throughout the development process, which included clinical experts (physicians, social workers), an adult health education specialist, a patient advocacy specialist, a video scriptwriter, and a medical illustrator.

Figure 1
figure 1

Decision aid development process.

Full size image

Methods: developing the video

We intended to provide patients and their families with specific testimonials from others who had already made treatment decisions. We attempted to ensure these testimonials addressed as many of the seven previously identified “concerns” as possible. When patients or families did not address “concerns” in their testimonials, or if deemed appropriate, we incorporated testimonials from health care professionals who could provide additional views regarding common patient experiences relevant to the “concerns.”

Video development stage 1: selecting patients and families for the video, writing the script

Engaging a scriptwriter

We engaged a professional video scriptwriter with experience creating medical education videos. The scriptwriter reviewed findings from our foundational research and worked with our investigative team to develop questions designed to elicit positive and negative viewpoints from patients and families about their chosen RRTs.

Selecting patients and families and writing the script. We worked with local nephrologists in Baltimore, MD to identify a diverse group of patients who might potentially participate in the video. We asked nephrologists to select patients receiving different forms of RRT with specific demographic characteristics (e.g., minority female who received a transplant) to ensure diversity. The scriptwriter conducted 90 to 120 minute directed interviews with patients and family members and recorded the content of interviews for review.

The scriptwriter then drafted a script by incorporating views she obtained from patients and families. In our previously conducted focus groups, patients and families identified health care professionals they thought would be appropriate to address various “concerns.” For example, patients and family members felt physicians would be appropriate for discussions about the health risks and benefits associated with treatment selections, while social workers would be more appropriate for discussing financial considerations. Our investigative team reviewed the script and refined it to ensure all seven “concerns” were adequately addressed by either patients and families or health professionals (which included a transplant nephrologist, transplant social worker, general nephrologist, and dialysis social worker).

We also ensured that the video addressed complementary information to the handbook. Because the handbook summarized findings from scientific studies about the various RRT options (e.g., differences of health risks associated with different therapies), we did not include this type of quantitative information in the video (Table1).

Video development stage 2: producing the video

Filming and producing the video

A professional video production crew filmed patients and family members in their homes during scheduled visits. To obtain testimonials on camera, an investigative team member interviewed them using the same questions the scriptwriter asked during initial directed interviews. The crew filmed health care personnel in a studio. An investigative team member asked health care providers questions to address issues that patient testimonials did not already address. After completing video filming and editing, we iteratively refined the video, focusing on decreasing length and improving clarity, consistency, and accuracy.

Video development stage 3: video screening and final edits

Screening groups

Prior to final editing, we recruited five patients receiving different RRTs from two local, academically affiliated nephrology practices in Baltimore, MD and seven of the patients’ family members to screen the video and provide their qualitative feedback during a two hour group meeting led by a trained moderator. The moderator elicited participants’ initial impressions of the video as well as feedback regarding length, amount of information presented, understandability, the balance in presentation of positive and negative aspects of the treatments, and areas for improvement. We audio-recorded and transcribed screening group discussions and we incorporated screening groups’ feedback in our final iterations.

Methods: developing the handbook

The handbook complemented the video (Table1) and summarized findings from scientific studies about differences in patient-reported and clinical outcomes among patients receiving different treatment options (in-center hemodialysis, peritoneal dialysis, kidney transplantation, and medical therapy with no dialysis or transplantation). We also provided information about the methodological quality of scientific studies. The handbook focused on the same seven “concerns” highlighted in the video.

Handbook development stage 1: preliminary handbook content design by our investigative team

We worked with specialists in medical illustration, adult education, health communications, and in adapting materials for low literacy readers to develop the handbook. We created several mock designs for presenting information before conducting the pilot study.

Handbook development stage 2: mixed-methods pilot study to guide iterative design

During early handbook development, we confronted three main challenges, including (1) presenting information about differences in multiple treatment types, (2) presenting information quantifying the magnitude and direction of scientific findings that did not incorporate risk probabilities and could not be described using standard methods (e.g. pictograms or bar graphs),[25] and (3) communicating information to patients about the quality of scientific evidence available to inform a decision.[26] To help address these challenges, we performed a pilot study among patients with kidney disease to obtain their input on the clarity of information we presented in various handbook iterations.

Pilot study goals, setting, and participants

In two study “phases,” we recruited a total of 48 English-speaking patients from two local nephrology practices in the Baltimore, MD metropolitan area to obtain qualitative feedback on handbook material (Phase 1 and 2) and quantitatively test patients’ comprehension of handbook material (Phase 2 only) during the iterative design process. For both phases, we asked nephrologists to identify patients either on dialysis or with progressive CKD (“Pre-ESRD,” defined as estimated glomerular filtration rate of less than 30 ml/min/1.73 m2) but not yet receiving dialysis or a kidney transplant. We then approached patients to assess their interest in participating in cognitive interviews.

Phase 1 cognitive interviews

During study Phase 1, we performed cognitive interviews with 12 patients with kidney disease to obtain feedback regarding the understandability of the information and ways to improve information presentation. We provided participants with a binder that included 8 to14 sample handbook pages. After participants reviewed each sample handbook page a trained interviewer asked six open-ended questions: 1) “Please tell me what you think the information on this page means,” 2) “What does (fill in topic name) mean to you?” 3) “We’re not sure if these are the best words to use, if you think there is a better way, we’d like to hear it,” 4) “Is any part of this page unclear?” 5) “Do you have any other suggestions?” and, 6) “On a scale of 1–5, with 1 being very hard to use and 5 being very easy to use, how would you score your ability to use this picture and the words to understand the information?” We audio-recorded and transcribed each interview. We reviewed transcriptions and made iterative changes to the handbook to address participants’ comments.

Phase 2 cognitive interviews and assessment of comprehension

During study Phase 2, we performed 36 additional cognitive interviews. As with Phase 1, we asked participants to provide qualitative feedback regarding the understandability and usability of information. In study Phase 2, we also assessed participants’ “gist” comprehension[27, 28] of five key aspects of scientific evidence presented on each page, including whether they could understand: (1) the topic (“What issue are we talking about on this page?”), (2) the treatment modalities (“What treatments are being compared on this page?”), (3) the direction of differences in outcomes between RRTs being portrayed (“Which treatment is better?”), (4) the magnitude of difference in outcomes between RRTs (“How much better or worse is one treatment versus the other?”), and (5) the research quality of scientific studies to inform that aspect of the decision (“Does the page tell people how good the information is?”).

We asked participants to view 7 to 9 different sample handbook pages displaying similar types of information (including information addressing five key aspects of scientific evidence: topic, treatment modalities being compared, direction of differences between treatments, magnitude of differences, and research quality). We assessed participants’ gist comprehension of each key aspect on each page. Two independent investigators reviewed transcripts of each audio-recorded interview to assess participants’ gist comprehension and scored participants’ responses as correct or incorrect. A third research team member adjudicated any discrepancies among the two reviewers by also reviewing the transcripts.

For each key aspect of scientific evidence, we calculated the number of pages for which participants correctly demonstrated comprehension. For example, if a participant demonstrated correct comprehension of the treatments displayed on 7 of 9 pages, we assessed that participant as having correct comprehension of this element 78% of the time. We then summarized the total group’s comprehension of each element by calculating the median (range) of all participants’ percentages indicating correct comprehension of that element. We also assessed patient participants’ demographic characteristics; health literacy (Rapid Estimate of Adult Literacy in Medicine)[29]; numeracy level (Risk Numeracy)[30]; cognitive function (Trails B)[31]; and knowledge and awareness of kidney disease[7].

Handbook development stage 3: handbook screening and final edits

Patients and family members who participated in the video screening (described above under “Video Development, Stage 3”) also reviewed the handbook and provided opinions to the same open-ended questions.

Results

Feedback we received during each phase resulted in numerous iterative refinements of both the video and the handbook. We iteratively refined the video 17 times (11 script iterations in Stage 1, three initial video iterations in Stage 2, and three final film iterations in Stage 3). We iteratively refined the handbook 26 times (three iterations in Stage 1, 22 iterations in Stage 2, and one final iteration in Stage 3). Examples of preliminary handbook presentations, reflecting major iterative design approaches from Stages 1 and 2, are presented in Figure2.

Figure 2
figure 2

Representative sample iterations from stages 1 and 2 of the handbook development process.

Full size image

Results: findings from mixed methods pilot study

Of the 12 participants in pilot study Phase 1, 67% were male, 92% were African American, and 75% had at most a high school degree or GED. The majority of participants were on hemodialysis (n=9, 75%), while the remaining participants were classified as pre-ESRD (n=3, 25%). Of the 36 participants in Phase 2, 44% were male, 83% were African American, and 47% had at most a high school degree or GED. Fifty-four percent of phase 2 participants were on hemodialysis, 33% were considered pre-ESRD, 29% on peritoneal dialysis, and 17% on home hemodialysis. Participants in both phases were diverse with respect to their knowledge of kidney disease, literacy, numeracy, and cognitive function (Table2).

Table 2 Participant characteristics for handbook development Stage 2 (Phases 1 and 2)
Full size table

Qualitative feedback obtained during cognitive interviews (during Phase 1 and Phase 2) focused on three key areas: (1) difficulties identifying the multiple treatment types presented in the handbook, (2) the handbook containing an intimidating amount of complex scientific information, and (3) their desires to have numerical or quantitative information from scientific studies presented in a way that was clear and understandable. Through a total of 22 iterations (16 iterations in Phase 1 and six iterations in Phase 2) we addressed these concerns through numerous adjustments (including the use of color coding to consistently identify RRT options when discussed throughout the handbook, providing explanatory guides for how to read pages, eliminating the use of abbreviations, and supplementing graphical numerical presentations with text written at a 4th to 6th grade reading level) (Table3).

Table 3 Qualitative feedback from handbook development Stage 2 (Phases 1 and 2)
Full size table

During study Phase 2, we measured gist comprehension of materials among a total of 36 participants. Twenty-four (24) participants provided feedback on iteration 20 of the handbook and 12 participants provided feedback on iteration 21 of the handbook. Among all items viewed by participants, participants demonstrated median gist comprehension of 78% (range: 15%–100%) in iteration 20. Participants’ median overall gist comprehension rose to a median of 91% (range: 9%–100%) in iteration 21. Participants demonstrated greatest gist comprehension on aspects of pages reflecting the topic being discussed, the treatment types (modalities) being compared, and the direction of scientific findings discussing treatment differences (i.e., they could correctly identify one treatment was better than the other). Participants demonstrated less gist comprehension regarding the magnitude of treatment differences (i.e., they had difficulty understanding how much better one treatment was than another) and information about the research study quality. However, gist comprehension scores improved in these areas between iterations 20 and 21 (Table4).

Table 4 Median percentage** of times participants correctly comprehended key aspects of scientific evidence during pilot testing (handbook development Stage 2, Phase 2)
Full size table

Results: feedback from decision Aid screening

Five patients and seven family members participated in two different screening groups. Patient participants were 80% female, 60% African American, and 60% had at least two years of college education. All family member participants were female, 71% were African American, and 43% had at least two years of college education.

Patients and family members provided mostly positive feedback after screening both the video and handbook. (Table5) Participants often commented that they liked the organization of the video and the breadth of topics it covered. Participants also commented on the complementary nature of the two materials. Screening group participants constructively criticized the large amount of information presented in the handbook. To address this concern, we created a mini-book entitled, “All of the Facts,” to accompany the handbook, which provides readers with a brief summary of key information presented in the longer handbook.

Table 5 Positive and constructive feedback obtained from screening the decision aid
Full size table

Results: final decision Aid

The final decision aid consists of a 50-minute video and a comprehensive, 159-page handbook accompanied by a 14-page mini-book (“All of the Facts”) which summarizes the key information in the handbook. Together, the video and handbook satisfied 92% of all quality criteria outlined in the three subject areas designated by IPDAS (satisfied 96% or 22 of 23 content items, 87% or 20 of 23 development process items, and 100% or 6 of 6 effectiveness items) (Additional file1).

The video provides testimonials from minority and non-minority (male and female) patients, their family members, and health care professionals (physicians and social workers) about “concerns” patients and their families might have which could influence their RRT selection decisions. These “concerns” correspond to the seven key content areas patients and their families identified as important to include in the decision aid during our foundational research studies. Patients and family members receiving different RRTs share subjective testimonials describing their positive and negative experiences with these “concerns” in the video. Health care providers discuss additional information regarding “concerns” to balance patient and family views and to contribute additional information.

The handbook provides evidence-based information regarding risks and benefits of different treatment options of peritoneal dialysis, in-center hemodialysis, home hemodialysis, kidney transplant, and conservative management (treatment with no transplant or dialysis) as they pertain to these same patient concerns. The introduction includes a section that defines kidney disease (“What Is Kidney Disease?”), describes and color codes the different treatment options (“What are the Treatments?”), presents a value clarification exercise to help patients determine which treatment is best for them (“How Do I Choose a Treatment?”), and orients the reader to the page layout (“What is on Each Page?”). Subsequent sections of the handbook describe summaries of scientific studies relevant to each “concern.” Handbook pages are 8.5” by 11” and oriented toward a 4th to 6th grade reading level. The handbook delivers content on a double-page spread layout to appeal to diverse learning preferences: the left-hand page includes brief statements summarizing key messages from scientific studies while each right-hand page presents a more detailed summary of study findings and a graphical presentation of data (Figure3).

Figure 3
figure 3

Sample pages from final handbook.

Full size image

Discussion

Patients need high quality decision aids to help guide them through the decision-making process about RRT selection. To our knowledge, rigorous and systematic approaches to developing decision aids for patients with CKD have not been previously undertaken. We developed our decision aid to accommodate patients with varying literacy, numeracy, and cognitive needs. The final decision aid achieved the vast majority of IPDAS criteria for engaging patients in informed decision-making. Our refinement process incorporated 11 video script iterations and 26 handbook iterations, reflecting the extensive effort we undertook to develop materials that addressed patients’ self-reported informational needs. Initial screening of our decision aid by patients and their families indicates that these materials are likely to be well received. Pilot testing in our mixed-methods study suggested patients found the final handbook understandable. However, further testing is still needed to determine how effective or useful the decision aid is for ESRD patients’ treatment decisions.

Our development process and findings highlight some of the challenges of developing comprehensive materials for a diverse population that address the positive and negative features of various RRT options. First, we based our content on rigorous foundational studies (20 focus groups and systematic literature reviews), which required substantial time and effort to conduct. Translating our findings from these studies to inform decision aid development also proved challenging as we sought to address multiple concerns identified by patients and invested additional significant time and effort performing the mixed-methods pilot study to identify ways to convey scientific information in a way that can be easily interpreted by the vast majority of patients. Further, factors such as the layout of information and the inclusion of figures are known to be potential sources of bias in decision making. For instance, the final handbook (150 pages) and video (50 minutes) were lengthy. Although we did create a shorter “All the Facts” 14-page handbook in response to participants’ comments during the screening sessions, we did not formally compare how participants viewed this distilled version of information compared to the complete handbook and video. Further testing will help clarify the optimum format for information. We are currently studying these materials in a randomized controlled trial to examine their influence on treatment decisions in African Americans[32], and we are designing additional studies to help further refine the decision aid.

Incorporating subjective testimonials about treatment could introduce bias as patients may identify with video participants rather than synthesize information on preference sensitive topics.[33–35] Nonetheless, treatments for ESRD are quite complex, requiring many practical considerations that significantly alter patients’ daily lives. We therefore felt it was very important to show viewers examples of patients and families undergoing these treatments. For instance, our video shows a patient who receives home hemodialysis inserting a needle into his dialysis fistula. It shows another patient receiving hemodialysis in a treatment unit, and yet another patient administering peritoneal dialysis in her home with help from her spouse. The video depicts patients storing boxes of treatment supplies, handling medications, and traveling to the dialysis treatment facility. Without direct visualization, many patients considering these treatments may not fully comprehend the significant differences between therapies. Future studies will be needed to assess whether viewers of our video felt compelled to choose a treatment because of their subjective identification with video participants.

Some additional limitations of our process are important to consider. Although our team comprised experts from multiple disciplines, it did not include a decision scientist, which may have influenced our development approach. We did, however, refer to the decision sciences literature extensively as well as the IPDAS criteria throughout the development process[19, 25–28, 36–40]. We also developed our decision aid with input from a largely minority (African American) population of patients with kidney disease living in the Baltimore, MD metropolitan area. It is possible patients with different characteristics could have different viewpoints about RRT which might have altered our development process. Further, we developed our decision aid for English-speaking patients. Materials may require significant adaptation to accommodate reading and cultural needs of non-English speakers.

Conclusion

We developed a decision aid to help patients with kidney disease and their families make informed decisions about RRT selections aligned with their values. While our development process helped to ensure completeness and readability of our decision aid, its effectiveness on aiding patients’ treatment decisions warrants further study.