Introduction

Evidence to inform the optimal care of patients at risk of chronic kidney disease (CKD) incidence or progression is greatly needed. Patients, clinicians, and policy makers [1–7] want evidence about effective strategies to: (1) prevent incident CKD, (2) improve the accuracy and timely detection of CKD, (3) treat CKD effectively to limit progression to end stage renal disease (ESRD), and (4) decrease morbidity from CKD-related comorbid illnesses. Comparative effectiveness research (CER) compares “the benefits and harms of various interventions and strategies for preventing, diagnosing, treating and monitoring health conditions in real-world settings.” [8] The American Recovery and Reinvestment Act (ARRA) of 2009 provided $1.1 billion for CER which was divided and disbursed to the National Institutes of Health, the Agency for Healthcare Research and Quality and the Office of the Secretary of Health and Human Services [9]. The Institute of Medicine was tasked to recommend national priorities for research questions to be addressed by CER and supported by ARRA funds. Among the Institute’s list of 100 initial priority topics, two addressed kidney disease [8].

A core research method of CER is systematic review and synthesis of existing literature, but this has not been widely employed to study CKD care. Findings from systematic reviews can help decision-makers draw conclusions about effective care strategies employed among heterogeneous populations and clinical settings. Systematic reviews also help to identify gaps in the existing evidence that need to be addressed by future primary research studies [10].

CER studies may be distinguished from other types of clinical research by their explicit goal of being highly responsive to priorities of community stakeholders (persons or groups who have a vested interest in a clinical decision and the evidence that supports that decision) who will use study findings to support their decisions about care [11]. Research funding agencies increasingly recommend formal solicitation of stakeholder input as an important initial step in designing CER studies. [11, 12]. To date, there has been little effort to identify priorities for CER among stakeholders in the CKD community and to share this with the CKD community at large. We engaged stakeholders within the CKD community to identify and prioritize topics for future CER systematic reviews and to help set an agenda for future primary CER studies of CKD care.

Methods

General approach

The Johns Hopkins University Evidence-Based Practice Center (EPC) assembled our team under a contract from the Agency for Healthcare Research and Quality (AHRQ) of the Department of Health and Human Services. Our primary goal was to use the input of community stakeholders to identify topics for systematic reviews of primary literature comparing the effectiveness of strategies to prevent, detect, and treat CKD or its complications. We also sought to identify topics for future CER studies of CKD care where the performance of systematic reviews might not be currently feasible due to lack of sufficient primary literature to review. Our team had clinical expertise in pediatric nephrology, adult nephrology, and general internal medicine, and research expertise in CER, clinical epidemiology and prevention of CKD, clinician practice patterns in the care of patients with CKD, health disparities, and systematic review methodology.

We established a preliminary protocol for identifying and prioritizing topics, which we later revised with input from the stakeholders. The protocol consisted of three main activities: (1) identifying and categorizing potential topics of interest to the CKD community for systematic review and primary CER studies, (2) asking stakeholders to rank topics that would be of greatest interest to the CKD community, and (3) exploring the feasibility of performing CER systematic reviews on these high priority topics (Figure 1).

Figure 1
figure 1

Chronic Kidney Disease Comparative Effectiveness Research (CER) Topic Identification Process.

Full size image

Preliminary categorization and identification of topics for CER studies

From the outset, our team acknowledged the breadth of topics potentially relevant to the care of patients with CKD, ranging from studies of strategies to prevent CKD, to studies of strategies to prevent morbidity and mortality among patients with end stage renal disease (ESRD). We decided a priori to restrict our efforts to identifying priorities for CER reviews relevant to the care of patients with CKD stages I-IV because of time limitations (the process was funded to occur in less than 1 year) and differing considerations relevant to the care of patients with early or progressive CKD versus the care of patients with ESRD. To categorize the topics, we developed a preliminary ‘map’ reflecting broad domains of CKD care (prevention, detection, progression and complications) prior to engaging stakeholders. Within each broad domain, we developed a preliminary list of potential topics through an iterative process informed by our review of recent clinical practice guidelines [12–14] and perspectives pieces published by CKD thought leaders. We identified topics without regard to whether they were the subject of ongoing reviews. For many of the topics, we also identified numerous subtopics that could be of interest to the CKD community.

Selection of CKD community stakeholder representatives

We sought to identify individuals representing varied stakeholder perspectives relevant to the performance of CER in CKD care. Many of the individuals we identified were employed or had active memberships in organizations representing relevant decision-makers in the CKD community and/or had clinical expertise relevant to the care of patients with CKD. (Table 1) Because we hypothesized government agencies such as the National Institutes of Health (NIH) might use the results of systematic reviews to guide funding decisions, we also engaged program officers overseeing clinical studies for their input.

Table 1 Organization Types and Clinical Expertise Represented by Expert Stakeholders*
Full size table

Community Stakeholder Engagement

We asked stakeholders to provide input on our approach to identifying potential CER research topics and to rank the importance of topics. We engaged them via two conference calls and several electronic and mailed communications.

We reviewed definitions of CER as presented by AHRQ, the Institute of Medicine and the Federal Coordinating Council for Comparative Effectiveness Research [8] with stakeholders. We then presented the preliminary map of domains for CER topics to stakeholders. The map reflected four areas of potential interest to the CKD community: (1) prevention of CKD, (2) detection of CKD, (3) progression of CKD, and (4) complications of CKD. We also presented stakeholders with a preliminary list of 30 potential topics for CER research. We solicited stakeholders’ feedback regarding: (1) the relevance and appropriateness of our preliminary conceptual map, and (2) the relevance and comprehensiveness of the preliminary list of topics we proposed. We asked stakeholders to add to our preliminary list of topics.

We also asked stakeholders to establish a set of criteria for ranking topics. We proposed four criteria which they might consider, including (1) the burden of the topic to patients with CKD or patients at risk of developing CKD (including consideration of whether that topic was of particular relevance to priority subpopulations such as patients with diabetes, pediatric patients, racial/ethnic minorities or other vulnerable populations); (2) the extent to which there is uncertainty regarding optimal approaches to the topic; (3) the extent to which a systematic review with definitive findings could impact decision-making by patients, clinicians, or policy makers, and (4) the feasibility of performing a systematic review (i.e., adequacy of literature facilitating a rigorous review and the absence of recent previous systematic reviews). However, stakeholders expressed concern about the availability of literature to perform systematic reviews in several areas, and therefore decided to prioritize topics without regard to feasibility. Our team subsequently carried out a feasibility assessment of the stakeholder-prioritized topics.

Stakeholders revised our preliminary list of CER topics by refining the proposed topics and adding additional topics. Stakeholders then each independently ranked their 10 highest priority topics for CER studies (10 for highest, 1 for lowest). We calculated a summary score reflecting each topic’s overall standing among stakeholders (Additional file 1: Appendix A). We allowed ties for overall rankings and shared the final ranked list of topics with stakeholders to learn whether the composite rankings reflected the consensus of the entire group. We then asked them to identify topics they thought should be recommended for systematic reviews by AHRQ’s Evidence Based Practice Centers.

Assessment of CER systematic review feasibility

We conducted preliminary literature searches to estimate the feasibility of performing CER systematic reviews on topics ranked highly by stakeholders. We searched publication titles in PubMed to identify relevant studies using terms sufficient to identify a reasonable core of the studies on a topic. We did not intend for these searches to be as exhaustive as would be required for formal systematic reviews. We assessed the availability of published systematic reviews/meta-analyses (within the past 5 years), randomized controlled trials (RCTs), and observational studies (if no systematic reviews/meta-analyses or RCTs were identified on a topic) for each top tier topic area. For RCTs and observational studies, we reviewed up to 300 of the most recent PubMed records for each topic. We noted whether studies pertained to specific priority populations and whether trial interventions were pharmacological or non-pharmacological. Search protocol and search terms are contained in Additional file 2: Appendix B.

Findings

Final categorization and list of potential topics for CER studies

Stakeholders agreed that the four proposed domains for categorizing topics for CER appropriately reflected the range of potential CER topics about CKD care, and added 16 additional topics to our preliminary list of 30 topics. Table 2 lists the 46 topics (grouped by domain, in no particular order) and example CER questions. The example questions are intended to illustrate ways in which CER topics could be framed, but they are not intended to limit the range of CER questions which could be asked regarding CER topic areas.

Table 2 List of topics considered by stakeholders for Comparative Effectiveness Systematic Reviews
Full size table

Final Stakeholder Prioritization of Topics for CER Studies

After stakeholders collectively reviewed the summary rankings, they agreed that topics with a global rank of 12 or better should be considered ‘high priority’ for future funded CER reviews. Because of ties, this cutoff resulted in 18 topics being considered ‘high priority.’ There was diversity in the opinions of the stakeholders regarding the appropriateness of topic areas for CER systematic reviews, with some stakeholders expressing concern regarding the likely paucity of evidence to compare existing strategies of care in certain topic areas. However, in general, stakeholders’ rankings were consistent, reflecting overall consensus regarding the appropriateness of topics for which CER systematic reviews would be of greatest interest to the CKD community. One anomaly was that one stakeholder’s top ranked topic, ‘inflammation in the progression of CKD’, was not ranked at all by any other stakeholder. Yet that top ranking by a single stakeholder was sufficient for it to be included among the top tier of topics. (Table 3) Half (n = 9) of top tier topics focused on strategies to slow or stop progression of CKD, while fewer top tier topics focused on prevention of CKD (n = 6), detection of CKD (n = 2), and complications of CKD (n = 1). Two topics emerged among the top tier priorities in more than one area of the conceptual map, patient safety (in prevention and progression areas) and hypertension (in prevention and progression areas).

Table 3 Topics receiving highest priority rankings for funded comparative Effectiveness Systematic Reviews in CKD
Full size table

Feasibility of performing CER systematic reviews on priority topics

Our preliminary assessment suggested some systematic reviews could be feasibly performed, with some topics having sufficient trial data and others having only observational data. (Table 3) Some of the priority topics identified, including ‘screening for CKD’ and ‘management of hypertension and lipid levels’, were previously reviewed in the AHRQ-funded evidence report Screening for and Management of Chronic Kidney Disease Stages 1-3 [15].

Discussion

This project is one of the first attempts to systematically assess priorities for CER studies among stakeholders in the CKD community. Stakeholders’ high priority rankings reflect topics posing significant burden to patients with CKD or at risk of CKD, and aspects of CKD care for which there is significant uncertainty regarding effectiveness. High rankings also reflect stakeholders’ views that high-quality systematic reviews could help inform clinical practice or policy. While topics identified through this process may not include all topics that warrant CER systematic reviews or primary CER studies, this process may provide a preliminary road map for researchers seeking to perform CER studies relevant to the CKD community.

Many topics receiving a ‘high priority’ designation by stakeholders focused on studying the effectiveness of strategies to prevent CKD incidence and to slow CKD progression. Some highly ranked topic areas, such as ‘patient safety’, were reflected in multiple domains of the conceptual map. Because they are viewed as relevant to several aspects of CKD care and prevention, rigorous CER reviews on these topics could have high impact on patient care and clinical outcomes. Other topics, such as studies evaluating the comparative effectiveness of health information technology and the study of collaborative care strategies (both ranked as ‘top tier’ by our stakeholders), represent novel areas of inquiry. We identified few studies of these topics during our feasibility assessment, potentially reflecting a potential need for primary CER studies in these areas.

There are several caveats to our engagement of stakeholders for their input. First, the group of stakeholders was relatively small. Thus, individual stakeholders’ rankings could substantially affect topics’ overall final rankings and might not reflect fully the range of opinions that might be present among the entire CKD community. Indeed, some stakeholders did express concerns regarding the appropriateness of some topics for CER reviews, given a paucity of evidence to address these topics. Second, the composition of our stakeholder group and the manner in which we contacted stakeholders for participation in this exercise could also affect our findings. Third, while our stakeholder group included representatives from two patient advocacy organizations, we did not include actual patients with CKD. Methods for selecting patients with appropriate levels of expertise (i.e., medical knowledge, awareness of issues affecting patients with CKD relevant to comparative effectiveness research) for this type of scientific effort are not yet well defined. Fourth, stakeholders’ input may also have been further enriched by our inclusion of non-physician medical professionals such as nurses, social workers and dietitians. Future efforts to identify topics for CER reviews should consider broadening stakeholder representation to include patients and non-physician medical professionals. Fifth, some stakeholders did submit additional comments in with their rankings, which provided additional context to their rankings. However, because all stakeholders did not comment to the same extent, we did not formally analyze these comments. Finally, as this was one of the first funded efforts to identify priorities for CER systematic reviews in CKD, diversity in stakeholders’ opinions could also reflect their varying interpretations of the ultimate goal of activities such as this.

Other characteristics of our approach could also have influenced our findings. For instance, we developed the map and topic lists via iterative discussions prior to obtaining stakeholders’ input, and without regard to ongoing systematic reviews, which may have influenced the range of the final list of topics that the stakeholders ranked. Future projects of this kind may benefit from assessing and comparing independently determined priorities of different groups of stakeholders, including policy makers, funding agencies, clinicians and patients. Also, while our preliminary findings suggest several topics may have enough evidence to make CER systematic reviews feasible, several factors determine whether a systematic review will add value to inform medical decision-makers, including the number and quality of studies available to review, how definitively identified studies answer the question of interest, and how answers to questions will reduce uncertainty or add to the current knowledge and ultimately impact care. These factors should all be considered prior to undertaking CER systematic reviews.

In conclusion, we systematically identified priorities for CER research relevant to the care of patients with Stages I-IV CKD among stakeholders in the CKD community. Future efforts such as ours might benefit from identification of an even more broadly defined group of stakeholders (e.g., patients and non-physician CKD health professionals), further refinement of protocols for engaging stakeholders throughout the topic identification process, and development of strategies for identifying areas in need of both CER systematic reviews as well as primary CER studies. Findings from this project may be a useful guide for researchers and research funders seeking to address CER questions highly relevant to improving clinical care and outcomes of patients with CKD.