Background

Systematic reviews are valuable not only to synthesize evidence to inform policy or practice, but also to identify the ‘evidence gaps’ or ‘research gaps’ that limit their ability to answer the questions posed. Structured approaches for examining the results of systematic reviews and prioritizing research gaps from systematic reviews have recently become more common and developed [1]. The main advantages of these approaches include the transparency of the process, results that are easily interpretable, and the need for modest resources in terms of funds and time required. As part of a new effort in 2010, the Agency for Healthcare Research and Quality (AHRQ) supported a series of Future Research Needs (FRN) projects, in which stakeholders were engaged after the completion of comparative effectiveness reviews to help develop and prioritize the future research needed by decision makers in a variety of healthcare content areas [2]. Concurrently a series of research papers on various methodologic issues identified through the initial pilot FRN projects was published to inform eventual guidance on the determination and prioritization of future research needs for the AHRQ’s Effective Health Care (EHC) Program [3]. In the AHRQ EHC program, the FRN process begins by identifying a list of evidence gaps from a systematic review. Then the systematic review team works with a stakeholder panel to first elaborate and then consolidate the evidence gaps. Potential research questions are then formalized following the PICOTS (Population, Intervention, Comparator, Outcome, Timing, and Study Design) framework with the exception of methodological questions. Finally, the research questions are prioritized by the stakeholders according to a set of criteria on potential value of the research [1].

The purpose of this paper is to present the methods and results of an FRN project which utilized a modified version of the AHRQ’s FRN framework for identifying research priorities [1]. Specifically, in addition to the research gaps identified from published systematic reviews (which is the current AHRQ FRN process), we integrated an evidence mapping of primary research studies with input from a multidisciplinary stakeholder panel. The panel members identified and prioritized research needs linking dietary sugars and potentially related health outcomes, based on their broad range of expertise. To our knowledge, the present project is the first FRN project with a goal to cover research questions beyond those that were identified by systematic reviews because of the need for covering research questions that are important for public health nutrition.

Dietary sugars commonly refer to monosaccharides and disaccharides. Monosaccharides include glucose, galactose, and fructose. Fructose is the most common naturally occurring monosaccharide, found in fruits and vegetables. Common disaccharides include sucrose (glucose plus fructose), lactose (glucose plus galactose), and maltose (glucose plus glucose). Associations among dietary sugars and a variety of chronic diseases such as obesity have spawned interest in investigating the metabolic effects of sugars in general. Several systematic reviews have been published focusing on the effects of fructose on cardiometabolic outcomes in controlled trials [49]. Other systematic reviews have examined the effects of dietary sugars on body weight [10, 11], and non-alcoholic fatty liver disease or indices of liver health [12], including both controlled trials and observational studies. Due to the complex relationships between dietary sugars and health outcomes there is a need to review a large array of populations, outcomes of interest, and types of studies to identify and prioritize the research gaps in this research area. This paper will summarize the FRN approach we used and the stakeholder panel’s perspectives on the top three research needs identified by this approach.

Methods

The initial scope of work was developed by the sponsor of this project (International Life Sciences Institute (ILSI)) in a request for proposals. Aside from their initial feedback on our funded research proposal, the sponsor was not involved in the later refinement of the scope of work or the process of creating the evidence-mapping database, other than providing a list of selected citations for cross-referencing purposes. The list of selected citations suggested by the sponsor was merged with the citations identified by our literature search, and were evaluated in the same way based on the study eligibility criteria (Table 1).

Table 1 Study eligibility criteria

Overall approach of this FRN project

An overview of the FRN approach we used is shown in Fig. 1. The project consisted of a research team and a stakeholder panel. The stakeholder panel was tasked with overseeing and advising on the development of the search strategy and study eligibility criteria for the evidence mapping, as well as prioritization of FRN research questions. The research team was tasked with data screening, abstraction and analysis, and facilitating stakeholder engagement and discussions. The roles and responsibilities of different parties in the FRN project are described in Appendix A of the Additional file 1.

Fig. 1
figure 1

Overview of the FRN assessment approach

A draft report underwent a period of public review in August 2014. All public reviewers’ comments and suggestions were considered in revising the report.

Evidence mapping

The process and methods for creating an evidence-map database of published studies linking dietary sugars and selected health outcomes are described in the Additional file 2. There are currently no methodological standards for evidence mapping. The steps of our evidence mapping included: 1) developing a comprehensive literature search strategy; 2) establishing study eligibility criteria and a systematic study selection process; 3) extracting data; 4) developing groups of health outcomes with input from the stakeholder panel; and 5) tabulating data using descriptive analyses [13]. One of the most important features of the evidence mapping process is the cataloging of a large number and variety of outcomes reported in the published literature. For this project, the research team solicited feedback from the stakeholder panel to ensure that the results would be useful for their discussion on research needs. Outcomes were grouped into clinically and biologically relevant categories based on stakeholder input and the research team’s expertise. (Fig. 2) A summary report of the descriptive evidence-mapping analysis was then presented to the stakeholder panel to generate discussion about research gaps and future research needs, which took place during a one-day in-person meeting (see Appendix B of the Additional file 1). All data were stored on SRDR™ (available on http://srdr.ahrq.gov/projects/136). The goal of the stakeholder panel’s discussions was to bring out different perspectives on the research questions and priorities. Therefore, scientific evidence was not required to support the statements or opinions raised by the stakeholder panel.

Fig. 2
figure 2

Analysis of outcomes reported in the studies in step-2 evidence map database by the outcome groupsa ranked as highest priority by the stakeholder panelb. Legends: a Seven outcome groups were formed, namely ‘cardiovascular disease risks,’ ‘diabetes risks,’ ‘body weight/body composition,’ ‘appetite,’ ‘liver health,’ ‘diet quality,’ and ‘mortality.’ Note top 7 categories include mortality but no studies reporting mortality outcome were identified in our evidence-mapping database. Total outcomes included 367 outcomes in the “other” category. b Outcome groups are displayed in different colors -- Cardiovascular Disease Risks (green bubbles), Diabetes Risks (light orange bubbles), Body weight/ Body Composition (combined groups, yellow bubbles), Diet Quality (red bubbles), Appetite (pink bubbles), and Liver Health (light blue bubbles). Numeric number in each bubble shows the number of studies in each outcome category. Abbreviations: Liver health: ALT- alanine aminotransferase, AST- aspartate aminotransferase, GGT- Gamma-glutamyl transpeptidase, GOT- serum glutamic-oxaloacetic Transaminase, DNL- de novo lipogenesis, IHCL-intrahepatic content of lipid, Cardiovascular disease risk: LDL- low density lipoprotein cholesterol, HDL- high-density lipoprotein cholesterol, VLDL-very low-density lipoprotein

Stakeholder panel

Stakeholders with various areas of expertise were identified and assembled according to a framework for stakeholder engagement in patient-centered outcomes research [14]. Stakeholders were selected to provide broad expertise and perspectives specific to the topic of dietary sugars and health. A total of 14 stakeholders with different backgrounds and expertise were initially identified through existing networks of both the sponsor and the research team (Table 2). Two members (1 lay audience and 1 statistical expert) dropped out due to personal reasons. The final stakeholder panel consisted of 12 members across seven stakeholder groups. Ten members participated as coauthors of the full report and this publication.

Table 2 Initial FRN Stakeholder Panel

Stakeholders engagement and research needs prioritization

Stakeholders participated in a one-day in-person meeting, chaired by the principal investigator (M.C.), to discuss, refine, and formulate research questions based on the top eight outcome groups. Prior to the in-person meeting, 25 preliminary research questions were drafted based on the research recommendations made by the authors of the published systematic reviews and the research gaps identified by the evidence-mapping analyses. Stakeholders provided input through comments and discussions posted on a web-based discussion forum. During the in-person meeting, the stakeholder panel discussed and revised these preliminary research questions. After the in-person meeting, the stakeholder panel provided additional input through emails and a webinar to refine and combine the research questions before the final topic prioritization. A total of 14 sets of research questions were agreed to by the stakeholder panel. The Effective Health Care (EHC) Program Selection Criteria [15] (with slight modifications for applications to the field of public health nutrition) was then used to prioritize the research questions. Each stakeholder was asked to prioritize the research questions considering four dimensions (importance, desirability, feasibility, and potential impact), giving a score ranging from 1 (not very important) to 5 (very important) for each dimension. An overall priority score was then calculated by averaging the four dimension scores from all stakeholders for each FRN research question.

Results

The literature search identified over 13,000 citations. Of these, a total of 708 potentially relevant full-text articles were screened, and 213 studies (202 intervention studies and 11 prospective cohort studies) were included in our final evidence-mapping database. (Fig. 3) Table 3 summarizes the study design and population characteristics of the intervention studies in the evidence map. Briefly, the study populations were mostly adults (90%), 52% were healthy at study entry, and 62.6% were males. Study sample sizes ranged from 5 to 2,026 (mean 45) with mean age of 35.4 and mean BMI of 23.8 kg/m2. Over a third (36%) of these trials were studies with a duration of <1 day. All 11 cohort studies [1626] were conducted in adults, of which eight were in healthy populations, one in pregnant women, one in diabetics, and one in a study population with mixed health status. The sample sizes of these cohorts were large (mean sample size = 35,738) with a wide range of follow-up time (4 to 48 years).

Fig. 3
figure 3

Literature Search and Selection Process for Evidence Mapping. Legends: *Exclusion reasons were not tracked at this stage. A low threshold was used to exclude irrelevant abstracts, such as animal or in vitro studies, studies with no sugar exposures or outcomes of interest

Table 3 Summary of study design and population characteristics of the included intervention studies (N = 202)

The complete FRN prioritization results are described in Table 4. Specific findings related to the top three research questions are summarized below and in Table 5.

Table 4 Research Question Prioritization Results

High priority research question 1

What is the long-term (>1 year) effect of a reduction in the intake of sugars on body weight or body composition in overweight or obese adults and children?

Evidence mapping

A total of 53 intervention studies in our evidence map reported on body weight or body composition outcomes. Of these, 13 studies were conducted in overweight or obese populations. There were an equal number of studies examining the effects of isolated fructose or sucrose on body weight, but fewer studies examining the effects of sucrose or other sugars on body composition outcomes compared to studies of isolated fructose. Only two studies had intervention durations of longer than three months. None lasted for more than one year. Of the 11 cohort studies [1626], one study in adults investigated the relationship between simple or added sugars and change in waist circumference after five years [20].

A published systematic review and meta-analysis of randomized controlled trials and prospective cohort studies concluded that among free living people on ad libitum diets, intake of dietary sugars or sugar sweetened beverages is a determinant of body weight [10]. The findings suggested that the change in body composition that occurs with modifying intakes is mediated by changes in energy intake, since iso-energetic exchange of sugars with other carbohydrates was not associated with weight change. Note that this systematic review and meta-analysis was used to support the 2014 draft WHO guidelines on intake of sugars for adults and children [27].

Stakeholder discussions

There are several challenges in conducting research on this topic that need greater emphasis. The stakeholder panel agreed that in addition to body weight, body composition should also be measured in future studies. Currently, there are several tools available for measuring body composition, such as Dual energy X-ray absorptiometry (DEXA). DEXA is used as the gold standard (over 99% accuracy) for measuring whole body adiposity but requires specialized equipment and is therefore expensive and cannot be measured in large, field trials. Waist to hip ratio is commonly used to measure body composition.

The stakeholder panel discussed the possibility that healthy people maintain body weight through physical activity. Thus it is important to objectively measure physical activity when designing studies to address this research question (also see Table 6 for cross-cutting study design considerations). Furthermore, there is a need to examine whether there are differences in the long-term (>1 year) effects of dietary sugars on body weight or body composition between healthy and overweight or obese populations. The stakeholder panel considered the possibility that overweight or obese adults and children might be more susceptible to the effects of “high” sugars intake (not well defined) on body weight compared to healthy counterparts, but there are no studies addressing this question.

High priority research question 2

Do dietary sugars have a different impact on the location of body fat deposition than other macronutrients?

Evidence mapping

Body fat deposition is related to body composition but specifically refers to the patterns or locations of body fat distribution. There were no studies investigating the effects of dietary sugars on locations of body fat deposition in the evidence-mapping database, and no published systematic reviews on this topic as of May 2013.

Stakeholder discussions

The question regarding the effects of dietary sugars versus other nutrients on fat deposition was first brought up by the lay audience stakeholder in lay terms (“whether consuming sugars would put more fat on my waist than on other parts of my body?”). This question was then translated into a research question by the expert stakeholders in the panel. The panel agreed that basic science in this area is limited, especially on how sugars may selectively drive fat deposition and whether sugars are different in terms of their effects on fat deposition compared to other energy containing nutrients such as protein or fat. The stakeholder panel discussed the fact that fructose, and possibly sugars in general, would likely drive fat deposition under hyper-caloric conditions based on the findings from studies of hyper-caloric fructose feeding on de novo lipogenesis. Age, gender, ethnicity, physical activity and exercise are also likely to play a role in individual metabolism. A related research gap is whether disease risks change with the location of fat deposition (subcutaneous versus visceral) in response to sugars intake.

High priority research question 3

What is the effect of sugars intake on satiety and hunger mechanisms?

Does sugars intake affect leptin and ghrelin levels, appetite, or fullness?

What are the different effects on satiety and appetite between different types of sugars (fructose, sucrose, high fructose corn syrup, honey, added sugar vs. intrinsic)?

Evidence mapping

Currently, the appetite control literature includes mainly short-term experimental studies. In the evidence map, a total of 32 intervention studies reported outcomes related to appetite, such as satiety, ghrelin, leptin, hunger, appetite score, caloric compensation, peptide tyrosine tyrosine (PYY), saliva flow, and taste. Of these, 31 studies were conducted in young adults (mean age of 30 years) with an average BMI of 26.0 kg/m2 (average of mean BMI across studies) and trial durations ranging from 1 day to 16 weeks. There were no published systematic reviews on this topic as of May 2013.

Stakeholder discussions

It appears that little is known about the relationships among sugars, appetite, and brain responses in humans, although this has been examined in animal models [28, 29]. One stakeholder pointed out that the current evidence shows that appetite is mainly driven by brain mechanisms. Other stakeholders mentioned that obesity and body composition are also related to appetite. The literature from animal studies suggest that sugars may attenuate the brain’s satiety responses and thus alter appetite. However there is little data supporting this in humans and more research is needed in this area.

The focus of this research question was whether people compensate for calories from sugars by eating less of other foods or if sugars had a particular characteristic that facilitated individuals to overcome the usual compensatory mechanisms, thus resulting in people who consume sugars eating more calories than they would otherwise. Some researchers relate these mechanisms to the overstimulation of reward systems similar to drug addiction, and feel strongly that the form (liquid versus solid) in which sugars are presented may be important for this question. However, this hypothesis has not been verified by the level of hunger or by the level of biological mechanism, e.g. satiety hormones or other satiety signal levels. There is currently only one experimental study (crossover design) in humans that compared the effect of liquid versus solid forms of sugars on weight or adiposity and appetite in seven males and eight females [30]. No significant differences in hunger (self-reported measures using a hunger questionnaire) between liquid and solid load periods were found in this study. A larger study is needed to address this question.

There are many methodological issues in the assessment of satiety. Objective tools for consistently measuring satiety, hunger or appetite outcomes are greatly needed. Factors such as culture, level of anxiety, and physical activity can impact eating behavior but are technically difficult to incorporate in studies with currently available measures of appetite or satiety such as leptin, ghrelin, or adipokine levels. The need for objective tools that can be obtained on larger study samples for consistently measuring satiety, hunger or appetite outcomes is enormous.

The stakeholder panel agreed that this research topic has sufficient evidence from animal studies, but not from human studies. Although it is feasible to address this research question in humans, this topic was suggested to be of lower priority unless objective measurement tools that can be used in large population-based studies for measuring satiety, hunger or appetite outcomes are developed.

The stakeholder panel also discussed considerations for future studies related to the top three high priority research questions (Table 5), and identified crosscutting study design considerations for future research (Table 6).

Table 5 Considerations for future studies related to the top three high priority research questions
Table 6 Cross-cutting study design considerations

Discussion

This FRN report is intended to be used by researchers and funders to help improve the usefulness of the body of research evidence for decision makers [2]. This FRN assessment ranked research questions regarding dietary sugars that are of interest to general consumers, health care professionals, and scientific researchers. Research questions on body weight and appetite were ranked higher than questions relating to disease risks in this project. To our knowledge, this is the first FRN project to integrate evidence mapping to support an FRN assessment. Evidence mapping appears to be a useful process to inform the stakeholder panel on the scope, breadth, characteristics, and quantity of research on a broad topic. A birds eye view of a broad research landscape can provide information on trends in known research topics and emerging new areas of research. The evidence mapping also highlights where the research gaps exist, and may empower stakeholders who are not familiar with the topic areas to participate in the FRN discussions. Because this structured FRN approach is systematic and transparent, it can be translated and used in other fields.

Several challenges were encountered in this FRN project, highlighting the importance of engaging a diverse stakeholder panel in the process. Since the evidence mapping covers a large number of topics, it was challenging to organize the large number of outcomes. The evidence-mapping research team alone did not have sufficient expertise to categorize the large number of biological markers and endpoints. This challenge was resolved by seeking stakeholders’ input. However, our stakeholders also had different opinions in grouping and analyzing outcomes because of their diverse backgrounds and expertise. Final decisions were made based on a majority of the stakeholders’ opinions. Therefore it is likely that a different panel of stakeholders could reach different decisions. Moreover, it was difficult to define the ideal composition of a stakeholder panel when the topic areas of FRN discussions are broad. Efforts were made to create a diverse stakeholder panel across seven areas of expertise but it is inevitable that our panel did not represent all perspectives. Due to budgetary constraints, we did not use the Delphi method [31] to facilitate the consensus process of the stakeholder panel nor formally evaluate this FRN process. Significantly more resources (time and money) are needed to accomplish this goal. Our evidence mapping implemented a single screening process for eligible studies, so it is possible that a few abstracts were mistakenly rejected during this screening phase. In an effort to ensure all key studies were included, all included citations were cross-referenced with the reference list of relevant systematic reviews and meta-analyses. Finally, including members of the lay public in the FRN discussions was important to ensure that research questions were meaningful to general consumers, but engaging a lay audience in the panel discussions was challenging. Some lay members may have felt intimidated or may have been unable to follow the FRN discussions due to lack of familiarity with scientific language. Additional training and orientation of the roles of a lay audience in FRN discussions may mitigate this challenge. Although our FRN approach seems promising, there is still a need to evaluate whether this process is reproducible with a different group of equivalently representative stakeholders, and to formally conduct process evaluations to identify areas for improvement.

Conclusions

Involving stakeholders of different backgrounds in the research process, and using a multidisciplinary stakeholder panel to help prioritize the direction of future research may have the potential to add diverse perspectives on how research needs are defined or determined, and to inform public health policy. Too often researchers are removed from the people affected by their research –general public/patients, practitioners/providers, or policy makers. Our experience is in line with the Patient-Centered Outcome Research Institute (PCORI), with a mission to bring together all healthcare stakeholders -- focusing on the community of patients -- to help set research priorities to ensure that the most relevant research is funded and conducted. A methodological research study found that AHRQ’s FRN projects demonstrated greater short-term (e.g., greater awareness of the issues) than medium-term outcomes (e.g., the generation of new knowledge) or long-term impact outcomes (e.g., changes in patient practice or health outcome) [32]. We expect our FRN project would have similar impacts. Moreover, our open evidence-map database can be used by researchers, research funders and policy makers to identify research gaps and to evaluate the quantity of evidence accumulated for a specific research questions. Hence it has the potential to promote dissemination and generation of new knowledge. With continuous updating and data quality control, our open evidence-map database would have the potential to promote knowledge translation from nutrition science to policy.