Introduction

The need for a scoping review on physical activity evidence types is two-fold: First, the World Health Organization (WHO) Regional Office for Europe and its Member States are implementing the first European Strategy on Physical Activity to support the reversal of trends in rates of certain non-communicable diseases and obesity (World Health Organization 2014). They seek to ground the strategy on current scientific evidence. Second, in a WHO working group convened to inform the new strategy, experts agreed that the rapid increase of published research on the subject of physical activity necessitates a fresh look at the body of available evidence. Of special interest is the distinction between three types of evidence: Type I evidence pertains to physical activity and health, i.e. studies that link physical activity to risk factors or health outcomes. Type II evidence pertains to physical activity interventions, i.e. studies that link interventions to physical activity behavior. Type III evidence pertains to physical activity policy, i.e. studies that link policymaking to physical activity (cf. Brownson et al. 2009; Martin-Diener et al. 2014).

Some assumptions were made regarding evidence types before starting the scoping review. Policy-makers and researchers involved agreed that Type I and Type II evidence might be much more developed than Type III evidence. For example, the landmark review of the Physical Activity Advisory Committee (United States Department of Health and Human Services 2008) provided an extensive overview of Type I evidence. Likewise, Type II evidence has been broadly covered in previous publications e.g. Heath et al. (2012) and World Health Organization (2009). In contrast, very few attempts have been made to specifically establish the evidence-base for physical activity policy-making. Some approaches have developed good practice criteria based on literature reviews (Bull et al. 2014). Others completed content analyses of national-level policy documents (Daugbjerg et al. 2009; Vestmark et al. 2011) or focused on the use of evidence in physical activity policies (Aro et al. 2015).

To our knowledge no review systematically integrates and compares the literature available across the three different evidence types. A comparison of this kind would increase understanding of the extent, range and nature of the available evidence and the evidence gaps in physical activity research. It could also be used to ground policies on all types of current scientific evidence. Moreover, knowledge derived from the comparison might be useful to guide future research, support efforts of pivotal organizations (Martin et al. 2006) and encourage innovative funding schemes.

We first explain our conceptual approach. Second, we give an overview of the scoping review methodology. Third, we present comparative results of the three evidence types regarding (a) number of reviews found, (b) kinds of reviews, (c) first author country affiliation, (d) dates of publications, and (e) most frequent and/or trending topics. Fourth, in the discussion and concluding remarks we relate results to previous work, discuss limitations of the study and indicate future research needs.

Conceptual approach

A key aim of this paper is to refine the distinction between three types of evidence relating to physical activity. A second aim is to compare evidence types for the purpose of identifying potential research gaps. Type I evidence focuses on the link between physical activity and health status, i.e. the type, amount and intensity of physical activity and its effects on different health outcomes. For example, this type of evidence might demonstrate how physical activity reduces the risk of diabetes. Type II evidence links interventions with physical activity, i.e. the type of intervention, setting and/or environment that influences physical activity. For example, this type of evidence might demonstrate how settings-based interventions support increased physical activity. Effective environmental and policy interventions to promote physical activity also belong to Type II evidence. Type III evidence focuses on the effects of policy-making on physical activity interventions, i.e. the policy agendas, structures, funding and processes that affect development, implementation or adaptation of physical activity interventions. For example, this type of evidence may demonstrate how cross-sectoral approaches to policy-making help position physical activity promotion on the agendas of different policy sectors and policy levels.

Regarding Type II and Type III evidence, policy plays a role in both. Policy is involved in Type II evidence as a part of interventions to increase physical activity. Policy-involved interventions, however, result from the dynamics of policy-making processes (Type III evidence). For example, tax incentives that encourage active transportation may result in more active lifestyles in the general population. These incentives, as part of interventions, can be deemed Type II evidence. The policy-making process of reaching consensus on the incentives and examining the political environment can be deemed Type III evidence.

The distinction between Type I and Type II evidence is widely addressed in the literature. Making the distinction between Type II and III evidence is more innovative. Doing so may help overcome conceptual hurdles in previously published literature, which often conflates “policy and environmental approaches” or deals with policy approaches as a type of (physical activity) intervention (Brownson et al. 2006; Matson-Koffman et al. 2005; Rychetnik et al. 2002; Sallis et al. 1998). We propose to better distinguish research about policy interventions (including specific programs and projects) from policy-process oriented research that may more often be explored by the disciplines of social or political science. We recognize that some reviews may contain evidence relevant to more than one evidence type. This scoping review was designed to identify the main emphasis in each of the review articles that were analyzed to ascertain how the different types of evidence are being prioritized in the literature. This distinction is being made for conceptual, research-driven, and policy-driven reasons.

First, a conceptualization of the evidence of physical activity policy-making is needed. It will lend to a better understanding, for example, of how policy-making structures and processes in different countries influence common factors associated with effective physical activity interventions. This conceptual perspective builds on previous models and frameworks that connected policy issues with physical activity. For example, Sallis et al.’s (2006) ecological model for active living refers to a distinct category of the policy environment. Likewise, Schmid et al.’s (2006) framework for physical activity policy research underlines the need for research on policy agenda setting (“the determinants of establishing policy”, pS22) and policy processes (“process of developing and implementing policies” pS22). Most recently, Rütten et al.’s (2013) theory-driven model focuses on the interplay of policy, the environment, and physical activity behavior.

Second, there is an emerging body of research outside the physical activity context, but highly relevant to it, with a special focus on policy agenda setting, policy structures and processes in public health (Commission on Social Determinants of Health 2008). This research points to distinct evidence types regarding policy-making. For example, Lin et al. (2012) synthesized the evidence on “how governance structures can trigger governance action to support Health in All Policies”. Similar to our focus on a distinct evidence type related to physical activity policy-making, this evidence synthesis “was developed to advance knowledge on how to effect” policies (in this case Health in All Policies) through policy-making (here: “through intersectoral governance”) (Lin et al. 2012, p23). Moreover, there are several recently published reviews that focus on the evidence of intersectoral policy-making on social determinants of health and on health equity (e.g. Chircop et al. 2014; Ndumbe-Eyoh and Moffatt 2013; Rantala et al. 2014; Shankardass et al. 2012).

Third, there is a need for specific recommendations based on Type III evidence for physical activity policy development and implementation both at international and national levels. One example is the aforementioned WHO European Physical Activity Strategy. Inclusion of detailed recommendations about evidence-based strategies regarding agenda setting and other policy processes stands to strengthen and complement recommendations associated with Type I and Type II evidence. Another example is related to the European Commission’s efforts to implement the EU Physical Activity Guidelines (2008) as a framework for policy development (The Council of the European Union 2013, C 354/2).

Finally, when establishing a conceptual approach to reviewing three types of evidence similar perspectives were sought within public health literature. Comparable distinctions of evidence types have been made by Brownson et al. (2009) and Martin-Diener et al. (2014). Type III evidence in Brownson et al. (2009), however, does not specifically relate to policy (which is essential for our approach). Instead, it relates to “context” in a much broader sense. Their Type III pertains to evidence “needed to adapt and implement an evidence-based intervention” (2009, 179). In their framework Type III “political and economic” evidence describes just one kind of context variable at a particular level, which is distinguishable from other Type III context variables at other levels (e.g. “individual, interpersonal, organizational, and sociocultural). Martin-Diener et al. (2014, 8) integrated the typology of Brownson et al. (2009) in their “HEPA Europe Framework”. They relate Type III evidence to a much broader type of information on “How can be done what should be done?” They explicitly mention “policy process” as a category related to this evidence type but do not further conceptualize in this paper what that means related to their broad-ranging question quoted above.

Research question

The research question was clarified in an August 2014 sub-committee meeting of the WHO working group on a European Physical Activity Strategy. It was designed to foster three different, yet comparable, lines of systematic inquiry regarding the evidence types, with a pre-established recognition that Type III evidence is increasingly relevant: The question was also designed to ensure that research was gathered from studies across the lifespan.

What are the extent, range and nature of scientific evidence that exists relating to physical activity in three different areas?

  • Type I—evidence that links physical activity to health outcomes.

  • Type II—evidence that links interventions to physical activity behavior.

  • Type III—evidence that links policy-making to interventions.

Methods

Scoping reviews systematically synthesize knowledge (Grimshaw 2010) to ascertain the extent, range and nature of research in a particular area of inquiry (Arksey and O’Malley 2005). They usually involve iterative, qualitative approaches to analyzing, describing, mapping and reinterpreting available research (Levac et al. 2010). This is useful for gaining a better understanding of gaps in evidence, pivotal concepts, perspectives and trends in research. It is also useful for determining whether further exploration and analysis of the literature might be prudent (Armstrong et al. 2011).

This scoping review was conducted by a research sub-committee of the working group from the WHO European Strategy on Physical Activity for the purpose of overcoming the conceptual vagueness and blurry distinctions between the three types of evidence mentioned above, and to advance the knowledge base regarding the place of policy in the increasingly relevant implementation of physical activity interventions to combat social inequalities and non-communicable diseases.

In July 2014 the research sub-committee began a six-step scoping review process, adhering to the methodological framework set forth in Arksey and O’Malley (2005) and expanded by Levac et al. (2010): (1) clarify the research question, (2) identify relevant studies, (3) select studies, (4) chart and describe studies and data, (5) summarize and report results, (6) engage in consultation process with sub-committee members and key stakeholders. Sub-committee members represented a multi-disciplinary team of academics, policymakers and practitioners. Library scientists at Friedrich Alexander University, Erlangen-Nuremberg were consulted throughout in reference to the selection of key search terms, databases and pilot search processes. As part of Step 6 all scoping review activities involved ongoing consultation (e.g. in-person, telephone and Skype meetings) with stakeholders on the research sub-committee. Meetings were held at the outset of the project and at the key intervals of data collection and analysis addressed in Fig. 1 below. It was at these meetings that it was determined how to proceed with data analysis and reporting.

Fig. 1
figure 1

Scoping review search process

Full size image

In this section we focus on Arksey and O’Malley (2005) steps 1, 2 and 3. Steps 4 and 5 are addressed in the following section, where results are charted and summarized. Step 6 (consultation with relevant stakeholders) is addressed throughout the discussion.

Identifying relevant studies

Relevant studies were identified between September 2014 and November 2014. Pilot searches took place between October 8 and 31, 2014. Final searches took place between November 1 and 15, 2014. Selection and analysis of relevant studies took place between December 2014 and May 2015. Key search terms were selected based on results from more than 200 pilot searches along with a backwards keyword search (Levy and Ellis 2006). The backwards keyword search was done using sets of pre-identified core documents compiled for each evidence type. Final search terms were considered in relation to grammatical variation, i.e. singular and plural form, tense, etc. They were selected and defined based on each evidence type:

  • Type I combined variations of the terms “health, physical activity, evidence, effect and review”.

  • Type II combined variations of the terms “intervention, physical activity, evidence, effect and review”.

  • Type III combined variations of the terms “policy, physical activity, evidence, effect and review”.

Titles and abstracts in PubMed, Scopus, SportDiscus, PsycInfo, ERIC (Education Resource Information Center) and IBSS (International Bibliography of the Social Sciences) were searched independently by two researchers and in consultation with an institutional librarian. Searching incorporated Boolean operators, advanced search features and grammatical variations of search terms. While the type and number of search terms could easily be expanded in any direction, especially in relation to synonyms for physical activity (e.g. exercise), doing so would make the comparison of results across evidence types inconsistent. This review, however, is intended to be the first of a series of reviews that will build upon results in a step-wise process. The next round of reviews may tailor new combinations of keywords based upon what was learned from this review.

Inclusion criteria and exclusion criteria

At the outset, it was recognized that there might exist a vast amount of research in one area and a limited amount of research in another. Therefore, it was determined that prior to exploring single studies, the literature would be searched for various types of reviews. A variety of kinds of reviews was purposefully sought (e.g. systematic reviews, narrative reviews, non-systematic reviews), which fostered openness to identifying evidence produced from different disciplinary sources (e.g. public health, clinical medicine, social science, political science). All inclusion/exclusion criteria for the initial search, which sought to compare across the three different types of evidence, was the same. Any document type in English, French, German or Spanish was considered. Any study focusing on humans that addressed physical activity and was discussed by the authors in relation to effectiveness of evidence was considered, regardless of outcome, type of intervention, target group or exposure.

Documents were excluded if they did not address humans, made no connection to physical activity or were not considered to be an evidence review. For example, while some documents may have been retrieved that contained the word “review”, they may have, in fact, been a single study. Those remaining were included as part of conceptual mapping and comparative analysis.

Selecting relevant studies

Relevant studies were selected and analyzed between December 2014 and May 2015. A team of three researchers checked for duplicate references using EndNote plus a manual duplicate search. Once duplicates (Type I n = 958, Type II n = 1106, Type III n = 146) were removed, the research team conducted a preliminary screening of the remaining documents (Type I n = 1054, Type II n = 1035 Type III n = 163) in each category. After the preliminary screening it was found that the titles and abstracts contained a significant overlap in relation to the different evidence types (i.e. health, intervention and policy) (see Fig. 1 for overlap details). Because of this overlap, it was decided to combine the documents (n = 2252), eliminate duplicates (n = 756) in a second step and then conduct a screening and sorting process on the remaining documents (n = 1496).

References were then imported into EPPI Reviewer 4 where a team of researchers independently assessed a sample of the documents. A final screening of all references eliminated false positives (n = 250) (e.g. those that were not reviews, did not address physical activity, or did not relate to humans). All documents not identified as false positives were deemed relevant and selected for qualitative comparison (n = 1246).

Results

According to the conceptual approach described above, 856 reviews were classified as Type I, 350 as Type II, and 40 as Type III. The clear difference in totals confirms the initial hypothesis of the working group that Type III evidence is less developed and less differentiated in terms of conceptual variables that might advance the field of physical activity policy-making research.

For more detailed description, the results were sorted into cascading sub-categories: (1) kind of review, (2) country of first author’s institution, (3) date of publication, and (4) emergent themes.

There are a variety of kinds of reviews to consider when ascertaining the extent, range and nature of research that originates from different disciplines (Coughlan et al. 2013). In relation to physical activity, systematic reviews are often used to synthesize results from quantitative studies such as clinical trials and appraise effectiveness of interventions (Grant and Booth 2009). They tend to adhere to guidelines such as those put forth by Cochrane (Higgins and Green 2011). Meta-analyses combine results from systematic reviews and present additional analyses. In contrast, qualitative reviews, concept analyses and narrative reviews do things such as clarify conceptual approaches to interventions, describe existing qualitative evidence and effectiveness of interventions and critique claims of effectiveness of interventions. Different types of reviews contribute to the evidence-base in meaningful ways. Systematic reviews and meta-analyses are most prominent regarding Type II evidence (more than 50 %) and represent almost 40 % of selected reviews of Type I evidence (see Additional File 1, Table 1). In contrast, these approaches only represent 20 % of the Type III reviews. The bulk of research in each evidence category was generated in the United States, United Kingdom, Australia and Canada. Two thirds of Type I reviews, three-quarters of Type II reviews and almost 90 % of Type III reviews had first author institutions from these four countries (see Fig. 2).

Fig. 2
figure 2

First author institutional affiliation by evidence type and country

Full size image

Fifty-seven percent of Type I reviews were published between 2010 and 2014 and twenty-five percent were published between 2005 and 2009. The remaining eighteen percent were published between 1980 and 2004. Sixty-five percent of Type II reviews were published between 2010 and 2014 and twenty-three percent were published between 2005 and 2009. The remaining eleven percent were published between 1980 and 2004.Footnote 1 Seventy-three percent of Type III reviews were published between 2010 and 2014 and twenty-three percent were published between 2005 and 2009. The remaining four percent were published between 1980 and 2004 (Fig. 3).

Fig. 3
figure 3

Distribution of studies according to evidence type and year of publication

Full size image

Eighteen percent of Type I reviews (Fig. 4) have a primary emphasis on obesity/overweight/weight management. Fifteen percent had a primary emphasis on mental/cognitive/neurological issues. Following these in descending order the combination of: (1) cardiovascular/circulatory/stroke issues (2) musculoskeletal issues (3) cancer/neoplasms and 4) diabetes types 1 and 2 represent thirty-two percent of reviews. Ten percent of reviews were categorized as “broad” because they addressed a variety of diseases/illnesses and/or impairments.

Fig. 4
figure 4

Classifications of type I (health) reviews relating to disease/illness/impairment

Full size image

Regarding Type II (Fig. 5), the most prevalent target groups that reviews focused on were children and adolescents (23 %), followed closely by adults (21 %) and chronically ill or disabled (20 %). Older adults (5 %) and special groups (5 %) were less often addressed. Ten percent of reviews were classified as “broad” because they included studies relating to more than one target group or were not focused on addressing target groups.

Fig. 5
figure 5

Classifications of type II (intervention) reviews based on target groups

Full size image

The most prevalent setting in Type II was health care (31 %, see Additional File 1, Fig. 1). This was followed by community and city (13 %), school, after school (i.e. the after school period during the day), university (11 %), worksite (6 %) and home (3 %), respectively. Thirty-five percent of reviews were categorized as broad if they addressed multiple settings or were not focused on settings. The most prevalent intervention type in Type II reviews was counseling/education/referral (24 %, see Additional File 1, Fig. 2). This was followed by technology and computer-based (10 %), exercise and training (9 %), information-based (8 %), environmental (7 %), social and organizational (7 %), community-based (3 %) and mass media (1 %). Thirty percent of reviews were categorized as “broad” if they addressed multiple intervention types or were not focused on interventions.

The majority of Type III policy reviews focused on school-based polices (20 %) (Fig. 6). This was followed by broad-range policy interventions (i.e. those that reach across more than one topic) (17.5 %), child/youth policies (12.5 %), urban design/transport policies (12.5 %) and environmental policy (7.5 %). Broader range policy interventions that address a specific target group, policy instruments, adults and health care sector each represented 5 % of reviews. Child care policies, sport/competitive policies, workplace policies and dissemination represented 2.5 % of reviews.

Fig. 6
figure 6

Classification of type III (policy) reviews

Full size image

Discussion

This discussion addresses how the results of this scoping review achieve the aims of clarifying the distinction between evidence types and helping understand where potential research gaps can be filled. It also provides insights into evidence-based policy-making in the areas of health and health promotion in several ways.

First, the outcomes confirmed the hypothesis that evidence for the three types of physical activity research differs in terms of quantity and characteristics of reviews available in scientific journals. For example, the number of reviews on health effects of physical activity (Type I) was more than twenty times higher than those on physical activity policy-making (Type III). The kind of reviews ranked highest in the hierarchy (Higgins and Green 2011), e.g. meta-analyses and systematic reviews, were most prominent in research on physical activity interventions (Type II) and were least found in policy-related reviews. Some of this difference may be attributed to the appropriateness of study relevant to each evidence type. Type III evidence may be more appropriate for analysis in reviews that are not deemed systematic. Even so, the number of Type III reviews found was much lower than Type I and Type II reviews. These outcomes mirror the conclusion from Breton and de Leeuw’s (2008) systematic review that policy research in health promotion is still in its infancy. They also demonstrate that previous calls from researchers and public health organizations (e.g. Schmid et al. 2006) to give higher priority to policy-related research on physical activity are still very relevant today.

Second, although this scoping review was not limited to English language publications, most results show a first author affiliation for the United States, United Kingdom, Australia, or Canada. This implies a need to: (1) extend the scoping review to other than English languages/data bases to determine the availability of evidence from non-English language sources, (2) consider placing more emphasis on policy-related physical activity research in low and middle income countries (e.g. revisit the distribution of funding support, increase access to technological resources and implement capacity development measures for researchers and their institutions) (see Kohl et al. 2012) and (3) draw attention to this dynamic when conducting or reviewing research that accompanies current policy strategies in Europe such as implementation of the WHO Physical Activity Recommendations, EU Physical Activity Guidelines (EU Working Group “Sport and Health” 2008) and development of the new WHO European Physical Activity Strategy.

Third, the results demonstrate the current dynamic of physical activity research. It started to grow in the mid-1990s but developed more rapidly in the mid-2000s. The initial development may reflect the research impact of landmark scientific publications and policy documents, e.g. the US shift in recommendations towards the concept of health-enhancing physical activity (Pate et al. 1995), the Surgeon General’s Report on Physical Activity and Health (United States Department of Health and Human Services 2008), and the WHO Global Strategy on Diet and Physical Activity (World Health Organization 2004). More than half of the reviews in each evidence type were published in the last five years. While the absolute numbers of reviews found for this time period are the highest in Type I (about 500), compared to Type II (230) and Type III (about 30), the relative increase is highest in the policy related category where almost three quarters of the reviews were published since 2010. This may reflect an increasing emphasis on physical activity policy research by important public health organizations (see Schmid et al. 2006).

Fourth, the majority of Type I reviews primarily focused on obesity/overweight and weight management (e.g. Astrup 1999; Bonfioli et al. 2012; McPherson et al. 2014). These reviews merit further consideration in relation to the link between physical activity, weight management and different types of health outcomes. Further analysis might, for example, identify what types of interventions were used in the studies that were reviewed to promote obesity reduction. Thirteen of the Type I reviews reflect a potential growing interest in inflammation/immunity, as reflected in the literature, (Harvey et al. 2011; Lee and Pratley 2005), and how physical activity may play a key role (Packer et al. 2010, Romeo et al. 2010). Eleven percent of Type I reviews contained the word “sedentary” in either the title or abstract, reflecting increasing trends in relation to health outcomes and sedentary behavior.

Fifth, Type II evidence reviews showed an emphasis on health care in terms of settings (31 %) (e.g. Hinrichs and Brach 2012; Vancampfort et al. 2012) and young people in terms of target groups (23 %) (e.g. Chaput et al. 2013; Floriani and Kennedy 2007). In terms of intervention types, counseling/education/referral reviews (e.g. Cramp et al. 2013) are highly represented (24 %) as traditional forms of physical activity and health promotion. The presence of technology/computer focused intervention types (10 %) (e.g. Hamel et al. 2011) may signal their increasing role in physical activity promotion. Fourteen percent of Type II reviews contained the word “sedentary” in either the title or abstract, also reflecting trends in relation to physical activity interventions and sedentary behavior.

Sixth, as anticipated prior to this review, the results in the Type III category tend to address policy effectiveness (e.g. de Nazelle et al. 2011; Heath et al. 2006; Kumanyika et al. 2014), conflate policy with environmental approaches (e.g. Barton 2009; Sallis et al. 1998; Shill et al. 2012) and focus more on research as a method to influence creation of policy rather than on research as a method to assess policy-making processes, with the caveat that some studies do touch on this aspect (e.g. Patrick et al. 2009; Wiseman 2010). It may be that fewer Type III reviews were retrieved because conducting Type III studies would logically occur after Type I and Type II studies on a particular subject have already been completed, putting such studies further into the future and possibly out of reach of practical time constraints. Additionally, studies differ in their levels of complexity, and therefore feasibility, making Type III studies more challenging to implement in certain social, economic or political environments. It is the intention of the WHO European Strategy for Physical Activity research sub-committee to continue scoping the Type III evidence regarding physical activity and policy-making.

Limitations

There are limitations to this study. First, the heuristic of the three types of evidence was limited. An overlap of evidence types exists within some of the reviews. For example, reviews that focused on policy and environmental interventions also included evidence about policy-making processes. To address this issue reviews were categorized by the evidence type that was most emphasized. Second, the three types of evidence explored were not intended to capture other important physical activity evidence review types, such as physical activity determinants (e.g. Bauman et al. 2012). This review is a first step in looking to new avenues of extracting and selecting what is “relevant” from the large quantity of literature that is retrieved through electronic resources used for this type of qualitative review.

Conclusions

The distinction between three types of evidence serves as a useful heuristic to review the scope, dynamic and diversity of physical activity research in public health. This review demonstrates that this field is rapidly developing. Nonetheless, the production of Type III evidence is still in its infancy. It has been indicated that certain policy influences (e.g. Surgeon General Report (US DHHS 1996), WHO Global Strategy (WHO 2004) may have supported the production of Type I and Type II evidence in the past. Likewise, on-going policy initiatives such as the new WHO Physical Activity Strategy for the European Region could support the production of Type III evidence by emphasizing respective policy-related research. Continued analysis of documents within this scoping review is planned. It will focus on key studies that can inform recommendations regarding how to increase the quantity and the quality of Type III evidence with an emphasis on integrating stronger theoretical foundations and improving policy-process research methodology for the area of physical activity. While not within the purview of this review, it is important to mention that in the last decade sedentary behavior has emerged as a health risk, independent of physical activity. Further research will consider this topic. It will also seek to clarify the distinction between Type II evidence, i.e. what works in physical activity promotion, and Type III evidence, i.e. what works in policy-making for the purpose of developing concepts and providing an appropriate framework to guide future implementation and research.