Introduction

Obesity is a pressing public health issue. In the past three decades rates of overweight and obesity have increased dramatically in most industrialised countries, with increases also observed in several low-income contexts [1]. This global epidemic is of particular concern for children and young people. Almost a third of children in America and a fifth in Europe are overweight or obese [2,3]. Childhood obesity strongly ‘tracks’ into adulthood [4], with implications for morbidity and premature mortality [5].

Obesity is a complex condition requiring equally complex solutions. The World Health Organization (WHO) suggests this requires action in multiple settings, using a variety of approaches and involving diverse stakeholders [6]:p16. A key element is complex, multi-component interventions implemented in schools targeting key determinants of obesity: namely, physical activity and nutrition [7].

One such approach is the WHO’s Health Promoting Schools (HPS) framework. The HPS framework recognises the inherent, reciprocal link between health and education: healthy children achieve better educational outcomes which, in turn, are associated with better health later in life [8]. Inspired by the principles of the Ottawa charter and cognisant of the failure of health education alone to improve health outcomes, the HPS framework takes an eco-holistic approach to creating school environments conducive to health and healthy behaviours [9].

While definitions vary [8-14], HPS initiatives comprise: (1) health education promoted through the formal school curriculum; (2) changes to the school’s physical and/or social environment; and (3) engagement with families and the wider community in recognition of the influence of these on children’s health.

The HPS framework has proved popular in tackling obesity and other important public health issues such as cardio-vascular disease and Type II diabetes [11,15-17]. Half of the 67 trials included in our recent Cochrane review of the HPS framework targeted physical activity and/or nutrition [17]. Overall, we found intervention effects for improvements in students’ levels of physical activity, physical fitness, and fruit and vegetable intake. We found no overall effect for reducing students’ fat intake. The evidence for BMI and zBMI (standardised by age and gender) was equivocal. Surprisingly, given the underlying aim of the HPS framework, no study presented data on student academic attainment or attendance [17].

Within these studies we identified considerable heterogeneity in intervention effects. Given the complexity of these interventions and the variability between studies, it is important to consider why some were effective while others were not. Process evaluations can suggest explanations, helping to identify what works, for whom, in what contexts and why [18].

The aim of this paper was to synthesise process evaluation data presented in these studies to identify factors that helped or hindered implementation and/or success. Our findings have implications for the development of future trials and the implementation of programmes beyond the trial context.

Methods

Inclusion criteria

Full details of the methods can be found in the Cochrane review [17]. We included cluster randomised controlled trials (RCTs), with clusters at the level of school, district or other geographical area. Participants were students aged four to 18 years attending schools/colleges. As the HPS framework is not necessarily a term recognised in all countries, we did not require interventions to be explicitly based on the HPS framework. Rather, to be eligible interventions had to demonstrate active engagement in all three HPS domains, namely: curriculum, environment, and families and/or communities. Control schools offered no intervention or standard practice, or implemented an alternative intervention that included only one or two of the HPS criteria. For the purposes of this synthesis of process data, studies were included if they: took an HPS approach; focused on physical activity and/or nutrition; and presented process evaluation data.

Search strategy

We searched the following databases and trials registries using broad and inclusive search terms: ASSIA, Australian Education Index, British Education Index, BiblioMap, CAB Abstracts, Campbell Library, CENTRAL, CINAHL, Database of Educational Research, EMBASE, Education Resources Information Centre, Global Health Database, International Bibliography of Social Sciences, Index to Theses in Great Britain and Ireland, MEDLINE, PsycINFO, System for Information on Grey Literature in Europe, Social Science Citation Index, Sociological Abstracts, TRoPHI, Clinicaltrials.gov, Current Controlled Trials, and International Clinical Trials Registry Platform. We also searched relevant websites and reference lists of relevant articles. Searches were conducted in 2011 and 2013. No date or language restrictions were applied. One author performed an initial title screen, with a second screening a randomly-selected 10% of these for quality assurance (kappa score = 0.88). Thereafter, two reviewers independently screened abstracts and full texts to determine eligibility.

Data extraction

For each study, two reviewers independently extracted data pertaining to: study location, target age group, study duration, intervention content, outcome data and basic process data.

We undertook a thematic synthesis, adapting existing methods described by Thomas and Harden [19] to identify themes relating to programme implementation. More detailed descriptions of methods and quantitative and qualitative process findings were extracted verbatim from each study report by RL, that is extracting exactly the same words as the study author(s)’ used. Data were also extracted from discussion sections of reports when this addressed implementation or reasons for intervention success or failure. These extracts were read and re-read, an initial set of codes being developed and applied to the data. Some codes were identified a priori, focusing on aspects of process (acceptability, fidelity) while others arose inductively from the data (family involvement, barriers/facilitators). A Framework [20,21] approach was used to manage the data and assist analysis, whereby data from each study were summarised within a matrix under the following themes: intervention acceptability; implementation fidelity; family involvement; barriers to implementation; facilitators of implementation. This method allowed identification of similarities and differences between studies within themes.

Results

Our searches yielded 48,551 records (after deduplication), from which we identified 67 eligible studies (Figure 1). Of these, 34 focused on physical activity and/or nutrition. Twenty-six reported some process data and are the focus of this paper. Four studies focused solely on promoting physical activity, 11 on improving nutrition and 11 on physical activity and nutrition. Key characteristics of the interventions, including intervention activities carried out under the three HPS domains are presented in Table 1.

Figure 1
figure 1

Flow chart of study selection process.

Full size image
Table 1 Characteristics of studies
Full size table

Countries

Thirteen studies were conducted in the USA [24,25,27,28,30,33,43,46,47,51,54,55,58]. A further 10 were implemented in Europe; two in the UK [23,52], two in Norway [22,49], one multi-country study (involving The Netherlands, Spain and Norway) [35] and one study each in Belgium [50], Finland [26], France [39], Germany [42] and Switzerland [38]. Three studies were conducted in Australia [32,37,40].

Age groups

Of the 26 included studies, there were almost three times as many interventions conducted with younger children (≤12 years) compared to older children (19 vs. 7 studies, respectively). Among the former, most targeted students aged between 8-12 years (11 studies), while five included younger children. Studies focusing on older students tended to be conducted with 12-14 year olds. Only one study was implemented in grade 9 (14-15 years).

Quality of process data

The methods used to collect process data are summarised in Table 1. The quality and extent of process data varied greatly. Some studies conducted extensive process evaluations, examining different elements of the intervention implementation. Most studies only focused on fidelity and acceptability using quantitative data, with diverse methods and scales bespoke for each intervention (e.g. questionnaires, log-books, structured observations). No study reported on the reliability or validity of these scales. Other studies provided extremely limited process data; it was often unclear how these were collected. Authors’ conclusions about facilitators and barriers to implementation were included in some discussion sections but the evidence for these was generally unclear. However, these insights appeared to us useful and are included in our analysis. Few studies provided qualitative data. We summarise key themes below.

Key themes

Acceptability

Where reported, acceptability of the intervention to students, parents, and teaching or catering staff was generally high. Some studies provided quantitative assessments of acceptability. For example, teachers in the study by Bere et al. [22] all rated the intervention as ‘good’ or ‘very good’, while 70% of teachers and 90% of students participating in the KISS study reported they enjoyed the programme and wanted it to continue the following year [38]. Other studies reported acceptability in more general terms. For example, Hoppu et al. merely stated that ‘most of the feedback [on the intervention] was positive’ [26]:p975. Resources (such as sports equipment [48], stickers to promote fruit and vegetable intake [25] or pedometers [41]) were often highly rated by teachers and students. Training and support provided to teachers were also highly appreciated [44,52]. One study conducted structured interviews with Physical Education (PE) teachers to assess the intervention’s acceptability [48]. Some teachers were initially resistant to the new curriculum and implementation was lower in the early stages of the project. However, resistance lessened as they became familiar with the curriculum and could see the positive effect on student behaviour and activity levels.

Fidelity

Where reported, intervention fidelity varied. Some studies reported high levels of intervention fidelity [30,42,43,51,52]. In most studies, this was expressed as a percentage of intervention activities successfully implemented. For example, the CATCH trial reported 90% of food guidelines were met, 80% of PE activities were implemented and 88% of curriculum sessions were completed without modification [51]. Other studies reported much lower rates of implementation [22,23,27,34,46,54,58]. For example, one study [23] reported that despite high levels of acceptability, only 21% of intervention materials were implemented. Another [54] reported numerous problems with implementation, such as lack of volunteers or food preparation guidelines not being followed. Reynolds et al. [34] noted that taste-testing sessions were less likely to be undertaken because of the effort and disruption these caused. Importantly, they also noted that African-American and low socio-economic-status (SES) students were likely to receive lower doses of intervention activities, raising equity concerns. In the absence of additional contextual data, the authors were unable to explain this difference.

Family involvement

Despite being one of the three HPS domains, studies consistently identified engaging families as the most challenging and least successful intervention element. Almost all studies reporting on this indicated family engagement was low (typically only one-third to one-half of parents participating in any intervention activity) and authors frequently commented on the challenges of involving parents. The ICAPS trial noted parental attendance at meetings was poor (25-40%), particularly in low SES areas [39]. Similarly, Eather and colleagues acknowledged ‘parents are notoriously difficult to engage’ and many students were not supported at home in completing their intervention’s home-based activities [37]:p16. Two studies [36,41] identified language barriers in engaging non-English speaking parents. Even in the CATCH study where 70% of parents participated in some intervention activities, the authors noted the intensity of the family component was very low and thus unable to produce significant changes in children’s health behaviours [59].

A report on the Bienestar programme targeting Mexican Americans [55] identified very low family participation rates (17%), and study investigators conducted focus groups with parents to explore why [56]. Explanations included: misunderstanding the purpose of the programme and the family events offered; practical issues such as lack of transportation, babysitting or limited income; time constraints and conflicts with work schedules; embarrassment concerning parents’ own education and/or literacy levels; expectations of ‘boring’, didactic, teacher-led meetings; too many meetings offered; and assumptions that meetings were just for mothers and excluded other family members. In response, the program was modified to encourage greater participation leading to an increase in parental involvement, but nonetheless still only resulting in about a third of parents becoming involved (increase from 17% to 37%).

Facilitators of implementation

Making interventions relevant to the specific school context was noted as important. Needs assessments were undertaken in schools in some studies [32,52] helping to create interest and motivation within schools. Several studies encouraged schools to tailor intervention components to local needs. Equally, materials and messages needed to be culturally relevant, especially when targeting specific ethnic groups [43,55], helping create local ownership.

Good communication and thorough training were also noted as essential. Nicklas and O’Neill [29] explained that schools need to feel confident they are able to carry out the required tasks and adequate training is essential in achieving implementation fidelity. Researchers on the 5 A Day Power Plus study similarly recognised training and staff development as critical to implementation [31]. Teachers in this study received on-going training with one-on-one feedback and support. To facilitate this, the study reimbursed schools for the cost of substitute teachers and paid catering staff to attend training.

Understanding schools’ ‘core business’ also appeared critical. As Nicklas and O’Neill pointed out, successful school health interventions’ objectives aligned with teachers’ goals for their students [29]. Radcliffe et al. [32], for example, explained that schools in their study saw the intervention programme as related to the core business of the school. A needs assessment identified lack of breakfast as a particular problem and teachers were convinced that this was associated with poorer concentration and classroom behaviour. Disappointingly this study did not go on to measure the intervention’s impact on these educational outcomes.

Working with schools to develop programmes was similarly important. Teachers could help develop both the intervention programme and its delivery plan to ensure relevancy and increase implementation fidelity [36]. Participation from students could also be useful [26]. Brandstetter et al. [42] advocated the need for pragmatic programmes that fit within the existing curriculum and school structures without creating additional demands on teachers.

Barriers to implementation

Competing priorities and lack of institutional support were noted as barriers to successful implementation. Both the HEALTHY [48] and Pathways [45] studies cited the emphasis on academic subjects over PE as hindering implementation. One programme co-ordinator from the HEALTHY study noted that ‘at times, the administration was pulling students out of PE to do some academic testing, and we had situations where students were working on writing assignments during PE because of pressures from administration’([48]:p313). Similarly, Story and colleagues suggested that implementation of the 5 A DAY Power Plus programme in the fifth grade dropped because media reports of low academic test scores within the district meant teachers refocused only on ‘teaching the basics’ [31]. The intervention programme was thus an additional pressure which could easily be dropped. Other studies suggested that preparation for a forthcoming educational inspection or a general lack of time compromised teachers’ ability to engage in and deliver the intervention [27,36,45,52].

Numerous practical issues also presented challenges to implementation including: lack of space to deliver PE lessons [45]; difficulties in delivering hands-on taste testing sessions [34]; teacher absences or rapid staff turn-over [41,52]; high student-to-teacher ratios [48]; and lack of volunteers to run after-school physical activities [54]. Concerns over teacher burn-out and disruptive student behaviour were also mentioned [45,48].

Some issues that affected intervention delivery and success were beyond schools’ control. Fry and colleagues [41], for example, described how creating safe walking route to schools meant tackling local infrastructure, traffic management and access to public transport. Some school principals talked to local authorities about addressing these issues with varying success. Sallis et al. [54] identified the need for school food services to be financially self-supporting as the greatest barrier to improving student nutrition. Offering unfamiliar (and potentially unpopular) healthy foods posed too great a financial risk for catering services and thus disincentivised change. In addition, a centralized kitchen system meant schools had little control over ingredients or preparation.

Finally, although interventions must be of sufficient length and intensity to enable behavioural change and health impacts, optimal duration is not clear from these studies, there being no clear association between intervention duration and health impact.

Discussion

The HPS framework is generally effective at increasing physical activity, fitness and fruit and vegetable intake in school students [17]. This paper looked in more detail at the implementation of physical activity and/or nutrition interventions and identified key factors helping or hindering implementation and/or success.

Summary of main findings

Process evaluations revealed high levels of acceptability among teachers and students, but implementation fidelity varied considerably across trials. In particular, involving families, despite being a key part of the HPS approach, was reported as highly challenging. Essential elements of interventions included: tailoring programmes to individual schools’ needs; aligning interventions with schools’ core aims; working with teachers to develop programmes and increase ownership; and providing on-going training, support and communication. The emphasis on academic subjects (and the corresponding low value placed on health initiatives), and lack of institutional support were cited as barriers to implementation.

Many of these findings are congruent with conclusions from other studies examining effective elements of school-based interventions. A recent narrative synthesis of qualitative studies adopting the HPS approach [60] identified the importance of institutional support, assessment of school needs, ownership of programmes, adequate training and tailoring of intervention components to local contexts. This review also noted the low value placed on health versus academic achievement as a barrier to effective implementation. Similar findings regarding intervention development and implementation are also noted in Peters et al.’s recent review of school health promotion [61] and in guidelines for HPS produced by the International Union for Health Promotion and Education [14].

Limitations

Most studies lacked detailed description of intervention components and activities which would enable replication in other contexts. Equally, the quality of process data varied considerably and was often poorly reported. While providing some useful insights, much of the process data presented by these studies was disappointing in terms of its scope and depth. Process data generally consisted of quantitative assessments of acceptability and/or fidelity. While important, these are insufficient to explain why some interventions failed while others succeeded. For example, while studies which reported positive intervention impacts also reported high levels of implementation fidelity, so did many other studies which found no such positive effects. The most useful insights into intervention success often came from authors’ reflections reported in the discussion sections of papers, despite the evidence for many such claims being unclear. However, it should be noted that findings reported here are often based on reports from just one or two studies.

Better designed, more comprehensive process evaluations that go beyond mere quantitative assessments of acceptability and/or fidelity are required. These should provide greater insight into the context in which interventions are implemented and how this can affect intervention success. It is also notable that almost a quarter of the 34 physical activity and/or nutrition interventions identified by the Cochrane review provided no process data and were thus excluded from this analysis.

The publication of the template for intervention description and replication (TIDieR) checklist and guide [62] and the recently published process evaluation guidance [63] from the UK’s Medical Research Council are welcome developments. The adoption of these guidelines by scientific journals may prove important in raising both the profile and quality of reporting of trials and associated process evaluations. This is particularly important for multi-component, complex interventions where it is important to identify, first, what is being standardised in the intervention (intervention components or steps in the change process [64]) and, second, what works, for whom, in what circumstances and why [18].

That this study focuses only on RCT evidence is both a strength and a limitation. While evidence from RCTs provides the most reliable means of assessing intervention effectiveness, we acknowledge that many evaluations of the HPS approach do not use this methodology and thus were excluded from this review [11,14,15]. However, findings reported here are congruent with reviews of the wider evidence base [14,16] and contribute to the emerging picture of how best to improve physical activity and nutrition in schools. Implementation trials (including process evaluations) that evaluate the roll-out of successful programmes would extend our understanding of how to implement such interventions in ‘real world’ settings.

Implications for policy and research

Our findings raise three challenges for both policy makers and researchers. First, we need greater integration between health and education [65]. As suggested by the findings described above, schools are more likely to engage in health interventions if they fit with institutional priorities, namely improving educational attainment. It is disappointing that none of these HPS interventions measured outcomes such as academic test scores, attendance, attention, concentration, behaviour in the classroom or attitude towards school. Overweight and obesity have been found to be associated with poor academic performance [66]. There is also some evidence to suggest physical activity [67,68] and nutrition (particularly breakfast schemes) [68-71] can improve academic achievement. However, these data are often from methodologically weak studies and causality has yet to be demonstrated [72]. It is imperative that rigorous RCT evaluation studies include both health and educational outcomes to determine effectiveness, thus speaking to policy makers in health and education sectors alike.

Second, school health researchers need to carefully consider the importance of family involvement within HPS interventions. Evidence for the impact of family involvement in school-based obesity interventions remains inconclusive; some reviews suggest its importance [7,73] while others report no consistent pattern [74,75]. It may be that this aspect of the HPS framework is impractical in some or all schools and interventions would do better to focus resources on ‘in-school’ activities. Or it may simply be, as this review has found, that current approaches to parental involvement are inadequate (for example, newsletters or information evenings) and more innovative methods are required. Further research – both qualitative and quantitative – is needed to address these questions.

Finally, we need more evaluations (including process evaluations that go beyond quantitative measures of fidelity and acceptability) of interventions to promote physical activity and nutrition during adolescence. Physical activity levels are known to decline during teenage years, particularly in young women [76,77]. Adolescence also represents a period when young people start to make their own choices over the food they eat and how they spend their time [78]. The lack of research into this critical age period is therefore disappointing and represents a missed opportunity for public health impact.

Conclusion

The HPS framework has been shown to be effective overall in improving physical activity and nutrition, the key determinants of overweight and obesity [17]. The process data reported in these trials offer important insights into essential elements of success, as well as the challenges to implementation which need to be addressed at the outset of any new programme. These data suggest that the success of the HPS approach lies in creating effective partnerships between researchers, schools and families.