Background

Regular physical activity (PA) participation has been shown to be beneficial for the physical and mental health of children and youth [1,2,3,4] and additionally is associated with academic achievement and increased concentration in school [5,6,7]. These benefits indicate the importance of participating in sufficient PA to meet international guidelines for children and youth specifying 60 min of moderate to vigorous PA (MVPA) per day [8, 9]. Yet engaging in PA on a regular basis is challenging for many children and youth and there are documented declines in PA with increasing age and among females [10, 11].

Since children and youth spend a considerable amount of time in school over several years, this is an important setting in which to provide structured opportunities for PA [4, 12, 13]. By specifying the type and amount of PA offered, school-based policies, curricula and related programs can potentially serve to level the playing field for student PA opportunities. Thus, to the degree that these initiatives are adopted and implemented, schools can provide students with an important contribution to meeting the 60 min of MVPA specified in the guidelines [14].

In order to provide school-based opportunities for children and youth to engage in PA, a number of Canadian provinces have adopted Daily Physical Activity (DPA) policies [15]. These initiatives are normally intended to augment physical and health education classes which are not, for the most part, offered each school day. In 2005 the Ontario Ministry of Education (EDU) released Policy/Program Memorandum (PPM) No. 138: Daily Physical Activity (DPA). This policy requires publicly-funded school boards to “ensure that all elementary students (grades 1-8), including students with special needs, have a minimum of twenty minutes of sustained MVPA each school day during instructional time” [16]. Unique to Ontario is the requirement that DPA be provided during instructional time – not during recess, lunch hour or after school. This provision can be seen as strengthening the initiative since it is incorporated into the curriculum as well as being a provincial policy [17].

A 2012 joint report by Cancer Care Ontario and Public Health Ontario recommended that Ontario’s DPA policy be evaluated since, although the policy was originally released in 2005, no Ontario-wide evaluation had previously been conducted and also to enhance provincial government accountability for this ongoing initiative [18]. In response to this recommendation, a research team conducted a number of sequential studies evaluating the development and implementation of DPA in Ontario [17, 19]. Evaluation of policy implementation is considered to be highly important, since implementation is conceptualized in the broader literature as a central determinant of subsequent intervention impacts and outcomes [20,21,22]. Implementation fidelity is described as “the degree to which an intervention was implemented as it was prescribed in the original protocol or as intended by the program developers” [23]. In the context of the current paper, fidelity refers to the degree to which DPA was implemented as specified in the policy requirements.

Consistent with Social Ecological Theory, [24] a number of factors have been previously described as being associated with implementation of school-based PA policy and curriculum initiatives. For example, intrapersonal factors, particularly at the teacher level, include relevant education and experience with PA, beliefs about the initiative’s importance, and self-efficacy/confidence [25,26,27,28,29]. Interpersonal factors associated with implementation include the degree to which the initiative is supported and prioritized within schools and by other organizations and individuals, such as parents [12, 25, 30, 31]. Organizational-level factors influencing implementation include training, provision of resources, access to a physical education specialist, available time, space and equipment, and lack of accountability such as monitoring [25,26,27, 30, 32,33,34,35,36]. Also, a systematic review of randomized controlled trials examining factors influencing implementation of PA interventions found time, availability of resources, school support, training, self-efficacy, teacher characteristics, and scheduling to be prominent barriers and/or facilitating factors [37].

Findings from a recent study, based on representative random sample surveys of Ontario elementary schools and classrooms, indicated that 61% of schools were implementing DPA in fidelity with the required policy, while 50% of classrooms were implementing DPA in fidelity with the requirements [17]. Additionally, survey findings (analyzed separately for schools and classrooms) indicated that several factors were significantly associated with implementation fidelity at both the school and classroom levels: awareness of policy requirements; scheduling; monitoring; use of resources and supports; perceptions that the policy is realistic and achievable; and a number of specific barriers to implementation [17]. Other factors were significantly associated with implementation fidelity at the classroom level but not the school level, such as self-efficacy for planning and implementing DPA (significant for teachers but not administrators) [17].

Descriptive findings from these surveys provided important policy-relevant information on several salient factors associated with school-level and classroom-level implementation fidelity. However, the findings from analyses of the separate surveys did not address the question of the extent to which school-level (as reported by administrators) and classroom-level (as reported by teachers) are associated with classroom-level DPA policy implementation fidelity. That is, what school-level factors and classroom-level factors are associated with classroom-level fidelity when controlling for their simultaneous effects? Related to this is the question of what is the most parsimonious model predicting classroom-level fidelity? These questions are highly important on both theoretical and applied levels. Conceptual models examining the various factors influencing implementation fidelity implicitly pose questions of the extent to which structural, organizational and individual factors influence implementation outcomes such as fidelity [21].

The current paper addresses these questions using multi-level analysis (MLA) of merged data from the school and classroom surveys. In relation to previous school-based studies of PA and physical education implementation, MLA has been used almost entirely in examining school and classroom influences on student self-reported PA behaviour as an outcome [31, 38]. The closest study to the current one, in both content and methods, is a report of the factors influencing classroom implementation of 15 min of PA each day in elementary schools in British Columbia (Canada) as part of the Action Schools!BC comprehensive school health initiative [25]. In that study, MLA of characteristics of schools and teachers, based on data derived from online surveys of school principals and classroom teachers, indicated that four factors were associated with implementation: level of institutionalization; receiving training; teacher self-efficacy; and attributes of the innovations [25]. Beyond this example, however, there is a gap in understanding the multi-level effects of school and classroom influences on classroom-level PA (including DPA) policy implementation. Our study contributes to a further understanding of these important questions.

In addition to their theoretical importance, these issues are also highly relevant to government and other organizations responsible for providing evidence-informed policy and program interventions/initiatives. For example, should government approaches focus more on initiatives to support and improve implementation fidelity at the school level or the classroom level? By extension, should these approaches be directed to school administrators or classroom teachers? The study reported here addresses these issues by examining both school and classroom influences on classroom-level DPA policy implementation fidelity.

Methods

Study design & sample

As mentioned earlier, a cross-sectional study of Ontario’s elementary schools was conducted between February and June 2014. Participants included school administrators (principals and vice principals) and grade 3, 5, and 7 teachers. These specific grades were selected in order to represent primary, junior and intermediate grade classrooms and to decrease the schools’ administrative burden of including more grades. The study used proportionate stratified random sampling to obtain a representative sample of Ontario’s publicly-funded elementary schools, based on four attributes: school board language (French versus English); school board type (public versus Catholic); location (urban versus rural), and enrolment size (≤200, 201–400, and ≥ 401).

Participant recruitment occurred in multiple stages. First, 40 school boards were approached to conduct the study in randomly sampled schools within their board and, of those, 30 agreed to participate (75% approval rate). Second, 43% of the randomly sampled schools within the respective school boards agreed to participate in the study (228 of 532 schools). An online survey was sent to one administrator from each of the 228 schools and, of those, 209 responded to the survey, resulting in a response rate of 39% (209 of 532) at the administrator level. Afterward, a random sample of 508 teachers from grade 3, 5, and 7 classrooms in schools where a school administrator responded to the survey were invited to participate in an online survey. This resulted in a 60% response rate at the teacher level (307 of 508). Additional details regarding the study methods are provided in a previous paper [17]. Ethical approval for the study was obtained from Public Health Ontario’s Ethics Review Board (ID: 2013–039.01).

Measures

Survey measures for administrators and teachers were developed based on our adapted Chaudoir framework, [17, 21] as well as findings from interviews with key informants who were involved in the initial development and implementation of the DPA policy, [19] existing survey instruments, [39] and DPA materials published by the EDU [16, 40,41,42,43]. The survey measures and process were pre-tested and revised accordingly. The pre-test, which assessed question comprehension, skip patterns, flow and completion time, was completed by the members of the study team and the study Advisory Committee, which included representation from government, education, and public health.

The content of both administrator and teacher surveys were similar but adapted to each respective type of position. Both survey instruments were comprised of seven sections: (1) awareness of DPA policy requirements; (2) perception of DPA policy; (3) scheduling and monitoring of DPA within schools and classrooms; (4) use of DPA resources and supports; (5) perceptions of barriers; (6) self-efficacy; and (7) personal characteristics. Additional questions regarding DPA planning at the school level were also asked of school administrators. School administrators responded on behalf of their school and teachers responded on behalf of their classroom.

Outcome variable: Fidelity to DPA policy

The outcome variable for the current analysis was a dichotomous measure of fidelity to the DPA policy at the classroom level based on the required components of the policy [16]. To assess fidelity to the DPA policy at the classroom level, teachers were first asked whether DPA had been implemented in their classroom at least once during the 2013–2014 school year. Respondents indicating “yes” were asked an additional six questions pertaining to the requirements of the DPA policy: 1) duration; 2) frequency; 3) scheduling during instructional time; 4) intensity; 5) continuity; and 6) inclusivity of children with special needs. Responses to the frequency question were measured in number of days (1–5 days) and the other five questions were measured using a five-point Likert scale (never-always). The responses for all six questions were later assigned a value of 1–5: 1 = never/1 day; 2 = rarely/2 days; 3 = sometimes/3 days; 4 = often/4 days; and 5 = always/5 days. The internal consistency reliability of the 6-item initial scale was high (Cronbach’s coefficient alpha = 0.98) [17]. Afterward, an overall score of fidelity was calculated by taking the sum of the participants’ fidelity scores across the six items. The resulting composite score ranged from 0 to 30. A value of zero was assigned when teachers responded that they had not implemented DPA in their classroom at least once during the 2013–2014 school year. The overall fidelity score was later reclassified into “met DPA policy requirements” (scores 24–30) and “did not meet DPA policy requirements” (scores 0 or 6–23.99). Further details regarding this measure can be found in our previous overview paper [17].

Independent variables

Independent variables at the school and classroom levels were chosen on the basis of theory, existing literature, and findings from our previous studies [17, 19]. At the school level, five variables were selected from the school administrator survey, while at the classroom-level, eight variables were chosen from the teacher survey. These variables are described, along with their respective coding categories in Table 1.

Table 1 Description of independent variables
Full size table

Data analysis

As a preliminary step, descriptive analyses were conducted to explore the distribution of responses by school administrators and teachers. Two-level logistic regression models, with random effect on schools, were then carried out to account for the nesting of classrooms within schools. The analysis examined the relative contribution of classroom/teacher characteristics (level 1) and school/administrator characteristics (level 2) on implementation fidelity to DPA policy at the classroom level. Multicollinearity was examined separately for school and classroom predictors using Variance Inflation Factor (VIF) and none of the predictors showed signs of strong multicollinearity (VIF < 10). At first, an unconditional model (i.e., model without any predictors) was fit to calculate the intraclass correlation coefficient (ICC), which shows the amount of variation in the outcome that is attributable to the group level [44]. ICC was calculated using the formula ICC=Va/(Va + 3.29)*100, where Va is the area level variance and 3.29 is the estimated individual level variance [44]. The ICC was then used to calculate the design effect. The design effect is the ratio of actual variance for a given design to the variance computed under simple random sampling [45] and it is approximately equal to 1 + (average cluster size - 1) x ICC [46].

Series of models were conducted before arriving at a parsimonious model for implementation fidelity to DPA policy. Initially, a saturated model (Model 1) was fit using all of the school-level and classroom-level predictors. Then independent models were fit with solely school-level predictors (Model 2) and classroom-level (Model 3) predictors. Several other models were carried out by removing predictors that were conceptually less relevant and showed lower effect size compared to other predictors before arriving at the final model (Model 4). The contribution of each model was assessed by comparing the difference in deviance (− 2*log-likelihood) [47] and other model fit statistics. Since deviance tests can only be conducted with nested models, all models presented in this study were compared to the saturated model. Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) were used to assess the goodness-of-fit of all four models, including non-nested models. AIC and BIC are both based on the maximum likelihood estimates and control for over-fitting in complex models by penalizing free parameters. Smaller values for both information criteria indicate a better fit model. Statistical significance for all models was determined using alpha level of 0.05. All analyses were carried out in SAS 9.3.

Results

A total of 209 school administrators and 307 classroom teachers were included in the original study. The current analyses excluded 39 of the school administrators due to lack of participation at the teacher level in corresponding schools. Thus, the analytic sample for the analyses here included 170 school administrators and 307 classroom teachers from the corresponding schools. The number of participating teachers per school ranged from 1 to 3 (mean = 1.8; SD = 0.7).

The distribution of characteristics for the school boards and schools were similar to that of publicly funded elementary schools in Ontario. The majority of school boards for schools included in the study were English (94%) and publicly funded (68%) (Table 2). Approximately three-fourths of the schools were located in urban areas and about half of the schools had 296 students or more enrolled.

Table 2 School and administrator characteristics (n = 170)
Full size table

The majority of administrators were females (69%), principals (91%), and had 6–15 years of experience (57%). Many administrators (59%) viewed PA as a high priority in their daily life, though 77% had little or no health and physical education training. Eighty-three percent of administrators reported awareness of more than half of the DPA policy components. Only a small percentage of administrators reported the presence of a DPA monitoring procedure in their schools (28%) and frequent use of DPA supports (9%). The majority of administrators (77%) perceived competing curriculum priorities as a barrier for DPA implementation.

The demographics and experience levels of teachers were also similar to that of the administrators. The majority of teachers were females (72%) and had 6–15 years of experience (50%). Details of the teachers’ personal characteristics have been published in our previous paper [17]. Table 3 shows teacher-level predictors, including awareness, perceptions, and use of resources related to DPA, by fidelity to DPA policy. As expected, teachers who reported greater confidence level (self-efficacy) in implementation, increased awareness of DPA policy requirements, and frequent use of DPA resources showed higher percentages for meeting DPA policy requirements compared to their counterparts. Three-fourths of the teachers who perceived DPA policy as realistic and achievable met DPA policy requirements in their classrooms. Time and space were amongst the top implementation barriers reported by teachers (data not shown) [17]. Teachers who strongly agreed/agreed that lack of time and lack of space are implementation barriers reported lower percentage of fidelity to DPA policy compared to those who strongly disagreed/disagreed.

Table 3 Teacher-level predictors, by fidelity to DPA policy requirements (n = 307)
Full size table

Multi-level models showed significant effect by teacher-level predictors on implementation fidelity compared to administrator-level predictors. The ICC value calculated from the unconditional model showed that administrator-level predictors accounted for 5.4% of the variance in implementation of DPA in classrooms. The design effect was 1.04 (1 + (1.8–1)*0.054). MLA is generally not warranted when the design effect is less than two [46]. However, MLA was still carried out in this study due to small cluster sizes and our interest in examining the effects of level-2 variables on fidelity to DPA policy [48].

Findings from the multi-level logistic regression models are shown in Table 4. The saturated model (model 1), with all identified variables, showed teachers’ perception of DPA as realistic and achievable (AOR = 7.40; 95% CI: 2.52–21.76) and teachers’ high confidence in implementing DPA (AOR = 3.08; 95% CI = 1.22–7.76) as the only significant predictors for implementation fidelity to DPA policy. The model with administrator-level predictors alone (model 2) showed no significant findings while the model with only teacher-level predictors (model 3) showed the same significant predictors as the saturated model. In the final model (model 4), three additional variables emerged as significant predictors for implementation fidelity to DPA policy. Teachers’ perception of DPA as realistic and achievable showed the highest odds for implementation. Teachers who strongly agreed/agreed that DPA is realistic and achievable reported 4.63 (95% CI: 1.88–11.44) higher odds for implementing DPA in their classrooms compared to those who strongly disagreed/disagreed. Teachers who reported high confidence level (self-efficacy) in implementing DPA showed 3.39 (95%: 1.47–7.85) higher odds for implementing DPA compared to teachers with low-to-moderate confidence level. Teachers who scheduled DPA in their timetables also reported higher odds (AOR = 2.51; 95% CI: 1.03–6.12) for DPA implementation compared to their counterparts. Those who strongly agreed/agreed that lack of space is a barrier for DPA implementation were 65% less likely to implement DPA policy requirements compared to those who strongly disagreed/disagreed. Teachers who reported a neutral response to lack of time as a barrier were 86% less likely to implement DPA compared to those who disagreed/strongly disagreed.

Table 4 Multi-level model for school-level and classroom-level predictors of implementation fidelity to DPA policy
Full size table

Deviance tests showed model 4 as a better fitting model compared to the saturated model (p-value< 0.01). When comparing the fit of all four models, there was disagreement between AIC and BIC. AIC value was lowest for model 1 and BIC value was lowest for model 4. Given that BIC penalizes complex models more than AIC, [49] model 4 was chosen as the more parsimonious model.

Discussion

The underlying study was the first provincial-level assessment of the status of DPA policy implementation in Canada [17]. The study benefited from access to representative samples of schools and classrooms in Ontario, outcome measures designed to assess fidelity on the six components of the DPA policy, and (for the current analyses) merged data from the two surveys.

A central purpose of the MLA used here was to examine the influences of both school (as reported by administrators) and classroom (as reported by teachers) factors predicting policy fidelity at the classroom level. This is an important consideration since it could be argued that offering DPA (or not) depends largely on decisions of classroom teachers who are the individuals most directly involved in planning and implementing this activity. Yet it is also theoretically and empirically plausible that higher-level (school and administrator) factors could predict classroom fidelity. While a larger number of factors were identified in our earlier examination of predictors of DPA policy implementation in Ontario schools and classrooms, [17] results from MLA suggest that a more limited number of factors predicted implementation in classrooms. Moreover, these are all classroom/teacher-level factors: perception that DPA is realistic and achievable; confidence (self-efficacy) in implementing DPA; scheduling DPA in timetables; lack of space (inversely related) and; lack of time (neutral). None of the school/administrator-level factors that were shown to be significant predictors of school-level policy fidelity in earlier bi-variate analyses proved to be significant predictors of classroom-level policy implementation fidelity when taking both levels of factors into account.

Given these findings, it is tempting to conclude that the significant classroom/teacher- level predictors of classroom policy fidelity are simply reflecting their proximal relationship to the decision to plan and implement DPA in classrooms. To some extent, this may be true. Yet, whether or not the policy is considered to be realistic and achievable may also reflect important (though unmeasured) issues around the structure of schools and schooling: the multiple curricular demands on teachers; emphasis on academic success; and other school-based health and social priorities such as mental health, bullying and safety – all of which may compete to some extent with expectations to offer DPA. Similarly, scheduling DPA on a daily basis indicates attempts to formalize expectations for this activity among a challenging array of externally mandated curriculum expectations and other demands (field trips, assemblies, special events). Lack of space is also clearly situated in objective circumstances and constraints on the extent to which DPA can be accomplished in classrooms containing many desks, chairs and various ages (sizes) of students.

This issue is important on a theoretical level since the results may otherwise appear to indicate that teacher factors alone are the key determinant of policy implementation fidelity. The act of implementing DPA (or not) may be rooted in organizational or structural factors, as suggested by our adapted theoretical framework [17, 21].

Several of the specific factors emerging as significant predictors of DPA in the current study have been identified in other studies of DPA implementation or related initiatives, such as physical education. For example, teacher self-efficacy has been shown to be predictive of implementation of a school-based PA initiative in a British Columbia (Canada) MLA study [25]. As mentioned earlier in the current paper, self-efficacy has also been shown to be associated with PA implementation in a number of descriptive studies and reviews [27, 37, 50]. Studies indicating that availability of time in the curriculum is related to implementation attest to the importance of scheduling DPA and similar initiatives [27, 29, 32, 37]. Also a number of earlier studies found space/facilities to be related to implementation of school PA initiatives [26, 27, 33, 34, 37].

Limitations

Data from the study were based on self-reports of school administrators and classroom teachers, those most directly responsible for offering DPA in these settings. Their assessments of DPA implementation fidelity were based on responses to questions on each of the component requirements of the policy, thus providing a degree of content validity. However, the data were not validated through direct observation or other means.

Since DPA policies are largely unique to provincial settings in Canada, generalization of the findings to other jurisdictions and initiatives beyond Canada is limited. However, the use of activity breaks in some schools in the US and other countries provides some basis of comparison, and some of the challenges of these are similar to those of DPA. Application of the findings to physical education class is more difficult, since these are usually more highly structured components of the curriculum and more likely to be led by specialist rather than generalist teachers. Even so, there are well-documented reports of the challenges of implementing physical education on a regular basis due to such organizational factors as exemptions and other issues [51, 52].

The present study does not include measures that assess structural factors and influences representing the broader context of these issues. However, we know from our earlier assessment of the development and implementation of DPA in Ontario that there were several political, economic and logistical factors influencing how this initiative was planned and rolled out in Ontario school boards [19]. A key factor was negotiation around providing school boards and schools with sufficient flexibility as to how DPA is delivered, while also adhering to the required components of the policy. Also, while DPA is a provincial policy and curriculum requirement, it “competes” with a number of other priorities and requirements at least partly related to the nature of the education system itself. Understanding these contextual factors helps to explain the current findings – especially with respect to such predictors of fidelity as teachers’ perception about whether or not DPA is realistic and achievable, to what extent it is regularly scheduled in teachers’ timetables, and the availability of adequate space to successfully enact DPA sessions.

While the MLA analyses provide new insights as to the salient factors predicting classroom level DPA, we have not examined here the potential mediating and moderating effects of factors within the model. In order to conduct that analysis, structural equation modelling would be required. Future analysis may address these further relationships. In addition, not yet reported qualitative data from open-ended question items in the teacher survey provide additional insights from teachers as to their practical experiences and constraints in planning and implementing DPA in their classrooms.

Conclusions

This study provides important contributions to theory and research, with high relevance to application in school health policy and programs in Canada and elsewhere. Since the MLA identified a more parsimonious model of factors predictive of DPA policy fidelity, a clearer picture emerges of the implications for promoting DPA within the education system. Clearly, teacher self-efficacy is a key positive factor, theoretically grounded, and amenable to modification through training, resources and support. Teachers with higher levels of self-efficacy in implementing DPA are more likely to implement it in fidelity with the policy requirements.

Similarly, findings from the MLA indicate that teacher scheduling of DPA is a key predictor of fidelity. Additional support (and monitoring) by school administrators, school boards and the provincial EDU are likely to increase the practice of regular scheduling of DPA in weekly, monthly and annual timetables. Finally, the provision of sufficient space for DPA presents logistical challenges, indicating the need for innovative and creative approaches by administrators, teachers, and others to address this issue. Our findings suggest that these potential changes, accompanied by teacher perspectives that the policy is realistic and achievable, may be related to higher levels of implementation fidelity to the DPA policy in elementary school classrooms. Since several of these factors are imbedded in complex structural and organizational contexts, a systems approach to support these efforts is recommended.