Skip to main content
Log in

Testing an Integrated Model of Program Implementation: the Food, Health & Choices School-Based Childhood Obesity Prevention Intervention Process Evaluation

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

Childhood obesity is a complex, worldwide problem. Significant resources are invested in its prevention, and high-quality evaluations of these efforts are important. Conducting trials in school settings is complicated, making process evaluations useful for explaining results. Intervention fidelity has been demonstrated to influence outcomes, but others have suggested that other aspects of implementation, including participant responsiveness, should be examined more systematically. During Food, Health & Choices (FHC), a school-based childhood obesity prevention trial designed to test a curriculum and wellness policy taught by trained FHC instructors to fifth grade students in 20 schools during 2012–2013, we assessed relationships among facilitator behaviors (i.e., fidelity and teacher interest); participant behaviors (i.e., student satisfaction and recall); and program outcomes (i.e., energy balance-related behaviors) using hierarchical linear models, controlling for student, class, and school characteristics. We found positive relationships between student satisfaction and recall and program outcomes, but not fidelity and program outcomes. We also found relationships between teacher interest and fidelity when teachers participated in implementation. Finally, we found a significant interaction between fidelity and satisfaction on behavioral outcomes. These findings suggest that individual students in the same class responded differently to the same intervention. They also suggest the importance of teacher buy-in for successful intervention implementation. Future studies should examine how facilitator and participant behaviors together are related to both outcomes and implementation. Assessing multiple aspects of implementation using models that account for contextual influences on behavioral outcomes is an important step forward for prevention intervention process evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Berkel, C., Mauricio, A. M., Schoenfelder, E., & Sandler, I. N. (2011). Putting the pieces together: an integrated model of program implementation. Prevention Science, 12(1), 23–33.

    Article  PubMed  Google Scholar 

  • Christian, M. S., Evans, C. E., Ransley, J. K., Greenwood, D. C., Thomas, J. D., & Cade, J. E. (2012). Process evaluation of a cluster randomised controlled trial of a school-based fruit and vegetable intervention: project tomato. Public Health Nutrition, 15(03), 459–465.

    Article  PubMed  Google Scholar 

  • Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2(1), 40.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chen, H. T. (2010). The bottom up approach to integrative validity: a new perspective for program evaluation. Evaluation and Program Planning, 33, 205–214.

    Article  PubMed  Google Scholar 

  • Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2002). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Contento, I. R. (2015). Nutrition education: Linking research, theory, and practice (3rd ed.). Burlington: Jones & Bartlett Learning.

    Google Scholar 

  • Contento, I., Koch, P., & Gray, H. L. (2015). Reducing childhood obesity: an innovative curriculum with wellness policy support. Journal of Nutrition Education and Behavior, 47(4), S98.

    Article  Google Scholar 

  • Contento, I. R., Koch, P. A., Lee, H., & Calabrese-Barton, A. (2010). Adolescents demonstrate improvement in obesity risk behaviors after completion of Choice, Control & Change, a curriculum addressing personal agency and autonomous motivation. Journal of the American Dietetic Association, 110(12), 1830–1839.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cote, A. T., Harris, K. C., Panagiotopoulos, C., Sandor, G. G., & Devlin, A. M. (2013). Childhood obesity and cardiovascular dysfunction. Journal of the American College of Cardiology, 62, 1309–19.

    Article  PubMed  Google Scholar 

  • de Onis, M., Blossner, M., & Borghi, E. (2010). Global prevalence and trends of overweight and obesity among preschool children. American Journal of Clinical Nutrition, 92, 1257–1264.

    Article  PubMed  Google Scholar 

  • Durlak, J., & DuPre, E. (2008). Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.

    Article  PubMed  Google Scholar 

  • Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Education Research, 18(2), 237–256.

    Article  PubMed  Google Scholar 

  • Fisher, J. O., Johnson, R. K., Lindquist, C., Birch, L. L., & Goran, M. I. (2000). Influence of body composition on the accuracy of reported energy intake in children. Obesity Research, 8, 597–603.

    Article  CAS  PubMed  Google Scholar 

  • Gray, H. L., Burgermaster, M., Tipton, E., Contento, I. R., Koch, P. A., & Di Noia, J. (2015a). Intraclass correlation coefficients for obesity indicators and energy balance–related behaviors among New York City public elementary schools. Health Education & Behavior, 43(2), 172–181.

    Article  Google Scholar 

  • Gray, H. L., Contento, I. R., & Koch, P. A. (2015b). Linking implementation process to intervention outcomes in a middle school obesity prevention curriculum, Choice, Control and Change. Health Education Research, 30(2), 248–261.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gray, H. L., Koch, P. A., Contento, I. R., Bandelli, L. N., Ang, I., & Di Noia, J. (2016). Validity and reliability of behavior and theory-based psychosocial determinants measures, using audience response system technology in urban upper-elementary schoolchildren. Journal of Nutrition Education and Behavior, 48(7), 437–452.

    Article  PubMed  Google Scholar 

  • Greaney, M.L., Hardwick, C.K., Spadano-Gasbaro, J.L., Mezgebu, S., Horan, C.M., Schlotterbeck, S., …Peterson, K.E. (2014). Implementing a multicomponent school-based obesity prevention intervention: A qualitative study. Journal of Nutrition Education and Behavior, 46, 576–582.

  • Griffin, T. L., Pallan, M. J., Clarke, J. L., Lancashire, E. R., Lyon, A., Parry, J. M., & Adab, P. (2014). Process evaluation design in a cluster randomised controlled childhood obesity prevention trial: the WAVES study. International Journal of Behavioral Nutrition and Physical Activity, 11, 112.

    Article  PubMed  PubMed Central  Google Scholar 

  • Herouvil, D., Karanasios, D., Karayianni, C., & Karavanaki, K. (2013). Review: cardiovascular disease in childhood: the role of obesity. European Journal of Pediatrics, 172, 721–732.

    Article  Google Scholar 

  • Kam, C. M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4(1), 55–63.

    Article  PubMed  Google Scholar 

  • Koth, C. W., Bradshaw, C. P., & Leaf, P. J. (2008). A multilevel study of predictors of student perceptions of school climate: the effect of classroom-level factors. Journal of Educational Psychology, 100(1), 96–104.

    Article  Google Scholar 

  • Kuczmarski, R.J., Ogden, C.L., Grummer-Strawn, M., Flegal, K.M., Guo, S.S., Wei, R. …Johnson, C.L. (2000). CDC growth charts: United States. Advance Data, 314, 1–27.

  • Lee, H., Contento, I. R., & Koch, P. (2013). Using a systematic conceptual model for a process evaluation of a middle school obesity risk-reduction nutrition curriculum intervention: choice, control & change. Journal of Nutrition Education and Behavior, 45, 126–136.

    Article  PubMed  PubMed Central  Google Scholar 

  • Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202.

    Article  Google Scholar 

  • Martens, M., van Assema, P., Paulussen, T., Schaalma, H., & Brug, J. (2006). Krachtvoer: process evaluation of a Dutch programme for lower vocational schools to promote healthful diet. Health Education Research, 21(5), 695–704.

    Article  PubMed  Google Scholar 

  • Ogden, C.L., & Carroll, M.D. (2010). Prevalence of obesity among children and adolescents: United States, trends 1963–1965 through 2007–2008. Atlanta, GA: US Department of Health and Human Services, CDC, National Center for Health Statistics. Retrieved from: http://www.cdc.gov/nchs/data/hestat/obesity_child_07_08/obesity_child_07_08.pdf.

  • Paulson, D. S. (2006). Handbook of regression and modeling: applications for the clinical and pharmaceutical industries. Boca Raton: Chapman and Hall/CRC.

    Book  Google Scholar 

  • Peets, K., Pöyhönen, V., Juvonen, J., & Salmivalli, C. (2015). Classroom norms of bullying alter the degree to which children defend in response to their affective empathy and power. Developmental Psychology, 51(7), 913.

    Article  PubMed  Google Scholar 

  • Pettigrew, J., Graham, J. W., Miller-Day, M., Hecht, M. L., Krieger, J. L., & Shin, Y. J. (2015). Adherence and delivery: implementation quality and program outcomes for the seventh-grade keepin’ it REAL program. Prevention Science, 16(1), 90–99.

    Article  PubMed  PubMed Central  Google Scholar 

  • Pratt, C. A., Arteaga, S., & Loria, C. (2014). Forging a future of better cardiovascular health: addressing childhood obesity. Journal of the American College of Cardiology, 63, 369–371.

    Article  PubMed  Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., … & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76.

  • Raudenbrush, S. W., & Byrk, A. S. (2002). Hierarchical linear models: applications and data analysis methods (2nd ed.). Thousand Oaks: Sage.

    Google Scholar 

  • Raudenbush, S. W., Bryk, A. S., Cheong, A. S., Fai, Y. F., Congdon, R. T., & du Toit, M. (2011). HLM 7: Hierarchical linear and nonlinear modeling. Lincolnwood: Scientific Software International.

    Google Scholar 

  • Rohrbach, L. A., Dent, C. W., Skara, S., Sun, P., & Sussman, S. (2007). Fidelity of implementation in project towards No drug abuse (TND): a comparison of classroom teachers and program specialists. Prevention Science, 8(2), 125–132.

    Article  PubMed  Google Scholar 

  • Sánchez, V., Steckler, A., Nitirat, P., Hallfors, D., Cho, H., & Brodish, P. (2007). Fidelity of implementation in a treatment effectiveness trial of reconnecting youth. Health Education Research, 22, 95–107.

    Article  PubMed  Google Scholar 

  • Saunders, R. P., Ward, D., Felton, G. M., Dowda, M., & Pate, R. R. (2006). Examining the link between program implementation and behavior outcomes in the lifestyle education for activity program (LEAP). Evaluation and Program Planning, 29(4), 352–364.

    Article  PubMed  Google Scholar 

  • Schmied, E., Parada, H., Horton, L., Ibarra, L., & Ayala, G. (2015). A process evaluation of an efficacious family-based intervention to promote healthy eating: the Entre Familia: Reflejos de Salud study. Health Education and Behavior, 42, 583–592.

    Article  PubMed  Google Scholar 

  • Schneider, M., Hall, W.J., Hernandez, A.E., Hindes, K., Montez, G., Pham, T., … & Zeveloff, A. (2009). Rationale, design and methods for process evaluation in the HEALTHY study. International Journal of Obesity, 33, S60-S67.

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2001). Experimental and quasi-experimental designs for generalized causal inference (2nd ed.). Independence: Cengage.

    Google Scholar 

  • Singh, A. S., Chinapaw, M. J. M., Brug, J., & Van Mechelen, W. (2009). Process evaluation of a school-based weight gain prevention program: the Dutch Obesity Intervention in Teenagers (DOiT). Health Education Research. doi:10.1093/her/cyp011.

    PubMed  Google Scholar 

  • van Stralen, M. M., de Meij, J., Te Velde, S. J., Van der Wal, M. F., Van Mechelen, W., Knol, D. L., & Chinapaw, M. J. (2012). Mediators of the effect of the JUMP-in intervention on physical activity and sedentary behavior in Dutch primary schoolchildren from disadvantaged neighborhoods. International Journal of Behavioral Nutrition and Physical Activity, 9(1), 131.

    Article  PubMed  PubMed Central  Google Scholar 

  • Waters, E., de Silva-Sanigorski, A., Hall, B.J., Brown, T., Campbell, K.J., Gao, Y. … Summerbell, C. D. (2011). Interventions for preventing obesity in children. Cochrane Database Systematic Reviews, 12, CD001871.

Download references

Acknowledgements

The authors would like to thank the FHC Instructors Emily Abrams, Lorraine Bandelli, Casey Luber, Jennifer Markowitz, Betsy Ginn, Greta Kollman, Shien Chiou; FHC Research assistants/data collectors; FHC schools, teachers, and students. The authors would also like to thank the reviewers for their insightful comments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Marissa Burgermaster.

Ethics declarations

Funding

USDA AFRI NIFA grant #2010-85215-20661

Conflict of Interest

The authors state they have no conflict of interest. Dr. Burgermaster’s involvement in the preparation of this manuscript was supported by training grants T15LM007079 and T32HL007343; at the time of this study, she was a doctoral candidate at Teachers College Columbia University.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent

We obtained informed consent from all participants included in this study.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(DOCX 128 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Burgermaster, M., Gray, H.L., Tipton, E. et al. Testing an Integrated Model of Program Implementation: the Food, Health & Choices School-Based Childhood Obesity Prevention Intervention Process Evaluation. Prev Sci 18, 71–82 (2017). https://doi.org/10.1007/s11121-016-0736-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-016-0736-2

Keywords

Navigation