Prevention Science

, Volume 20, Issue 4, pp 510–520 | Cite as

Preparing Teens to Stay Safe and Healthy on the Job: a Multilevel Evaluation of the Talking Safety Curriculum for Middle Schools and High Schools

  • Rebecca J. GuerinEmail author
  • Andrea H. Okun
  • John P. Barile
  • James G. Emshoff
  • Michelle D. Ediger
  • Devin S. Baker


US adolescents experience a higher rate of largely preventable job-related injuries compared with adults. Safety education is considered critical to the prevention of these incidents. This study evaluates the effectiveness of a foundational curriculum from the National Institute for Occupational Safety and Health (NIOSH), Talking Safety, to change adolescents’ workplace safety and health knowledge, attitude, subjective norm, self-efficacy, and behavioral intention to engage in workplace safety actions. The study also examines the impact of teacher fidelity of curriculum implementation on student outcomes. A multilevel evaluation, based on a modified theory of planned behavior, was conducted in 2016 with 1748 eighth-graders in Miami-Dade, Florida. Post-intervention, students had statistically significant increases (p < .05) in mean scores across outcomes: workplace safety knowledge (34%), attitude (5%), subjective norm (7%), self-efficacy (7%), and behavioral intention (7%). Consistent with theory, gains in attitude (b = 0.25, p < .001), subjective norm (b = 0.07, p < .01), and self-efficacy (b = 0.55, p < .001) were associated with gains in behavioral intention. Higher levels of implementation fidelity were associated with significant gains across outcome measures: knowledge (b = 0.60, p < .001), attitude (b = 0.08, p < .01), subjective norm (b = 0.04, p < .001), self-efficacy (b = 0.07, p < .01) and behavioral intention (b = 0.07, p < .01). Findings demonstrate the effectiveness of Talking Safety, delivered with fidelity, at positively changing measured outcomes, and provide support for using this curriculum as an essential component of any school-based, injury prevention program for young workers.


Young worker Occupational safety and health Injury prevention Middle school Theory of planned behavior Fidelity of implementation Multilevel modeling 



We thank the Miami-Dade County Public School System for facilitating this research, especially Mr. Cristian Carranza, Administrative Director, Division of Academics (STEAM); Dr. Ava D. Rosales, Executive Director, Department of Mathematics and Science; Mr. Dane Jaber, Instructional Supervisor, Department of Mathematics and Science; and the School Board of Miami-Dade County, Florida. For assistance with teacher training, we thank Robin Dewey, LOHP, University of California, Berkeley. For reviews of and thoughtful feedback on this manuscript, we thank Dr. Lehua Choy and Dr. Charlene Baker, University of Hawai‘i.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

The NIOSH Institutional Review Board (IRB) waived the documentation of informed consent because the project occurred within a regularly established educational setting, used a publically available curriculum adopted by the school district as part of established and ongoing classroom studies, presented no risk of harm to participants, and involved no procedures for which written consent is normally required outside of the research context.


The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the National Institute for Occupational Safety and Health. This manuscript’s data will not be deposited.


  1. Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211.CrossRefGoogle Scholar
  2. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.Google Scholar
  3. Boini, S., Colin, R., & Grzebyk, M. (2017). Effect of occupational safety and health education received during schooling on the incidence of workplace injuries in the first 2 years of occupational life: A prospective study. BMJ Open.
  4. Brener, N. D., Kann, L., Shanklin, S., Kinchen, S., Eaton, D. K., Hawkins, J., et al. (2013). Methodology of the youth risk behavior surveillance system—2013. Morbidity and Mortality Weekly Report. Recommendations and Reports, 62(1), 1–20.Google Scholar
  5. Buckley, L., Sheehan, M., & Shochet, I. (2010). Short-term evaluation of a school-based adolescent injury prevention program: Determining positive effects or iatrogenic outcomes. Journal of Early Adolescence, 30, 834–853.CrossRefGoogle Scholar
  6. Bureau of Labor Statistics. (2005). Work activity of high school students: Data from the National Longitudinal Survey of Youth 1997. Accessed 6 July 2018.
  7. Centers for Disease Control and Prevention. (2010). Occupational injuries and deaths among younger workers: United States, 1998–2007. Morbidity and Mortality Weekly Report, 59(15), 449–455.Google Scholar
  8. Centers for Disease Control and Prevention. (2016). National health education standards. Accessed 18 May 2018.
  9. Department of Labor, Wage, and Hour Division. (2016). Child labor provisions of the Fair Labor Standards Act (FLSA) for nonagricultural occupations. Accessed 6 July 2018.
  10. Domina, T., Brummet, Q., Pharris-Ciurej, N., Porter, S. R., Penner, A., Penner, E., et al. (2017). Capturing more than poverty: School free and reduced-price lunch data and household income (CARRA Working Paper Series Working Paper 2017–09). Retrieved from Accessed 9 January 2019.
  11. Downes, A., Novicki, E., & Howard, J. (2018). Using the contribution analysis approach to evaluate science impact: A case study of the National Institute for Occupational Safety and Health. American Journal of Evaluation.
  12. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3), 327–350.CrossRefGoogle Scholar
  13. Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12(2), 121–138.CrossRefGoogle Scholar
  14. Flay, B. R., Allred, C. G., & Ordway, N. (2001). Effects of the Positive Action program on achievement and discipline: Two matched-control comparisons. Prevention Science, 2(2), 71–89.CrossRefGoogle Scholar
  15. Graham, J. W., Olchowski, A. E., & Gilreath, T. D. (2007). How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science, 8(3), 206–213.CrossRefGoogle Scholar
  16. Guerin, R. J., Okun, A. H., & Kelley, P. (2016). Development and validation of an assessment tool for a national young worker curriculum: Assessment development for a young worker curriculum. American Journal of Industrial Medicine, 59(11), 969–978.CrossRefGoogle Scholar
  17. Guerin, R. J., Toland, M. D., Okun, A. H., Rojas-Guyler, L., & Bernard, A. L. (2018). Using a modified theory of planned behavior to examine adolescents’ workplace safety and health knowledge, perceptions, and behavioral intention: A structural equation modeling approach. Journal of Youth and Adolescence.
  18. Herbert, P. C., & Lohrmann, D. K. (2011). It’s all in the delivery! An analysis of instructional strategies from effective health education curricula. Journal of School Health, 81, 258–264.CrossRefGoogle Scholar
  19. Houck, C. D., Hadley, W., Barker, D., Brown, L. K., Hancock, E., & Almy, B. (2016). An emotion regulation intervention to reduce risk behaviors among at-risk early adolescents. Prevention Science, 17(1), 71–82.CrossRefGoogle Scholar
  20. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling.
  21. Kaiser, F. G., & Fuhrer, U. (2003). Ecological behavior’s dependency on different forms of knowledge. Applied Psychology. An International Review, 52(4), 598–613.CrossRefGoogle Scholar
  22. Klassen, R. M., & Chiu, M. M. (2010). Effects on teachers’ self-efficacy and job satisfaction: Teacher gender, years of experience, and job stress. Journal of Educational Psychology, 102(3), 741–756.CrossRefGoogle Scholar
  23. Koehoorn, M., Breslin, F. C., & Xu, F. (2008). Investigating the longer-term health consequences of work-related injuries among youth. Journal of Adolescent Health, 43(5), 466–473.CrossRefGoogle Scholar
  24. Lai, M. H., & Kwok, O. M. (2015). Examining the rule of thumb of not using multilevel modeling: The “design effect smaller than two” rule. The Journal of Experimental Education, 83(3), 423–438.CrossRefGoogle Scholar
  25. Linker, D., Miller, M. E., Freeman, K. S., & Burbacher, T. (2005). Health and safety awareness for working teens: Developing a successful, statewide program for educating teen workers. Family & Community Health, 28(3), 225–238.CrossRefGoogle Scholar
  26. MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Lawrence Erlbaum Associates.Google Scholar
  27. Mardis, A. L., & Pratt, S. G. (2003). Nonfatal injuries to young workers in the retail trades and services industries in 1998. Journal of Occupational and Environmental Medicine, 45(3), 316–323.CrossRefGoogle Scholar
  28. Marsiglia, F. F., Ayers, S. L., Baldwin-White, A., & Booth, J. (2016). Changing Latino adolescents’ substance use norms and behaviors: The effects of synchronized youth and parent drug use prevention interventions. Prevention Science, 17(1), 1–12.CrossRefGoogle Scholar
  29. M-DCPS. (2017). Statistical highlights 2016–2017. Accessed 9 Jan 2019.
  30. Miara, C., Gallagher, S., Bush, D., & Dewey, R. (2003). Developing an effective tool for teaching teens about workplace safety. American Journal of Health Education, 34(sup5), S–30.CrossRefGoogle Scholar
  31. Montaño, D. E., & Kasprzyk, D. (2015). Theory of reasoned action, theory of planned behavior, and the integrated behavioral model. In K. Glanz, B. K. Rimer, & K. Viswanath (Eds.), Health behavior and health education: Theory, research, and practice (5th ed., pp. 95–124). Philadelphia: Wiley.Google Scholar
  32. Mortimer, J. T. (2010). The benefits and risks of adolescent employment. The Prevention Researcher, 17(2), 8–11.Google Scholar
  33. Mowbray, C., Holter, M., Teague, G., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.CrossRefGoogle Scholar
  34. Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus user’s guide (7th ed.). Los Angeles: Muthén & Muthén.Google Scholar
  35. Nicholson, L. M., Slater, S. J., Chriqui, J. F., & Chaloupka, F. (2014). Validating adolescent socioeconomic status: Comparing school free or reduced price lunch with community measures. Spatial Demography, 2(1), 55–65.CrossRefGoogle Scholar
  36. NIOSH. (2018a). The work-related injury statistics query system (Work-RISQS). Accessed 8 July 2018.
  37. NIOSH (2018b). Talking safety. Accessed 6 July 2018.
  38. O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of Educational Research, 78(1), 33–84.CrossRefGoogle Scholar
  39. Okun, A. H., Guerin, R. J., & Schulte, P. A. (2016). Foundational workplace safety and health competencies for the emerging workforce. Journal of Safety Research, 59, 43–51.CrossRefGoogle Scholar
  40. Pisaniello, D. L., Stewart, S. K., Jahan, N., Pisaniello, S. L., Winefield, H., & Braunack-Mayer, A. (2013). The role of high schools in introductory occupational safety education—Teacher perspectives on effectiveness. Safety Science, 55, 53–61.CrossRefGoogle Scholar
  41. Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209–233.CrossRefGoogle Scholar
  42. Preacher, K. J., Zhang, Z., & Zyphur, M. J. (2011). Alternative methods for assessing mediation in multilevel data: The advantages of multilevel SEM. Structural Equation Modeling, 18, 161–182.CrossRefGoogle Scholar
  43. Rauscher, K. J., & Myers, D. J. (2008). Socioeconomic disparities in the prevalence of work-related injuries among adolescents in the United States. Journal of Adolescent Health, 42(1), 50–57.CrossRefGoogle Scholar
  44. Rauscher, K. J., Runyan, C. W., Schulman, M. D., & Bowling, J. M. (2008). U.S. child labor violations in the retail and service industries: Findings from a national survey of working adolescents. American Journal of Public Health, 98(9), 1693–1699.CrossRefGoogle Scholar
  45. Rauscher, K. J., Casteel, C., Bush, D., & Myers, D. J. (2015). Factors affecting high school teacher adoption, sustainability, and fidelity to the “Youth @ Work: Talking Safety” curriculum. American Journal of Industrial Medicine, 58(12), 1288–1299.CrossRefGoogle Scholar
  46. Rimal, R. N. (2000). Closing the knowledge-behavior gap in health promotion: The mediating role of self-efficacy. Health Communication, 12(3), 219–237.CrossRefGoogle Scholar
  47. Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., et al. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation’s middle schools. Health Education & Behavior, 30(3), 375–391.CrossRefGoogle Scholar
  48. Runyan, C. W., Schulman, M., Dal Santo, J., Bowling, J. M., Agans, R., & Ta, M. (2007). Work-related hazards and workplace safety of US adolescents employed in the retail and service sectors. Pediatrics, 119(3), 526–534.CrossRefGoogle Scholar
  49. Runyan, C. W., Lewko, J., & Rauscher, K. (2012). Setting an agenda for advancing young worker safety in the U.S. and Canada. Public Health Reports, 127(3), 246–252.CrossRefGoogle Scholar
  50. Smith, J., Purewal, B. P., Macpherson, A., & Pike, I. (2018). Metrics to assess injury prevention programs for young workers in high-risk occupations: A scoping review of the literature. Health Promotion and Chronic Disease Prevention in Canada: Research, Policy and Practice, 38(5), 191–199.CrossRefGoogle Scholar
  51. Webb, T. L., & Sheeran, P. (2006). Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychological Bulletin, 132(2), 249–268.CrossRefGoogle Scholar
  52. Zierold, K. M., & Anderson, H. A. (2006). Severe injury and the need for improved safety training among working teens. American Journal of Health Behavior, 30(5), 525–532.CrossRefGoogle Scholar
  53. Zierold, K. M., Garman, S., & Anderson, H. (2004). Summer work and injury among middle school students, aged 10–14 years. Occupational and Environmental Medicine, 61(6), 518–522.CrossRefGoogle Scholar

Copyright information

© This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2019

Authors and Affiliations

  1. 1.National Institute for Occupational Safety and Health (NIOSH)Centers for Disease Control and Prevention (CDC)CincinnatiUSA
  2. 2.Department of PsychologyUniversity of Hawai‘i at MānoaHonoluluUSA
  3. 3.EMSTAR Research, Inc.AtlantaUSA

Personalised recommendations