Preparing Teens to Stay Safe and Healthy on the Job: a Multilevel Evaluation of the Talking Safety Curriculum for Middle Schools and High Schools

Abstract

US adolescents experience a higher rate of largely preventable job-related injuries compared with adults. Safety education is considered critical to the prevention of these incidents. This study evaluates the effectiveness of a foundational curriculum from the National Institute for Occupational Safety and Health (NIOSH), Talking Safety, to change adolescents’ workplace safety and health knowledge, attitude, subjective norm, self-efficacy, and behavioral intention to engage in workplace safety actions. The study also examines the impact of teacher fidelity of curriculum implementation on student outcomes. A multilevel evaluation, based on a modified theory of planned behavior, was conducted in 2016 with 1748 eighth-graders in Miami-Dade, Florida. Post-intervention, students had statistically significant increases (p < .05) in mean scores across outcomes: workplace safety knowledge (34%), attitude (5%), subjective norm (7%), self-efficacy (7%), and behavioral intention (7%). Consistent with theory, gains in attitude (b = 0.25, p < .001), subjective norm (b = 0.07, p < .01), and self-efficacy (b = 0.55, p < .001) were associated with gains in behavioral intention. Higher levels of implementation fidelity were associated with significant gains across outcome measures: knowledge (b = 0.60, p < .001), attitude (b = 0.08, p < .01), subjective norm (b = 0.04, p < .001), self-efficacy (b = 0.07, p < .01) and behavioral intention (b = 0.07, p < .01). Findings demonstrate the effectiveness of Talking Safety, delivered with fidelity, at positively changing measured outcomes, and provide support for using this curriculum as an essential component of any school-based, injury prevention program for young workers.

This is a preview of subscription content, log in to check access.

Fig. 1

Notes

  1. 1.

    The child labor provisions of the 1938 federal Fair Labor Standards Act (FLSA) prohibit the employment of minors in jobs and under conditions harmful to their health or well-being. These provisions restrict work hours for youth under age 16 and define hazardous occupations that the Secretary of Labor has determined to be too dangerous for minors under age 18 to perform. Certain businesses are exempt from the FLSA, and varying standards apply to agricultural and nonagricultural industries (Department of Labor, Wage, and Hour Division 2016).

References

  1. Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211.

    Article  Google Scholar 

  2. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.

    Google Scholar 

  3. Boini, S., Colin, R., & Grzebyk, M. (2017). Effect of occupational safety and health education received during schooling on the incidence of workplace injuries in the first 2 years of occupational life: A prospective study. BMJ Open. https://doi.org/10.1136/bmjopen-2016-015100.

  4. Brener, N. D., Kann, L., Shanklin, S., Kinchen, S., Eaton, D. K., Hawkins, J., et al. (2013). Methodology of the youth risk behavior surveillance system—2013. Morbidity and Mortality Weekly Report. Recommendations and Reports, 62(1), 1–20.

    PubMed  Google Scholar 

  5. Buckley, L., Sheehan, M., & Shochet, I. (2010). Short-term evaluation of a school-based adolescent injury prevention program: Determining positive effects or iatrogenic outcomes. Journal of Early Adolescence, 30, 834–853.

    Article  Google Scholar 

  6. Bureau of Labor Statistics. (2005). Work activity of high school students: Data from the National Longitudinal Survey of Youth 1997. http://www.bls.gov/news.release/archives/nlsyth_04272005.pdf. Accessed 6 July 2018.

  7. Centers for Disease Control and Prevention. (2010). Occupational injuries and deaths among younger workers: United States, 1998–2007. Morbidity and Mortality Weekly Report, 59(15), 449–455.

    Google Scholar 

  8. Centers for Disease Control and Prevention. (2016). National health education standards. https://www.cdc.gov/healthyschools/sher/standards/index.htm. Accessed 18 May 2018.

  9. Department of Labor, Wage, and Hour Division. (2016). Child labor provisions of the Fair Labor Standards Act (FLSA) for nonagricultural occupations. https://www.dol.gov/whd/regs/compliance/whdfs43.pdf. Accessed 6 July 2018.

  10. Domina, T., Brummet, Q., Pharris-Ciurej, N., Porter, S. R., Penner, A., Penner, E., et al. (2017). Capturing more than poverty: School free and reduced-price lunch data and household income (CARRA Working Paper Series Working Paper 2017–09). Retrieved from https://www.census.gov/library/working-papers/2017/adrm/carra-wp-2017-09.html. Accessed 9 January 2019.

  11. Downes, A., Novicki, E., & Howard, J. (2018). Using the contribution analysis approach to evaluate science impact: A case study of the National Institute for Occupational Safety and Health. American Journal of Evaluation. https://doi.org/10.1177/1098214018767046.

  12. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3), 327–350.

    Article  Google Scholar 

  13. Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12(2), 121–138.

    Article  Google Scholar 

  14. Flay, B. R., Allred, C. G., & Ordway, N. (2001). Effects of the Positive Action program on achievement and discipline: Two matched-control comparisons. Prevention Science, 2(2), 71–89.

    Article  CAS  PubMed  Google Scholar 

  15. Graham, J. W., Olchowski, A. E., & Gilreath, T. D. (2007). How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science, 8(3), 206–213.

    Article  Google Scholar 

  16. Guerin, R. J., Okun, A. H., & Kelley, P. (2016). Development and validation of an assessment tool for a national young worker curriculum: Assessment development for a young worker curriculum. American Journal of Industrial Medicine, 59(11), 969–978.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Guerin, R. J., Toland, M. D., Okun, A. H., Rojas-Guyler, L., & Bernard, A. L. (2018). Using a modified theory of planned behavior to examine adolescents’ workplace safety and health knowledge, perceptions, and behavioral intention: A structural equation modeling approach. Journal of Youth and Adolescence. https://doi.org/10.1007/s10964-018-0847-0.

  18. Herbert, P. C., & Lohrmann, D. K. (2011). It’s all in the delivery! An analysis of instructional strategies from effective health education curricula. Journal of School Health, 81, 258–264.

    Article  PubMed  Google Scholar 

  19. Houck, C. D., Hadley, W., Barker, D., Brown, L. K., Hancock, E., & Almy, B. (2016). An emotion regulation intervention to reduce risk behaviors among at-risk early adolescents. Prevention Science, 17(1), 71–82.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. https://doi.org/10.1080/10705519909540118.

  21. Kaiser, F. G., & Fuhrer, U. (2003). Ecological behavior’s dependency on different forms of knowledge. Applied Psychology. An International Review, 52(4), 598–613.

    Article  Google Scholar 

  22. Klassen, R. M., & Chiu, M. M. (2010). Effects on teachers’ self-efficacy and job satisfaction: Teacher gender, years of experience, and job stress. Journal of Educational Psychology, 102(3), 741–756.

    Article  Google Scholar 

  23. Koehoorn, M., Breslin, F. C., & Xu, F. (2008). Investigating the longer-term health consequences of work-related injuries among youth. Journal of Adolescent Health, 43(5), 466–473.

    Article  PubMed  Google Scholar 

  24. Lai, M. H., & Kwok, O. M. (2015). Examining the rule of thumb of not using multilevel modeling: The “design effect smaller than two” rule. The Journal of Experimental Education, 83(3), 423–438.

    Article  Google Scholar 

  25. Linker, D., Miller, M. E., Freeman, K. S., & Burbacher, T. (2005). Health and safety awareness for working teens: Developing a successful, statewide program for educating teen workers. Family & Community Health, 28(3), 225–238.

    Article  Google Scholar 

  26. MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Lawrence Erlbaum Associates.

    Google Scholar 

  27. Mardis, A. L., & Pratt, S. G. (2003). Nonfatal injuries to young workers in the retail trades and services industries in 1998. Journal of Occupational and Environmental Medicine, 45(3), 316–323.

    Article  PubMed  Google Scholar 

  28. Marsiglia, F. F., Ayers, S. L., Baldwin-White, A., & Booth, J. (2016). Changing Latino adolescents’ substance use norms and behaviors: The effects of synchronized youth and parent drug use prevention interventions. Prevention Science, 17(1), 1–12.

    Article  PubMed  PubMed Central  Google Scholar 

  29. M-DCPS. (2017). Statistical highlights 2016–2017. http://drs.dadeschools.net/StatisticalHighlights/SH1617.pdf. Accessed 9 Jan 2019.

  30. Miara, C., Gallagher, S., Bush, D., & Dewey, R. (2003). Developing an effective tool for teaching teens about workplace safety. American Journal of Health Education, 34(sup5), S–30.

    Article  Google Scholar 

  31. Montaño, D. E., & Kasprzyk, D. (2015). Theory of reasoned action, theory of planned behavior, and the integrated behavioral model. In K. Glanz, B. K. Rimer, & K. Viswanath (Eds.), Health behavior and health education: Theory, research, and practice (5th ed., pp. 95–124). Philadelphia: Wiley.

    Google Scholar 

  32. Mortimer, J. T. (2010). The benefits and risks of adolescent employment. The Prevention Researcher, 17(2), 8–11.

    PubMed  PubMed Central  Google Scholar 

  33. Mowbray, C., Holter, M., Teague, G., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.

    Article  Google Scholar 

  34. Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus user’s guide (7th ed.). Los Angeles: Muthén & Muthén.

    Google Scholar 

  35. Nicholson, L. M., Slater, S. J., Chriqui, J. F., & Chaloupka, F. (2014). Validating adolescent socioeconomic status: Comparing school free or reduced price lunch with community measures. Spatial Demography, 2(1), 55–65.

    Article  Google Scholar 

  36. NIOSH. (2018a). The work-related injury statistics query system (Work-RISQS). https://wwwn.cdc.gov/wisards/workrisqs/. Accessed 8 July 2018.

  37. NIOSH (2018b). Talking safety. https://www.cdc.gov/niosh/talkingsafety/default.html. Accessed 6 July 2018.

  38. O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of Educational Research, 78(1), 33–84.

    Article  Google Scholar 

  39. Okun, A. H., Guerin, R. J., & Schulte, P. A. (2016). Foundational workplace safety and health competencies for the emerging workforce. Journal of Safety Research, 59, 43–51.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Pisaniello, D. L., Stewart, S. K., Jahan, N., Pisaniello, S. L., Winefield, H., & Braunack-Mayer, A. (2013). The role of high schools in introductory occupational safety education—Teacher perspectives on effectiveness. Safety Science, 55, 53–61.

    Article  Google Scholar 

  41. Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209–233.

    Article  Google Scholar 

  42. Preacher, K. J., Zhang, Z., & Zyphur, M. J. (2011). Alternative methods for assessing mediation in multilevel data: The advantages of multilevel SEM. Structural Equation Modeling, 18, 161–182.

    Article  Google Scholar 

  43. Rauscher, K. J., & Myers, D. J. (2008). Socioeconomic disparities in the prevalence of work-related injuries among adolescents in the United States. Journal of Adolescent Health, 42(1), 50–57.

    Article  PubMed  Google Scholar 

  44. Rauscher, K. J., Runyan, C. W., Schulman, M. D., & Bowling, J. M. (2008). U.S. child labor violations in the retail and service industries: Findings from a national survey of working adolescents. American Journal of Public Health, 98(9), 1693–1699.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Rauscher, K. J., Casteel, C., Bush, D., & Myers, D. J. (2015). Factors affecting high school teacher adoption, sustainability, and fidelity to the “Youth @ Work: Talking Safety” curriculum. American Journal of Industrial Medicine, 58(12), 1288–1299.

    Article  PubMed  Google Scholar 

  46. Rimal, R. N. (2000). Closing the knowledge-behavior gap in health promotion: The mediating role of self-efficacy. Health Communication, 12(3), 219–237.

    Article  CAS  PubMed  Google Scholar 

  47. Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., et al. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation’s middle schools. Health Education & Behavior, 30(3), 375–391.

    Article  Google Scholar 

  48. Runyan, C. W., Schulman, M., Dal Santo, J., Bowling, J. M., Agans, R., & Ta, M. (2007). Work-related hazards and workplace safety of US adolescents employed in the retail and service sectors. Pediatrics, 119(3), 526–534.

    Article  PubMed  Google Scholar 

  49. Runyan, C. W., Lewko, J., & Rauscher, K. (2012). Setting an agenda for advancing young worker safety in the U.S. and Canada. Public Health Reports, 127(3), 246–252.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Smith, J., Purewal, B. P., Macpherson, A., & Pike, I. (2018). Metrics to assess injury prevention programs for young workers in high-risk occupations: A scoping review of the literature. Health Promotion and Chronic Disease Prevention in Canada: Research, Policy and Practice, 38(5), 191–199.

    Article  Google Scholar 

  51. Webb, T. L., & Sheeran, P. (2006). Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychological Bulletin, 132(2), 249–268.

    Article  Google Scholar 

  52. Zierold, K. M., & Anderson, H. A. (2006). Severe injury and the need for improved safety training among working teens. American Journal of Health Behavior, 30(5), 525–532.

    Article  PubMed  Google Scholar 

  53. Zierold, K. M., Garman, S., & Anderson, H. (2004). Summer work and injury among middle school students, aged 10–14 years. Occupational and Environmental Medicine, 61(6), 518–522.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

We thank the Miami-Dade County Public School System for facilitating this research, especially Mr. Cristian Carranza, Administrative Director, Division of Academics (STEAM); Dr. Ava D. Rosales, Executive Director, Department of Mathematics and Science; Mr. Dane Jaber, Instructional Supervisor, Department of Mathematics and Science; and the School Board of Miami-Dade County, Florida. For assistance with teacher training, we thank Robin Dewey, LOHP, University of California, Berkeley. For reviews of and thoughtful feedback on this manuscript, we thank Dr. Lehua Choy and Dr. Charlene Baker, University of Hawai‘i.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Rebecca J. Guerin.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

The NIOSH Institutional Review Board (IRB) waived the documentation of informed consent because the project occurred within a regularly established educational setting, used a publically available curriculum adopted by the school district as part of established and ongoing classroom studies, presented no risk of harm to participants, and involved no procedures for which written consent is normally required outside of the research context.

Disclaimer

The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the National Institute for Occupational Safety and Health. This manuscript’s data will not be deposited.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Guerin, R.J., Okun, A.H., Barile, J.P. et al. Preparing Teens to Stay Safe and Healthy on the Job: a Multilevel Evaluation of the Talking Safety Curriculum for Middle Schools and High Schools. Prev Sci 20, 510–520 (2019). https://doi.org/10.1007/s11121-019-01008-2

Download citation

Keywords

  • Young worker
  • Occupational safety and health
  • Injury prevention
  • Middle school
  • Theory of planned behavior
  • Fidelity of implementation
  • Multilevel modeling