Advertisement

School Mental Health

, Volume 5, Issue 2, pp 96–109 | Cite as

Evaluation of a School-wide Preventive Intervention for Adolescents: The Secondary Social and Emotional Aspects of Learning (SEAL) Programme

  • Michael Wigelsworth
  • Neil Humphrey
  • Ann Lendrum
Original Paper

Abstract

Adolescence is a time during which youth are at an increased risk of developing mental health disorders, yet there are relatively few empirically validated preventive interventions designed for implementation in the secondary/high school phase of education. The aim of the current study was to evaluate the impact of one such programme, entitled ‘social and emotional aspects of learning (SEAL)’, over a 2-year period using a quasi-experimental pre-test–post-test control group design. Our sample consisted of 4,443 students (aged 11–12 at the start of the study) attending 41 secondary schools across England—2,442 at 22 SEAL schools and 2,001 at 19 matched control schools. Our outcome measures were the emotional symptoms and conduct problems subscales of the self-report version of the Strengths and Difficulties Questionnaire. Hierarchical linear modelling indicated that the SEAL programme had no discernible impact upon either of these domains of mental health. Further analysis using data from a subset of youth whose baseline scores placed them ‘at-risk’ demonstrated a reduction in difficulties in those attending SEAL schools, but this was matched by a similar trend in those attending control schools. Finally, although there was evidence that student outcomes for conduct problems were mediated by implementation quality, the associated effect size was very small. Variability in student outcomes for emotional symptoms appeared to be unrelated to implementation quality. The implications of these findings for school-based preventive interventions are discussed using three explanatory frameworks—theory failure, implementation failure and research failure.

Keywords

Social and emotional learning School-based prevention Universal Evaluation 

References

  1. Blank, L., Baxter, S., Goyder, L., Guillaume, L., Wilkinson, A. S. H., & Chilcott, J. (2010). Promoting wellbeing by changing behaviour: A systematic review and narrative synthesis of the effectiveness of whole secondary school behavioural interventions. Mental Health Review Journal, 15, 43–53.CrossRefGoogle Scholar
  2. Bourdon, K. H., Goodman, R., Rae, D. S., Simpson, G., & Koretz, D. S. (2005). The Strengths and Difficulties Questionnaire: U.S. normative data and psychometric properties. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 557–564.PubMedCrossRefGoogle Scholar
  3. Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2, 40–49.PubMedCrossRefGoogle Scholar
  4. Catalano, R. F., Berglund, M. L., Ryan, J. A. M., Lonczak, H. S., & Hawkins, J. D. (2004). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. The Annals of the American Academy of Political and Social Science, 591, 98–124.CrossRefGoogle Scholar
  5. Century, J., Rudnick, M., & Freeman, C. (2010). A framework for measuring fidelity of implementation: A foundation for shared language and accumulation of knowledge. American Journal of Evaluation, 31, 199–218.CrossRefGoogle Scholar
  6. Clark, A. F., O’Malley, A., Woodham, A., Barrett, B., & Byford, S. (2005). Children with complex mental health problems: Needs, costs and predictors over one year. Child and Adolescent Mental Health, 10, 170–178.CrossRefGoogle Scholar
  7. Colman, I., Murray, J., Abbott, R. A., Maughan, B., Kuh, D., Croudace, T. J., et al. (2009). Outcomes of conduct problems in adolescence: 40 year follow-up of national cohort. British Medical Journal, 338, a2981.PubMedCrossRefGoogle Scholar
  8. Davis, S. K., & Humphrey, N. (2012). Emotional intelligence as a moderator of stressor–mental health relations in adolescence: Evidence for specificity. Personality and Individual Differences, 52, 100–105.CrossRefGoogle Scholar
  9. Denham, S., & Brown, C. (2010). “Plays nice with others”: Social–emotional learning and academic success. Early Education & Development, 21, 652–680.CrossRefGoogle Scholar
  10. Department for Children, Schools and Families. (2007). Social and emotional aspects of learning in secondary schools: Guidance for schools. London: DCSF Publications.Google Scholar
  11. Department for Children, Schools and Families. (2009). Schools, pupils, and their characteristics, January 2009. London: DCSF Publications.Google Scholar
  12. Department for Children, Schools and Families. (2010a). Key stage 4 attainment by pupil characteristics, in England 2008/9. London: DCSF Publications.Google Scholar
  13. Department for Children, Schools and Families. (2010b). Pupil absence in schools, including pupil characteristics: 2008/9. London: DCSF Publications.Google Scholar
  14. Department of Health. (2007). Introduction to the national healthy schools programme. London: Department of Health.Google Scholar
  15. Dix, K. L., Slee, P. T., Lawson, M. J., & Keeves, J. P. (2012). Implementation quality of whole-school mental health promotion and students’ academic performance. Child and Adolescent Mental Health, 17, 45–51.PubMedCrossRefGoogle Scholar
  16. Domitrovich, C., Gest, S. D., Jones, D., Gill, S., & Sanford-DeRousie, R. M. (2010). Implementation quality: Lessons learned in the context of the Head Start REDI trial. Early Childhood Research Quarterly, 25, 284–298.PubMedCrossRefGoogle Scholar
  17. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.PubMedCrossRefGoogle Scholar
  18. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432.PubMedCrossRefGoogle Scholar
  19. Elias, M. J., Zins, J. E., Graczyk, P., & Weissberg, R. P. (2003). Implementation, sustainability, and scaling up of social–emotional and academic innovations in public schools. School Psychology Review, 32, 303–319.Google Scholar
  20. Farrington, D. P., Healey, A., & Knapp, M. (2004). Adult labour market implications of antisocial behaviour in childhood and adolescence: Findings from a UK longitudinal study. Applied Economics, 36, 93–105.CrossRefGoogle Scholar
  21. Field, A. (2009). Discovering statistics using SPSS. London: Sage.Google Scholar
  22. Goleman, D. (1996). Emotional intelligence: Why it can matter more than IQ. London: Bloomsbury.Google Scholar
  23. Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581–586.PubMedCrossRefGoogle Scholar
  24. Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345.PubMedCrossRefGoogle Scholar
  25. Green, H., McGinnity, A., Meltzer, H., Ford, T., & Goodman, R. (2005). Mental health of children and young people in Great Britain. Cardiff: Office for National Statistics.Google Scholar
  26. Greenberg, M. T. (2010). School-based prevention: Current status and future challenges. Effective Education, 2, 27–52.CrossRefGoogle Scholar
  27. Gutman, L., & Feinstein, L. (2008). Children's well-being in primary school: Pupil and school effects. London: Centre for Research on the Wider Benefits of Learning.Google Scholar
  28. Hawes, D. J., & Dadds, M. R. (2004). Australian data and psychometric properties of the Strengths and Difficulties Questionnaire. Australian and New Zealand Journal of Psychiatry, 38, 644–651.PubMedCrossRefGoogle Scholar
  29. Horowitz, J. L., & Garber, J. (2006). The prevention of depressive symptoms in children and adolescents: A meta-analytic review. Journal of Consulting and Clinical Psychology, 74, 401–415.PubMedCrossRefGoogle Scholar
  30. Humphrey, N. (in press). Social and emotional learning: A critical appraisal. London: Sage.Google Scholar
  31. Humphrey, N., Lendrum, A., & Wigelsworth, M. (2010). Secondary social and emotional aspects of learning (SEAL): National evaluation. Nottingham: Department for Education.Google Scholar
  32. Johnston, C., & Gowers, S. (2005). Routine outcome measurement: A survey of UK Child and Adolescent Mental Health Services. Child and Adolescent Mental Health, 10, 133–139.CrossRefGoogle Scholar
  33. Kam, C. M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63.PubMedCrossRefGoogle Scholar
  34. Maughan, B., Iervolino, A. C., & Collishaw, S. (2005). Time trends in child and adolescent mental disorders. Current Opinion in Psychiatry, 18, 381–385.PubMedCrossRefGoogle Scholar
  35. McLaughlin, K. A. (2011). The public health impact of major depression: A call for interdisciplinary prevention efforts. Prevention Science, 12, 361–371.PubMedCrossRefGoogle Scholar
  36. Mclaughlin, C., & Clarke, B. (2010). Relational matters: A review of the impact of school experience on mental health in early adolescence. Educational and Child Psychology, 27, 91–103.Google Scholar
  37. Menard, S. (1995). Applied logistic regression analysis. Thousand Oaks: Sage.Google Scholar
  38. Merrell, K. W., & Gueldner, B. A. (2010). Social and emotional learning in the classroom: Promoting mental health and academic success. London: Guilford Press.Google Scholar
  39. Myers, R. (1990). Classical and modern regression with applications. Boston: Duxbury.Google Scholar
  40. O’Connell, E., Boat, T., & Warner, K. (2009). Preventing mental, emotional and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Academies Press.Google Scholar
  41. Office for Standards in Education. (2007). Developing social, emotional and behavioural skills in secondary schools. London: Office for Standards in Education.Google Scholar
  42. Paterson, L., & Goldstein, H. (2007). New statistical methods for analysing social structures: An introduction to multilevel models. British Educational Research Journal, 17, 387–393.CrossRefGoogle Scholar
  43. Raudenbush, S. (2008). Advancing educational policy by advancing research on instruction. American Educational Research Journal, 45, 206–230.CrossRefGoogle Scholar
  44. Roth, P. (1994). Missing data: A conceptual review for psychologists. Personnel Psychology, 47, 537–560.CrossRefGoogle Scholar
  45. Schick, A., & Cierpka, M. (2005). Faustlos: Evaluation of a curriculum to prevent violence in elementary schools. Applied and Preventive Psychology, 11, 157–165.CrossRefGoogle Scholar
  46. Shafer, J. (1999). Multiple imputation: a primer. Statistical Methods in Medical Research, 8, 3–15.CrossRefGoogle Scholar
  47. Sheffield, J. K., Spence, S. H., Rapee, R. M., Kowalenko, N. K., Wignall, A., Davis, A., et al. (2006). Evaluation of universal, indicated and combined cognitive-behavioural approaches to the prevention of depression among adolescents. Journal of Consulting and Clinical Psychology, 74, 66–79.PubMedCrossRefGoogle Scholar
  48. Smith, P., O’Donnell, L., Easton, C., & Rudd, P. (2007). Secondary social, emotional and behavioural skills pilot evaluation. Nottingham: DCSF Publications.Google Scholar
  49. Social and Character Development Research Consortium. (2010). Efficacy of school-wide programs to promote social and character development and reduce problem behavior in elementary school children. Washington, DC.Google Scholar
  50. Stewart-Brown, S., & Anthony, R. (2011). Should randomised controlled trials be the “gold standard” for research on preventive interventions for children? Journal of Children’s Services, 6, 228–235.CrossRefGoogle Scholar
  51. Torgeson, C., & Torgeson, D. (2001). The need for randomised controlled trials in educational research. British Journal of Educational Studies, 49, 316–328.CrossRefGoogle Scholar
  52. Twisk, J. (2006). Applied multi-level analysis. Cambridge: Cambridge University Press.Google Scholar
  53. Tymms, P., Merrell, C., & Coe, R. (2008). Educational policies and randomised controlled trials. The Psychology of Education Review, 32, 3–7 & 26–29.Google Scholar
  54. UNICEF. (2007). An overview of child well-being in rich countries. Florence, Italy: UNICEF.Google Scholar
  55. Waterhouse, L. (2006). Multiple intelligences, the Mozart effect, and emotional intelligence: A critical review. Educational Psychologist, 41, 207–225.CrossRefGoogle Scholar
  56. Weare, K. (2010). Mental health and social and emotional learning: Evidence, principles, tensions, balances. Advances in School Mental Health Promotion, 3, 5–17.CrossRefGoogle Scholar
  57. Wells, J., Barlow, J., & Stewart-Brown, S. (2003). A systematic review of universal approaches to mental health promotion in schools. Health Education, 103, 197–220.CrossRefGoogle Scholar
  58. Wigelsworth, M., Humphrey, N., & Lendrum, A. (2012). A national evaluation of the impact of the secondary social and emotional aspects of learning (SEAL) programme. Educational Psychology, 32, 213–238.CrossRefGoogle Scholar
  59. Wilkinson, L. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594–604.CrossRefGoogle Scholar
  60. Wilson, S. J., & Lipsey, M. W. (2007). School-based interventions for aggressive and disruptive behavior: Update of a meta-analysis. American Journal of Preventive Medicine, 33(Suppl), S130–S143.PubMedCrossRefGoogle Scholar
  61. Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (2004). The scientific base linking social and emotional learning to school success. In J. E. Zins, R. P. Weissberg, M. C. Wang, & H. J. Walberg (Eds.), Building academic success on social and emotional learning: What does the research say? (pp. 3–22). New York: Teachers College Press.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2012

Authors and Affiliations

  • Michael Wigelsworth
    • 1
  • Neil Humphrey
    • 1
  • Ann Lendrum
    • 1
  1. 1.School of EducationUniversity of ManchesterManchesterUK

Personalised recommendations