Abstract
A systematic review was conducted to identify emergent trends in the assessment of secondary school climate. Comparisons of assessments published in the past 6 years (2016–2021) and those published prior to 2016 (total N = 46) revealed recent shifts related to conceptualization, validity, accessibility, and recommendations for improving school climate. Notably, newer assessments were more likely than older assessments to use technology to administer surveys and disseminate reports. Based on a general lack of psychometric data, we call for more attention towards the validation of school climate assessments, especially with attention towards school climate as reflecting both school-level and individual-level differences. As an exploratory analysis, we also compared features between assessments that were published in peer-reviewed journals (n = 33) to those published in the gray literature (whitepapers, evaluation reports; n = 10). More of the peer-reviewed assessments had undergone psychometric testing (88%) than those that were not peer-reviewed (60%), but fewer peer-reviewed assessments offered reports (49%) than those that were not peer-reviewed (80%), likely reflecting differing goals. Integrating features from across assessments could serve to create a single omnibus assessment with maximum utility for both research and practice.
Similar content being viewed by others
Data Availability
Data is available upon request.
References
Aldridge, J. M., Fraser, B. J., Fozdar, F., Ala’i, K., Earnest, J., & Afari, E. (2016). Students’ perceptions of school climate as determinants of wellbeing, resilience and identity. Improving Schools, 19, 5–26.
Alexander, P. A. (2020). Methodological guidance paper: The art and science of quality systematic reviews. Review of Educational Research, 90, 6–23.
Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84, 261–271. https://doi.org/10.1037/0022-0663.84.3.261
Anderson, M., & Jiang, J. (2018). Teens, social media and technology. Pew Research Center. https://www.pewresearch.org/internet/2018/05/31/teens-social-media-technology-2018/. Accessed 1 May 2022.
Assessment Work Group. (2019). Student social and emotional competence assessment: The current state of the field and a vision for its future. Collaborative for Academic, Social, and Emotional Learning.
Bandyopadhyay, S., Cornell, D. G., & Konold, T. R. (2009). Validity of three school climate scales to assess bullying, aggressive attitudes, and help seeking. School Psychology Review, 38, 338–355.
Bardach, L., Oczlon, S., Pietschnig, J., & Lüftenegger, M. (2020). Has achievement goal theory been right? A meta-analysis of the relation between goal structures and personal achievement goals. Journal of Educational Psychology, 112(6), 1197–1220.
Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., ... Tsai, Y. M. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American educational research journal, 47(1), 133–180.
Baumsteiger, R., Hoffmann, J., Castillo Gualda, R., & Brackett, M. A. (2021). Enhancing school climate through social and emotional learning: Effects of RULER in Mexican secondary schools. Learning Environments Research, 25, 465–483.
Bear, G. G., Gaskins, C., Blank, J., & Chen, F. F. (2011). Delaware school climate survey-student: Its factor structure, concurrent validity, and reliability. Journal of School Psychology, 49, 157–174.
Bellmore, A., Nishina, A., You, J.-I., & Ma, T.-L. (2012). School context protective factors against peer ethnic discrimination across the high school years. American Journal of Community Psychology, 49, 98–111.
Berkowitz, R., Moore, H., Astor, R., & Benbenishty, R. (2017). A research synthesis of the associations between socioeconomic background, inequality, school climate, and academic achievement. Review of Educational Research, 87, 425–469.
Birkett, M., Espelage, D. L., & Koenig, B. (2009). LGB and questioning students in schools: The moderating effects of homophobic bullying and school climate on negative outcomes. Journal of Youth and Adolescence, 38, 989–1000.
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149.
Bradshaw, C. P., Waasdorp, T. E., Debnam, K. J., & Lindstrom Johnson, S. (2014). Measuring school climate in high schools: A focus on safety, engagement, and the environment. Journal of School Health, 84, 593–604.
Brand, S., Felner, R., Shim, M., Seitsinger, A., & Dumas, T. (2003). Middle school improvement and reform: Development and validation of a school-level assessment of climate, cultural pluralism, and school safety. Journal of Educational Psychology, 95, 570–588.
Braskamp, L. A., & Maehr, M. L. (1988). Instructional climate inventory: Form S. MetriTech.
Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Harvard University Press.
Buros Center for Testing–Spencer Foundation Project Scholars. (2020). Social-emotional learning assessment technical guidebook. https://buros.org/selassessment-technical-guidebook. Accessed 20 Apr 2022.
National Center on Safe Supportive Learning Environments. (n.d.). School climate survey compendium. https://safesupportivelearning.ed.gov/topic-research/school-climate-measurement/school-climate-survey-compendium. Accessed 25 Apr 2022.
Charlton, C. T., Moulton, S., Sabey, C. V., & West, R. (2020). A systematic review of the effects of schoolwide intervention programs on student and teacher perceptions of school climate. Journal of Positive Behavior Interventions, 23, 185–200.
Cipriano, C., Naples, L. H., Eveleigh, A., Funaro, M., Cook, A., Cassidy, C., Rappolt-Schlichtmann, G., & McCarthy, M. (2023). A systematic review of student disability and race representation in universal school-based SEL interventions for elementary school students. Review of Educational Research, 93, 73–102.
Cohen, J., McCabe, E. M., Michelli, N., & Pickeral, T. (2009). School climate: Research, policy, practice, and teacher education. Teachers College Records, 111, 180–213.
Collins, T. N., & Parson, K. A. (2010). School climate and student outcomes. Journal of Cross-Disciplinary Perspectives Education, 3, 34–39.
Cornell, D., Shukla, K., & Konold, T. R. (2016). Authoritative school climate and student academic engagement, grades, and aspirations in middle and high Schools. AERA Open, 2, 2332858416633184.
Daily, S. M., Mann, M. J., Lilly, C. L., Dyer, A. M., Smith, M. L., & Kristjansson, A. L. (2020). School climate as an intervention to reduce academic failure and educate the whole child: A longitudinal study. Journal of School Health, 90, 182–193.
Dary, T., & Pickeral, T. (2013). School climate: Practices for implementation and sustainability. National School Climate Center.
Demirtas-Zorbaz, S., Akin-Arikan, C., & Terzi, R. (2021). Does school climate that includes students’ views deliver academic achievement? A multilevel meta-analysis. School Effectiveness and School Improvement, 32, 543–563.
Deng, L., & Chan, W. (2017). Testing the difference between reliability coefficients alpha and omega. Educational and Psychological Measurement, 77, 185–203.
Divecha, D., & Brackett, M. A. (2020). Rethinking school-based bullying prevention through the lens of social and emotional learning: A bioecological perspective. International Journal of Bullying Prevention, 2, 93–113.
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432.
Panorama Education. (n.d.). User guide: Panorama student survey. https://panorama-www.s3.amazonaws.com/files/panorama-student-survey/User-Guide.pdf?__hssc=17850406.1.1590524619628&__hstc=17850406.cbeb9c75f190a19c9cf4a6ac0a888161.1590524619620.1590524619620.1590524619620.1&__hsfp=1315046140h&hsCtaTracking=ef4fd9f4-f6c2-474a-9d37-0a95a80b09ae%7C36a69aba-bb58-4f6a-addf-89c106ad7b85. Accessed 5 May 2022.
Evans, C. B., & Smokowski, P. R. (2016). Understanding weaknesses in bullying research: How school personnel can help strengthen bullying research and practice. Children and Youth Services Review, 69, 143–150.
Evans, D., Borriello, G. A., & Field, A. P. (2018). A review of the academic and psychological impact of the transition to secondary education. Frontiers in Psychology, 9, 1482.
Every Student Succeeds Act, Public Law No. 114.95, S. 1177, 114th Cong. §1111(c)(4)(B) (Dec. 10, 2015).
Faster, D., & Lopez, D. (2013). School climate measurement and analysis. In T. Dary & T. Pickeral (Eds.), School climate practices for implementation and sustainability (pp. 17–21). New York, NY: National School Climate Center.
Fatou, N., & Kubiszewski, V. (2018). Are perceived school climate dimensions predictive of students’ engagement? Social Psychology of Education: An International Journal, 21, 427–446.
Floden, R., Stephens, A., & Schere, L. (2020). Changing expectations for the K-12 teacher workforce: Policies, preservice education, professional development, and the workplace. Washington D. C.: National Academic Press.
Frederiksen, J. R., & Collins, A. (1989). A systems approach to educational testing. Educational Researcher, 18, 27–32.
Gillen, A., Wright, A., & Spink, L. (2011). Student perceptions of a positive climate for learning: A case study. Educational Psychology in Practice, 27, 65–82.
Grazia, V., & Molinari, L. (2020). School climate multidimensionality and measurement: A systematic literature review. Research Papers in Education, 36, 561–587.
Greenberg, M. T., Domitrovich, C. E., Weissberg, R. P., & Durlak, J. A. (2017). Social and emotional learning as a public health approach to education. The Future of Children, 27, 13–32.
Harbour, K. E., Evanovich, L. L., Sweigart, C. A., & Hughes, L. E. (2015). A brief review of effective teaching practices that maximize student engagement. Preventing School Failure: Alternative Education for Children and Youth, 59, 5–13.
Harter, S. (2012). The construction of the self: Developmental and sociocultural foundations (2nd ed.). The Guilford Press.
Hoffmann, J. D., Baumsteiger, R., Seibyl, J., Hills, E., Bradley, C., Cipriano, C., & Brackett, M. A. (2022). Building useful, web-based educational measures for students, with students: An illustrative demonstration with the School Climate Walkthrough Tool for high schools. Assessment in Education: Principles, Policy, & Practice, 29, 95–120.
Hoffmann, J. D., McGarry, J. A., Baumsteiger, R., Seibyl, J., & Brackett, M. A. (in press). Emotional empowerment in high school life. In G. Misra & I. Misra (Eds.), Emotions in cultural context. Springer.
Search Institute. (2016). Technical summary: Search Institute’s REACH survey. http://www.search-institute.org/sites/default/files/a/REACH-Survey-Tech-Summary.pdf. Accessed 3 May 2022.
Jackson, C. K., Rockoff, J. E., & Staiger, D. O. (2014). Teacher effects and teacher-related policies. Annual Review of Economics, 6, 901–825.
Jiang, Y., Perry, D. K., & Hesser, J. E. (2010). Suicide patterns and association with predictors among Rhode Island public high school students: A latent class analysis. American Journal of Public Health, 100, 1701–1707.
Jimerson, S. R., Nickerson, A. B., Mayer, M. J., & Furlong, M. J. (Eds.). (2012). Handbook of school violence and school safety. Routledge.
Klaczynski, P. A. (2000). Motivated scientific reasoning biases, epistemological beliefs, and theory polarization: A two-process approach to adolescent cognition. Child Development, 71, 1347–1366.
Klieme, E., Schümer, G., & Knoll, S. (2001). Mathematikunterricht in der Sekundarstufe I: Aufgabenkultur und Unterrichtsgestaltung. In E. Klieme, & J. Baumert (Eds.), TIMSS – Impulse für Schule und Unterricht (pp. 43–57). Bundesministerium für Bildung und Forschung.
Klostermann, B., White, B., Lichtenberger, E., & Holt, J. (2014). Use of the Illinois 5Essentials Survey data (IERC 2014–2). Illinois Education Research Council at Southern Illinois University Edwardsville.
Konold, T. R., & Sanders, E. A. (2020). The nature of rater effects and differences in multilevel MTMM latent variable models. Measurement: Interdisciplinary Research and Perspectives, 18, 177–195.
Konold, T., Cornell, D., Huang, F., Meyer, P., Lacey, A., Nekvasil, E., & Shukla, K. (2014). Multilevel multi-informant structure of the authoritative school climate survey. School Psychology Quarterly, 29, 238.
Krogstad, J. M., & Fry, R. (2014). Dept. of Ed. projects public schools will be ‘majority-mintority’ this fall. Pew Research Center. https://www.pewresearch.org/fact-tank/2014/08/18/u-s-public-schools-expected-to-be-majority-minority-starting-this-fall/. Accessed 7 May 2022.
Kwong, D., & Davis, J. R. (2015). School climate for academic success: A multilevel analysis of school climate and student outcomes. Journal of Research in Education, 25, 68–81.
Lambie, G. W., Blount, A. J., & Mullen, P. R. (2017). Establishing content-oriented evidence for psychological assessments. Measurement and Evaluation in Counseling and Development, 50, 210–216.
Legislative Analyst’s Office. (2020). The 2020–21 budget: Proposition 98 education analysis. https://lao.ca.gov/reports/2020/4174/prop98-ed-analysis-022420.pdf
Lenz, A. S., Soler Gomez, I., Dell’Aquila, J., & Uribe, P. (2017). Translation and cross-cultural adaptation of assessments for use in counseling research. Measurement and Evaluation in Counseling and Development, 50, 224–231.
Lenz, A. S., Rocha, L., & Aras, Y. (2021). Measuring school climate: A systematic review of initial development and validation studies. International Journal for the Advancement of Counselling, 43, 1–15.
Lipowsky, F., Rakoczy, K., Pauli, C., Drollinger-Vetter, B., Klieme, E., & Reusser, K. (2009). Quality of geometry instruction and its short-term impact on students’ understanding of the Pythagorean Theorem. Learning and Instruction, 19(6), 527–537.
Loukas, A., Suzuki, R., & Horton, K. D. (2006). Examining school connectedness as a mediator of school climate effects. Journal of Research on Adolescence, 16, 491–502.
Lucarelli, J. F., Alaimo, K., Mang, E., Martin, C., Miles, R., Bailey, D., Kelleher, D., Drzal, N., & Liu, H. (2014). Facilitators to promoting health in schools: Is school health climate the key? School Health, 84, 133–140.
Malinen, O.-P., & Savilainen, H. (2016). The effect of perceived school climate and teacher efficacy in behavior management on job satisfaction and burnout: A longitudinal study. Teaching and Teacher Education, 60, 144–152.
Marraccini, M. E., Fang, Y., Levine, S. P., Chin, A. J., & Pittleman, C. (2020). Measuring student perceptions of school climate: A systematic review and ecological content analysis. School Mental Health, 12, 195–221.
Marsh, H. W., Ludtke, O., Nagengast, B., Trautwein, U., Morin, A. J. S., Abduljabbar, A. S., & Koller, O. (2012). Classroom climate and contextual effects. Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47, 106–124.
Marx, S., & Byrnes, D. (2012). Multicultural school climate inventory. Current Issues in Education, 15, 1–16.
Midgley, J. (1995). Social development: The developmental perspective in social welfare. Sage.
Milsom, A. (2006). Creating positive school experiences for students with disabilities. Professional School Counseling, 10, 66–72.
Moeller, J., Brackett, M. A., Ivcevic, Z., & White, A. E. (2020). High school students’ feelings: Discoveries from a large national survey and an experience sampling study. Learning and Instruction, 66, 101301.
National Center for Education Statistics. (n.d.). Status and trends in the education of racial and ethnic groups. https://nces.ed.gov/programs/raceindicators/index.asp. Accessed 2 May 2022.
Nickerson, A. B., Singleton, D., Schnurr, B., & Collen, M. H. (2014). Perceptions of school climate as a function of bullying involvement. Journal of Applied School Psychology, 30, 157–181.
O’Brennan, L. M., Bradshaw, C. P., & Furlong, M. J. (2014). Influence of classroom and school climate on teacher perceptions of student problem behavior. School Mental Health, 6, 125–136.
Olsen, J., Preston, A. I., Algozzine, B., Algozzine, K., & Cusumano, D. (2018). A review and analysis of selected school climate measures. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 91, 47–58.
Orpinas, P., & Raczynski, K. (2016). School climate associated with school dropout among tenth graders. Pensamiento Psicológico, 14, 9–20.
Osher, D., Bear, G. B., Sprague, J. R., & Doyle, W. (2010). How can we improve school discipline? Educational Researcher, 39, 48–58.
Osher, D., Kidron, Y., Brackett, M., Dymnicki, A., Jones, S., & Weissberg, R. P. (2016). Advancing the science and practice of social and emotional learning: Looking back and moving forward. Review of Research in Education, 40, 644–681.
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D.,… Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. British Medical Journal (Clinical research ed.), 372, n71
Panorama Education. (2015). Validity brief: Panorama student survey. Panorama Education. https://go.panoramaed.com/hubfs/Panorama_January2019%20/Docs/validity-brief.pdf. Accessed 5 May 2022.
Pekrun, R. (2020). Self-report is indispensable to assess students’ learning. Frontline Learning Research, 8, 185–193.
Phillips, S. F. (2015, July 6). Tripod’s seven-item climate survey: A concise assessment of school-level conditions. https://tripoded.com/2015/07/tripods-seven-item-climate-survey/. Accessed 20 Apr 2022.
Roeser, R. W., & Eccles, J. S. (2000). Schooling and mental health. In A. J. Sameroff, M. Lewis, & S. M. Miller (Eds.), Handbook of Developmental Psychopathology (pp. 135–156). Springer.
Romero, L. S., & O’Malley, M. D. (2020). An examination of classes of school climate perceptions among Latinx middle school students. Journal of School Psychology, 82, 70–84.
Rudasill, K. M., Snyder, K. E., Levinson, H., & Adelson, J. L. (2018). Systems view of school climate: A theoretical framework for research. Educational Psychology Review, 30, 35–60.
Ryberg, R., Her, S., Temkin, D., Madill, R., Kelley, C., Thompson, J., & Gabriel, A. (2020). Measuring school climate: Validating the Education Department School Climate Survey in a sample of urban middle and high school students. AERA Open, 6. https://doi.org/10.1177/2332858420948024
Schneider, B., Krajcik, J., Lavonen, J., Salmela-Aro, J., Broda, M., Spicer, J., & Viljaranta, J. (2016). Investigating optimal learning moments in U.S. and Finnish science classes. Journal of Research in Science Teaching, 53, 400–421.
Šimůnková, K. (2019). Being hybrid: A conceptual update of consumer self and consumption due to online/offline hybridity. Journal of Marketing Management, 35, 40–74.
Spurgeon, S. L. (2017). Evaluating the unintended consequences of assessment practices: Construct irrelevance and construct underrepresentation. Measurement and Evaluation in Counseling and Development, 50, 275–281.
Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41, 481–520.
Strand, G. M. (2019). Experiencing the transition to lower secondary school: Students’ voices. International Journal of Educational Research, 97, 13–21.
Taylor, R. D., Oberle, E., Durlak, J. A., & Weissberg, R. P. (2017). Promoting positive youth development through school-based social and emotional learning interventions: A meta-analysis of follow-up effects. Child Development, 88, 1156–1171.
Thapa, A., Cohen, J., Guffey, S., & Higgins-D’Alessandro, A. (2013). A review of school climate research. Review of Educational Research, 83, 357–385.
Twenge, J. M., Martin, G. N., & Spitzberg, B. H. (2019). Trends in U.S. Adolescents’ media use, 1976–2016: The rise of digital media, the decline of TV, and the (near) demise of print. Psychology of Popular Media Culture, 8, 329.
U.S. Department of Education. (n.d.). Safe and supportive schools model. https://safesupportivelearning.ed.gov/. Accessed Apr 23 2022.
U.S. Education Council. (2019). Measuring school climate and social and emotional development: A navigation guide for states and districts. https://ccsso.org/sites/default/files/2019-03/CCSSO-EdCounsel%20SE%20and%20School%20Climate%20measurement_0.pdf. Accessed 3 May 2022.
Urdan, T. (2010). The challenges and promise of research on classroom goal structures. In J. Meece & J. Eccles (Eds.), Handbook of research on classroom motivation (pp. 92–108). Erlbaum.
van der Aar, L. P. E., Peters, S., & Crone, E. A. (2018). The development of self-views across adolescence: Investigating self-descriptions with and without social comparison using a novel experimental paradigm. Cognitive Development, 48, 256–270.
Wang, M.-T., & Degol, J. L. (2016). School climate: A review of the construct, measurement, and impact on student outcomes. Educational Psychology Review, 28, 315–352.
Wang, M., & Dishion, T. J. (2012). The trajectories of adolescents’ perceptions of school climate, deviant peer affiliation, and behavioral problems during the middle school years. Journal of Research on Adolescence, 22, 40–53.
WestEd. (2014). California school climate, health, and learning survey. WestEd.
White, N., La Salle, T., Ashby, J. S., & Meyers, J. (2014). A brief measure of adolescent perceptions of school climate. School Psychology Quarterly, 29, 349–359.
Zimmer-Gembeck, M. J., & Collins, W. A. (2008). Autonomy development during adolescence. Blackwell Handbook of Adolescence, 8, 174–204.
Zitzmann, S., Wagner, W., Hecht, M., Helm, C., Fischer, C., Bardach, L., & Göllner, R. (2021). How many classes and students should ideally be sampled when assessing the role of classroom climate via student ratings on a limited budget? An optimal design perspective. Educational Psychology Review, 34, 511–536.
Zullig, K. J., Koopman, T. M., Patton, J. M., & Ubbes, V. A. (2010). School climate: Historical review, instrument development, and school assessment. Journal of Psychoeducational Assessment, 28, 139–152.
Funding
This work was funded by the Chan Zuckerberg Initiative (grant number GR101828).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics Approval and Consent to Participate
We confirm that this research was approved by the Yale Institutional Review Board and that we obtained the appropriate informed consent from participants.
Conflict of Interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Baumsteiger, R., Hoffmann, J.D., Seibyl, J. et al. A Systematic Review of Secondary School Climate Assessments. Educ Psychol Rev 35, 47 (2023). https://doi.org/10.1007/s10648-023-09748-y
Accepted:
Published:
DOI: https://doi.org/10.1007/s10648-023-09748-y