Skip to main content
SpringerLink
Log in

The development of an instrument to measure school readiness for a prevention program

  • Original Paper
  • Published:
Learning Environments Research Aims and scope Submit manuscript

Abstract

There is a need to understand how environmental characteristics of a school influence the effectiveness of prevention efforts. Although researchers have discussed the importance of conducting needs assessments before implementing interventions, few of these types of assessments focus on the relational aspect. Furthermore, few assessments have the appropriate psychometric evidence to support that they are reliably assessing an environmental concept (e.g., a construct at the school level versus at the individual level). To fill this void, we developed a scale assessing the level of school problems and the quality of relationships among principals, teachers and students. Test data were collected from 368 teachers in 37 schools who participated in a randomised controlled trial of a violence prevention program. The many-faceted Rasch rating scale model was used to analyse the data. Information on construct dimensionality, school-level reliability, internal consistency, concurrent validity and predictive validity indicated satisfactory psychometric properties for this measure. We discuss contributions of this study to the measurement of school settings and the potential benefit of implementing more effective school-based interventions by collecting this type of baseline information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Anderson, C. S. (1982). The search for school climate: A review of the research. Review of Educational Research, 52, 368–420. doi:10.3102/00346543052003368.

    Article  Google Scholar 

  • Anderson, A., Thomas, D. R., Moore, D. W., & Kool, B. (2008). Improvements in school climate associated with enhanced health and welfare services for students. Learning Environments Research, 11(3), 245–256. doi:10.1007/s10984-008-9044-5.

  • Atara, S., & Chan, D. W. K. (2013). Teacher interpersonal behavior and secondary students’ cognitive, affective and moral outcomes in Hong Kong. Learning Environments Research, 16, 23–36. doi:10.1007/s10984-012-9123-5.

    Article  Google Scholar 

  • Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: The Guilford Press.

    Google Scholar 

  • Conduct Problems Prevention Research Group. (2009). The difficulty of maintaining positive intervention effects: A look at disruptive behavior, deviant peer relations, and social skills during the middle school years. Journal of Early Adolescence, 30, 593–624. doi:10.1177/0272431609340513.

    Google Scholar 

  • Debnam, K. J., Pas, E., & Bradshaw, C. P. (2013). Factors influencing staff perceptions of administrator support for tier 2 and tier 3 interventions: A multilevel perspective. Journal of Emotional and Behavioral Disorders, 21(2), 116–126. doi:10.1177/1063426611410571.

  • DeVoe, J. F., Peter, K., Noonan, M., Snyder, T. D., & Baum, K. (2005). Indicators of school crime and safety: 2005 (NCES 2006–001/NCJ 210697). U. S. Departments of Education and Justice. Washington, DC: U. S. Government Printing Office.

  • Downer, J. T., Rimm-Kaufman, S. E., & Pianta, R. C. (2007). How do classroom conditions and children’s risk for school problems contribute to children’s engagement in learning? School Psychology Review, 36, 413–432.

    Google Scholar 

  • Dymnicki, A. B., & The Multisite Violence Prevention Project. (2013). Moderating effects of school climate on outcomes for the Multi-Site Violence Prevention Project universal program. Journal of Research on Adolescence, 24, 383–398. doi:10.1111/jora.12073.

    Article  Google Scholar 

  • Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286–299. doi:10.1037/1040-3590.7.3.286.

    Article  Google Scholar 

  • Fox, R. S., Boies, H. E., Brainard, E., Fletcher, E., Huge, J. S., Martin, C. L., et al. (1973). School climate improvement: A challenge to the school administrator. Bloomington, IN: Phi Delta Kappa.

    Google Scholar 

  • Fraser, B. J., & Walberg, H. J. (2006). Research on teacher–student relationships and learning environments: Context, retrospect and prospect. International Journal of Educational Research, 43(1–2), 103–109. doi:10.1016/j.ijer.2006.03.001.

    Google Scholar 

  • Gottfredson, G. D., Gottfredson, D. C., Payne, A. A., & Gottfredson, N. C. (2005). School climate predictors of school disorder: Results from a national study of delinquency prevention in schools. Journal of Research in Crime and Delinquency, 42, 412–444. doi:10.1177/0022427804271931.

    Article  Google Scholar 

  • Greenberg, M. T., Domitrovich, C. E., Graczyk, P. A., & Zins, J. E. (2005). The study of implementation in school-based preventive interventions: Theory, research, and practice (Vol. 3). Washington, DC: U.S. Department of Health and Human Services.

    Google Scholar 

  • Gregory, A. E., Henry, D. B., Schoeny, M. E., & The Metropolitan Area Child Study Research Group. (2007). School climate and implementation of a preventive intervention. American Journal of Community Psychology, 40, 250–260. doi:10.1007/s10464-007-9142-z.

    Article  Google Scholar 

  • Hawkins, J. D., Kosterman, R., Catalano, R. F., Hill, K. G., & Abbott, R. D. (2008). Effects of social development intervention in childhood 15 years later. Archives of Pediatrics and Adolescent Medicine, 162, 1133–1141. doi:10.1001/archpedi.162.12.1133.

    Article  Google Scholar 

  • Henry, D. B., & Chan, W. (2010). Cross-sectional and longitudinal effects of sixth-grade setting-level norms for nonviolent problem solving on aggression and associated attitudes. Journal of Community Psychology, 38, 1007–1022. doi:10.1002/jcop.20411.

    Article  Google Scholar 

  • Henry, D. B., Farrell, A. D., & The Multisite Violence Prevention Project. (2004). The study designed by a committee: Design of the Multisite Violence Prevention Project. American Journal of Preventive Medicine, 26, 12–19. doi:10.1016/j.amepre.2003.09.027.

    Article  Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. doi:10.1080/10705519909540118.

    Article  Google Scholar 

  • Johnson, W. L., Johnson, A. M., Gott, R., & Zimmerman, K. (1997). Assessing the validity of scores on the Charles F. Kettering Scale for the junior high school. Educational and Psychological Measurement, 57, 858–869. doi:10.1177/0013164497057005010.

    Article  Google Scholar 

  • Johnson, B., & Stevens, J. J. (2001). Exploratory and confirmatory factor analysis of the School Level Environment Questionnaire. Learning Environments Research, 4, 323–344. doi:10.1023/A:101448682171423.

    Article  Google Scholar 

  • Johnson, B., & Stevens, J. J. (2006). Student achievement and elementary teachers’ perceptions of school climate. Learning Environments Research, 9, 111–122. doi:10.1007/s10984-006-9007-7.

    Article  Google Scholar 

  • Kallestad, J. H., & Olweus, D. (2003). Predicting teachers’ and schools’ implementation of the Olweus Bullying Prevention Program: A multilevel study. Prevention & Treatment, 6, 1–31. doi:10.1037/1522-3736.6.1.621a.

    Article  Google Scholar 

  • Kallestad, J. H., Olweus, D., & Alsaker, F. (1998). School climate reports from Norwegian teachers: A methodological and substantive study. School Effectiveness and School Improvement, 9, 70–94. doi:10.1080/0924345980090104.

    Article  Google Scholar 

  • Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795.

    Article  Google Scholar 

  • Kellam, S. G., Brown, C. H., Poduska, J. M., Ialongo, N. S., Wang, W., Toyinbo, P., et al. (2008). Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence, 95(Supple 1), S5–S28. doi:10.1016/j.drugalcdep.2008.01.004.

    Article  Google Scholar 

  • Linacre, J. M. (1989). Many-facet Rasch measurement. Chicago, IL: MESA Press.

    Google Scholar 

  • Linacre, J. M. (2004). Optimizing rating scale category effectiveness. In E. V. Smith Jr & R. M. Smith (Eds.), Introduction to Rasch measurement: Theory, models and applications (pp. 258–279). Maple Grove, MN: JAM Press.

    Google Scholar 

  • Linacre, J. M. (2009). Facets: Version 3.66. Chicago, IL: MESA Press.

    Google Scholar 

  • Lüdtke, O., Robitzsch, A., Trautwein, U., & Kunter, M. (2006a). Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling. Contemporary Educational Psychology, 34, 120–131. doi:10.1016/j.cedpsych.2008.12.001.

    Article  Google Scholar 

  • Lüdtke, O., Trautwein, U., Kunter, M., & Baumert, J. (2006b). Reliability and agreement of student ratings of the classroom environment: A reanalysis of TIMSS data. Learning Environments Research, 9, 215–230. doi:10.1007/s10984-006-9014-8.

    Article  Google Scholar 

  • Lunz, M. E., Wright, B. D., & Linacre, J. M. (1990). Measuring the impact of judge severity on examination scores. Applied Measurement in Education, 3, 331–345. doi:10.1207/s15324818ame0304_3.

    Article  Google Scholar 

  • Miller-Johnson, S., Sullivan, T. N., Simon, T. R., & The Multisite Violence Prevention Project. (2004). Evaluating the impact of interventions in the Multisite Violence Prevention Study: Samples, procedures, and measures. American Journal of Preventive Medicine, 26, 48–61. doi:10.1016/j.amepre.2003.09.015.

    Article  Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (2007). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.

    Google Scholar 

  • Orpinas, P., Horne, A., & The Multisite Violence Prevention Project. (2004). A teacher-focused approach to prevent and reduce students’ aggressive behavior: The GREAT Teacher Program. American Journal of Preventive Medicine, 26, 29–38. doi:10.1016/j.amepre.2003.09.021.

    Article  Google Scholar 

  • Payne, A. A., Gottfredson, D. C., & Goffredson, G. D. (2006). School predictors of the intensity of implementation of school-based prevention programs: Results from a national study. Prevention Science, 7, 225–237. doi:10.1007/s11121-006-0029-2.

    Article  Google Scholar 

  • Pryce, S., & Frederickson, N. (2013). Bullying behavior, intentions and classroom ecology. Learning Environments Research, 16, 183–199. doi:10.1007/s10984-013-9137-7.

    Article  Google Scholar 

  • Raudenbush, S. W., Bryk, A. S., & Congdon, R. (2004). HLM 6 for Windows [computer software]. Skokie, IL: Scientific Software International Inc.

    Google Scholar 

  • Raudenbush, S. W., & Sampson, R. (1999). Ecometrics: Toward a science of assessing ecological settings, with application to the systematic social observations of neighborhoods. Sociological Methodology, 29, 1–41. doi:10.1111/0081-1750.00059.

    Article  Google Scholar 

  • Rentoul, J., & Fraser, B. J. (1983). Development of a school-level environment questionnaire. Journal of Educational Administration, 21(1), 21–39. doi:10.1108/eb009866.

    Article  Google Scholar 

  • Rimm-Kaufman, S. E., Storm, M. D., Sawyer, B. E., Pianta, R. C., & LaParo, K. M. (2006). The Teacher Belief QSort: A measure of teachers’ priorities in relation to disciplinary practices, teaching practices, and beliefs about children. Journal of School Psychology, 44, 141–165. doi:10.1016/j.jsp.2006.01.003.

    Article  Google Scholar 

  • Rowan, B., Raudenbush, S. W., & Kang, S. J. (1991). School climate in secondary schools. In S. W. Raudenbush & J. D. Willms (Eds.), Schools, classrooms, and pupils: International studies of a schooling from a multilevel perspective (pp. 203–233). San Diego, CA: Academic Press.

    Google Scholar 

  • SAS Institute Inc. (2008). SAS/GRAPH software: Version 9.2. Cary, NC: SAS Institute Inc.

    Google Scholar 

  • Shinn, M., & Yoshikawa, H. (2008). Towards positive youth development: Transforming schools and communities. New York: Oxford University Press.

    Book  Google Scholar 

  • Sirotnik, K. A. (1980). Psychometric implications of the unit-of-analysis problem (with examples from the measurement of organizational climate). Journal of Educational Measurement, 17, 245–282. doi:10.1111/j.1745-3984.1980.tb00831.x.

    Article  Google Scholar 

  • Stewart, E. A. (2003). School social bonds, school climate, and school misbehavior: A multilevel analysis. Justice Quarterly, 20, 575–604. doi:10.1080/07418820300095621.

    Article  Google Scholar 

  • The Multisite Violence Prevention Project. (2009). The ecological effects of universal and selective violence prevention programs for middle school students: A randomized trial. Journal of Consulting and Clinical Psychology, 77, 526–542. doi:10.1037/a0014395.

    Article  Google Scholar 

  • Thomas, D. R., Anderson, A., Kool, B., & Moore, D. W. (2004). Evaluation of the Healthy Community Schools Initiative in the AIMHI schools: Final evaluation report to the Ministry of Education, New Zealand. Auckland, NZ: UniServices Limited, The University of Auckland.

    Google Scholar 

  • Tobin, K., Ritchie, S. M., Hudson, P., Oakley, J., & Mergard, V. (2013). Relationships between emotional climate and the fluency of classroom interactions. Learning Environments Research, 16, 71–89. doi:10.1007/s10984-013-9125-y.

    Article  Google Scholar 

  • Tseng, V., & Seidman, E. (2007). A systems framework for understanding social settings. American Journal of Community Psychology Special Issue: Systems Change, 39, 217–228. doi:10.1007/s10464-007-9101-8.

    Article  Google Scholar 

  • U. S. Department of Education. Public School Teacher Questionnaire. School and Staffing Survey: 1999–2000 School Year. Washington, DC: National Center for Educational Statistics. Retrieved from http://nces.ed.gov/surveys/sass/question9900.asp.

  • Vessels, G. (1998). Character and community development: A school planning and teacher training handbook. Westport, CT: Praeger.

    Google Scholar 

  • Welsh, N. (2000). The effects of school climate on school disorder. Annals of the American Academy of Political and Social Science, 567(1), 88–107. doi:10.1177/0002716200567001007.

    Article  Google Scholar 

  • Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370. Retrieved from. http://www.rasch.org/rmt/contents.htm.

  • Zedan, R. (2010). New dimensions in the classroom climate. Learning Environments Research, 13, 75–88. doi:10.1007/s10984-009-9068-5.

    Article  Google Scholar 

Download references

Acknowledgments

This article was funded by a Dean’s Scholar Fellowship at the University of Illinois at Chicago awarded to the first author. We also wish to express our appreciation to Roger Weissberg, who provided helpful comments on an earlier draft of this manuscript.

Author information

Authors and Affiliations

  1. American Institutes for Research, 1000 Thomas Jefferson NW, Washington, DC, 20003, USA

    Allison B. Dymnicki

  2. University of Illinois at Chicago, Chicago, IL, USA

    David B. Henry & Carol M. Myford

Authors
  1. Allison B. Dymnicki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David B. Henry
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carol M. Myford
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Allison B. Dymnicki.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dymnicki, A.B., Henry, D.B. & Myford, C.M. The development of an instrument to measure school readiness for a prevention program. Learning Environ Res 18, 267–287 (2015). https://doi.org/10.1007/s10984-015-9183-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10984-015-9183-4

Keywords

Search

Navigation