Skip to main content

Advertisement

SpringerLink
Log in

Convergent, Criterion and Social Validity of the Emotional and Behavioral Screener

  • Original Paper
  • Published:
Journal of Child and Family Studies Aims and scope Submit manuscript

Abstract

Recent reports suggest that up to 20 % of all school-aged students demonstrate characteristics related to emotional disturbance. However, <1 % of these students actually receive special education services for ED, a percentage that has remained unchanged for nearly 30 years. Screening procedures have been developed to identify the behavioral needs of students within school-wide systems such as Multitiered System of Support, Positive Behavior Support, and Response to Intervention. While several screening tools are available at this time, many require significant time, training, and/or effort, factors which limit their practical use in schools. A new efficient assessment tool Emotional and Behavioral Screener (EBS) has recently been developed as a universal screening instrument. The purpose of this study was to examine the validity of the EBS scores, with a sample of over 600 elementary students, in comparison to scores from the Social Skills Improvement System assessment tool. Results indicated high levels of convergent validity between the scores (social skills; ρ = −0.70; and problem behavior; ρ = 0.79); strong criterion validity as assessed by differences in EBS scores between individuals with normal versus problematic scores (based on established cut-off scores) on the SSIS (social skills; d = 2.23; and problem behavior; d = 3.58) and assessed by binary classification between cut-off scores of the EBS and SSIS (AUROC range .86–.88); and adequate social validity. Implications and limitations of these results are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • American Educational Research Association, American Psychological Association, & National Council of Measurement in Education. (1999). The standards for educational and psychological testing. Washington, DC: American Educational Research Association.

    Google Scholar 

  • Bradshaw, C. P., Buckley, J., & Lalongo, N. (2008). School-based service utilization among urban children with early-onset educational and mental health problems: The squeaky wheel phenomenon. School Psychology Quarterly, 23, 169–186.

    Article  Google Scholar 

  • Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York City: Guilford Press.

    Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New Jersey: Lawrence Erlbaum Associates.

    Google Scholar 

  • Compton, D. L., Fuchs, D., Fuchs, L. S., & Bryant, J. D. (2006). Selecting at-risk readers in first grade for early intervention: A two-year longitudinal study of decision rules and procedures. Journal of Educational Psychology, 98, 394–409.

    Article  Google Scholar 

  • Conroy, M. A., Hendrickson, J. M., & Hester, P. P. (2004). Early identification and prevention of emotional and behavioral disorders. In R. B. Rutherford Jr, M. M. Quinn, & S. Mather (Eds.), Handbook of research in behavior disorders (pp. 199–215). New York, NY: Guilford Press.

    Google Scholar 

  • Costello, E. J., Mustillo, S., Erkanli, A., Keeler, G., & Angold, A. (2003). Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry, 60, 837–844.

    Article  PubMed  Google Scholar 

  • Crick, N. R., Grotpeter, J. K., & Bigbee, M. A. (2002). Relationally and physically aggressive children’s intent attributions and feelings for relational and instrumental peer provocations. Child Development, 73, 1134–1142.

    Article  PubMed  Google Scholar 

  • Cullinan, D., & Epstein, M. H. (2013a). Development, reliability, and construct validity of the emotional and behavioral screener. Preventing School Failure, 57, 223–230.

    Article  Google Scholar 

  • Cullinan, D., & Epstein, M. H. (2013b). Emotional and behavioral screener (EBS). Austin, TX: PRO-ED.

    Google Scholar 

  • Eisenberg, N., Valiente, C., Spinrad, T. L., Cumbers-land, A., Liew, J., Reiser, M., et al. (2009). Longitudinal relations of children’s effortful control, impulsivity, and negative emotionality to their externalizing, internalizing and co- occurring behavior problems. Developmental Psychology, 45, 988–1008.

    Article  PubMed  PubMed Central  Google Scholar 

  • Epstein, M. H., & Cullinan, D. (2010). Scales for assessing emotional disturbance (2nd ed.). Austin, TX: PRO-ED.

    Google Scholar 

  • Essex, M., Kraemer, H. C., Slattery, M. J., Burk, L. R., Boyce, W. T., Woodward, H. R., & Kupfer, D. J. (2009). Screening for childhood mental health problems: Outcomes and early identification. Journal of Child Psychology and Psychiatry, 50, 562–570.

    Article  PubMed  PubMed Central  Google Scholar 

  • Federal Register. (2006). Assistance to states for the education of children with disabilities and preschool grants for children with disabilities; Final Rule 34 (CFR Parts 300 and 301, 71 Fed. Reg. 46540). Washington, DC: U.S. Department of Education.

  • Feldt, L. S. (1969). A test of the hypothesis that Cronbach’s alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363–373.

    Article  Google Scholar 

  • Green, S. B., & Hershberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7, 251–270.

    Article  Google Scholar 

  • Gresham, F. M., & Elliott, S. N. (2008). Social skills rating system—rating scales. Minneapolis: NCS Pearson Inc.

    Google Scholar 

  • Gresham, F. M., Elliott, S. N., & Kettler, R. J. (2010). Base rates of social skills acquisition/performance deficit, strengths, and problem behaviors: An analysis of the social skills improvement system—rating scales. Psychological Assessment, 22, 809–815.

    Article  PubMed  Google Scholar 

  • Gresham, F. M., Elliott, S. N., Vance, M. J., & Cook, C. R. (2011). Comparability of the social skills rating system to the social skills improvement system: Content and psychometric comparisons across elementary and secondary age levels. School Psychology Quarterly, 26, 27–44. doi:10.1037/a0022662.

    Article  Google Scholar 

  • Hammill, D. D., Brown, L., Bryant, B. R., & Cullinan, D. (1989). A consumer’s guide to tests in print (2nd ed.). Austin, TX: PRO-ED.

    Google Scholar 

  • Hawken, L. S., Vincent, C. G., & Schumann, J. (2008). Response to intervention for social behavior: Challenges and opportunities. Journal of Emotional and Behavioral Disorders, 16, 213–225.

    Article  Google Scholar 

  • Hopkins, W. G. (2002). A new view of statistics: A scale of magnitudes for effect statistics. Retrieved from http://www.sportsci.org/resource/stats/.

  • Horner, R. H., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A. W., et al. (2009). A randomized, wait-list controlled effectiveness trial assessing schoolwide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–144.

    Article  Google Scholar 

  • Johnson, E. S., Jenkins, J. R., Petscher, Y., & Catts, H. W. (2009). How can we improve the accuracy of screening instruments? Learning Disabilities Research & Practice, 24, 174–185.

    Article  Google Scholar 

  • Joint Committee on Standards for Educational and Psychological Testing. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

    Google Scholar 

  • Kamphaus, R. W., & Reynolds, C. R. (2007). BASC-2: Behavioral and emotional screening system. Minneapolis, MN: Pearson.

    Google Scholar 

  • Kamphaus, R. W., Thorpe, J. S., Winsor, A. P., Kroncke, A. P., Dowdy, E. T., & VanDeventer, M. (2007). Development and predictive validity of a teacher screener for child behavioral and emotional problems at school. Educational and Psychological Measurement, 6, 1–15.

    Google Scholar 

  • Kauffman, J. M., & Landrum, T. J. (2009). Characteristics of emotional and behavioral disorders of children and youth (9th ed.). Upper Saddle River, NJ: Pearson Education Inc.

    Google Scholar 

  • Lambert, M. C., Epstein, M. H., & Cullinan, D. (2014a). The diagnostic quality of the Emotional and Behavioral Screener. Journal of Psychoeducational Assessment, 32, 51–61. doi:10.1177/0734282913485541.

    Article  Google Scholar 

  • Lambert, M. C., Epstein, M., Ingram, S., Simpson, A., & Bernstein, S. (2014b). Psychometrics and measurement invariance of the emotional and behavioral screener. Behavior Disorders, 39, 89–101.

    Google Scholar 

  • Lane, K. L., Menzies, H. M., Oakes, W. P., Lambert, W., Cox, M., & Hankins, K. (2012). A validation of the student risk screening scale for internalizing and externalizing behaviors: Patterns in rural and urban elementary schools. Behavioral Disorders, 37(4), 244–270.

    Google Scholar 

  • Lane, K. L., Oakes, W., & Menzies, H. (2010). Systematic screenings to prevent the development of learning and behavioral problems: Considerations for practitioners, researchers, and policy makers. Journal of Disability Policy Studies, 21, 160–172.

    Article  Google Scholar 

  • McDonald, R. P. (1978). Generalizability in factorable domains: Domain validity and generalizability. Educational and Psychological Measurement, 38, 75–79.

    Article  Google Scholar 

  • Morris, R. J., Shah, K., & Morris, Y. P. (2002). Internalizing behavior disorders. In K. Lane, F. Gresham, & T. O’Shaughnessy (Eds.), Interventions for children with or at risk for emotional and behavioral disorders (pp. 223–241). Boston: Allyn & Bacon.

    Google Scholar 

  • Mrazek, D., & Mrazek, P. J. (2005). Prevention of psychiatric disorders in children and adolescents. In B. J. Sadock & V. A. Sadock (Eds.), Kaplan & Sadock’s comprehensive textbook of psychiatry (Vol. II, pp. 3513–3518). Philadelphia, PA: Lippincott Williams & Wilkins.

    Google Scholar 

  • National Research Council and Institute of Medicine. (2009). Preventing mental, emotional, and behavioraldisorders among young people: Progress and possibilities. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults: Research Advances and Promising Interventions. In M. E. O’Connell, T. Boat, & K. E. Warner (Eds.). Board on Children, Youth, and Families, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academies Press.

  • Nordness, P. D., Epstein, M. H., Cullinan, D., & Pierce, C. D. (2014). Emotional and behavioral screener: Test-retest reliability, inter-rater reliability, and convergent validity. Remedial and Special Education., 35, 211–217.

    Article  Google Scholar 

  • Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Pastor, P. N., Reuben, C. A., & Duran, C. R. (2012). Identifying emotional and behavioral problems in children aged 4–17 years: United States, 2001–2007. National Health Statistics Reports. Number 48. National Center for Health Statistics.

  • Salvia, J., Yesseldyke, J., & Bolt, S. (2013). Assessment in special and inclusive education (12th ed.). Belmont, CA: Wadsworth.

    Google Scholar 

  • Severson, H. H., Walker, H. M., Hope-Doolittle, J., Kratochwill, T. R., & Gresham, F. M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193–223.

    Article  Google Scholar 

  • Spearman, C. C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 3, 271–295.

    Google Scholar 

  • Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245–251.

    Article  Google Scholar 

  • Sugai, G., & Horner, R. (2006). A promising approach for expanding and sustaining the implementation of school-wide positive behavior support. School Psychology Review, 35, 245–259.

    Google Scholar 

  • Sugai, G., & Horner, R. H. (2009). Defining and describing schoolwide positive behavior support. In W. Sailor, G. Dunlap, G. Sugai, & R. H. Horner (Eds.), Handbook of positive behavior support (pp. 307–326). New York, NY: Springer. doi:10.1007/978-0-387-09632-2_13.

    Chapter  Google Scholar 

  • Walker, H. M., & Severson, H. (1992). Systematic screening for behavior disorders: User’s guide and technical manual. Longmont, CO: Sopris West.

    Google Scholar 

  • Wood, F., Flowers, L., Meyer, M., & Hill, D. (2002). How to evaluate and compare screening tests: Principles of science and good sense. Paper presented at the meeting of the International Dyslexia Association, Atlanta, GA.

Download references

Author information

Authors and Affiliations

  1. School of Special Education, University of Northern Colorado, Greeley, CO, 80639, USA

    Corey D. Pierce & Hussain Alamer

  2. University of Nebraska, Lincoln, NE, USA

    Matthew Lambert

Authors
  1. Corey D. Pierce
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew Lambert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hussain Alamer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Corey D. Pierce.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pierce, C.D., Lambert, M. & Alamer, H. Convergent, Criterion and Social Validity of the Emotional and Behavioral Screener. J Child Fam Stud 25, 77–85 (2016). https://doi.org/10.1007/s10826-015-0206-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10826-015-0206-7

Keywords

Search

Navigation