Skip to main content
SpringerLink
Log in

Accuracy and Validity of Methods for Identifying Learning Disabilities in a Response-to-Intervention Service Delivery Framework

  • Chapter
  • First Online:
Handbook of Response to Intervention

Abstract

This chapter addresses the accuracy and validity of methods of learning disabilities (LD) identification, particularly methods based on a response-to-intervention (RTI) service delivery framework. Recently, classification frameworks have shifted from cognitive discrepancy models towards instructional models utilizing low achievement and instructional response criteria. All actuarial methods for LD identification, including methods based on RTI, demonstrate limited reliability for individual decisions, because they: (a) apply strict cut points that dichotomize a dimensional attribute and (b) rely on tests with imperfect reliability and validity. The resulting group membership is inherently unstable. However, methods based on RTI demonstrate good validity, because emergent groups can be differentiated on attributes not utilized to form groups, a critical test of validity. In contrast, methods based on identifying cognitive discrepancies fail because resulting groups cannot be differentiated reliably on variables not used to form groups. The authors suggest that instructional models for LD identification can be improved by limiting use of rigid cut points on single tests. Instead, identification processes should incorporate multiple academic measures, utilize confidence intervals, and move towards a system focused on ongoing assessments of risk or probability of academic difficulty and timely intervention, rather than issues of identification and entitlement.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 269.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 349.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 499.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Al Otaiba, S., & Fuchs, D. (2006). Who are the young children for whom best practices in reading are ineffective? An experimental and longitudinal study. Journal of Learning Disabilities, 39, 414–431.

    Article  PubMed  Google Scholar 

  • Barth, A. E., Stuebing, K. K., Anthony, J. L., Denton, C. A., Mathes, P. G., Fletcher, J. M., & Francis, D. J. (2008). Agreement among response to intervention criteria for identifying responder status. Learning and Individual Differences, 18, 296–307.

    Article  PubMed Central  PubMed  Google Scholar 

  • Bereiter, C. (1967). Some persisting dilemmas in the measurement of change. In C. W. Harris (Ed.), Problems in the measurement of change. Madison: U of Wisconsin Press.

    Google Scholar 

  • Bradley, R., Danielson, L., & Hallahan, D. P. (Eds.). (2002). Identification of learning disabilities: Research to practice. Mahwah: Erlbaum.

    Google Scholar 

  • Brown-Waesche, J. S., Schatschneider, C., Maner, J. K., Ahmed, Y., & Wagner, R. K. (2011). Examining agreement and longitudinal stability among traditional and RTI-based definitions of reading disability using the affected-status agreement statistic. Journal of Learning Disabilities, 44, 296–307.

    Article  PubMed Central  PubMed  Google Scholar 

  • Burns, M. K., & Senesac, S. V. (2005). Comparison of dual discrepancy criteria to assess response to intervention. Journal of School Psychology, 43(5), 393–406.

    Article  Google Scholar 

  • Burns, M. K., Scholin, S. E., Kosciolek, S., & Livingston, J. (2010). Reliability of decision-making frameworks for response to intervention for reading. Journal of Psychoeducational Assessment, 28, 102–114.

    Article  Google Scholar 

  • Campbell, D. T., & Kenny, D. A. (1999). A primer on regression artifacts. New York: Guilford Press.

    Google Scholar 

  • Cicchetti, D. V., & Sparrow, S. S. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American journal of Mental Deficiency, 86, 127–137.

    PubMed  Google Scholar 

  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.

    Article  Google Scholar 

  • Cohen, J. (1983). The cost of dichotomization. Applied Psychological Measurement, 7, 249–253.

    Article  Google Scholar 

  • Connor, C. M., Piasta, S. B., Fishman, B., Glasney, S., Schatschneider, C., Crowe, E., Underwood, P., & Morrison, F. J. (2009). Individualizing student instruction precisely: Effects of child by instruction interactions on first graders’ literacy development. Child Development, 80(1), 77–100.

    Article  PubMed Central  PubMed  Google Scholar 

  • Consortium for Evidence-Based Early Intervention Practices. (2010). A response to the Learning Disabilities Association of America (LDA) white paper on specific learning disabilities (SLD) identification. www.isbe.state.il.us/spec-ed/LDA_SLD_white_paper_response.pdf. Accessed 16 May 2012.

  • Ellis, A. W. (1984). The cognitive neuropsychology of developmental (and acquired) dyslexia: A critical survey. Cognitive Neuropsychology, 2, 169–205.

    Article  Google Scholar 

  • Fisher, S. E., & DeFries, J. C. (2002). Developmental dyslexia: Genetic dissection of a complex cognitive trait. Neuroscience, 3, 767–780.

    PubMed  Google Scholar 

  • Flanagan, D. P., Ortiz, S. O., & Alphonso, V. C. (Eds.). (2007). Essentials of cross- battery assessment. New York: Wiley.

    Google Scholar 

  • Fletcher, J. M., & Vaughn, S. (2009). Response to intervention: Preventing and remediating academic deficits. Child Development Perspectives, 3, 30–37.

    Article  PubMed Central  PubMed  Google Scholar 

  • Fletcher, J. M., Shaywitz, S. E., Shankweiler, D., Katz, L., Liberman, I. Y., Stuebing, K. K., Francis, D. J., Fowler, A. E., & Shaywitz, B. A. (1994). Cognitive profiles of reading disability: Comparisons of discrepancy and low achievement definitions. Journal of Educational Psychology, 86, 6–23.

    Article  Google Scholar 

  • Fletcher, J. M., Lyon, G. R., Fuchs, L. S., & Barnes, M. A. (2007). Learning disabilities: From identification to intervention. New York: Guilford Press.

    Google Scholar 

  • Fletcher, J. M., Stuebing, K. K., Barth, A. E., Denton, C. A., Cirino, P. T., Francis, D. J., & Vaughn, S. (2011). Cognitive correlates of inadequate response to intervention. School Psychology Review, 40, 2–22.

    Google Scholar 

  • Fletcher, J. M., Stuebing, K. K., Morris, R. D., & Lyon, G. R. (2012). Classification and definition of learning disabilities: A hybrid model. In H. L. Swanson, K. R. Harris, & S. Graham (Eds.), Handbook of learning disabilities (2nd ed., pp. 33–50). New York: Guilford Press.

    Google Scholar 

  • Fletcher, J. M., Stuebing, K. K., Barth, A. E., Miciak, J., Francis, D. J., & Denton, C. A. (2014). Agreement and coverage of indicators of response to intervention: A multi-method comparison and simulation. Topics in language disorders, 34(1), 74.

    Google Scholar 

  • Francis, D. J., Shaywitz, S. E., Stuebing, K. K., Shaywitz, B. A., & Fletcher, J. M. (1996). Developmental lag versus deficit models of reading disability: A longitudinal, individual growth curves analysis. Journal of Educational Psychology, 88, 3–17.

    Article  Google Scholar 

  • Francis, D. J., Fletcher, J. M., Stuebing, K. K., Lyon, G. R., Shaywitz, B. A., & Shaywitz, S. E. (2005). Psychometric approaches to the identification of learning disabilities: IQ and achievement scores are not sufficient. Journal of Learning Disabilities, 38, 98–110.

    Article  PubMed  Google Scholar 

  • Fuchs, D., & Deshler, D. K. (2007). What we need to know about responsiveness to intervention (and shouldn’t be afraid to ask). Learning Disabilities Research & Practice, 20, 129–136.

    Article  Google Scholar 

  • Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research & Practice, 13, 204–219.

    Google Scholar 

  • Geary, D. C., Hoard, M. K., Byrd-Craven, M., Nugent, L., & Chattavee, N. (2007). Cognitive mechanisms underlying achievement deficits in children with mathematical learning disability. Child Development, 78, 1343–1359.

    Article  PubMed Central  PubMed  Google Scholar 

  • Grigorenko, E. L. (2001). Developmental dyslexia: An update on genes, brains and environments. Journal of Child Psychology and Psychiatry, 42, 91–125.

    Article  PubMed  Google Scholar 

  • Grigorenko, E. L. (2005). A conservative meta-analysis of linkage and linkage-association studies of developmental dyslexia. Scientific Studies of Reading, 9, 285–316.

    Article  Google Scholar 

  • Hale, J. B., & Fiorello, C. A. (2004). School neuropsychology: A practitioner’s handbook. New York: Guilford Press.

    Google Scholar 

  • Hale, J. B., Alfonso, V., Berninger, B., Bracken, B., Christo, C., Clark, E., et al. (2010). Critical issues in response-to-intervention, comprehensive evaluation, and specific learning disabilities identification and intervention: An expert white paper consensus. Learning Disability Quarterly, 33(3), 223–236.

    Google Scholar 

  • Hoskyn, M., & Swanson, H. L. (2000). Cognitive processing of low achievers and children with reading disabilities: A selective meta-analytic review of the published literature. The School Psychology Review, 29, 102–119.

    Google Scholar 

  • Jorm, A. F., share, D. L., Matthews, M., & Matthews, R. (1986). Cognitive factors at school entry predictive of specific reading retardation and general reading backwardness: A research note. Journal of Child Psycholgoy, 27, 45–54.

    Article  Google Scholar 

  • Kearns, D. M., & Fuchs, D. (2013). Does cognitively focused instruction improve the academic performance of low-achieving students? Exceptional Children, 79, 263–290.

    Google Scholar 

  • Kovaleski, J. F., VanDerHeyden, A. M., & Shapiro, E. S. (2013). The RTI approach to evaluating learning disabilities. New York: Guilford Press.

    Google Scholar 

  • MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7(1), 19.

    Google Scholar 

  • Macmann, G. M., & Barnett, D. W. (1985). Discrepancy score analysis: A computer simulation of classification stability. Journal of Psychoeducational Assessment, 4, 363–375.

    Article  Google Scholar 

  • Macmann, G. M., Barnett, D. W., Lombard, T. J., Belton-Kocher, E., & Sharpe, M. N. (1989). On the actuarial classification of children: Fundamental studies of classification agreement. The Journal of Special Education, 23(2), 127–149.

    Article  Google Scholar 

  • Mathes, P. G., Denton, C. A., Fletcher, J. M., Anthony, J. A., Francis, D. J., & Schatschneider, C. (2005). The effects of theoretically different instruction and student characteristics on the skills of struggling readers. Reading Research Quarterly, 40, 148–182.

    Article  Google Scholar 

  • Miciak, J., Stuebing, K. K., Vaughn, S., Roberts, G., Barth, A. E., & Fletcher, J. M. (2014a). Cognitive attributes of adequate and inadequate responders to reading intervention in middle school. School Psychology Review, 43(4).

    Google Scholar 

  • Miciak, J., Fletcher, J. M., Stuebing, K. K., Vaughn, S., & Tolar, T. D. (2014b). Patterns of cognitive strengths and weaknesses: Identification rates, agreement, and validity for learning disabilities identification. School Psychology Quarterly.

    Google Scholar 

  • Molfese, P. J., Fletcher, J. M., & Denton, C. A. (2013). Adequate versus inadequate response to reading intervention: An event-related potentials assessment. Developmental Neuropsychology, 38(8), 534–549.

    Google Scholar 

  • Morris, R. D., & Fletcher, J. M. (1988). Classification in neuropsychology: A theoretical framework and research paradigm. Journal of Clinical and Experimental Neuropscyhology, 10, 640–658.

    Article  Google Scholar 

  • Morris, R., Lovett, M. W., Wolf, M., Sevcik, R., Steinbach, K., Frijters, J., & Shapiro, M. (2012). Multiple-component remediation for developmental reading disabilities: IQ, socioeconomic status, and race as factors in remedial outcome. Journal of Learning Disabilities, 45(2), 99–127.

    Article  PubMed  Google Scholar 

  • Naglieri, J. A. (1999). Essentials of CAS assessment. New York: Wiley.

    Google Scholar 

  • Naglieri, J. A., & Das, J. P. (1997). Cognitive assessment system. Chicago: Riverside Publishing.

    Google Scholar 

  • Nelson, R. J., Benner, G. J., & Gonzalez, J. (2003). Learner characteristics that influence the treatment effectiveness of early literacy interventions: A meta-analytic review. Learning Disabilities Research & Practice, 18, 255–267.

    Article  Google Scholar 

  • Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2009). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9(3), 105–119.

    Google Scholar 

  • Plomin R., & Kovas, Y. (2005). Generalist genes and learning disabilities. Psychological Bulletin, 131, 592–617.

    Article  PubMed  Google Scholar 

  • Reschly, D. J., & Tily, W. D. (1999). Reform trends and system design alternatives. In D. Reschly, W. Tilly, & J. Grimes (Eds.), Special education in transition (pp. 19–48). Longmont: Sopris West.

    Google Scholar 

  • Reynolds, C. R., & Shaywitz, S. E. (2009). Response to intervention: Ready or not? Or watch- them- fail. School Psychology Quarterly, 24, 130–145.

    Article  PubMed Central  PubMed  Google Scholar 

  • Rezaie, R., Simos, P., Fletcher, J., Cirino, P., Vaughn, S., & Papanicolaou, A. C. (2011). Temporo-parietal brain activity as a longitudinal predictor of response to educational interventions among middle school struggling readers. Journal of the International Neuropsychological Society, 17, 875–885.

    Article  PubMed Central  PubMed  Google Scholar 

  • Rodgers, B. (1983). The identification and prevalence of specific reading retardation. British Journal of Educational Psychology, 53, 369–373.

    Article  PubMed  Google Scholar 

  • Ross, A. D. (1976). Psychological aspects of learning disabilities and reading disorders. New York: McGraw-Hill.

    Google Scholar 

  • Rutter, M. (1982). Syndromes attributed to “minimal brain dysfunction” in childhood. American Journal of Psychiatry, 139, 21–33.

    Article  PubMed  Google Scholar 

  • Satz, P., & Fletcher, J. M. (1980). Minimal brain dysfunctions: An appraisal of research concepts and methods. In H. Rie & E. Rie (Eds.), Handbook of minimal brain dysfunctions: A critical view (pp. 669–715). New York: Wiley.

    Google Scholar 

  • Schatschneider, C., Wagner, R. K., & Crawford, E. C. (2008). The importance of measuring growth in response to intervention models: Testing a core assumption. Learning and Individual Differences, 18, 308–315.

    Article  PubMed Central  PubMed  Google Scholar 

  • Shaywitz, S. E., Escobar, M. D., Shaywitz, B. A., Fletcher, J. M., & Makuch, R. (1992). Evidence that dyslexia may represent the lower tail of a normal distribution of reading ability. New England Journal of Medicine, 326, 145–150.

    Article  PubMed  Google Scholar 

  • Siegel, L.S. (1992). An evaluation of the discrepancy definition of dyslexia. Journal of Learning Disabilities, 25, 618–629.

    Article  PubMed  Google Scholar 

  • Silva, P. A., McGee, R., & Williams, S. (1985). Some characteristics of 9-year-old boys with general reading backwardness or specific reading retardation. Journal of Child Psychology and Psychiatry, 26, 407–421.

    Article  PubMed  Google Scholar 

  • Simos, P. G., Rezaie, R., Fletcher, J. M., & Papanicolaou, A. C. (2013). Time constrained functional connectivity analysis of cortical networks underlying phonological decoding in typically developing school-aged children: A magnetoencephalography study. Brain and Language, 125(2), 156–64.

    Article  PubMed Central  PubMed  Google Scholar 

  • Stanovich, K. E., & Siegel, L. S. (1994). Phenotypic performance profile of children with reading disabilities: A regression-based test of the phonological-core variable-difference model. Journal of Educational Psychology, 86, 24–53.

    Article  Google Scholar 

  • Stuebing, K. K., Fletcher, J. M., LeDoux, J. M., Lyon, G. R., Shaywitz, S. E., & Shaywitz, B. A. (2002). Validity of IQ-discrepancy classifications of reading disabilities: A meta-analysis. American Educational Research Journal, 39, 469–518.

    Article  Google Scholar 

  • Stuebing, K. K., Barth, A. E., Molfese, P. J., Weiss, B., & Fletcher, J. M. (2009). IQ is not strongly related to response to reading instruction: A meta-analytic interpretation. Exceptional Children, 76, 31–51.

    PubMed Central  PubMed  Google Scholar 

  • Stuebing, K. K., Fletcher, J. M., Branum-Martin, L., & Francis, D. J. (2012). Evaluation of the technical adequacy of three methods for identifying specific learning disabilities based on cognitive discrepancies. School Psychology Review, 41, 3–21.

    PubMed Central  PubMed  Google Scholar 

  • Tanaka, H., Black, J., Hulme, C., Leanne, S., Kesler, S., Whitfield, G., Reiss, A. L., Gabrieli, J. D., & Hoeft, F. (2011). The brain basis of the phonological deficit in dyslexia is independent of IQ. Psychological Science, 22, 1442–1451.

    Article  PubMed Central  PubMed  Google Scholar 

  • Taylor, H. G., Satz, P., & Friel, J. (1979). Developmental dyslexia in relation to other childhood reading disorders: Significance and clinical validity. Reading Research Quarterly, 15, 84–101.

    Article  Google Scholar 

  • Tolar, T. D., Barth, A. E., Fletcher, J. M., Francis, D. J., & Vaughn, S. (2014). Predicting reading outcomes with progress monitoring slopes among middle grade students. Learning and Individual Differences, 30, 46–57.

    Google Scholar 

  • Torgesen, J., Wagner, R., & Rashotte, C. (1999). Test of word reading efficiency. Austin: Pro-Ed.

    Google Scholar 

  • Vellutino, F. R., Scanlon, D. M., & Lyon, G. R. (2000). Differentiating between difficult to remediate and readily remediated poor readers: More evidence against the IQ Achievement discrepancy definition of reading disability. Journal of Learning Disabilities, 33, 223–238.

    Article  PubMed  Google Scholar 

  • Vellutino, F. R., Scanlon, D. M., Small, S., & Fanuele, D. P. (2006). Response to intervention as a vehicle for distinguishing between children with and without reading disabilities: Evidence for the role of kindergarten and first-grade interventions. Journal of Learning Disabilities, 39, 157–169.

    Article  PubMed  Google Scholar 

  • Wechsler, D. (2003). Wechsler intelligence scale for children, fourth edition: Technical manual. San Antonio: The Psychological Corporation.

    Google Scholar 

  • Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson III tests of achievement. Itasca: Riverside.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology and Texas Institute for Measurement, Evaluation, and Statistics, University of Houston, Houston, TX, USA

    Jeremy Miciak, Jack M. Fletcher & Karla K. Stuebing

Authors
  1. Jeremy Miciak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jack M. Fletcher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karla K. Stuebing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeremy Miciak .

Editor information

Editors and Affiliations

  1. University of California, Santa Barbara Gevirtz Graduate School of Education, Santa Barbara, California, USA

    Shane R. Jimerson

  2. University of Missouri, Columbia, Missouri, USA

    Matthew K. Burns

  3. Education Research & Consulting, Inc., FAIRHOPE, Alabama, USA

    Amanda M. VanDerHeyden

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media New York

About this chapter

Cite this chapter

Miciak, J., Fletcher, J., Stuebing, K. (2016). Accuracy and Validity of Methods for Identifying Learning Disabilities in a Response-to-Intervention Service Delivery Framework. In: Jimerson, S., Burns, M., VanDerHeyden, A. (eds) Handbook of Response to Intervention. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7568-3_25

Download citation

Publish with us

Policies and ethics

Search

Navigation