Abstract
Diagnosticians who recommend educational accommodations for postsecondary students with learning, cognitive, and psychiatric disabilities often reference specific diagnostic test scores as a basis for the recommended accommodations. Moreover, accommodation decision-makers often follow diagnosticians’ lead and/or rely on the diagnostic scores themselves to make and justify accommodation determinations. The present paper considers the ecological validity of these diagnostic test scores, focusing on their generalizability across time, setting, and dimension of performance. A wide variety of research suggests a need for circumspection and care when using diagnostic test scores to make accommodation decisions. Illustrative data are presented showing that scores from diagnostic cognitive tests do not significantly predict students’ ability to access a realistic test. Diagnostic tests should not be unduly criticized, and data from these tests can be helpful, but both clinicians and accommodation decision-makers should carefully consider issues of diagnostic test scores’ ecological validity and make nuanced conclusions about the needs of a client/applicant based on a wide variety of evidence.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adelman, H. S., Lauber, B. A., Nelson, P., & Smith, D. C. (1989). Toward a procedure for minimizing and detecting false positive diagnoses of learning disability. Journal of Learning Disabilities, 22(4), 234–244.
Arnett, P. A. (Ed.). (2013). Secondary influences on neuropsychological test performance. New York: Oxford University Press.
Brown, J. I., Fishco, V. V., & Hanna, G. (1993). Nelson-Denny Reading Test. Itasca, IL: Riverside Publishing.
Bush, S. S., Ruff, R. M., Tröster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., ... & Silver, C. H. (2005). Symptom validity assessment: Practice issues and medical necessity NAN policy & planning committee. Archives of Clinical Neuropsychology, 20(4), 419–426.
Caye, A., Swanson, J., Thapar, A., Sibley, M., Arseneault, L., Hechtman, L., Arnold, L. E., Niclasen, J., Moffitt, T., & Rohde, L. A. (2016). Life span studies of ADHD—conceptual challenges and predictors of persistence and outcome. Current Psychiatry Reports, 18(12), 111.
Chafetz, M. D., Prentkowski, E., & Rao, A. (2011). To work or not to work: motivation (not low IQ) determines symptom validity test findings. Archives of Clinical Neuropsychology, 26(4), 306–313.
Chaytor, N., & Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological tests: a review of the literature on everyday cognitive skills. Neuropsychology Review, 13(4), 181–197.
Cone, J. D. (1977). The relevance of reliability and validity for behavioral assessment. Behavior Therapy, 8(3), 411–426.
Cronbach, L. J. (1984). Essentials of psychological testing (4th ed.). New York: Harper & Row.
Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, NY: John Wiley.
Dawes, R. (1994). House of cards. New York: Free Press.
Dawes, P., Sirimanna, T., Burton, M., Vanniasegaram, I., Tweedy, F., & Bishop, D. V. M. (2009). Temporal auditory and visual motion processing of children diagnosed with auditory processing disorder and dyslexia. Ear and Hearing, 30, 675–686.
Duckworth, A. L., Quinn, P. D., Lynam, D. R., Loeber, R., & Stouthamer-Loeber, M. (2011). Role of test motivation in intelligence testing. Proceedings of the National Academy of Sciences, 108(19), 7716–7720.
Eckert, T. L., & Lovett, B. J. (2013). Principles of behavioral assessment. In D. H. Saklofske, C. R. Reynolds, & V. L. Schwean (Eds.), Oxford handbook of child psychological assessment (pp. 366–384). New York: Oxford University Press.
Epstein, J. N., Langberg, J. M., Rosen, P. J., Graham, A., Narad, M. E., Antonini, T. N., Brinkman, W. B., Froehlich, T., Simon, J. O., & Altaye, M. (2011). Evidence for higher reaction time variability for children with ADHD on a range of cognitive tasks including reward and event rate manipulations. Neuropsychology, 25(4), 427–441.
Gordon, M., Barkley, R. A., & Lovett, B. J. (2006). Tests and observational measures. In R. A. Barkley (Ed.), Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (3rd ed., pp. 369–388). New York: Guilford.
Gould, S. J. (1981). The mismeasure of man. New York: Norton.
Green, P., Rohling, M. L., Lees-Haley, P. R., & III, L. M. A. (2001). Effort has a greater effect on test scores than severe brain injury in compensation claimants. Brain Injury, 15(12), 1045–1060.
Gresham, F. M., & Witt, J. C. (1997). Utility of intelligence tests for treatment planning, classification, and placement decisions: recent empirical findings and future directions. School Psychology Quarterly, 12(3), 249–267.
Gropper, R. J., & Tannock, R. (2009). A pilot student of working memory and academic achievement in college students with ADHD. Journal of Attention Disorders, 12, 574–581.
Harrison, A. G. (2015). Child and adolescent psychoeducational evaluations. In M. W. Kirkwood (Ed.), Validity testing in the assessment of children and adolescents: evaluating exaggeration, feigning, and noncredible effort (pp. 185–206). New York: Guilford Press.
Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Participants, C. (2009). American Academy of Clinical Neuropsychology Consensus Conference Statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23(7), 1093–1129.
Heward, W. L. (2013). Exceptional children (10th ed.). Boston: Pearson.
Hunsley, J., & Mash, E. J. (2018). Developing criteria for evidence-based assessment: an introduction to assessments that work. In J. Hunsley & E. J. Mash (Eds.), Assessments that work (2nd ed., pp. 3–16). New York: Oxford University Press.
Ketterlin-Geller, L. R., & Johnstone, C. (2006). Accommodations and universal design: supporting access to assessments in higher education. Journal of Postsecondary Education and Disability, 19(2), 163–172.
Kimberg, D. Y., & Farah, M. J. (1993). A unified account of cognitive impairments following frontal lobe damage: the role of working memory in complex, organized behavior. Journal of Experimental Psychology: General, 122(4), 411–428.
Kleinmann, A. E. (2005). Not so fast: Using speed to differentiate high and average readers. Unpublished dissertation, Syracuse University.
Larrabee, G. J. (2012). Performance validity and symptom validity in neuropsychological assessment. Journal of the International Neuropsychological Society: JINS, 18(4), 625–630.
Lerner, C. S. (2004). Accommodations for the learning disabled: a level playing field or affirmative action for elites? Vanderbilt Law Review, 57, 1043–1124.
Lewandowski, L. J., Lambert, T. L., Lovett, B. J., Panahon, C., & Sytsma, M. (2014). College students’ preferences for test accommodations. Canadian Journal of School Psychology, 29, 116–126.
Lewandowski, L. J., Lovett, B. J., Parolin, R., Gordon, M., & Codding, R. S. (2007). Extended time accommodations and the mathematics performance of students with and without ADHD. Journal of Psychoeducational Assessment, 25(1), 17–28.
Lovett, B. J. (2010). Extended time testing accommodations for students with disabilities: answers to five fundamental questions. Review of Educational Research, 80, 611–638.
Lovett, B. J. (2014). Testing accommodations under the amended ADA: the voice of empirical research. Journal of Disability Policy Studies, 25, 81–90.
Lovett, B. J., Gordon, M., & Lewandowski, L. J. (2016). Legal conceptions of impairment: implications for the assessment of psychiatric disabilities. In S. Goldstein & J. A. Naglieri (Eds.), Assessing impairment: from theory to practice (2nd ed., pp. 125–139). New York: Springer.
Lovett, B. J., & Lewandowski, L. J. (2015). Testing accommodations for students with disabilities: research-based practice. Washington, DC: American Psychological Association Press.
Lovett, B. J., Lewandowski, L. J., & Potts, H. E. (2017). Test-taking speed: predictors and implications. Journal of Psychoeducational Assessment, 35(4), 351–360.
Nelson, J. M., Whipple, B., Lindstrom, W., & Foels, P. A. (2014). How is ADHD assessed and documented? Examination of psychological reports submitted to determine eligibility for postsecondary disability. Journal of Attention Disorders, Advance online, available at: https://doi.org/10.1177/1087054714561860, 108705471456186.
Ofiesh, N., Mather, N., & Russell, A. (2005). Using speeded cognitive, reading, and academic measures to determine the need for extended test time among university students with learning disabilities. Journal of Psychoeducational Assessment, 23(1), 35–52.
Palladino, P., Cornoldi, C., De Beni, R., & Pazzaglia, F. (2001). Working memory and updating processes in reading comprehension. Memory & Cognition, 29(2), 344–354.
Pardy, B. (2016). Head starts and extra time: academic accommodation on post-secondary exams and assignments for students with cognitive and mental disabilities. Education Law Journal, 25(2), 191–208.
Roberts, B. L. (2012). Beyond psychometric evaluation of the student-task determinants of accommodation: why students with learning disabilities may not need to be accommodated. Canadian Journal of School Psychology, 27(1), 72–80.
Rutkowski, D., & Wild, J. (2015). Stakes matter: student motivation and the validity of student assessments for teacher evaluation. Educational Assessment, 20(3), 165–179.
Shanahan, M. A., Pennington, B. F., Yerys, B. E., Scott, A., Boada, R., Willcutt, E. G., Olson, R. K., & DeFries, J. C. (2006). Processing speed deficits in attention deficit/hyperactivity disorder and reading disability. Journal of Abnormal Child Psychology, 34(5), 585–602.
Sharma, M., Purdy, S. C., Newall, P., Wheldall, K., Beaman, R., & Dillon, H. (2006). Electrophysiological and behavioral evidence of auditory processing deficits in children with reading disorder. Clinical Neurophysiology, 117, 1130–1144.
Sireci, S. G., Scarpati, S. E., & Li, S. (2005). Test accommodations for students with disabilities: an analysis of the interaction hypothesis. Review of Educational Research, 75(4), 457–490.
Sliwinski, M. J., Smyth, J. M., Hofer, S. M., & Stawski, R. S. (2006). Intraindividual coupling of daily stress and cognition. Psychology and Aging, 21(3), 545–557.
Solis, M., Ciullo, S., Vaughn, S., Pyle, N., Hassaram, B., & Leroux, A. (2012). Reading comprehension interventions for middle school students with learning disabilities: a synthesis of 30 years of research. Journal of Learning Disabilities, 45(4), 327–340.
Sparks, R. L., & Lovett, B. J. (2009). Objective criteria for classification of postsecondary students as learning disabled: effects on prevalence rates and group characteristics. Journal of Learning Disabilities, 42(3), 230–239.
Steedle, J. T., & Grochowalski, J. (2017). The effect of stakes on accountability test scores and pass rates. Educational Assessment, 22(2), 111–123.
Suhr, J. A. (2015). Psychological assessment: a problem-solving approach. New York: Guilford.
Swanson, H. L., Howard, C. B., & Saez, L. (2006). Do different components of working memory underlie different subgroups of reading disabilities? Journal of Learning Disabilities, 39, 252–269.
von Stumm, S. (2016). Is day-to-day variability in cognitive function coupled with day-to-day variability in affect? Intelligence, 55, 1–6.
Wasserman, J. D. (2012). A history of intelligence assessment: the unfinished tapestry. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment (3rd ed., pp. 3–55). New York: Guilford.
Watkins, M. W., & Smith, L. G. (2013). Long-term stability of the Wechsler intelligence scale for children—fourth edition. Psychological Assessment, 25(2), 477–483.
Weyandt, L. L., & DuPaul, G. J. (2013). College students with ADHD: current issues and future directions. New York: Springer.
Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson III Tests of Cognitive Abilities. Itasca, IL: Riverside.
Funding
The illustrative data in the present paper were from a study funded by a Psi Chi Faculty Advisor Grant to the first author. The opinions as expressed in this paper are those of the authors and do not necessarily reflect those of the funders.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Informed Consent
All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional). Approval was obtained from the ethics board at the institution where the data was collected.
Animal Rights
No animal studies were carried out by the authors for this article.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lovett, B.J., Bizub, A.L. Pinpointing Disability Accommodation Needs: What Evidence Is Most Relevant?. Psychol. Inj. and Law 12, 42–51 (2019). https://doi.org/10.1007/s12207-019-09341-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12207-019-09341-1