Skip to main content

Advertisement

SpringerLink
Log in

Development and Initial Validation of a Reading-Specific Performance Validity Test: the College Assessment of Reading Effort (CARE)

  • Published:
Psychological Injury and Law Aims and scope Submit manuscript

Abstract

Although performance validity tests (PVTs) are routinely used in neuropsychological assessment to detect malingering or low-effort test-taking, they are seldom administered to college students seeking academic accommodations and other benefits for reading disabilities. Previous research indicates that between 9.5 and 31% of students seeking learning disability evaluations at university-based clinics provide noncredible test scores indicative of symptom exaggeration or low effort. We developed a brief reading–specific PVT designed for college students participating in reading disability testing: the College Assessment of Reading Effort (CARE). We administered the CARE and standardized reading tests to three groups of students: honest controls, students with documented reading disabilities, and students coached to simulate reading disabilities. Simulators displayed normative deficits on standardized reading measures, similar to the scores earned by students with actual reading disabilities and lower than the scores earned by honest controls. In contrast, CARE scores differentiated simulators from honest examinees with and without disabilities. ROC curve analysis showed that CARE composite scores could be used diagnostically to detect low effort with sensitivity, specificity, and predictive power ≥ 0.90. The CARE offers a time- and cost-effective way to assess performance validity during reading disability testing for postsecondary students.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. We conducted a multivariate analysis of variance (MANOVA) examining clinic-referred participants’ CARE scores as a function of whether they passed or failed the WMT. The test was significant, Wilks’ ƛ = 0.790, F(4,114) = 7.59, p < 0.001, η2 = 0.21. Follow-up analyses of variance (ANOVAs) indicated that participants who failed the WMT earned higher scores than students who passed the WMT on all four CARE measures, CARE 1 Duration, F(1,117) = 27.88, p < 0.001, η2 = 0.19; CARE 2 Duration, F(1,117) = 25.25, p < 0.001, η2 = 0.18; CARE 1 Errors, F(1,117) = 25.46, p < 0.001, η2 = 0.18; and CARE 2 Errors, F(1,117) = 25.57, p < 0.001, η2 = 0.18.

    A second MANOVA examined clinic-referred participants’ WJ-IV reading errors as a function of whether they passed or failed the WMT. The test was significant, Wilks’ ƛ = 0.693, F(2,115) = 25.44, p < 0.001, η2 = 0.31. Follow-up ANOVAs indicated that participants who failed the WMT made more reading errors than students who passed the WMT on the SRF, F(1,116) = 37.85, p < 0.001, η2 = 0.25, and WRF subtests, F(1,116) = 50.23, p < 0.001, η2 = 0.30.

  2. Qualified professionals may obtain a copy of the CARE by contacting the authors.

References

  • Alfano, K., & Boone, K. B. (2007). The use of effort tests in the context of actual versus feigned attention-deficit/hyperactivity disorder and learning disability. In K. B. Boone (Ed.), Assessment of feigned cognitive impairment (pp. 366–383). New York: Guilford.

    Google Scholar 

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Publishing.

    Book  Google Scholar 

  • Berger, C. L., & Lewandowski, L. (2013). The effect of a word processor as an accommodation for students with learning disabilities. Journal of Writing Research, 4, 261–280.

    Article  Google Scholar 

  • Boone, K. B. (2017). Self-deception in somatoform conditions. In K. B. Boone (Ed.), Neuropsychological evaluation of somatoform and other functional somatic conditions (pp. 3–42). New York: Routledge.

    Chapter  Google Scholar 

  • Bouck, E. C. (2009). Calculating the value of graphing calculators for seventh-grade students with and without disabilities. Remedial and Special Education, 30, 207–215.

    Article  Google Scholar 

  • Bouck, E. C., & Bouck, M. K. (2008). Does it add up? Calculators as accommodations for sixth grade students with disabilities. Journal of Special Education Technology, 23, 17–32.

    Article  Google Scholar 

  • Bouck, E. C., & Yadav, A. (2008). Assessing calculators as assessment accommodations for students with disabilities. Assistive Technology Outcomes and Benefits, 5, 19–28.

    Google Scholar 

  • Brueggemann, A. E., Kamphaus, R. W., & Dombrowski, S. C. (2008). An impairment model of learning disability diagnosis. Professional Psychology: Research and Practice, 39, 424–430.

    Article  Google Scholar 

  • Bush, S. S., Heilbronner, R. L., & Ruff, R. M. (2014). Psychological assessment of symptom and performance validity, response bias, and malingering: Official position of the Association for Scientific Advancement in Psychological Injury and Law. Psychological Injury and Law, 7, 197–205.

    Article  Google Scholar 

  • Chafetz, M. D., Williams, M. A., Ben-Porath, Y. S., Bianchini, K. J., Boone, K. B., Kirkwood, M. W., Larrabee, G. J., & Ord, J. S. (2015). Official position of the American Academy of Clinical Neuropsychology Social Security Administration policy on validity testing. The Clinical Neuropsychologist, 29, 723–740.

    Article  PubMed  Google Scholar 

  • Dombrowski, S. C., Kamphaus, R. W., & Reynolds, C. R. (2004). After the demise of the discrepancy: Proposed learning disabilities diagnostic criteria. Professional Psychology: Research and Practice, 35, 364–372.

    Article  Google Scholar 

  • DuPaul, G. J., Pinho, T. D., Pollack, B. L., Gormley, M. J., & Laracy, S. D. (2017). First-year college students with ADHD and/or LD. Journal of Learning Disabilities, 50, 238–251.

    Article  PubMed  Google Scholar 

  • Erdodi, L., & Roth, R. (2017). Low scores on BDAE Complex Ideational Material are associated with invalid performance in adults without aphasia. Applied Neuropsychology, 24, 264–274.

    Article  PubMed  Google Scholar 

  • Flanagan, D. P., Fiorello, C. A., & Ortiz, S. O. (2010). Enhancing practice through application of Cattell–Horn–Carroll theory and research: A third method approach to specific learning disability identification. Psychology in the Schools, 47, 739–760.

    Google Scholar 

  • Flanagan, D. P., Ortiz, S. O., & Alfonso, V. C. (2013). Essentials of cross-battery assessment. New York: Wiley.

    Google Scholar 

  • Frazier, T. W., Frazier, A. R., Busch, R. M., Kerwood, M. A., & Demaree, H. (2008). Detection of simulated ADHD and reading disorder using symptom validity measures. Archives of Clinical Neuropsychology, 23, 501–509.

    Article  PubMed  Google Scholar 

  • Green, P. (2005). Green’s Word Memory Test user’s manual. Edmonton, Alberta, Canada: Author.

    Google Scholar 

  • Green, P., & Flaro, L. (2003). Word Memory Test performance in children. Child Neuropsychology, 9, 189–207.

    Article  PubMed  Google Scholar 

  • Gregg, N. (2007). Underserved and unprepared: Postsecondary learning disabilities. Learning Disabilities Research & Practice, 22, 219–228.

    Article  Google Scholar 

  • Gregg, N. (2012). Increasing access to learning for the adult basic education learner with learning disabilities. Journal of Learning Disabilities, 45, 47–63.

    Article  PubMed  Google Scholar 

  • Greher, M. R., & Wodushek, T. R. (2017). Performance validity testing in neuropsychology. Journal of Psychiatric Practice, 23, 134–140.

    Article  PubMed  Google Scholar 

  • Harrison, A. G., & Edwards, M. J. (2010). Symptom exaggeration in post-secondary students. Applied Neuropsychology, 17, 135–143.

    Article  PubMed  Google Scholar 

  • Harrison, A. G., Edwards, M. J., & Parker, K. C. (2008). Identifying students feigning dyslexia. Dyslexia, 14, 228–246.

    PubMed  Google Scholar 

  • Harrison, A. G., Edwards, M. J., Armstrong, I., & Parker, K. C. (2010). An investigation of methods to detect feigned reading disabilities. Archives of Clinical Neuropsychology, 25, 89–98.

    Article  PubMed  Google Scholar 

  • Harrison, A. G., Green, P., & Flaro, L. (2012). The importance of symptom validity testing in adolescents and young adults undergoing assessments for learning or attention difficulties. Canadian Journal of School Psychology, 27, 98–113.

    Article  Google Scholar 

  • Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Participants, C. (2009). American Academy of Clinical Neuropsychology Consensus Conference Statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23, 1093–1129.

    Article  PubMed  Google Scholar 

  • Hurtubise, J. L., Scavone, A., Sagar, S., & Erdodi, L. A. (2017). Psychometric markers of genuine and feigned neurodevelopmental disorders in the context of applying for academic accommodations. Psychological Injury and Law, 10, 121–137.

    Article  Google Scholar 

  • Internal Revenue Service. (2017a). Publication 502: Medical and dental expenses. Washington, DC: US Treasury Department.

    Google Scholar 

  • Internal Revenue Service. (2017b). Publication 907: Tax highlights for persons with disabilities. Washington, DC: US Treasury Department.

    Google Scholar 

  • Joy, J. A., Julius, R. J., Akter, R., & Baron, D. A. (2010). Assessment of ADHD documentation from candidates requesting Americans with Disabilities Act (ADA) accommodations for the National Board of Osteopathic Medical Examiners COMLEX exam. Journal of Attention Disorders, 14, 104–108.

    Article  PubMed  Google Scholar 

  • Kraemer, H. C. (1992). Evaluating medical tests: Objective and quantitative guidelines. Newbury Park, CA: Sage.

    Google Scholar 

  • Larochette, A. C., & Harrison, A. G. (2012). Word Memory Test performance in Canadian adolescents with learning disabilities. Applied Neuropsychology, 1, 38–47.

    Article  PubMed  Google Scholar 

  • Lewandowski, L. J., Lovett, B. J., Codding, R. S., & Gordon, M. (2008a). Symptoms of ADHD and academic concerns in college students with and without ADHD diagnoses. Journal of Attention Disorders, 12, 156–161.

    Article  PubMed  Google Scholar 

  • Lewandowski, L. J., Lovett, B. J., & Rogers, C. L. (2008b). Extended time as a testing accommodation for students with reading disabilities: Does a rising tide lift all ships? Journal of Psychoeducational Assessment, 26, 315–324.

    Article  Google Scholar 

  • Lewandowski, L., Cohen, J., & Lovett, B. J. (2013). Effects of extended time allotments on reading comprehension performance of college students with and without learning disabilities. Journal of Psychoeducational Assessment, 31, 326–336.

    Article  Google Scholar 

  • Lewandowski, L., Lambert, T. L., Lovett, B. J., Panahon, C. J., & Sytsma, M. R. (2014). College students’ preferences for test accommodations. Canadian Journal of School Psychology, 29, 116–126.

    Article  Google Scholar 

  • Lewandowski, L. J., Berger, C., Lovett, B. J., & Gordon, M. (2016a). Test-taking skills of high school students with and without learning disabilities. Journal of Psychoeducational Assessment, 34, 566–576.

    Article  Google Scholar 

  • Lewandowski, L. J., Wood, W., & Miller, L. A. (2016b). Technological applications for individuals with learning disabilities and ADHD. In J. K. Luiselli & A. J. Fischer (Eds.), Computer-assisted and web-based innovations in psychology, special education, and health (pp. 61–93). San Diego, CA: Academic Press.

    Chapter  Google Scholar 

  • Lindstrom, W., & Lindstrom, J. H. (2017). College admissions tests and LD and ADHD documentation guidelines. Journal of Disability Policy Studies, 28, 32–42.

    Article  Google Scholar 

  • Lindstrom, W., Coleman, C., Thomassin, K., Southall, C. M., & Lindstrom, J. H. (2011). Simulated dyslexia in postsecondary students. The Clinical Neuropsychologist, 25, 302–322.

    Article  PubMed  Google Scholar 

  • Lovett, B. J. (2010). Extended time testing accommodations for students with disabilities. Review of Educational Research, 80, 611–638.

    Article  Google Scholar 

  • Lovett, B. J. (2014). Testing accommodations under the Amended Americans with disabilities act. Journal of Disability Policy Studies, 25, 81–90.

    Article  Google Scholar 

  • Lovett, B. J., & Lewandowski, L. J. (2015). Testing accommodations for students with disabilities. Washington, DC: American Psychological Association.

    Book  Google Scholar 

  • Lovett, B. J., & Nelson, J. M. (2017). Test anxiety and the Americans with Disabilities Act. Journal of Disability Policy Studies, 28, 99–108.

    Article  Google Scholar 

  • Lovett, B. J., & Sparks, R. L. (2010). Exploring the diagnosis of “Gifted/LD”. Journal of Psychoeducational Assessment, 28, 91–101.

    Article  Google Scholar 

  • Lovett, B. J., Nelson, J. M., & Lindstrom, W. (2015). Documenting hidden disabilities in higher education. Journal of Disability Policy Studies, 26, 44–53.

    Article  Google Scholar 

  • Mather, N., & Wendling, B. J. (2014). Examiner’s manual. Rolling Meadows, IL: Riverside: Woodcock-Johnson IV.

    Google Scholar 

  • McGregor, K. K., Langenfeld, N., Van Horne, S., Oleson, J., Anson, M., & Jacobson, W. (2016). The university experiences of students with learning disabilities. Learning Disabilities Research and Practice, 31, 90–102.

    Article  PubMed  Google Scholar 

  • Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194–216.

    Article  PubMed  Google Scholar 

  • Nelson, J.M., Whipple, B., Lindstrom, W., & Foels, P.A. (2014). How is ADHD assessed and documented? Journal of Attention Disorders. Online First. https://journals.sagepub.com/doi/10.1177/1087054714561860.

  • Newman, L., Wagner, M., Huang, T., Shaver, D., Knokey, A., Yu, J., et al. (2011). Secondary school programs and performance of students with disabilities. Washington, DC: National Center for Special Education Research.

    Google Scholar 

  • Newman, L., Wagner, M., Knokey, A., Marder, C., Nagle, K., Shaver, D.…., & Schwarting, M. (2012). The post-high school outcomes of young adults with disabilities up to 8 years after high school. Menlo Park, CA: SRI.

    Google Scholar 

  • Odland, A. P., Lammy, A. B., Martin, P. K., Grote, C. L., & Mittenberg, W. (2015). Advanced administration and interpretation of multiple validity tests. Psychological Injury and Law, 8, 46–63.

    Article  Google Scholar 

  • Osmon, D. C., Plambeck, E., Klein, L., & Mano, Q. (2006). The word reading test of effort in adult learning disability. The Clinical Neuropsychologist, 20, 315–324.

    Article  PubMed  Google Scholar 

  • Peterson’s Guide. (2017). Colleges for students with learning disabilities or ADHD. Lawrenceville, NJ: Author.

    Google Scholar 

  • Phillips, S. E. (1994). High-stakes testing accommodations. Applied Measurement in Education, 7, 93–120.

    Article  Google Scholar 

  • Princeton Review. (2016). The K&W guide to colleges for students with learning differences. Natick, MA: Author.

    Google Scholar 

  • Proctor, B., & Prevatt, F. (2003). Agreement among four models used for diagnosing learning disabilities. Journal of Learning Disabilities, 36, 459–466.

    Article  PubMed  Google Scholar 

  • Rayner, K., White, S. J., Johnson, R. L., & Liversedge, S. P. (2006). Raeding wrods with jubmled lettres: There is a cost. Psychological Science, 17, 192–193.

    Article  PubMed  Google Scholar 

  • Schrank, F.A., McGrew, K.S., & Mather, N. (2014). Woodcock-Johnson IV. Rolling Meadows, IL: Riverside.

  • Shaywitz, S. (2003). Overcoming dyslexia. New York: Vintage.

    Google Scholar 

  • Sireci, S. G., Scarpati, S. E., & Li, S. (2005). Test accommodations for students with disabilities. Review of Educational Research, 75, 457–490.

    Article  Google Scholar 

  • Sparks, R. L., & Lovett, B. J. (2009). Objective criteria for classification of postsecondary students as learning disabled. Journal of Learning Disabilities, 42, 230–239.

    Article  PubMed  Google Scholar 

  • Sparks, R. L., & Lovett, B. J. (2013). Applying objective diagnostic criteria to students in a college support program for learning disabilities. Learning Disability Quarterly, 36, 231–241.

    Article  Google Scholar 

  • Streiner, D. L. (2003). Diagnosing tests: Using and misusing diagnostic and screening tests. Journal of Personality Assessment, 81, 209–219.

    Article  PubMed  Google Scholar 

  • Suhr, J. A. (2016). Illness identity and its implications for neuropsychological assessment. National Academy of Neuropsychology Bulletin, 30, 14–16.

    Google Scholar 

  • Suhr, J. A., & Wei, C. (2013a). Response expectancies and their potential influence in neuropsychological evaluation. In P. A. Arnett (Ed.), Secondary influences on neuropsychological test performance (pp. 182–200). New York: Oxford.

    Google Scholar 

  • Suhr, J. A., & Wei, C. (2013b). Symptoms as an excuse. Journal of Social and Clinical Psychology, 32, 753–769.

    Article  Google Scholar 

  • Suhr, J. A., & Wei, C. (2017). Attention deficit/hyperactivity disorder as an illness identity. In K. Boone (Ed.), Neuropsychological evaluation of somatoform and other functional somatic conditions (pp. 251–273). New York: Routledge.

    Google Scholar 

  • Sullivan, B. K., May, K., & Galbally, L. (2007). Symptom exaggeration by college adults in attention-deficit hyperactivity disorder and learning disorder assessments. Applied Neuropsychology, 14, 189–207.

    Article  PubMed  Google Scholar 

  • UD Department of Education. (2018). The civil rights of students with hidden disabilities under Section 504 of the Rehabilitation Act of 1973. Washington, DC: Office for Civil Rights.

    Google Scholar 

  • US Department of Justice. (2016). Amendment of Americans with Disabilities Act Title II and Title III: Regulations to implement ADA Amendments Act of 2008. Federal Register, 81, 53204–53243.

    Google Scholar 

  • van den Boer, M., de Bree, E. H., & de Jong, P. F. (2018). Simulation of dyslexia. PLoS One, 13, e0196903.

    Article  PubMed  PubMed Central  Google Scholar 

  • Weis, R., Sykes, L., & Unadkat, D. (2012). Qualitative differences in learning disabilities across postsecondary institutions. Journal of Learning Disabilities, 45, 491–502.

    Article  PubMed  Google Scholar 

  • Weis, R., Speridakos, E. C., & Ludwig, K. (2014). Community college students with learning disabilities. Journal of Learning Disabilities, 47, 556–568.

    Article  PubMed  Google Scholar 

  • Weis, R., Dean, E. L., & Osborne, K. J. (2016). Accommodation decision making for postsecondary students with learning disabilities. Journal of Learning Disabilities, 49, 484–498.

    Article  PubMed  Google Scholar 

  • Weis, R., Erickson, C. P., & Till, C. H. (2017). When average is not good enough: Students with learning disabilities at selective, private colleges. Journal of Learning Disabilities, 50, 684–700.

    Article  PubMed  Google Scholar 

  • Wiederholt, J. L., & Bryant, B. R. (2013). Gray oral reading test: Examiner’s manual (5th ed.). Austin, TX: Pro-Ed.

    Google Scholar 

  • Ziegler, E. A., & Boone, K. B. (2013). Symptom invalidity on neuropsychological testing. In P. A. Arnett (Ed.), Secondary influences on neuropsychological test performance (pp. 7–38). Oxford: Oxford University Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Denison University, 100 College Road, Granville, OH, 43023, USA

    Robert Weis & Sarah J. Droder

Authors
  1. Robert Weis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarah J. Droder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert Weis.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (national and institutional). Approval was obtained from the ethics board at Denison University.

Animal Rights

No animal studies were carried out by the authors for this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research was supported by a grant from the Laurie Bukovac and David Hodgson Family Fund for disability research.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Weis, R., Droder, S.J. Development and Initial Validation of a Reading-Specific Performance Validity Test: the College Assessment of Reading Effort (CARE). Psychol. Inj. and Law 12, 29–41 (2019). https://doi.org/10.1007/s12207-019-09346-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12207-019-09346-w

Keywords

Search

Navigation