Advertisement

Neuropsychology Review

, Volume 13, Issue 4, pp 181–197 | Cite as

The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills

  • Naomi Chaytor
  • Maureen Schmitter-Edgecombe
Article

Abstract

Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic over the past decade. In this paper, we provide a comprehensive review of the research on the ecological validity of neuropsychological tests, as it pertains to everyday cognitive skills. This review is presented in the context of several theoretical issues facing ecological validity research. Overall, the research suggests that many neuropsychological tests have a moderate level of ecological validity when predicting everyday cognitive functioning. The strongest relationships were noted when the outcome measure corresponded to the cognitive domain assessed by the neuropsychological tests. Several other factors that may moderate the degree of ecological validity established for neuropsychological tests are in need of further exploration. These factors include the effects of the population being tested, the approach utilized (verisimilitude vs. veridicality), the person completing the outcome measure (significant other vs. clinician), illness severity, and time from injury until evaluation. In addition, a standard measurement of outcome for each cognitive domain is greatly needed to allow for comparison across studies.

ecological validity neuropsychological tests everyday cognitive skills 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

REFERENCES

  1. Acker, M. B. (1986). Relationships between test scores and everyday life functioning. In: Uzzell, B., and Gross, Y. (eds.), Neuropsychology of Intervention, Martinus Nijhoff Publishing, Leiden, The Netherlands, pp. 85–117.Google Scholar
  2. Bowman, M. L. (1996). Ecological validity of neuropsychological and other predictors following head injury. Clin. Neuropsychol. 10: 382–396.Google Scholar
  3. Brown, D. G., Burnett-Stolnack, M., Hashimoto, N., Hier-Wellmer, S., Perlman, O. Z., and Seigerman, C. (1996). The relationship of neuropsychological status and productive outcomes following traumatic brain injury. Brain Inj. 10: 663–676.PubMedGoogle Scholar
  4. Burgess, P. W., Alderman, N., Evans, J., Emslie, H., and Wilson, B. A. (1998). The ecological validity of tests of executive function. J. Int. Neuropsychol. Soc. 4: 547–558.PubMedGoogle Scholar
  5. Chelune, G. J. (1985). Toward a neuropsychological model of everyday functioning. Psychother. Priv. Pract. 3: 39–44.Google Scholar
  6. Cohen, J., and Cohen, P. (1983). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 2nd ed., Erlbaum, Hillsdale, NJ.Google Scholar
  7. Cripe, L. I. (1996). The ecological validity of executive function testing. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 171–202.Google Scholar
  8. Deloche, G., Dellatolas, G., Vendrell, J., and Bergego, C. (1996). Calculation and number processing: Neuropsychological assessment and daily life difficulties. J. Int. Neuropsychol. Soc. 2: 177–180.PubMedGoogle Scholar
  9. Dimitrov, M., Grafman, J., and Hollnagel, C. (1996). The effects of frontal lobe damage on everyday problem solving. Cortex 32: 357–366.PubMedGoogle Scholar
  10. Evans, J. J., Chua, S. E., McKenna, P. J., and Wilson, B. A. (1997). Assessment of the dysexecutive syndrome in schizophrenia. Psychol. Med. 27: 1–12.Google Scholar
  11. Farmer, J. E., and Eakman, A. M. (1995). The relationship between neuropsychological functioning and instrumental activities of daily living following acquired brain injury. Appl. Neuropsychol. 2: 107–115.Google Scholar
  12. Franzen, M. D., and Arnett, P. A. (1997). The validity of neuropsychological assessment procedures. In: Reese, H. W., and Franzen, M. D. (eds.), Biological and Neuropsychological Mechanisms: Life-Span Developmental Psychology, Erlbaum, Mahwah, NJ, pp. 51–69.Google Scholar
  13. Franzen, M. D., and Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in neuropsychology. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 91–112.Google Scholar
  14. Goldstein, G. (1996). Functional considerations in neuropsychology. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 75–89.Google Scholar
  15. Goldstein, G., and McCue, M. (1995). Differences between patient and informant functional outcome ratings in head-injured individuals. Int. J. Rehabil. Health, 1: 25–35.Google Scholar
  16. Goldstein, G., McCue, M., Rogers, J., and Nussbaum, P. D. (1992). Diagnostic differences in memory test based predictions of functional capacity in the elderly. Neuropsychol. Rehabil. 2: 307–317.Google Scholar
  17. Guilmette, T. J., and Kastner, M. P. (1996). The prediction of vocational functioning from neuropsychological data. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 387–411.Google Scholar
  18. Heaton, R. K., and Pendleton, M. G. (1981). Use of neuropsychological tests to predict adult patients' everyday functioning. J. Consult. Clin. Psychol. 49: 807–821.PubMedGoogle Scholar
  19. Heinrichs, R. W. (1990). Current and emergent applications of neuropsychological assessment: Problems with validity and utility. Prof. Psychol. Res. Pract. 21: 171–176.Google Scholar
  20. Higginson, C. I., Arnett, P. A., and Voss, W. D. (2000). The ecological validity of clinical tests of memory and attention in multiple sclerosis. Arch. Clin. Neuropsychol. 15: 185–204.PubMedGoogle Scholar
  21. Johnson, J. L. (1994). Episodic memory deficits in Alzheimer's disease: A behaviorally anchored scale. Arch. Clin. Neuropsychol. 9: 337–346.PubMedGoogle Scholar
  22. Johnstone, B., and Farmer, J. E. (1997). Preparing neuropsychologists for the future: The need for additional training guidelines. Arch. Clin. Neuropsychol. 12: 523–530.PubMedGoogle Scholar
  23. Kaitaro, T., Koskinen, S., and Kaipio, M. (1995). Neuropsychological problems in everyday life: A 5-year follow-up study of young severely closed-head-injured patients. Brain Inj. 9: 713–727.PubMedGoogle Scholar
  24. Kibby, M. Y., Schmitter-Edgecombe, M., and Long, C. J. (1998). Ecological validity of neuropsychological tests: Focus on the California verbal learning test and the Wisconsin card-sorting test. Arch. Clin. Neuropsychol. 13: 523–534.PubMedGoogle Scholar
  25. Long, C. J. (1996). Neuropsychological tests: A look at our past and the impact that ecological issues may have on our future. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 1–14.Google Scholar
  26. Long, C. J., and Collins, L. F. (1997). Ecological validity and forensic neuropsychological assessment. In: McCaffrey, R. J., Williams, A. D., Fisher, J. M., and Laing, L. C. (eds.), The Practice of Forensic Neuropsychology: Meeting Challenges in the Courtroom, Plenum, New York, pp. 153–164.Google Scholar
  27. Long, C. J., and Kibby, M. Y. (1995). Ecological validity of neuropsychological tests: A look at neuropsychology's past and the impact that ecological issues may have on its future. Adv. Med. Psychother. 8: 59–78.Google Scholar
  28. Makatura, T. J., Lam, C. S., Leahy, B. J., Castillo, M. T., and Kalpakjian, C. Z. (1999). Standardized memory tests and the appraisal of everyday memory. Brain Inj. 13: 355–367.PubMedGoogle Scholar
  29. McCue, M., and Pramuka, M. (1998). Functional assessment. In: Goldstein, G., and Beers, S. (eds.), Rehabilitation, Plenum, New York, pp. 113–129.Google Scholar
  30. McCue, M., Rogers, J. C., and Goldstein, G. (1990). Relationships between neuropsychological and functional assessment in elderly neuropsychiatric patients. Rehabil. Psychol. 35: 91–99.Google Scholar
  31. Nadolne, M. J., and Stringer, A. Y. (2001). Ecologic validity in neuropsychological assessment: Prediction of wayfinding. J. Neuropsychol. Soc. 7: 675–682.Google Scholar
  32. Naugle, R. I., and Chelune, G. J. (1990). Integrating neuropsychological and “real-life” data: A neuropsychological model for assessing everyday functioning. In: Tupper, D., and Cicerone, K. (eds.), The Neuropsychology of Everyday Life: Assessment and Basic Competencies, Kluwer Academic, Boston, MA, pp. 57–73.Google Scholar
  33. Nisbett, R. E., and Wilson, T. D. (1977). Telling more than we know: Verbal reports on mental processes. Psychol. Rev. 84: 231–259.Google Scholar
  34. Norris, G., and Tate, R. L. (2000). The behavioural assessment of the dysexecutive syndrome (BADS): Ecological, concurrent and construct validity. Neuropsychol. Rehabil. 10: 33–45.Google Scholar
  35. Osmon, D. C. (1999). Complexities in the evaluation of executive functions. In: Sweet, J. J. (ed.), Forensic Neuropsychology: Fundamentals and Practice, Swets and Zeitlinger, Lisse, The Netherlands, pp. 185–226.Google Scholar
  36. Poole, J. H., Ober, B. A., Shenaut, G. K., and Vinogradov, S. (1999). Independent frontal-system deficits in schizophrenia: Cognitive, clinical, and adaptive implications. Psychiatry Res. 85: 161–176.PubMedGoogle Scholar
  37. Possl, J., Jurgensmeyer, S., Karlbauer, F., Wenz, C., and Goldenberg, G. (2001). Stability of employment after brain injury: A 7-year follow-up study. Brain Inj. 15: 15–27.PubMedGoogle Scholar
  38. Ready, R. E., Stierman, L., and Paulsen, J. S. (2001). Ecological validity of neuropsychological and personality measures of executive functions. Clin. Neuropsychol. 15: 314–323.PubMedGoogle Scholar
  39. Robertson, I. H., Ward, T., Ridgeway, V., and Nimmo-Smith, I. (1996). The structure of normal human attention: The test of everyday attention. J. Int. Neuropsychol. Soc. 2: 525–534.PubMedGoogle Scholar
  40. Sbordone, R. J. (1996). Ecological validity: some critical issues for the neuropsychologist. In: Sbordone, R. J., and Long, C. J. (eds.), Ecological Validity of Neuropsychological Testing, GR Press/St. Lucie Press, Delray Beach, FL, pp. 15–41.Google Scholar
  41. Sbordone, R. J. (1997). The ecological validity of neuropsychological testing. In: Horton, A. M., Wedding, D., and Webster, J. (eds.), The Neuropsychology Handbook, Volume 1: Foundations and Assessment, 2nd edn., Springer, New York, pp. 365–392.Google Scholar
  42. Sbordone, R. J., and Guilmette, T. J. (1999). Ecological validity: Prediction of everyday and vocational functioning from neuropsychological test data. In: Sweet, J. J. (ed.), Forensic Neuropsychology: Fundamentals and Practice, Swets and Zeitlinger, Lisse, The Netherlands, pp. 227–254.Google Scholar
  43. Silver, C. H. (2000). Ecological validity of neuropsychological assessment in childhood traumatic brain injury. J. Head Trauma Rehabil. 15: 973–988.PubMedGoogle Scholar
  44. Sunderland, A., Harris, J. E., and Baddeley, A. D. (1983). Do laboratory tests predict everyday memory? A neuropsychological study. J. Verbal Learn. Verbal Behav. 22: 341–357.Google Scholar
  45. Tupper, D., and Cicerone, K. (1990). Introduction to the neuropsychology of everyday life. In: Tupper, D., and Cicerone, K. (eds.), The Neuropsychology of Everyday Life: Assessment and Basic Competencies, Kluwer Academic, Boston, MA, pp. 3–18.Google Scholar
  46. Williams, J. M. (1988). Everyday cognition and the ecological validity of intellectual and neuropsychological tests. In: Williams, J. M., and Long, C. J. (eds.), Cognitive Approaches to Neuropsychology, Plenum, New York, pp. 123–141.Google Scholar
  47. Wills, P., Clare, L., Shiel, A., and Wilson, B. A. (2000). Assessing subtle memory impairments in the everyday memory performance of brain injured people: Exploring the potential of the extended rivermead behavioural memory test. Brain Inj. 14: 693–704.PubMedGoogle Scholar
  48. Wilson, B. A. (1993). Ecological validity of neuropsychological assessment: Do neuropsychological indexes predict performance in everyday activities? Appl. Prev. Psychol. 2: 209–215.Google Scholar
  49. Wilson, B. A., Alderman, N., Burgess, P., Emslie, H., and Evans, J. J. (1996). Behavioural Assessment of the Dysexecutive Syndrome (BADS), Thames Valley Test Company, Bury St. Edmunds, England.Google Scholar
  50. Wilson, B. A., Cockburn, J., and Baddeley, A. D. (1985). The Rivermead Behavioural Memory Test Manual, Thames Valley Test Company, Bury St. Edmunds, England.Google Scholar

Copyright information

© Plenum Publishing Corporation 2003

Authors and Affiliations

  1. 1.Department of PsychologyWashington State University, PullmanWashington

Personalised recommendations