Skip to main content

Computerized Neurocognitive Testing in the Management of Sport-Related Concussion: An Update

Neuropsychology Review volume 23, pages 335–349 (2013)Cite this article

Abstract

Since the late nineties, computerized neurocognitive testing has become a central component of sport-related concussion (SRC) management at all levels of sport. In 2005, a review of the available evidence on the psychometric properties of four computerized neuropsychological test batteries concluded that the tests did not possess the necessary criteria to warrant clinical application. Since the publication of that review, several more computerized neurocognitive tests have entered the market place. The purpose of this review is to summarize the body of published studies on psychometric properties and clinical utility of computerized neurocognitive tests available for use in the assessment of SRC. A review of the literature from 2005 to 2013 was conducted to gather evidence of test-retest reliability and clinical validity of these instruments. Reviewed articles included both prospective and retrospective studies of primarily sport-based adult and pediatric samples. Summaries are provided regarding the available evidence of reliability and validity for the most commonly used computerized neurocognitive tests in sports settings.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Allen, B. J., & Gfeller, J. D. (2011). The Immediate Post-Concussion Assessment and Cognitive Testing battery and traditional neuropsychological measures: a construct and concurrent validity study. Brain Injury, 25(2), 179–191.

    PubMed  Article  Google Scholar 

  • ANAM4 Sports Medicine Battery: Automated Neuropsychological Assessment Metrics (2010). Norman: Center for the Study of Human Operator Performance.

  • Anastasi, A. (1998). Pyschological testing (6th ed.). New York: Macmillan.

    Google Scholar 

  • Association, A. E. R. (1999). Standards for educational and psychological testing. Washington, D.C.: American Education Research Association.

    Google Scholar 

  • Aubry, M., Cantu, R., Dvorak, J., Graf-Baumann, T., Johnston, K. M., Kelly, J., et al. (2002). Summary and agreement statement of the 1st International Symposium on Concussion in Sport, Vienna 2001. Clinical Journal of Sports Medicine, 12(1), 6–11.

    Article  Google Scholar 

  • Bauer, R. M., Iverson, G. L., Cernich, A. N., Binder, L. M., Ruff, R. M., & Naugle, R. I. (2012). Computerized neuropsychological assessment devices: joint position paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology. The Clinical Neuropsychologist, 26(2), 177–196.

    PubMed  Article  Google Scholar 

  • Baumgartner, T. A., & Hensley, L. D. (2006). Conducting and reading research in health and human performance (4th ed.). New York: McGraw Hill.

    Google Scholar 

  • Belanger, H. G., & Vanderploeg, R. D. (2005). The neuropsychological impact of sports-related concussion: a meta-analysis. Journal of International Neuropsychological Society, 11(4), 345–357.

    Article  Google Scholar 

  • Broglio, S. P., & Puetz, T. W. (2008). The effect of sport concussion on neurocognitive function, self-report symptoms and postural control : a meta-analysis. Sports Medicine, 38(1), 53–67.

    PubMed  Article  Google Scholar 

  • Broglio, S. P., Ferrara, M. S., Macciocchi, S. N., Baumgartner, T. A., & Elliott, R. (2007a). Test-retest reliability of computerized concussion assessment programs. Journal of Athletic Training, 42(4), 509–514.

    PubMed  PubMed Central  Google Scholar 

  • Broglio, S. P., Macciocchi, S. N., & Ferrara, M. S. (2007b). Sensitivity of the concussion assessment battery. Neurosurgery, 60(6), 1050–1057.

    PubMed  Article  Google Scholar 

  • Cernich, A. N., Brennana, D. M., Barker, L. M., & Bleiberg, J. (2007). Sources of error in computerized neuropsychological assessment. Archives of Clinical Neuropsychology, 22(Suppl 1), S39–S48.

    PubMed  Article  Google Scholar 

  • CogState (2011). CogState Research Manual (Version 6.0). New Haven: CogState Ltd.

  • Cole, W. R., Arrieux, J. P., Schwab, K., Ivins, B. J., Qashu, F. M., & Lewis, S. C. (2013). Test-retest reliability of four computerized neurocognitive assessment tools in an active duty military population. Archives of Clinical Neuropsychology. doi:10.1093/arclin/act040.

    PubMed  Google Scholar 

  • Collie, A., Maruff, P., Makdissi, M., McCrory, P., McStephen, M., & Darby, D. (2003a). CogSport: reliability and correlation with conventional cognitive tests used in postconcussion medical evaluations. Clinical Journal of Sport Medicine, 13(1), 28–32.

    PubMed  Article  Google Scholar 

  • Collie, A., Maruff, P., McStephen, M., & Darby, D. (2003b). Are Reliable Change (RC) calculations appropriate for determining the extent of cognitive change in concussed athletes? British Journal of Sports Medicine, 37(4), 370–372.

    PubMed  Article  CAS  PubMed Central  Google Scholar 

  • Collie, A., Maruff, P., Makdissi, M., McStephen, M., Darby, D. G., & McCrory, P. (2004). Statistical procedures for determining the extent of cognitive change following concussion. British Journal of Sports Medicine, 38(3), 273–278.

    PubMed  Article  CAS  PubMed Central  Google Scholar 

  • Comstock, R. D., Collins, C. L., Corlette, J. D., & Fletcher, E. N. (2012). National high school sports- related injury surveillance study: 2011–2012 school year. Columbus: The Research Institute at Nationwide Children’s Hosptial.

    Google Scholar 

  • Covassin, T., Elbin, R. J., 3rd, Stiller-Ostrowski, J. L., & Kontos, A. P. (2009). Immediate post-concussion assessment and cognitive testing (ImPACT) practices of sports medicine professionals. Journal of Athletic Training, 44(6), 639–644.

    PubMed  Article  PubMed Central  Google Scholar 

  • Covassin, T., Elbin, R., Kontos, A., & Larson, E. (2010). Investigating baseline neurocognitive performance between male and female athletes with a history of multiple concussion. Journal of Neurology, Neurosurgery, and Psychiatry, 81(6), 597–601.

    PubMed  Article  Google Scholar 

  • Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory. Mason: Cengage Learning.

    Google Scholar 

  • Dancey, C., & Reidy, J. (2004). Statistics without maths for psychology: using SPSS for windows. London: Prentice Hall.

    Google Scholar 

  • Echemendia, R. J., Iverson, G. L., McCrea, M., Broshek, D. K., Gioia, G. A., Sautter, S., et al. (2012). Role of neuropsychologists in the evaluation and management of sport-related concussion: an inter-organization position statement. The Clinical Neuropsychologist, 25(8), 1289–1294.

    Article  Google Scholar 

  • Elbin, R., Schatz, P., & Covassin, T. (2011). One-year test-retest reliability of the online version of ImPACT in high school athletes. The American Journal of Sports Medicine, 39(11), 2319–2324.

    PubMed  Article  CAS  Google Scholar 

  • Erdal, K. (2012). Neuropsychological testing for sports-related concussion: how athletes can sandbag their baseline testing without detection. Archives of Clinical Neuropsychology, 27(5), 473–479.

    PubMed  Article  Google Scholar 

  • Erlanger, D., Saliba, E., Barth, J., Almquist, J., Webright, W., & Freeman, J. (2001). Monitoring resolution of postconcussion symptoms in athletes: preliminary results of a web-based neuropsychological test protocol. Journal of Athletic Training, 36(3), 280–287.

    PubMed  PubMed Central  Google Scholar 

  • Falleti, M. G., Maruff, P., Collie, A., & Darby, D. G. (2006). Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals. Journal of Clinical and Experimental Neuropsychology, 28(7), 1095–1112.

    PubMed  Article  Google Scholar 

  • Ferrara, M. S., McCrea, M., Peterson, C. L., & Guskiewicz, K. M. (2001). A survey of practice patterns in concussion assessment and management. Journal of Athletic Training, 36(2), 145–149.

    PubMed  PubMed Central  Google Scholar 

  • Giza, C. C., Kutcher, J. S., Ashwal, S., Barth, J., Getchius, T. S., Gioia, G. A., et al. (2013). Summary of evidence-based guideline update: evaluation and management of concussion in sports: Report of the Guideline Development Subcommittee of the American Academy of Neurology. Neurology, 80(24), 2250–2257.

    PubMed  Article  Google Scholar 

  • Guskiewicz, K. M., Riemann, B. L., Perrin, D. H., & Nashner, L. M. (1997). Alternative approaches to the assessment of mild head injury in athletes. Medicine and Science in Sports and Exercise, 29(7 Suppl), S213–S221.

    PubMed  Article  CAS  Google Scholar 

  • Guskiewicz, K. M., McCrea, M., Marshall, S. W., Cantu, R. C., Randolph, C., Barr, W., et al. (2003). Cumulative effects associated with recurrent concussion in collegiate football players: the NCAA Concussion Study. Journal of the American Medical Association, 290(19), 2549–2555.

    PubMed  Article  CAS  Google Scholar 

  • Guskiewicz, K. M., Bruce, S. L., Cantu, R. C., Ferrara, M. S., Kelly, J. P., McCrea, M., et al. (2004). National athletic trainers’ association position statement: management of sport-related concussion. Journal of Athletic Training, 39(3), 280–297.

    PubMed  PubMed Central  Google Scholar 

  • Harmon, K. G., Drezner, J. A., Gammons, M., Guskiewicz, K. M., Halstead, M., Herring, S. A., et al. (2013). American Medical Society for Sports Medicine position statement: concussion in sport. British Journal of Sports Medicine, 47(1), 15–26.

    PubMed  Article  Google Scholar 

  • Erlanger, D. M. (2002). HeadMinder concussion resolution index (CRI): professional manual. New York, NY: HeadMinder.

  • Hinton-Bayre, A. D. (2012). Choice of reliable change model can alter decisions regarding neuropsychological impairment after sports-related concussion. Clinical Journal of Sport Medicine, 22(2), 105–108.

    PubMed  Article  Google Scholar 

  • Hunt, T. N., Ferrara, M. S., Miller, L. S., & Macciocchi, S. (2007). The effect of effort on baseline neuropsychological test scores in high school football athletes. Archives of Clinical Neuropsychology, 22(5), 615–621.

    PubMed  Article  Google Scholar 

  • Iverson, G. L., Lovell, M. R., & Collins, M. W. (2003). Interpreting change on ImPACT following sport concussion. Clinical Neuropsychologist, 17(4), 460–467.

    PubMed  Article  Google Scholar 

  • Lovell, M. (2007). ImPACT 2007 (6.0) software user’s manual. Pittsburgh: ImPACT Applications Inc.

    Google Scholar 

  • Lovell, M. R. (2013). Immediate post-concussion assessment testing (ImPACT) test: clinical interpretive manual online ImPACT 2007–2012. Pittsburgh: ImPACT Applications Inc.

  • Lovell, M., Echemendia, R. J., Barth, J. T., & Collins, M. W. (Eds.). (2004). Traumatic brain injury in sports: A neuropsychological and international perspective. Lisse: Swets & Zeitlinger B.V.

    Google Scholar 

  • Macciocchi, S. N., Barth, J. T., Alves, W., Rimel, R. W., & Jane, J. A. (1996). Neurological functioning and recovery after mild head injury in collegiate athletes. Neursurgery, 39, 510–514.

    CAS  Google Scholar 

  • Maerlender, A., Flashman, L., Kessler, A., Kumbhani, S., Greenwald, R., Tosteson, T., et al. (2010). Examination of the construct validity of ImPACT computerized test, traditional, and experimental neuropsychological measures. The Clinical Neuropsychologist, 24(8), 1309–1325.

    PubMed  Article  CAS  PubMed Central  Google Scholar 

  • Makdissi, M., Darby, D., Maruff, P., Ugoni, A., Brukner, P., & McCrory, P. R. (2010). Natural history of concussion in sport: markers of severity and implications for management. The American Journal of Sports Medicine, 38(3), 464–471.

    PubMed  Article  Google Scholar 

  • Maruff, P., Thomas, E., Cysique, L., Brew, B., Collie, A., Snyder, P., et al. (2009). Validity of the CogState brief battery: relationship to standardized tests and sensitivity to cognitive impairment in mild traumatic brain injury, schizophrenia, and AIDS dementia complex. Archives Clinical Neuropsychology, 24(2), 165–178.

    Article  Google Scholar 

  • McCrea, M. (2001). Standardized mental status testing on the sideline after sport-related concussion. Journal of Athletic Training, 36(3), 274–279.

    PubMed  PubMed Central  Google Scholar 

  • McCrea, M., Iverson, G. L., Echemendia, R. J., Makdissi, M. & Raftery, M. (2013). Day of injury assessment of sport-related concussion. British Journal of Sports Medicine, 47(5), 272–284.

    Google Scholar 

  • McCrory, P., Collie, A., Anderson, V., & Davis, G. (2004). Can we manage sport related concussion in children the same as in adults? British Journal of Sports Medicine, 38(5), 516–519.

    PubMed  Article  CAS  PubMed Central  Google Scholar 

  • McCrory, P., Johnston, K., Meeuwisse, W., Aubry, M., Cantu, R., Dvorak, J., et al. (2005). Summary and agreement statement of the 2nd International Conference on Concussion in Sport, Prague 2004. British Journal of Sports Medicine, 39(4), 196–204.

    PubMed  CAS  PubMed Central  Google Scholar 

  • McCrory, P., Meeuwisse, W., Johnston, K., Dvorak, J., Aubry, M., Molloy, M., et al. (2009). Consensus Statement on Concussion in Sport: the 3rd International Conference on Concussion in Sport held in Zurich, November 2008. British Journal of Sports Medicine, 43(Suppl 1), i76–i90.

    PubMed  Article  Google Scholar 

  • McCrory, P., Meeuwisse, W. H., Aubry, M., Cantu, B., Dvorak, J., Echemendia, R. J., et al. (2013). Consensus statement on concussion in sport: the 4(th) international conference on concussion in sport held in Zurich, November 2012. Journal of the American College of Surgeons, 216(5), e55–e71.

    PubMed  Article  Google Scholar 

  • Meehan, W. P., 3rd, d’Hemecourt, P., Collins, C. L., Taylor, A. M., & Comstock, R. D. (2012). Computerized neurocognitive testing for the management of sport-related concussions. Pediatrics, 129(1), 38–44.

    PubMed  Article  PubMed Central  Google Scholar 

  • Moser, R. S., Schatz, P., Neidzwski, K., & Ott, S. D. (2011). Group versus individual administration affects baseline neurocognitive test performance. The American Journal of Sports Medicine, 39(11), 2325–2330.

    PubMed  Article  Google Scholar 

  • Notebaert, A. J., & Guskiewicz, K. M. (2005). Current trends in athletic training practice for concussion assessment and management. Journal of Athletic Training, 40(4), 320–325.

    PubMed  PubMed Central  Google Scholar 

  • Nunnally, J. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Pedhazur, E. J., & Pedhazur Schmelkin, L. (1991). Measurement, design and analysis: An integrated approach. Hillsdale: Lawrence Erilbaum Associates, Inc., Publishers.

    Google Scholar 

  • Portney, L. G., & Watkins, M. P. (1993). Foundations of clinical research: Applications to practice. Norwalk: Appleton & Lange.

    Google Scholar 

  • Randolph, C. (2011). Baseline neuropsychological testing in managing sport-related concussion: does it modify risk? Current Sports Medicine Reports, 10(1), 21–26.

    PubMed  Article  Google Scholar 

  • Randolph, C., McCrea, M., & Barr, W. B. (2005). Is neuropsychological testing useful in the management of sport-related concussion? Journal of Athletic Training, 40(3), 139–152.

    PubMed  PubMed Central  Google Scholar 

  • Register-Mihalik, J. K., Guskiewicz, K. M., Mihalik, J. P., Schmidt, J. D., Kerr, Z. Y., & McCrea, M. A. (2012a). Reliable change, sensitivity, and specificity of a multidimensional concussion assessment battery: implications for caution in clinical practice. The Journal of Head Trauma Rehabilitation, 28(4), 274–283.

    Article  Google Scholar 

  • Register-Mihalik, J. K., Kontos, D. L., Guskiewicz, K. M., Mihalik, J. P., Conder, R., & Shields, E. W. (2012b). Age-related differences and reliability on computerized and paper-and-pencil neurocognitive assessment batteries. Journal of Athletic Training, 47(3), 297–305.

    PubMed  PubMed Central  Google Scholar 

  • Resch, J., Driscoll, A., McCaffrey, N., Brown, C., Ferrara, M. S., Macciocchi, S., et al. (2013). ImPact test-retest reliability: reliably unreliable? Journal of Athletic Training, 48(4), 506–511.

    PubMed  Article  Google Scholar 

  • Roebuck-Spencer, T., Sun, W., Cernich, A. N., Farmer, K., & Bleiberg, J. (2007). Assessing change with the Automated Neuropsychological Assessment Metrics (ANAM): issues and challenges. Archives of Clinical Neuropsychology, 22(Suppl 1), S79–S87.

    PubMed  Article  Google Scholar 

  • Schatz, P. (2009). long-term test-retest reliability of baseline cognitive assessments using ImPACT. American Journal of Sports Medicine, 38(1), 47–53.

    PubMed  Google Scholar 

  • Schatz, P., & Ferris, C. S. (2013). One-month test-retest reliability of the ImPACT test battery. Archives of Clinical Neuropsychology, 28(5), 499–504.

    PubMed  Article  Google Scholar 

  • Schatz, P., & Glatts, C. (2013). “Sandbagging” baseline test performance on ImPACT, without detection, is more difficult than it appears. Archives of Clinical Neuropsychology, 28(3), 236–244.

    PubMed  Article  Google Scholar 

  • Schatz, P., & Putz, B. O. (2006). Cross-validation of measures used for computer-based assessment of concussion. Applied Neuropsychology, 13(3), 151–159.

    PubMed  Article  Google Scholar 

  • Schatz, P., & Sandel, N. (2013). Sensitivity and specificity of the online version of ImPACT in high school and collegiate athletes. The American Journal of Sports Medicine, 41(2), 321–326.

    PubMed  Article  Google Scholar 

  • Schatz, P., Pardini, J. E., Lovell, M. R., Collins, M. W., & Podell, K. (2006). Sensitivity and specificity of the ImPACT Test Battery for concussion in athletes. Archives of Clinical Neuropsychology, 21(1), 91–99.

    PubMed  Article  Google Scholar 

  • Schatz, P., Neidzwski, K., Moser, R. S., & Karpf, R. (2010). Relationship between subjective test feedback provided by high-school athletes during computer-based assessment of baseline cognitive functioning and self-reported symptoms. Archives of Clinical Neuropsychology, 25(4), 285–292.

    PubMed  Article  Google Scholar 

  • Schatz, P., Moser, R. S., Solomon, G. S., Ott, S. D., & Karpf, R. (2012). Prevalence of invalid computerized baseline neurocognitive test results in high school and collegiate athletes. Journal of Athletic Training, 47(3), 289–296.

    PubMed  PubMed Central  Google Scholar 

  • Schmidt, J. D., Register-Mihalik, J. K., Mihalik, J. P., Kerr, Z. Y., & Guskiewicz, K. M. (2012). Identifying Impairments after concussion: normative data versus individualized baselines. Medicine and Science in Sports and Exercise, 44(9), 1621–1628.

    PubMed  Article  Google Scholar 

  • Segalowitz, S. J., Mahaney, P., Santesso, D. L., MacGregor, L., Dywan, J., & Willer, B. (2007). Retest reliability in adolescents of a computerized neuropsychological battery used to assess recovery from concussion. NeuroRehabilitation, 22(3), 243–251.

    PubMed  Google Scholar 

  • Straume-Naesheim, T. M., Andersen, T. E., & Bahr, R. (2005). Reproducibility of computer based neuropsychological testing among Norwegian elite football players. British Journal of Sports Medicine, 39(Suppl 1), i64–i69.

    PubMed  Article  PubMed Central  Google Scholar 

  • Van Kampen, D. A., Lovell, M. R., Pardini, J. E., Collins, M. W., & Fu, F. H. (2006). The “value added” of neurcognitive testing after sports-related concussion. American Journal of Sports Medicine, 34(10), 1630–1635.

    Google Scholar 

  • Woodard, J. L., & Rahman, A. (2012). The human-computer interface in computer-based concussion assessment. Journal of Clnicial Sport Psychology, 6, 385–408.

    Google Scholar 

Download references

Funding sources or financial disclosures

The authors of the current manuscript have no funding sources or financial disclosures to report.

Author information

Authors and Affiliations

  1. Brain Injury Laboratory, Department of Kinesiology, The University of Texas at Arlington, Box 19259, 113 Maverick Activities Center, Arlington, TX, 76019-0259, USA

    Jacob E. Resch

  2. Departments of Neurosurgery and Neurology, Medical College of Wisconsin, Neuroscience Research Center, 8701 W. Watertown Plank Road, Milwaukee, WI, 53266, USA

    Michael A. McCrea

  3. Department of Psychiatry, The University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd., Dallas, TX, 75390-9044, USA

    C. Munro Cullum

Authors
  1. Jacob E. Resch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael A. McCrea
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. Munro Cullum
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jacob E. Resch.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Resch, J.E., McCrea, M.A. & Cullum, C.M. Computerized Neurocognitive Testing in the Management of Sport-Related Concussion: An Update. Neuropsychol Rev 23, 335–349 (2013). https://doi.org/10.1007/s11065-013-9242-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11065-013-9242-5

Keywords

  • Neurocognitive
  • Reliability
  • Concussion
  • Validity
  • Sensitivity
  • Specificity