Skip to main content

Optimizing Order of Administration for Concussion Baseline Assessment Among NCAA Student-Athletes and Military Cadets

Abstract

Background

Concussion pre-injury (i.e., baseline) assessments serve as a benchmark comparison point in the event an individual sustains a concussion and allows clinicians to compare to post-injury measures. However, baseline assessments must reflect the individual’s true and most optimized performance to serve as a useful comparison. Mental fatigue and motivation throughout baseline testing may alter individual assessment performance, indicating an order of administration (OoA) may play an influential role in assessment outcomes.

Objective

To examine the influence concussion baseline battery OoA has on symptom, postural stability, cognitive screening, and computerized neurocognitive test outcomes.

Methods

We employed a retrospective observational cohort study to examine healthy collegiate student-athletes and military cadets (n = 2898, 19.0 ± 1.4 years, 66.1% male, 75.6% white, 54.4% Division-I) baseline assessment performance on the Sport Concussion Assessment Tool (SCAT; total symptom number and severity), Balance Error Scoring System (BESS; total error scores), Standardized Assessment of Concussion (SAC; total score), and Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) domain scores (verbal and visual memory, visual-motor speed, reaction time). Assessments were binned to beginning, middle, or end tertiles based upon OoA. We used one-way ANOVAs with Tukey post-hoc t tests, 95% confidence intervals (CI), and Cohen’s d effect sizes for significant models (α = 0.05).

Results

SCAT total symptom number (mean difference = 2.23; 95% CI 1.76–2.70; d = 0.49, p < 0.001) and severity (mean difference = 5.58; 95% CI 4.42–6.74; d = 0.50; p < 0.001) were lower when completed at the end of baseline testing compared to the middle. Total BESS errors were 1.06 lower when completed at the middle relative to the end (95% CI 0.43–1.69; d = 0.17; p = 0.001). Total SAC scores were better at the beginning relative to middle (mean difference = 0.58; 95% CI 0.25–0.90; d = 0.33; p < 0.001) and end (mean difference = 0.44; 95% CI 0.16–0.73; d = 0.24; p = 0.001). Verbal memory, visual memory, and reaction time performance were highest at the beginning (p ≤ 0.002), while visual-motor speed performance was highest at the middle (p = 0.001).

Conclusion

Completing baseline assessments in the order of (1) ImPACT, (2) SAC, (3) BESS, and (4) SCAT symptom checklist may improve performance across assessments collectively. Clinicians and researchers should consider completing baseline assessments in this order when possible to potentially aid in optimizing concussion baseline assessment performance and maximize post-concussion comparisons.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

References

  1. 1.

    Guskiewicz KM, Weaver NL, Padua DA, Garrett WE. Epidemiology of concussion in collegiate and high school football players. Am J Sports Med. 2000;28:643–50.

    CAS  PubMed  Article  Google Scholar 

  2. 2.

    Halstead ME, Walter KD. The Council on sports medicine and fitness sport-related concussion in children and adolescents. Pediatrics. 2010;126:597–615.

    PubMed  Article  Google Scholar 

  3. 3.

    McCrory P, Meeuwisse W, Dvorak J, Aubry M, Bailes J, Broglio S, et al. Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. 2017;bjsports-2017-097699.

  4. 4.

    Broglio SP, Puetz TW. The effect of sport concussion on neurocognitive function, self-report symptoms and postural control. Sports Med. 2008;38:53–67.

    PubMed  Article  Google Scholar 

  5. 5.

    Broglio SP, Macciocchi SN, Ferrara MS. Sensitivity of the concussion assessment battery. Neurosurgery. 2007;60:1050–7 (discussion 1057–1058).

    PubMed  Article  Google Scholar 

  6. 6.

    Resch JE, Brown CN, Schmidt J, Macciocchi SN, Blueitt D, Cullum CM, et al. The sensitivity and specificity of clinical measures of sport concussion: three tests are better than one. BMJ Open Sport Exerc Med. 2016;2:e000012.

    PubMed  PubMed Central  Article  Google Scholar 

  7. 7.

    Broglio SP, Harezlak J, Katz B, Zhao S, McAllister T, McCrea M, et al. Acute sport concussion assessment optimization: a prospective assessment from the CARE consortium. Sports Med [Internet]. 2019 [cited 2019 Jul 28]. https://doi.org/10.1007/s40279-019-01155-0.

  8. 8.

    Lempke LB, Schmidt JD, Lynall RC. Athletic trainers’ concussion-assessment and -management practices: an update. J Athl Train. 2020;51:17–26.

    Article  Google Scholar 

  9. 9.

    Broglio SP, McCrea M, McAllister T, Harezlak J, Katz B, Hack D, et al. A National Study on the Effects of Concussion in Collegiate Athletes and US Military Service Academy Members: the NCAA–DoD Concussion Assessment, Research and Education (CARE) Consortium Structure and Methods. Sports Med [Internet]. 2017 [cited 2017 Mar 24]. https://doi.org/10.1007/s40279-017-0707-1.

  10. 10.

    Louey AG, Cromer JA, Schembri AJ, Darby DG, Maruff P, Makdissi M, et al. Detecting cognitive impairment after concussion: sensitivity of change from baseline and normative data methods using the CogSport/Axon cognitive test battery. Arch Clin Neuropsychol. 2014;29:432–41.

    PubMed  Article  Google Scholar 

  11. 11.

    Covassin T, Elbin R, Kontos A, Larson E. Investigating baseline neurocognitive performance between male and female athletes with a history of multiple concussion. J Neurol Neurosurg Psychiatry. 2010;81:597–601.

    PubMed  Article  Google Scholar 

  12. 12.

    Moser RS, Schatz P, Neidzwski K, Ott SD. Group versus individual administration affects baseline neurocognitive test performance. Am J Sports Med. 2011;39:2325–30.

    PubMed  Article  Google Scholar 

  13. 13.

    Anderson MN, Lempke LB, Bell DH, Lynall RC, Schmidt JD. The ability of CNS vital signs to detect coached sandbagging performance during concussion baseline testing: a randomized control trial. Brain Inj. 2020;34:369–74.

    CAS  PubMed  Article  Google Scholar 

  14. 14.

    Walton SR, Broshek DK, Freeman JR, Hertel J, Meyer JP, Erdman NK, et al. Institutionally based ImPACT Test® normative values may differ from manufacturer-provided normative values. Arch Clin Neuropsychol. 2020;35:275–82.

    PubMed  Article  Google Scholar 

  15. 15.

    Kuhn AW, Solomon GS. Supervision and computerized neurocognitive baseline test performance in high school athletes: an initial investigation. J Athl Train. 2014;49:800–5.

    PubMed  PubMed Central  Article  Google Scholar 

  16. 16.

    Sufrinko A, Pearce K, Elbin RJ, Covassin T, Johnson E, Collins M, et al. The effect of preinjury sleep difficulties on neurocognitive impairment and symptoms after sport-related concussion. Am J Sports Med. 2015;43:830–8.

    PubMed  Article  Google Scholar 

  17. 17.

    Bailey CM, Echemendia RJ, Arnett PA. The impact of motivation on neuropsychological performance in sports-related mild traumatic brain injury. J Int Neuropsychol Soc [Internet]. 2006 [cited 2019 Feb 4];12. http://www.journals.cambridge.org/abstract_S1355617706060619.

  18. 18.

    Covassin T, Weiss L, Powell J, Womack C. Effects of a maximal exercise test on neurocognitive function. Yearb Sports Med. 2008;2008:10–1.

    Article  Google Scholar 

  19. 19.

    Boksem MAS, Tops M. Mental fatigue: costs and benefits. Brain Res Rev. 2008;59:125–39.

    PubMed  Article  PubMed Central  Google Scholar 

  20. 20.

    Register-Mihalik JK, Kontos DL, Guskiewicz KM, Mihalik JP, Conder R, Shields EW. Age-related differences and reliability on computerized and paper-and-pencil neurocognitive assessment batteries. J Athl Train. 2012;47:297–305.

    PubMed  PubMed Central  Article  Google Scholar 

  21. 21.

    Lloyd KP, Higginson CI, Lating JM, Coiro MJ. Test-order effects: a measure of verbal memory influences performance on a measure of verbal fluency. Appl Neuropsychol Adult. 2012;19:299–304.

    PubMed  Article  Google Scholar 

  22. 22.

    Brandon AD, Chavez EL. Order and delay effects on neuropsychological test presentation: the Halstead Category and Wisconsin Card Sorting Tests. Int J Clin Neuropsychol. 1985;7:152–3.

    Google Scholar 

  23. 23.

    Franzen MD, Smith SS, Paul DS, Maclnnes WD. Order effects in the administration of the booklet category test and Wisconsin Card Sorting Test. Arch Clin Neuropsychol. 1993;8:105–10.

    CAS  PubMed  Article  Google Scholar 

  24. 24.

    Ackerman PL, Kanfer R. Test length and cognitive fatigue: an empirical examination of effects on performance and test-taker reactions. J Exp Psychol Appl. 2009;15:163–81.

    PubMed  Article  Google Scholar 

  25. 25.

    Boksem MAS, Meijman TF, Lorist MM. Effects of mental fatigue on attention: an ERP study. Cogn Brain Res. 2005;25:107–16.

    Article  Google Scholar 

  26. 26.

    von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344–9.

    Article  Google Scholar 

  27. 27.

    Broglio SP, Katz BP, Zhao S, McCrea M, McAllister T, CARE Consortium Investigators. Test-retest reliability and interpretation of common concussion assessment tools: findings from the NCAA-DoD CARE consortium. Sports Med Auckl NZ. 2017;48:1255–68.

    Article  Google Scholar 

  28. 28.

    Medicine BPGL and BA of S and E. Sport Concussion Assessment Tool—3rd Edition. Br J Sports Med. 2013;47:259.

  29. 29.

    Guskiewicz KM, Register-Mihalik J, McCrory P, McCrea M, Johnston K, Makdissi M, et al. Evidence-based approach to revising the SCAT2: introducing the SCAT3. Br J Sports Med. 2013;47:289–93.

    PubMed  Article  Google Scholar 

  30. 30.

    Guskiewicz KM, Ross SE, Marshall SW. Postural stability and neuropsychological deficits after concussion in collegiate athletes. J Athl Train. 2001;36:263–73.

    PubMed  PubMed Central  Google Scholar 

  31. 31.

    McCrea M, Kelly JP, Kluge J, Ackley B, Randolph C. Standardized Assessment of Concussion in football players. Neurology. 1997;48:586–8.

    CAS  PubMed  Article  Google Scholar 

  32. 32.

    McCrea M, Kelly JP, Randolph C, Kluge J, Bartolic E, Finn G, et al. Standardized Assessment of Concussion (SAC): On-Site Mental Status Evaluation of the Athlete: J Head Trauma Rehabil. 1998;13:27–35.

  33. 33.

    Resch JE, Schneider MW, Munro CC. The test-retest reliability of three computerized neurocognitive tests used in the assessment of sport concussion. Int J Psychophysiol. 2018;132:31–8.

    PubMed  Article  Google Scholar 

  34. 34.

    Schatz P, Pardini J, Lovell M, Collins M, Podell K. Sensitivity and specificity of the ImPACT Test Battery for concussion in athletes. Arch Clin Neuropsychol. 2006;21:91–9.

    PubMed  Article  Google Scholar 

  35. 35.

    Mucha A, Collins MW, Elbin RJ, Furman JM, Troutman-Enseki C, DeWolf RM, et al. A brief vestibular/ocular motor screening (VOMS) assessment to evaluate concussions: preliminary findings. Am J Sports Med. 2014;42:2479–86.

    PubMed  PubMed Central  Article  Google Scholar 

  36. 36.

    Sufrinko AM, Mucha A, Covassin T, Marchetti G, Elbin RJ, Collins MW, et al. Sex differences in vestibular/ocular and neurocognitive outcomes after sport-related concussion. Clin J Sport Med. 2017;27:133–8.

    PubMed  PubMed Central  Article  Google Scholar 

  37. 37.

    Caccese JB, Eckner JT, Franco-MacKendrick L, Hazzard JB, Ni M, Broglio SP, et al. Clinical reaction-time performance factors in healthy collegiate athletes. J Athl Train. 2020;55:601–7.

    PubMed  PubMed Central  Article  Google Scholar 

  38. 38.

    Lempke LB, Howell DR, Eckner JT, Lynall RC. Examination of reaction time deficits following concussion: a systematic review and meta-analysis. Sports Med. 2020;50:1341–59.

    PubMed  Article  Google Scholar 

  39. 39.

    Galetta KM, Brandes LE, Maki K, Dziemianowicz MS, Laudano E, Allen M, et al. The King-Devick test and sports-related concussion: study of a rapid visual screening tool in a collegiate cohort. J Neurol Sci. 2011;309:34–9.

    PubMed  Article  Google Scholar 

  40. 40.

    Tjarks BJ, Dorman JC, Valentine VD, Munce TA, Thompson PA, Kindt SL, et al. Comparison and utility of King-Devick and ImPACT(R) composite scores in adolescent concussion patients. J Neurol Sci. 2013;334:148–53.

    PubMed  Article  Google Scholar 

  41. 41.

    Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Erlbaum Associates; 1988.

    Google Scholar 

  42. 42.

    RStudio Team. RStudio: integrated development environment for R [Internet]. Boston, MA: RStudio, Inc.; 2016. Available from: http://www.rstudio.com/.

  43. 43.

    Bell DR, Guskiewicz KM, Clark MA, Padua DA. Systematic review of the balance error scoring system. Sports Health. 2011;3:287–95.

    PubMed  PubMed Central  Article  Google Scholar 

  44. 44.

    Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991;59:12–9.

    CAS  PubMed  Article  Google Scholar 

  45. 45.

    Van der Vaart AW. Asymptotic statistics. Cambridge: Cambridge University Press; 2000.

    Google Scholar 

  46. 46.

    Balasundaram AP, Athens J, Schneiders AG, McCrory P, Sullivan SJ. Day-to-day variability of post-concussion-like symptoms reported over time by a non-concussed cohort. Brain Inj. 2016;30:1599–604.

    PubMed  Article  Google Scholar 

  47. 47.

    Marcora SM, Staiano W, Manning V. Mental fatigue impairs physical performance in humans. J Appl Physiol. 2009;106:857–64.

    PubMed  Article  Google Scholar 

  48. 48.

    Ozinga SJ, Linder SM, Koop MM, Dey T, Figler R, Russman AN, et al. Normative performance on the balance error scoring system by youth, high school, and collegiate athletes. J Athl Train. 2018;53:636–45.

    PubMed  PubMed Central  Article  Google Scholar 

  49. 49.

    Wilkins JC, Valovich McLeod TC, Perrin DH, Gansneder BM. Performance on the balance error scoring system decreases after fatigue. J Athl Train. 2004;39:156–61.

    PubMed  PubMed Central  Google Scholar 

  50. 50.

    Chin EY, Nelson LD, Barr WB, McCrory P, McCrea MA. Reliability and validity of the sport concussion assessment tool-3 (SCAT3) in high school and collegiate athletes. Am J Sports Med. 2016;44:2276–85.

    PubMed  Article  Google Scholar 

  51. 51.

    Echemendia RJ, Meeuwisse W, McCrory P, Davis GA, Putukian M, Leddy J, et al. The Sport Concussion Assessment Tool 5th edition (SCAT5): background and rationale. Br J Sports Med. 2017;51:848–50.

    PubMed  PubMed Central  Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Jody Harland, Janetta Matesan, Larry Riggen (Indiana University); Ashley Rettmann (University of Michigan); Melissa Koschnitzke (Medical College of Wisconsin); Michael Jarrett, Vibeke Brinck and Bianca Byrne (Quesgen); and the research and medical staff at each of the participating sites.

Contributing CARE Consortium Investigators include: Thomas Buckley, EdD, ATC and Thomas W. Kaminski, PhD, ATC (University of Delaware); James T. Eckner MD, MS (University of Michigan); Micky Collins and Anthony P. Kontos, PhD (University of Pittsburgh); Stefan Duma, PhD (Virginia Tech); Louise A. Kelly, PhD (California Lutheran University); Alison Brooks, MD, MPH (University of Wisconsin-Madison); April Hoy, MS, ATC (Azusa Pacific University); Luis A. Feigenbaum, PT, DPT, ATC (University of Miami); Christina L. Master, MD, FAAP, CAQSM, FACSM (Children’s Hospital of Philadelphia); Laura Lintner, DO (Wake Forest University); Jason P. Mihalik, PhD, CAT(C), ATC (University of North Carolina at Chapel Hill); Justus Ortega, PhD (Humboldt State University); Nicholas Port, PhD (Indiana University); and Adam J. Susmarski, DO (United States Naval Academy). CARE Consortium Investigators are listed alphabetically by institution: April M. (Reed) Hoy, MS, ATC (Azusa Pacific University); Joseph B. Hazzard Jr., EdD, ATC (Bloomsburg University); Louise A. Kelly, PhD (California Lutheran University); Justus D. Ortega, PhD (Humboldt State University); Nicholas Port, PhD (Indiana University); Margot Putukian MD (Princeton University); T. Dianne Langford, PhD, and Ryan Tierney, PhD, ATC (Temple University); Christopher C. Giza, MD and Joshua T. Goldman, MD, MBA (University of California, Los Angeles); Holly J. Benjamin MD (University of Chicago); Thomas Buckley, EdD, ATC, and Thomas W. Kaminski, PhD, ATC (University of Delaware); James R. Clugston, MD, MS (University of Florida); Julianne D. Schmidt, PhD, ATC (University of Georgia); Luis A. Feigenbaum, DPT, ATC (University of Miami); Steven P. Broglio, PhD, ATC and James T. Eckner, MD, MS (University of Michigan); Kevin Guskiewicz, PhD, ATC, and Jason P. Mihalik, PhD, CAT(C), ATC (University of North Carolina at Chapel Hill); Jessica Dysart Miles, PhD, ATC (University of North Georgia); Scott Anderson, ATC (University of Oklahoma); Christina L. Master, MD (University of Pennsylvania); Micky Collins, PhD, and Anthony P. Kontos, PhD (University of Pittsburgh Medical Center); Jeffrey J. Bazarian, MD, MPH (University of Rochester); Sara P. O. Chrisman, MD, MPH (University of Washington); Alison Brooks, MD, MPH (University of Wisconsin-Madison); Jonathan Jackson, MD, and Gerald McGinty, DPT (United States Air Force Academy); Patrick O’Donnell, MHA (United States Coast Guard Academy); Kenneth Cameron, PhD, MPH, ATC; Megan N. Houston, PhD, ATC (United States Military Academy); Adam Susmarski, MD (United States Naval Academy); Stefan Duma, PhD and Steve Rowson, PhD (Virginia Tech); Christopher Todd Bullers, MD, and Christopher M. Miles, MD (Wake Forest University); Brian H. Dykhuizen, MS, ATC (Wilmington College); Laura Lintner DO (Winston-Salem University).

Author information

Affiliations

Authors

Consortia

Corresponding author

Correspondence to Landon B. Lempke.

Ethics declarations

Author Contributions

All authors contributed to the study conception, study design, critical review and interpretation of findings, and drafting and/or critique of the manuscript. All authors read and approved the final manuscript. Specifically, Landon B. Lempke contributed to conceptualization, methodology, data analysis, and original and revised manuscript drafts. Robert C. Lynall and Melissa N. Anderson contributed to conceptualization, analysis interpretation, and original and revised manuscript drafting. Michael A. McCrea, Thomas W. McAllister, and Steven P. Broglio contributed to conceptualization, methodology, data analysis and manuscript critique, and funding acquisition. Julianne D. Schmidt contributed to conceptualization, analysis interpretation, original and revised manuscript drafting, and funding acquisition. The CARE Consortium Investigators contributed to critical critique and review of methodology, data analysis, and original manuscript review.

Funding

This study was made possible, in part, with support from the Grand Alliance Concussion Assessment, Research, and Education Consortium, funded by the National Collegiate Athletic Association and the Department of Defense. The US Army Medical Research Acquisition Activity, 820 Chandler Street, Fort Detrick, MD 21702-5014, USA is the awarding and administering acquisition office. This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Psychological Health and Traumatic Brain Injury Program under Award no. W81XWH-14-2-0151. Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the Department of Defense (Defense Health Program funds).

Conflict of interest

Landon B. Lempke, Robert C. Lynall, Melissa N. Anderson, Michael A. McCrea, Thomas W. McAllister and Julianne D. Schmidt declare that they have no additional conflicts of interest relevant to the content of this manuscript.

Availability of data and material

The CARE Consortium datasets generated and analyzed during the current study are available in the FITBIR repository (https://ftbir.nih.gov/).

Research involving human participants

The study was performed in accordance with the standards of ethics outlined in the Declaration of Helsinki.

Informed consent

All study procedures were reviewed and approved by the University of Michigan IRB, the US Army Medical Research and Materiel Command Human Research Protection Office (HRPO), as well the local IRB at each of the performance sites. Participants provided written informed consent prior to participation.

Code availability

Not applicable.

Additional information

This article is part of a Topical Collection on The NCAA-DoD Concussion Assessment, Research and Education (CARE) Consortium.

Members of the ‘CARE Consortium Investigators’ are given in the Acknowledgements section.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lempke, L.B., Lynall, R.C., Anderson, M.N. et al. Optimizing Order of Administration for Concussion Baseline Assessment Among NCAA Student-Athletes and Military Cadets. Sports Med (2021). https://doi.org/10.1007/s40279-021-01493-y

Download citation