Quantifying the Value of Multidimensional Assessment Models for Acute Concussion: An Analysis of Data from the NCAA-DoD Care Consortium
Many concussion assessment methods exist, but few studies quantify the performance of these methods to determine which can best assess acute concussion alone or in combination.
The objectives of this study were to evaluate: (1) selected concussion assessments for acute concussion assessment; (2) the utility of change scores for acute concussion assessment; and (3) concussion assessment capabilities when constrained to limited clinical data or objective clinical measures.
The ‘acute concussion’ group contained assessments from < 6 h post-injury (n = 560) and 24–48 h post-injury (n = 733). The ‘normal performance’ group contained assessments from baseline testing (n = 842) and unrestricted return to play (n = 707) timepoints. Univariate and multivariate logistic regression models were created separately for < 6- and 24- to 48-h timepoints. Models were evaluated on sensitivity, specificity, and area under the receiver operating characteristic curve.
Within the univariate analysis, Sport Concussion Assessment Tool symptom assessments had the highest combination of sensitivity, specificity, and area under the receiver operating characteristic curve, with values up to 0.93, 0.97, and 0.98, respectively. Full models had a sensitivity, specificity, and area under the receiver operating characteristic curve up to 0.94, 0.97, and 0.99, respectively, and outperformed all univariate models, raw score models, and objective models. Objective models were outperformed by all multivariate models and the univariate models containing only Sport Concussion Assessment Tool symptom assessments.
Results support the use of multidimensional assessment batteries over single instruments and suggest the importance of self-reported symptoms in acute concussion assessment. Balance assessments, however, may not provide additional benefit when symptom and neurocognitive assessments are available. Additionally, change scores provide some clinical utility over raw scores, but the difference may not be clinically meaningful.
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE 1256260. This publication was made possible, in part, with support from the Grand Alliance Concussion Assessment, Research, and Education (CARE) Consortium, funded, in part, by the National Collegiate Athletic Association and the Department of Defense. The US Army Medical Research Acquisition Activity, Fort Detrick, MD, USA is the awarding and administering acquisition office. This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Psychological Health and Traumatic Brain Injury Program under Award No. W81XWH-14-2-0151. Opinions, interpretations, conclusions, and recommendations are those of the author(s) and are not necessarily endorsed by the Department of Defense (Defense Health Program funds).
The CARE Consortium Investigators are listed alphabetically by institution: April Marie (Reed) Hoy, MS, ATC (Azusa Pacific University); Louise A. Kelly, PhD (California Lutheran University); Justus D. Ortega, PhD (Humboldt State University); Nicholas Port, PhD (Indiana University); Margot Putukian MD (Princeton University); T. Dianne Langford, PhD (Temple University); Scott Anderson, ATC and Gerald McGinty, DPT (US Air Force Academy); Patrick O’Donnell, MHA (US Coast Guard Academy); Steven J. Svoboda, MD (US Military Academy); John P. DiFiori (University of California-Los Angeles); Holly J. Benjamin MD (University of Chicago); Thomas Buckley, EdD, ATC, and Thomas W. Kaminski, PhD, ATC (University of Delaware); James R. Clugston, MD, MS (University of Florida); Julianne D. Schmidt, PhD, ATC (University of Georgia); Jason P. Mihalik, PhD, CAT(C), ATC (University of North Carolina at Chapel Hill); Christina L. Master, MD (University of Pennsylvania); Micky Collins, PhD, and Anthony P. Kontos, PhD (University of Pittsburgh Medical Center); Sara P.D. Chrisman, MD, MPH (University of Washington); Christopher M. Miles, MD (Wake Forest University); Brian H. Dykhuizen, MS, ATC (Wilmington College); Alison Brooks MD, MPH (University of Wisconsin).
Compliance with Ethical Standards
Conflict of interest
Gian-Gabriel Garcia, Steven Broglio, Mariel Lavieri, Thomas McAllister, and Michael McCrea have no conflicts of interest directly relevant to the content of this study.
This study was approved by each local institutional review board and the Army Human Research Protection Office.
- 3.Giza CC, Kutcher JS, Ashwal S, Barth J, Getchius TSD, Gioia GA, et al. Summary of evidence-based guideline update: evaluation and management of concussion in sports: report of the Guideline Development Subcommittee of the American Academy of Neurology. Neurology. 2013;80:2250–7.CrossRefPubMedPubMedCentralGoogle Scholar
- 18.Echemendia RJ, Meeuwisse W, McCrory P, Davis GA, Putukian M, Leddy J, et al. The sport concussion assessment tool 5th Edition (SCAT5). Br J Sports Med. 2017;5:1–3.Google Scholar
- 23.Putukian M, Echemendia R, Dettwiler-Danspeckgruber A, Duliba T, Bruce J, Furtado JL, et al. Prospective clinical assessment using sideline concussion assessment tool-2 testing in the evaluation of sport-related concussion in college athletes. Clin J Sport Med. 2015;25:36–42.CrossRefPubMedGoogle Scholar
- 31.Broglio SP, McCrea M, McAllister T, Harezlak J, Katz B, Hack D, et al. A national study on the effects of concussion in collegiate athletes and US Military Service Academy members: the NCAA–DoD concussion assessment, research and education (CARE) consortium structure and methods. Sport Med. 2017;47:1437–51.CrossRefGoogle Scholar
- 33.McCrory P, Meeuwisse WH, Aubry M, Cantu RC, Dvorák J, Echemendia RJ, et al. Consensus statement on concussion in sport. In: the 4th international conference on concussion in sport held in Zurich, November 2012. Br J Sports Med. 2013;47:255–79.Google Scholar
- 37.Concussion in Sport Group. Sport concussion assessment tool: 3rd edition. Br J Sports Med. 2013;47(5):259.Google Scholar
- 56.Moreau MS, Langdon J, Buckley TA. The lived experience of an in-season concussion amongst NCAA Division I student-athletes. Int J Exerc Sci. 2014;7:62–74.Google Scholar