Many concussion assessment methods exist, but few studies quantify the performance of these methods to determine which can best assess acute concussion alone or in combination.
The objectives of this study were to evaluate: (1) selected concussion assessments for acute concussion assessment; (2) the utility of change scores for acute concussion assessment; and (3) concussion assessment capabilities when constrained to limited clinical data or objective clinical measures.
The ‘acute concussion’ group contained assessments from < 6 h post-injury (n = 560) and 24–48 h post-injury (n = 733). The ‘normal performance’ group contained assessments from baseline testing (n = 842) and unrestricted return to play (n = 707) timepoints. Univariate and multivariate logistic regression models were created separately for < 6- and 24- to 48-h timepoints. Models were evaluated on sensitivity, specificity, and area under the receiver operating characteristic curve.
Within the univariate analysis, Sport Concussion Assessment Tool symptom assessments had the highest combination of sensitivity, specificity, and area under the receiver operating characteristic curve, with values up to 0.93, 0.97, and 0.98, respectively. Full models had a sensitivity, specificity, and area under the receiver operating characteristic curve up to 0.94, 0.97, and 0.99, respectively, and outperformed all univariate models, raw score models, and objective models. Objective models were outperformed by all multivariate models and the univariate models containing only Sport Concussion Assessment Tool symptom assessments.
Results support the use of multidimensional assessment batteries over single instruments and suggest the importance of self-reported symptoms in acute concussion assessment. Balance assessments, however, may not provide additional benefit when symptom and neurocognitive assessments are available. Additionally, change scores provide some clinical utility over raw scores, but the difference may not be clinically meaningful.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
McCrory P, Meeuwisse W, Dvorak J, Aubry M, Bailes J, Broglio S, et al. Consensus statement on concussion in sport: the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. 2017;51:838–47.
Zuckerman SL, Kerr ZY, Yengo-Kahn A, Wasserman E, Covassin T, Solomon GS. Epidemiology of sports-related concussion in NCAA athletes from 2009-2010 to 2013-2014: incidence, recurrence, and mechanisms. Am J Sports Med. 2015;43:2654–62.
Giza CC, Kutcher JS, Ashwal S, Barth J, Getchius TSD, Gioia GA, et al. Summary of evidence-based guideline update: evaluation and management of concussion in sports: report of the Guideline Development Subcommittee of the American Academy of Neurology. Neurology. 2013;80:2250–7.
Broglio SP, Cantu RC, Gioia GA, Guskiewicz KM, Kutcher J, Palm M, et al. National Athletic Trainers’ Association position statement: management of sport concussion. J Athl Train. 2014;49:245–65.
Harmon KG, Drezner JA, Gammons M, Guskiewicz KM, Halstead M, Herring SA, et al. American Medical Society for Sports Medicine position statement: concussion in sport. Br J Sports Med. 2013;47:15–26.
McCrory P, Meeuwisse WH, Kutcher JS, Jordan BD, Gardner A. What is the evidence for chronic concussion-related changes in retired athletes: behavioural, pathological and clinical outcomes? Br J Sports Med. 2013;47:327–30.
Randolph C, Karantzoulis S, Guskiewicz K. Prevalence and characterization of mild cognitive impairment in retired national football league players. J Int Neuropsychol Soc. 2013;19:873–80.
Guskiewicz KM, Marshall SW, Bailes J, McCrea M, Cantu RC, Randolph C, et al. Association between recurrent concussion and late-life cognitive impairment in retired professional football players. Neurosurgery. 2005;57:719–26.
Daneshvar DH, Riley DO, Nowinski CJ, McKee AC, Stern RA, Cantu RC. Long-term consequences: effects on normal development profile after concussion. Phys Med Rehabil Clin N Am. 2011;22:683–700.
Kerr ZY, Marshall SW, Harding HP, Guskiewicz KM. Nine-year risk of depression diagnosis increases with increasing self-reported concussions in retired professional football players. Am J Sports Med. 2012;40:2206–12.
Guskiewicz KM, Marshall SW, Bailes J, Mccrea M, Harding HP, Matthews A, et al. Recurrent concussion and risk of depression in retired professional football players. Med Sci Sports Exerc. 2007;39:903–9.
De Beaumont L, Lassonde M, Leclerc S, Théoret H. Long-term and cumulative effects of sports concussion on motor cortex inhibition. Neurosurgery. 2007;61:329–36.
Barr WB, McCrea M. Sensitivity and specificity of standardized neurocognitive testing immediately following sports concussion. J Int Neuropsychol Soc. 2001;7:693–702.
Broglio SP, Macciocchi SN, Ferrara MS. Sensitivity of the concussion assessment battery. Neurosurgery. 2007;60:1050–7.
Broglio SP, Ferrara MS, Sopiarz K, Kelly MS. Reliable change of the sensory organization test. Clin J Sport Med. 2008;18:148–54.
Lovell MR, Iverson GL, Collins MW, Podell K, Johnston KM, Pardini D, et al. Measurement of symptoms following sports-related concussion: reliability and normative data for the post-concussion scale. Appl Neuropsychol. 2006;13:166–74.
McCrea M, Barr WB, Guskiewicz K, Randolph C, Marshall SW, Cantu R, et al. Standard regression-based methods for measuring recovery after sport-related concussion. J Int Neuropsychol Soc. 2005;11:58–69.
Echemendia RJ, Meeuwisse W, McCrory P, Davis GA, Putukian M, Leddy J, et al. The sport concussion assessment tool 5th Edition (SCAT5). Br J Sports Med. 2017;5:1–3.
Randolph C. Baseline neuropsychological testing in managing sport-related concussion. Curr Sports Med Rep. 2011;10:21–6.
Schmidt JD, Register-Mihalik JK, Mihalik JP, Kerr ZY, Guskiewicz KM. Identifying impairments after concussion: normative data versus individualized baselines. Med Sci Sports Exerc. 2012;44:1621–8.
Echemendia RJ, Bruce JM, Bailey CM, Sanders JF, Arnett P, Vargas G. The utility of post-concussion neuropsychological data in identifying cognitive change following sports-related MTBI in the absence of baseline data. Clin Neuropsychol. 2012;26:1077–91.
Valovich McLeod TC, Bay RC, Lam KC, Chhabra A. Representative baseline values on the sport concussion assessment tool 2 (SCAT2) in adolescent athletes vary by gender, grade, and concussion history. Am J Sports Med. 2012;40:927–33.
Putukian M, Echemendia R, Dettwiler-Danspeckgruber A, Duliba T, Bruce J, Furtado JL, et al. Prospective clinical assessment using sideline concussion assessment tool-2 testing in the evaluation of sport-related concussion in college athletes. Clin J Sport Med. 2015;25:36–42.
Covassin T, Elbin RJ, Harris W, Parker T, Kontos A. The role of age and sex in symptoms, neurocognitive performance, and postural stability in athletes after concussion. Am J Sports Med. 2012;40:1303–12.
Covassin T, Buz Swanik C, Sachs ML. Sex differences and the incidence of concussions among collegiate athletes. J Athl Train. 2003;38:238–44.
Covassin T, Schatz P, Swanik CB. Sex differences in neuropsychological function and post-concussion symptoms of concussed collegiate athletes. Neurosurgery. 2007;61:345–50.
Shehata N, Wiley JP, Richea S, Benson BW, Duits L, Meeuwisse WH. Sport concussion assessment tool: baseline values for varsity collision sport athletes. Br J Sports Med. 2009;43:730–4.
Bruce JM, Echemendia RJ. Concussion history predicts self-reported symptoms before and following a concussive event. Neurology. 2004;63:1516–8.
Covassin T, Stearne D, Elbin R. Concussion history and postconcussion neurocognitive performance and symptoms in collegiate athletes. J Athl Train. 2008;43:119–24.
Guskiewicz KM, Mccrea M, Marshall SW, Cantu RC, Randolph C, Barr W, et al. Cumulative effects associated with recurrent concussion in football players: the NCAA Concussion Study. J Am Med Assoc. 2003;290:2549–55.
Broglio SP, McCrea M, McAllister T, Harezlak J, Katz B, Hack D, et al. A national study on the effects of concussion in collegiate athletes and US Military Service Academy members: the NCAA–DoD concussion assessment, research and education (CARE) consortium structure and methods. Sport Med. 2017;47:1437–51.
Carney N, Ghajar J, Jagoda A, Bedrick S, Davis-O’Reilly C, Du Coudray H, et al. Concussion guidelines step 1: systematic review of prevalent indicators. Neurosurgery. 2014;75:3–15.
McCrory P, Meeuwisse WH, Aubry M, Cantu RC, Dvorák J, Echemendia RJ, et al. Consensus statement on concussion in sport. In: the 4th international conference on concussion in sport held in Zurich, November 2012. Br J Sports Med. 2013;47:255–79.
Asken BM, McCrea MA, Clugston JR, Snyder AR, Houck ZM, Bauer RM. “Playing through it”: delayed reporting and removal from athletic activity after concussion predicts prolonged recovery. J Athl Train. 2016;51:329–35.
Elbin RJ, Sufrinko AM, Schatz P, French J, Henry L, Burkhart S, et al. Removal from play after concussion and recovery time. Pediatrics. 2016;138:e20160910.
McCrea M, Kelly JP, Randolph C, Kluge J, Bartolic E, Finn G, et al. Standardized assessment of concussion (SAC): on-site mental status evaluation of the athlete. J Head Trauma Rehabil. 1998;13:27–35.
Concussion in Sport Group. Sport concussion assessment tool: 3rd edition. Br J Sports Med. 2013;47(5):259.
Bell DR, Guskiewicz KM, Clark MA, Padua DA. Systematic review of the balance error scoring system. Sports Health. 2011;3:287–95.
Van Buuren S, Boshuizen H, Knook D. Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med. 1999;18:681–94.
McCrea M, Guskiewicz KM, Marshall SW, Barr W, Randolph C, Cantu RC, et al. Acute effects and recovery time following concussions in collegiate football players. JAMA. 2004;290:2556–63.
McCrea M, Guskiewicz K, Randolph C, Barr WB, Hammeke TA, Marshall SW, et al. Incidence, clinical course, and predictors of prolonged recovery time following sport-related concussion in high school and college athletes. J Int Neuropsychol Soc. 2013;19:22–33.
Bozdogan H. Model selection and Akaike’s information criterion (AIC): the general theory and its analytical extensions. Psychometrika. 1987;52:345–70.
Williamson IJS, Goodman D. Converging evidence for the under-reporting of concussions in youth ice hockey. Br J Sports Med. 2006;40:128–32.
Pepe MS, Longton G, Janes H. Estimation and comparison of receiver operating characteristic curves. Stata J. 2009;9:1–16.
Meehan WP, Mannix RC, Stracciolini A, Elbin RJ, Collins MW. Symptom severity predicts prolonged recovery after sport-related concussion, but age and amnesia do not. J Pediatr. 2013;163:721–5.
Sufrinko A, McAllister-Deitrick J, Womble M, Kontos A. Do sideline concussion assessments predict subsequent neurocognitive impairment after sport-related concussion? J Athl Train. 2017;52:676–81.
Sufrinko AM, Marchetti GF, Cohen PE, Elbin RJ, Re V, Kontos AP. Using acute performance on a comprehensive neurocognitive, vestibular, and ocular motor assessment battery to predict recovery duration after sport-related concussions. Am J Sports Med. 2017;45:1187–94.
Register-Mihalik JK, Guskiewicz KM, Mihalik JP, Schmidt JD, Kerr ZY, McCrea MA. Reliable change, sensitivity, and specificity of a multidimensional concussion assessment battery. J Head Trauma Rehabil. 2013;28:274–83.
Resch JE, Brown CN, Schmidt J, Macciocchi SN, Blueitt D, Cullum CM, et al. The sensitivity and specificity of clinical measures of sport concussion: three tests are better than one. BMJ Open Sport Exerc Med. 2016;2:e000012.
Chin EY, Nelson LD, Barr WB, McCrory P, McCrea MA. Reliability and validity of the sport concussion assessment tool-3 (SCAT3) in high school and collegiate athletes. Am J Sports Med. 2016;44:2276–85.
Gessel LM, Fields SK, Collins CL, Dick RW, Comstock RD. Concussions among United States high school and collegiate athletes. J Athl Train. 2007;42:495–503.
Lincoln AE, Caswell SV, Almquist JL, Dunn RE, Norris JB, Hinton RY. Trends in concussion incidence in high school sports. Am J Sports Med. 2011;39:958–63.
Dick RW. Is there a gender difference in concussion incidence and outcomes? Br J Sports Med. 2009;43:i46–50.
Broshek DK, Kaushik T, Freeman JR, Erlanger D, Webbe F, Barth JT. Sex differences in outcome following sports-related concussion. J Neurosurg. 2005;102:856–63.
Valovich McLeod TC, Perrin DH, Guskiewicz KM, Shultz SJ, Diamond R, Gansneder BM. Serial administration of clinical concussion assessments and learning effects in healthy young athletes. Clin J Sport Med. 2004;14:287–95.
Moreau MS, Langdon J, Buckley TA. The lived experience of an in-season concussion amongst NCAA Division I student-athletes. Int J Exerc Sci. 2014;7:62–74.
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE 1256260. This publication was made possible, in part, with support from the Grand Alliance Concussion Assessment, Research, and Education (CARE) Consortium, funded, in part, by the National Collegiate Athletic Association and the Department of Defense. The US Army Medical Research Acquisition Activity, Fort Detrick, MD, USA is the awarding and administering acquisition office. This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Psychological Health and Traumatic Brain Injury Program under Award No. W81XWH-14-2-0151. Opinions, interpretations, conclusions, and recommendations are those of the author(s) and are not necessarily endorsed by the Department of Defense (Defense Health Program funds).
The CARE Consortium Investigators are listed alphabetically by institution: April Marie (Reed) Hoy, MS, ATC (Azusa Pacific University); Louise A. Kelly, PhD (California Lutheran University); Justus D. Ortega, PhD (Humboldt State University); Nicholas Port, PhD (Indiana University); Margot Putukian MD (Princeton University); T. Dianne Langford, PhD (Temple University); Scott Anderson, ATC and Gerald McGinty, DPT (US Air Force Academy); Patrick O’Donnell, MHA (US Coast Guard Academy); Steven J. Svoboda, MD (US Military Academy); John P. DiFiori (University of California-Los Angeles); Holly J. Benjamin MD (University of Chicago); Thomas Buckley, EdD, ATC, and Thomas W. Kaminski, PhD, ATC (University of Delaware); James R. Clugston, MD, MS (University of Florida); Julianne D. Schmidt, PhD, ATC (University of Georgia); Jason P. Mihalik, PhD, CAT(C), ATC (University of North Carolina at Chapel Hill); Christina L. Master, MD (University of Pennsylvania); Micky Collins, PhD, and Anthony P. Kontos, PhD (University of Pittsburgh Medical Center); Sara P.D. Chrisman, MD, MPH (University of Washington); Christopher M. Miles, MD (Wake Forest University); Brian H. Dykhuizen, MS, ATC (Wilmington College); Alison Brooks MD, MPH (University of Wisconsin).
Conflict of interest
Gian-Gabriel Garcia, Steven Broglio, Mariel Lavieri, Thomas McAllister, and Michael McCrea have no conflicts of interest directly relevant to the content of this study.
This study was approved by each local institutional review board and the Army Human Research Protection Office.
The members of the ‘CARE Consortium Investigators’ are given in Acknowledgements section. This article is part of the Topical Collection on The NCAA-DoD Concussion Assessment, Research and Education (CARE) Consortium.
About this article
Cite this article
Garcia, G.P., Broglio, S.P., Lavieri, M.S. et al. Quantifying the Value of Multidimensional Assessment Models for Acute Concussion: An Analysis of Data from the NCAA-DoD Care Consortium. Sports Med 48, 1739–1749 (2018). https://doi.org/10.1007/s40279-018-0880-x