Skip to main content

Advertisement

SpringerLink
Log in

Predicting Risk of Sport-Related Concussion in Collegiate Athletes and Military Cadets: A Machine Learning Approach Using Baseline Data from the CARE Consortium Study

  • Original Research Article
  • Published:
Sports Medicine Aims and scope Submit manuscript

Abstract

Objective

To develop a predictive model for sport-related concussion in collegiate athletes and military service academy cadets using baseline data collecting during the pre-participation examination.

Methods

Baseline assessments were performed in 15,682 participants from 21 US academic institutions and military service academies participating in the CARE Consortium Study during the 2015–2016 academic year. Participants were monitored for sport-related concussion during the subsequent season. 176 baseline covariates mapped to 957 binary features were used as input into a support vector machine model with the goal of learning to stratify participants according to their risk for sport-related concussion. Performance was evaluated in terms of area under the receiver operating characteristic curve (AUROC) on a held-out test set. Model inputs significantly associated with either increased or decreased risk were identified.

Results

595 participants (3.79%) sustained a concussion during the study period. The predictive model achieved an AUROC of 0.73 (95% confidence interval 0.70–0.76), with variable performance across sports. Features with significant positive and negative associations with subsequent sport-related concussion were identified.

Conclusion(s)

This predictive model using only baseline data identified athletes and cadets who would go on to sustain sport-related concussion with comparable accuracy to many existing concussion assessment tools for identifying concussion. Furthermore, this study provides insight into potential concussion risk and protective factors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Langlois JA, Rutland-Brown W, Wald MM. The epidemiology and impact of traumatic brain injury: a brief overview. J Head Trauma Rehabil. 2006;21(5):375–8.

    PubMed  Google Scholar 

  2. National Collegiate Athletic Association. Number of NCAA college athletes reaches all-time high. 2018.

  3. US Department of Defense. Special report—traumatic brain injury. 2015.

  4. BrainLine, How Many Service Members Have Sustained a TBI? Defense Medical Surveillance System, 2018.

  5. Guskiewicz KM, et al. Recurrent concussion and risk of depression in retired professional football players. Med Sci Sports Exerc. 2007;39(6):903–9.

    PubMed  Google Scholar 

  6. Guskiewicz KM, et al. Association between recurrent concussion and late-life cognitive impairment in retired professional football players. Neurosurgery. 2005;57(4):719–26 (discussion 719–2).

    PubMed  Google Scholar 

  7. McKee AC, et al. Chronic traumatic encephalopathy in athletes: progressive tauopathy after repetitive head injury. J Neuropathol Exp Neurol. 2009;68(7):709–35.

    PubMed  Google Scholar 

  8. McKee AC, et al. The spectrum of disease in chronic traumatic encephalopathy. Brain. 2013;136(Pt 1):43–64.

    PubMed  Google Scholar 

  9. McKee AC, et al. The neuropathology of chronic traumatic encephalopathy. Brain Pathol. 2015;25(3):350–64.

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Mez J, et al. Clinicopathological evaluation of chronic traumatic encephalopathy in players of American Football. JAMA. 2017;318(4):360–70.

    PubMed  PubMed Central  Google Scholar 

  11. Caccese JB et al. Does baseline concussion testing aid in identifying future concussion risk? Res Sports Med. 2019:1–6.

  12. Abrahams S, et al. Risk factors for sports concussion: an evidence-based systematic review. Br J Sports Med. 2014;48(2):91–7.

    PubMed  Google Scholar 

  13. Harmon KG, et al. American Medical Society for Sports Medicine position statement: concussion in sport. Br J Sports Med. 2013;47(1):15–26.

    PubMed  Google Scholar 

  14. Kerr HA. Concussion risk factors and strategies for prevention. Pediatr Ann. 2014;43(12):e309–15.

    PubMed  Google Scholar 

  15. Noble JM, Hesdorffer DC. Sport-related concussions: a review of epidemiology, challenges in diagnosis, and potential risk factors. Neuropsychol Rev. 2013;23(4):273–84.

    PubMed  Google Scholar 

  16. Finnoff JT, Jelsing EJ, Smith J. Biomarkers, genetics, and risk factors for concussion. PM R. 2011;3(10 Suppl 2):S452–9.

    PubMed  Google Scholar 

  17. Hollis SJ, et al. Incidence, risk, and protective factors of mild traumatic brain injury in a cohort of Australian nonprofessional male rugby players. Am J Sports Med. 2009;37(12):2328–33.

    PubMed  Google Scholar 

  18. Schneider KJ, et al. Preseason reports of neck pain, dizziness, and headache as risk factors for concussion in male youth ice hockey players. Clin J Sport Med. 2013;23(4):267–72.

    PubMed  Google Scholar 

  19. Curran-Sills G, Abedin T. Risk factors associated with injury and concussion in sanctioned amateur and professional mixed martial arts bouts in Calgary, Alberta. BMJ Open Sport Exerc Med. 2018;4(1):e000348.

    PubMed  PubMed Central  Google Scholar 

  20. Yeung A, Munjal V, Virji-Babul N. Development of the sports organization concussion risk assessment tool (SOCRAT). Brain Inj. 2017;31(4):542–9.

    CAS  PubMed  Google Scholar 

  21. Baugh CM, et al. Frequency of head-impact-related outcomes by position in NCAA division I collegiate football players. J Neurotrauma. 2015;32(5):314–26.

    PubMed  PubMed Central  Google Scholar 

  22. Black AM, et al. Policy change eliminating body checking in non-elite ice hockey leads to a threefold reduction in injury and concussion risk in 11- and 12-year-old players. Br J Sports Med. 2016;50(1):55–61.

    PubMed  Google Scholar 

  23. Blumenfeld RS, et al. The epidemiology of sports-related head injury and concussion in water polo. Front Neurol. 2016;7:98.

    PubMed  PubMed Central  Google Scholar 

  24. Hollis SJ, et al. Mild traumatic brain injury among a cohort of rugby union players: predictors of time to injury. Br J Sports Med. 2011;45(12):997–9.

    PubMed  Google Scholar 

  25. Teramoto M, et al. Style of play and rate of concussions in the national football league. Orthop J Sports Med. 2015;3(12):2325967115620365.

    PubMed  PubMed Central  Google Scholar 

  26. Collins CL, et al. Neck strength: a protective factor reducing risk for concussion in high school sports. J Prim Prev. 2014;35(5):309–19.

    PubMed  Google Scholar 

  27. Dretsch MN, et al. Evaluating the clinical utility of the Validity-10 for detecting amplified symptom reporting for patients with mild traumatic brain injury and comorbid psychological health conditions. Appl Neuropsychol Adult. 2017;24(4):376–80.

    PubMed  Google Scholar 

  28. Van Pelt KL, et al. A cohort study to identify and evaluate concussion risk factors across multiple injury settings: findings from the CARE Consortium. Inj Epidemiol. 2019;6(1):1.

    PubMed  PubMed Central  Google Scholar 

  29. Schulz MR, et al. Incidence and risk factors for concussion in high school athletes, North Carolina, 1996–1999. Am J Epidemiol. 2004;160(10):937–44.

    PubMed  Google Scholar 

  30. Broglio SP, et al. A national study on the effects of concussion in collegiate athletes and US military service academy members: the NCAA-DoD concussion assessment, research and education (CARE) consortium structure and methods. Sports Med. 2017;47(7):1437–51.

    PubMed  PubMed Central  Google Scholar 

  31. Carney N, et al. Concussion guidelines step 1: systematic review of prevalent indicators. Neurosurgery. 2014;75(Suppl 1):S3–15.

    PubMed  Google Scholar 

  32. Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–97.

    Google Scholar 

  33. Ustun B, Traca S, Rudin C. Cynthia supersparse linear integer models for interpretable classification. 2014. arXiv:1306.6677.

  34. Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th international joint conference on Artificial intelligence—Volume 2. 1995, Morgan Kaufmann Publishers Inc.: Montreal, Quebec, Canada. p. 1137–1143.

  35. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.

    CAS  PubMed  Google Scholar 

  36. Wasserman EB, et al. Academic dysfunction after a concussion among US High School and College Students. Am J Public Health. 2016;106(7):1247–53.

    PubMed  PubMed Central  Google Scholar 

  37. Manley G, et al. A systematic review of potential long-term effects of sport-related concussion. Br J Sports Med. 2017;51(12):969–77.

    PubMed  Google Scholar 

  38. Barlow M, et al. Differences in change scores and the predictive validity of three commonly used measures following concussion in the middle school and high school aged population. Int J Sports Phys Ther. 2011;6(3):150–7.

    PubMed  PubMed Central  Google Scholar 

  39. Randolph C, et al. Concussion symptom inventory: an empirically derived scale for monitoring resolution of symptoms following sport-related concussion. Arch Clin Neuropsychol. 2009;24(3):219–29.

    PubMed  PubMed Central  Google Scholar 

  40. Barr WB, McCrea M. Sensitivity and specificity of standardized neurocognitive testing immediately following sports concussion. J Int Neuropsychol Soc. 2001;7(6):693–702.

    CAS  PubMed  Google Scholar 

  41. Mucha A, et al. A Brief Vestibular/Ocular Motor Screening (VOMS) assessment to evaluate concussions: preliminary findings. Am J Sports Med. 2014;42(10):2479–86.

    PubMed  PubMed Central  Google Scholar 

  42. Schmidt JD, et al. Age at first concussion influences the number of subsequent concussions. Pediatr Neurol. 2018;81:19–24.

    PubMed  Google Scholar 

  43. Stamm JM, et al. Age at first exposure to football is associated with altered corpus callosum white matter microstructure in former professional football players. J Neurotrauma. 2015;32(22):1768–76.

    PubMed  PubMed Central  Google Scholar 

  44. Stamm JM, et al. Age of first exposure to football and later-life cognitive impairment in former NFL players. Neurology. 2015;84(11):1114–20.

    PubMed  PubMed Central  Google Scholar 

  45. Alosco ML, et al. Age of first exposure to American football and long-term neuropsychiatric and cognitive outcomes. Transl Psychiatry. 2017;7(9):e1236.

    CAS  PubMed  PubMed Central  Google Scholar 

  46. Schultz V, et al. Age at first exposure to repetitive head impacts is associated with smaller thalamic volumes in former professional american football players. J Neurotrauma. 2018;35(2):278–85.

    PubMed  PubMed Central  Google Scholar 

  47. Caccese JB, et al. Estimated age of first exposure to american football and neurocognitive performance amongst NCAA male student-athletes: a cohort study. Sports Med. 2019;49(3):477–87.

    PubMed  Google Scholar 

  48. Caccese JB et al. Estimated age of first exposure to contact sports is not associated with greater symptoms or worse cognitive functioning in male U.S. Service Academy Athletes. J Neurotrauma. 2020;37(2):334–9.

    PubMed  Google Scholar 

  49. Brett BL, et al. Age of first exposure to american football and behavioral, cognitive, psychological, and physical outcomes in high school and collegiate football players. Sports Health. 2019;11(4):332–42.

    PubMed  PubMed Central  Google Scholar 

  50. Koh JO, Cassidy JD, Watkinson EJ. Incidence of concussion in contact sports: a systematic review of the evidence. Brain Inj. 2003;17(10):901–17.

    PubMed  Google Scholar 

  51. Hootman JM, Dick R, Agel J. Epidemiology of collegiate injuries for 15 sports: summary and recommendations for injury prevention initiatives. J Athl Train. 2007;42(2):311–9.

    PubMed  PubMed Central  Google Scholar 

  52. Marar M, et al. Epidemiology of concussions among United States high school athletes in 20 sports. Am J Sports Med. 2012;40(4):747–55.

    PubMed  Google Scholar 

  53. Giza CC, et al. Summary of evidence-based guideline update: evaluation and management of concussion in sports: Report of the Guideline Development Subcommittee of the American Academy of Neurology. Neurology. 2013;80(24):2250–7.

    PubMed  PubMed Central  Google Scholar 

  54. Dick RW. Is there a gender difference in concussion incidence and outcomes? Br J Sports Med. 2009;43(Suppl 1):i46–50.

    PubMed  Google Scholar 

  55. Lipton RB, et al. A self-administered screener for migraine in primary care: the ID Migraine validation study. Neurology. 2003;61(3):375–82.

    CAS  PubMed  Google Scholar 

  56. Eckner JT, et al. Is migraine headache associated with concussion in athletes? A case-control study. Clin J Sport Med. 2017;27(3):266–70.

    PubMed  PubMed Central  Google Scholar 

  57. Gordon KE, Dooley JM, Wood EP. Is migraine a risk factor for the development of concussion? Br J Sports Med. 2006;40(2):184–5.

    CAS  PubMed  PubMed Central  Google Scholar 

  58. Alosco ML, Fedor AF, Gunstad J. Attention deficit hyperactivity disorder as a risk factor for concussions in NCAA division-I athletes. Brain Inj. 2014;28(4):472–4.

    PubMed  Google Scholar 

Download references

Acknowledgements

This project was supported, in part, with support from the Grand Alliance Concussion Assessment, Research, and Education (CARE) Consortium, funded, in part by the National Collegiate Athletic Association (NCAA) and the Department of Defense (DOD). The U.S. Army Medical Research Acquisition Activity, 820 Chandler Street, Fort Detrick MD 21702-5014 is the awarding and administering acquisition office. This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Combat Casualty Care Program, endorsed by the Department of Defense under Award No. W81XWH-BA170608. Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the Office of the Assistant Secretary of Defense for Health Affairs.

The authors thank Kaitlyn Carter (Azusa Pacific University); Jennifer Brewington Dickerson, Jody Harland, Nicole Johnson, Janetta Matesan, Nicholas Port, Larry Riggen (Indiana University); Margot Putukian (Princeton University); Gerald McGinty (United States Air Force Academy); Patrick G. O’Donnell, Carlos Esteves (United States Coast Guard Academy); Ken Cameron (United States Military Academy); Tom Kaminski (University of Delaware); Julianne Schmidt (University of Georgia); Josh Goldman (University of California Los Angeles); Ashley Rettmann (University of Michigan); Kevin Guskiewicz (University of North Carolina at Chapel Hill); Scott Anderson (University of Oklahoma); Jeffery J Bazarian (University of Rochester); Sara Chrisman (University of Washington); Alison Brooks (University of Wisconsin); Stefan Duma (Virginia Polytechnic Institute and State University); and research and medical staff at each of the CARE participation sites.

Care Consortium Investigators are listed alphabetically by institution: April (Reed) Hoy, Azusa Pacific University; Louise Kelly, California Lutheran University; Jonathan Jackson, United States Air Force Academy; Tim Kelly, United States Military Academy; Thomas Buckley, University of Delaware; James (Jay) R. Clugston, University of Florida; Justus Ortega, Humboldt State University; Anthony Kontos, University of Pittsburgh; Christopher C. Giza, University of California Los Angeles; Jason Mihalik, University of North Carolina at Chapel Hill; Steve Rowson, Virginia Polytechnic Institute and State University.

Author information

Author notes

  1. Joel Castellanos

    Present address: Anestheshiology, School of Medicine, University of California San Diego, San Diego, CA, USA

  2. Cheng Perng Phoo

    Present address: Computer Science, Cornell University, New York, USA

  3. Joel Castellanos and Cheng Perng Phoo are co-first authors.

Authors and Affiliations

  1. Department of Physical Medicine and Rehabilitation, Michigan Medicine, University of Michigan, 325 E. Eisenhower Parkway, Ann Arbor, MI, 48108, USA

    Joel Castellanos, James T. Eckner & Lea Franco

  2. Computer Science and Engineering, University of Michigan, Ann Arbor, MI, USA

    Cheng Perng Phoo & Jenna Wiens

  3. Kinesiology, University of Michigan, Ann Arbor, MI, USA

    Steven P. Broglio

  4. Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA

    Mike McCrea

  5. Psychiatry, Indiana University, Indianapolis, IN, USA

    Thomas McAllister

Authors
  1. Joel Castellanos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cheng Perng Phoo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James T. Eckner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lea Franco
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steven P. Broglio
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mike McCrea
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Thomas McAllister
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jenna Wiens
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

The CARE Consortium Investigators

  • April Hoy
  • , Louise Kelly
  • , Jonathan Jackson
  • , Tim Kelly
  • , Thomas Buckley
  • , James R. Clugston
  • , Justus Ortega
  • , Anthony Kontos
  • , Christopher C. Giza
  • , Jason Mihalik
  •  & Steve Rowson

Corresponding author

Correspondence to James T. Eckner.

Ethics declarations

Funding

This project was supported, in part, with support from the Grand Alliance Concussion Assessment, Research, and Education (CARE) Consortium, funded, in part by the National Collegiate Athletic Association (NCAA) and the Department of Defense (DOD). The U.S. Army Medical Research Acquisition Activity, 820 Chandler Street, Fort Detrick MD 21702-5014 is the awarding and administering acquisition office. This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Combat Casualty Care Program, endorsed by the Department of Defense under Award No. W81XWH-BA170608. Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the Office of the Assistant Secretary of Defense for Health Affairs.

Conflict of interest

The authors whose names are listed immediately below certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript: Castellanos and Wiens. The authors whose names are listed immediately below certify that grant support for this project was received from the Grand Alliance Concussion Assessment, Research, and Education (CARE) Consortium, funded, in part by the NCAA and the DoD: Phoo, Eckner, Franco, Broglio, McCrea, McAllister. The authors whose names are listed immediately below also certify that travel support was provided by the Grand Alliance Concussion Assessment, Research, and Education (CARE) Consortium, funded, in part by the NCAA and the DoD: Eckner, Franco, Broglio, and McCrea. The authors whose names are listed immediately below certify additional disclosures that do not have financial interest in the subject matter discussed in this manuscript as detailed in their author declaration forms: Eckner (patent, grant funding), Broglio (consultation, expert testimony, grant funding, advisory and editorial boards), and McCrea (grant funding, consultation).

Ethics approval

Institutional Review Board (IRB) approval was obtained at the University of Michigan (lead study site), with US Department of Defense Human Research Protection Office approval as well as local IRB approval at each participating site. This study was performed in accordance with the standards of ethics outlined in the Declaration of Helsinki.

Consent to participate

All participants provided informed written consent.

Consent for publication

Not applicable. No identifiable information or images are included in this publication.

Availability of data

CARE Consortium data are publically available upon request from the Federal Interagency Traumatic Brain Injury Research (FITBIR) Informatics System.

Code availability

The authors are willing to provide the data analysis code upon written request.

Author contributions

Dr. Castellanos contributed to the conception and design of the work; data interpretation; drafting and revision of the manuscript. He approved the final published version and agreed to be accountable for all aspects of the work. Mr. Phoo contributed to the design of the work; data analysis and interpretation; drafting and revision of the manuscript. He approved the final published version and agreed to be accountable for all aspects of the work. Dr. Eckner contributed to the conception and design of the work; data acquisition, analysis, and interpretation; drafting and revision of the manuscript. He approved the final published version and agreed to be accountable for all aspects of the work. Ms. Franco contributed to data acquisition and interpretation; critical revision of the manuscript for intellectual content. She approved the final published version and agreed to be accountable for all aspects of the work. Drs. Broglio, McCrea, and McAllister contributed to the design of the work; data interpretation; critical revision of the manuscript for intellectual content. They approved the final published version and agreed to be accountable for all aspects of the work. Dr. Wiens contributed to the conception and design of the work; data analysis and interpretation; drafting and revision of the manuscript. She approved the final published version and agreed to be accountable for all aspects of the work. The CARE Consortium Investigators (Ms. Hoy, Dr. Kelly, Dr. Jackson, Mr. Kelly, Dr. Buckley, Dr. Clugston, Dr. Ortega, Dr. Kontos, Dr. Giza, Dr. Mihalik, Dr. Rowson) contributed to data acquisition and interpretation and critical revision of the manuscript for intellectual content. They approved the final published version and agreed to be accountable for all aspects of the work.

Additional information

The members of The CARE Consortium Investigators are mentioned in “Acknowledgements” section.

This article is part of a collection on The NCAA-DoD Concussion Assessment, Research and Education (CARE) Consortium.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Castellanos, J., Phoo, C.P., Eckner, J.T. et al. Predicting Risk of Sport-Related Concussion in Collegiate Athletes and Military Cadets: A Machine Learning Approach Using Baseline Data from the CARE Consortium Study. Sports Med 51, 567–579 (2021). https://doi.org/10.1007/s40279-020-01390-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40279-020-01390-w

Search

Navigation