Advertisement

Journal of Pediatric Neuropsychology

, Volume 5, Issue 4, pp 177–187 | Cite as

A Clinician’s Guide to Machine Learning in Neuropsychological Research and Practice

  • Julian Keith
  • Mark Williams
  • Sasidharan Taravath
  • Len LecciEmail author
Article

Abstract

Machine learning (ML) techniques can help harness insights from data that complement and extend those that can be attained by traditional statistical methods. The current article introduces clinicians to concepts underlying ML and explores how it can be applied within the domain of neuropsychology. Specifically, we illustrate an application of ML to a dataset that includes a battery of standardized measures designed to provide diagnostic support for concussions, including standardized neurocognitive (CPT 3) and neurobehavioral (BESS, NIH 4 meter gait) measures, gait sensor data, and a CDC concussion symptom checklist. These variables were used to predict the decision-making of a pediatric neurologist evaluating a group of child/adolescent patients. With a sample of 111 cases, ML (using a general linear model and deep learning as illustrations) achieved accuracies of 91% and 84.8% and AUCs of 1.0 and .947, respectively, when predicting the neurologist’s binomial decision-making (safe/remove). In presenting the data and various considerations for interpretation, we attempt to balance both the promise and perils of ML.

Keywords

Machine learning concussion screening assessment decision support tool 

Notes

Compliance with Ethical Standards

Conflict of Interest

Julian Keith is a paid consultant for SportGait, which is the concussion battery utilized in the current research. Mark Williams is one of the founders and stockholder in SportGait. Len Lecci is a paid consultant and stockholder in SportGait. Sasiharan Taravath has no conflicts to report and he provided the clinical data herein reported.

References

  1. Battista, P., Salvatore, C., & Castiglioni, I. (2017, 2017). Optimizing neuropsychological assessments for cognitive, behavioral, and functional impairment classification: a machine learning study. Behavioural Neurology, 1–19.  https://doi.org/10.1155/2017/1850909.CrossRefGoogle Scholar
  2. Bleiberg, J., Cernich, A. N., Cameron, K., et al. (2004). Duration of cognitive impairment after sports concussion. Neurosurgery, 54, 1073–1080.  https://doi.org/10.1227/01.NEU.0000118820.33396.6A.CrossRefPubMedGoogle Scholar
  3. Broglio, S.P., Ferrara, M.S., Piland, S.G., Anderson, R.B. (2006). Concussion history is not a predictor of computerised neurocognitive performance. British Journal of Sports Medicine, 40(9), 802-805.CrossRefGoogle Scholar
  4. Broglio, S. P., & Puetz, T. W. (2008). The effect of sport concussion on neurocognitive function, self-report symptoms, and postural control: a meta-analysis. Sports Medicine, 38, 53–67.CrossRefGoogle Scholar
  5. Broglio, S. P., Ferrara, M. S., Macciocchi, S. N., Baumgartner, T. A., & Elliott, R. (2007). Test-retest reliability of computerized concussion assessment programs. Journal of Athletic Training, 42(4), 509–514.PubMedPubMedCentralGoogle Scholar
  6. Bruce, J., Echemendia, R., Meeuwisse, W., Comper, P., & Sisco, A. (2014). 1-year test–retest reliability of ImPACT in professional ice hockey players. The Clinical Neuropsychologist, 28(1), 14–25.CrossRefGoogle Scholar
  7. Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668–1674.CrossRefGoogle Scholar
  8. Frank, G. (1984). The Boulder Model: History, rationale, and critique. Professional Psychology: Research and Practice, 15(3), 417–435.  https://doi.org/10.1037/0735-7028.15.3.417.CrossRefGoogle Scholar
  9. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: a meta-analysis. Psychological Assessment, 12, 19–30.  https://doi.org/10.1037//1040-3590.12.1.19.CrossRefPubMedGoogle Scholar
  10. Henry, L. C., Elbin, R. J., Collins, M. W., et al. (2016). Examining recovery trajectories after sport-related concussion with a multimodal clinical assessment approach. Neurosurgery, 78, 232–241.  https://doi.org/10.1227/NEU.0000000000001041.CrossRefPubMedPubMedCentralGoogle Scholar
  11. Hinton, G. (2018). Deep Learning—a technology with the potential to transform health care. Journal of the American Medical Association, 320(11), 1101–1102.  https://doi.org/10.1001/jama.2018.11100.CrossRefPubMedGoogle Scholar
  12. Iverson, G. L., Gardner, A. J., Terry, D. P., et al. (2017). Predictors of clinical recovery from concussion: a systematic review. British Journal of Sports Medicine, 51, 941–948.CrossRefGoogle Scholar
  13. Kaye, A. J., Gallagher, R., Callahan, J. M., & Nance, M. L. (2010). Mild traumatic brain injury in the pediatric population: the role of the pediatrician in routine follow-up. Journal of Trauma, 68, 1396–1400.CrossRefGoogle Scholar
  14. Kirelik, S. B., & McAvoy, K. (2016). Acute concussion management with remove-reduce/educate/adjust-accommodate/pace (REAP). The Journal of emergency medicine, 50(2), 320–324.CrossRefGoogle Scholar
  15. Lau, B., Lovell, M. R., Collins, M. W., & Pardini, J. (2009). Neurocognitive and symptom predictors of recovery in high school athletes. Clinical Journal of Sport Medicine, 19(3), 216–221.CrossRefGoogle Scholar
  16. Lecci, L., Wiiliams, M., Taravath, S., Frank, H.G., Dugan, K., Page, G.R, & Keith, J.R. (in press). Validation of a concussion screening battery for use in medical settings. Archives of Clinical Neuropsychology.Google Scholar
  17. LeCun, Y., Bengio, Y., & Hinton, G. H. (2015). Deep learning. Nature, 521, 436–444.  https://doi.org/10.1038/nature14539.CrossRefPubMedGoogle Scholar
  18. Macciocchi, S. N., Barth, J. T., Alves, W., et al. (1996). Neuropsychological functioning and recovery after mild head injury in collegiate Athletes. Neurosurgery, 39, 510–514.CrossRefGoogle Scholar
  19. McCrea, M., Guskiewicz, K. M., Marshall, S. W., et al. (2003). Acute effects and recovery time following concussion in collegiate football players: The NCAA Concussion Study. JAMA, 290, 2556–2563.  https://doi.org/10.1001/jama.290.19.2556.CrossRefPubMedGoogle Scholar
  20. McCrory, P., Meeuwisse, W. H., Aubry, M., Cantu, R. C., Dvorak, J., Echemendia, R. J., et al. (2013). Consensus statement on concussion in sport—the 4th International Conference on Concussion in Sport held in Zurich, November 2012. PM&R, 5(4), 255–279.CrossRefGoogle Scholar
  21. McCrory, P., Meeuwisse, W., Dvorak, J., Aubry, M., Bailes, J., Broglio, S., et al. (2017). Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. British Journal of Sports Medicine, 51(11), 838–847.PubMedGoogle Scholar
  22. Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence (Vol. x 149 pp.). Minneapolis: University of Minnesota Press.  https://doi.org/10.1037/11281-000.CrossRefGoogle Scholar
  23. Parker, T. M., Osternig, L. R., van Donkelaar, P., & Chou, L. S. (2006). Gait stability after a concussion. Medicine and Science in Sports & Exercise, 38, 1031–1040.CrossRefGoogle Scholar
  24. Resch, J. E., Macciocchi, S., & Ferrara, M. S. (2013). Preliminary evidence of equivalence of alternate forms of the ImPACT. The Clinical Neuropsychologist, 27(8), 1265–1280.CrossRefGoogle Scholar
  25. Rosenblatt, F. (1957). The perceptron – a perceiving and recognizing automation. Report 85-460-1, Cornell Aeronautical Laboratory.Google Scholar
  26. Sahl, S. M. (2015). Estimating R 2 shrinkage in regression. International Journal of Technical Research and Applications, 3, 01–06.Google Scholar
  27. Schatz, P., & Putz, B. O. (2006). Cross-validation of measures used for computer-based assessment of concussion. Applied Neuropsychology, 13(3), 151–159.CrossRefGoogle Scholar
  28. Shatte, A., Hutchinson, D., & Teague, S. (2019). Machine learning in mental health: a scoping review of methods and applications. Psychological Medicine, 49(9), 1426–1448.  https://doi.org/10.1017/S0033291719000151.CrossRefPubMedGoogle Scholar

Copyright information

© American Academy of Pediatric Neuropsychology 2019

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of North Carolina WilmingtonWilmingtonUSA
  2. 2.Internal Medicine, New Hanover Regional Medical CenterWilmingtonUSA
  3. 3.Pediatric Neurology, New Hanover Regional Medical CenterWilmingtonUSA

Personalised recommendations