Abstract
Machine learning (ML) techniques can help harness insights from data that complement and extend those that can be attained by traditional statistical methods. The current article introduces clinicians to concepts underlying ML and explores how it can be applied within the domain of neuropsychology. Specifically, we illustrate an application of ML to a dataset that includes a battery of standardized measures designed to provide diagnostic support for concussions, including standardized neurocognitive (CPT 3) and neurobehavioral (BESS, NIH 4 meter gait) measures, gait sensor data, and a CDC concussion symptom checklist. These variables were used to predict the decision-making of a pediatric neurologist evaluating a group of child/adolescent patients. With a sample of 111 cases, ML (using a general linear model and deep learning as illustrations) achieved accuracies of 91% and 84.8% and AUCs of 1.0 and .947, respectively, when predicting the neurologist’s binomial decision-making (safe/remove). In presenting the data and various considerations for interpretation, we attempt to balance both the promise and perils of ML.
Similar content being viewed by others
Notes
In the context of artificial neural networks, a rectifier is an activation function defined as the positive part of its argument:
f(x) = x+ = max(0, x),
where x is the input to a neuron. Wikipedia
Softmax is a function that takes as input a vector of K real numbers, and normalizes it into a probability distribution consisting of K probabilities proportional to the exponentials of the input numbers. Wikipedia
References
Battista, P., Salvatore, C., & Castiglioni, I. (2017, 2017). Optimizing neuropsychological assessments for cognitive, behavioral, and functional impairment classification: a machine learning study. Behavioural Neurology, 1–19. https://doi.org/10.1155/2017/1850909.
Bleiberg, J., Cernich, A. N., Cameron, K., et al. (2004). Duration of cognitive impairment after sports concussion. Neurosurgery, 54, 1073–1080. https://doi.org/10.1227/01.NEU.0000118820.33396.6A.
Broglio, S.P., Ferrara, M.S., Piland, S.G., Anderson, R.B. (2006). Concussion history is not a predictor of computerised neurocognitive performance. British Journal of Sports Medicine, 40(9), 802-805.
Broglio, S. P., & Puetz, T. W. (2008). The effect of sport concussion on neurocognitive function, self-report symptoms, and postural control: a meta-analysis. Sports Medicine, 38, 53–67.
Broglio, S. P., Ferrara, M. S., Macciocchi, S. N., Baumgartner, T. A., & Elliott, R. (2007). Test-retest reliability of computerized concussion assessment programs. Journal of Athletic Training, 42(4), 509–514.
Bruce, J., Echemendia, R., Meeuwisse, W., Comper, P., & Sisco, A. (2014). 1-year test–retest reliability of ImPACT in professional ice hockey players. The Clinical Neuropsychologist, 28(1), 14–25.
Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668–1674.
Frank, G. (1984). The Boulder Model: History, rationale, and critique. Professional Psychology: Research and Practice, 15(3), 417–435. https://doi.org/10.1037/0735-7028.15.3.417.
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: a meta-analysis. Psychological Assessment, 12, 19–30. https://doi.org/10.1037//1040-3590.12.1.19.
Henry, L. C., Elbin, R. J., Collins, M. W., et al. (2016). Examining recovery trajectories after sport-related concussion with a multimodal clinical assessment approach. Neurosurgery, 78, 232–241. https://doi.org/10.1227/NEU.0000000000001041.
Hinton, G. (2018). Deep Learning—a technology with the potential to transform health care. Journal of the American Medical Association, 320(11), 1101–1102. https://doi.org/10.1001/jama.2018.11100.
Iverson, G. L., Gardner, A. J., Terry, D. P., et al. (2017). Predictors of clinical recovery from concussion: a systematic review. British Journal of Sports Medicine, 51, 941–948.
Kaye, A. J., Gallagher, R., Callahan, J. M., & Nance, M. L. (2010). Mild traumatic brain injury in the pediatric population: the role of the pediatrician in routine follow-up. Journal of Trauma, 68, 1396–1400.
Kirelik, S. B., & McAvoy, K. (2016). Acute concussion management with remove-reduce/educate/adjust-accommodate/pace (REAP). The Journal of emergency medicine, 50(2), 320–324.
Lau, B., Lovell, M. R., Collins, M. W., & Pardini, J. (2009). Neurocognitive and symptom predictors of recovery in high school athletes. Clinical Journal of Sport Medicine, 19(3), 216–221.
Lecci, L., Wiiliams, M., Taravath, S., Frank, H.G., Dugan, K., Page, G.R, & Keith, J.R. (in press). Validation of a concussion screening battery for use in medical settings. Archives of Clinical Neuropsychology.
LeCun, Y., Bengio, Y., & Hinton, G. H. (2015). Deep learning. Nature, 521, 436–444. https://doi.org/10.1038/nature14539.
Macciocchi, S. N., Barth, J. T., Alves, W., et al. (1996). Neuropsychological functioning and recovery after mild head injury in collegiate Athletes. Neurosurgery, 39, 510–514.
McCrea, M., Guskiewicz, K. M., Marshall, S. W., et al. (2003). Acute effects and recovery time following concussion in collegiate football players: The NCAA Concussion Study. JAMA, 290, 2556–2563. https://doi.org/10.1001/jama.290.19.2556.
McCrory, P., Meeuwisse, W. H., Aubry, M., Cantu, R. C., Dvorak, J., Echemendia, R. J., et al. (2013). Consensus statement on concussion in sport—the 4th International Conference on Concussion in Sport held in Zurich, November 2012. PM&R, 5(4), 255–279.
McCrory, P., Meeuwisse, W., Dvorak, J., Aubry, M., Bailes, J., Broglio, S., et al. (2017). Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. British Journal of Sports Medicine, 51(11), 838–847.
Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence (Vol. x 149 pp.). Minneapolis: University of Minnesota Press. https://doi.org/10.1037/11281-000.
Parker, T. M., Osternig, L. R., van Donkelaar, P., & Chou, L. S. (2006). Gait stability after a concussion. Medicine and Science in Sports & Exercise, 38, 1031–1040.
Resch, J. E., Macciocchi, S., & Ferrara, M. S. (2013). Preliminary evidence of equivalence of alternate forms of the ImPACT. The Clinical Neuropsychologist, 27(8), 1265–1280.
Rosenblatt, F. (1957). The perceptron – a perceiving and recognizing automation. Report 85-460-1, Cornell Aeronautical Laboratory.
Sahl, S. M. (2015). Estimating R 2 shrinkage in regression. International Journal of Technical Research and Applications, 3, 01–06.
Schatz, P., & Putz, B. O. (2006). Cross-validation of measures used for computer-based assessment of concussion. Applied Neuropsychology, 13(3), 151–159.
Shatte, A., Hutchinson, D., & Teague, S. (2019). Machine learning in mental health: a scoping review of methods and applications. Psychological Medicine, 49(9), 1426–1448. https://doi.org/10.1017/S0033291719000151.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
Julian Keith is a paid consultant for SportGait, which is the concussion battery utilized in the current research. Mark Williams is one of the founders and stockholder in SportGait. Len Lecci is a paid consultant and stockholder in SportGait. Sasiharan Taravath has no conflicts to report and he provided the clinical data herein reported.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Keith, J., Williams, M., Taravath, S. et al. A Clinician’s Guide to Machine Learning in Neuropsychological Research and Practice. J Pediatr Neuropsychol 5, 177–187 (2019). https://doi.org/10.1007/s40817-019-00075-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40817-019-00075-1