Avoid common mistakes on your manuscript.
We thank Prof. Molinski and collaborators for their interest in our research and their comments.
We agree with some of their remarks, especially regarding the risk of overfitting for a data set of 54 cases, and therefore we clearly acknowledge that limitation of our article [1]. Although we have attempted to reduce the risk of overfitting in the methodology adopted, it indeed remains a possibility, particularly when considering the strong performance of our pilot data. Since the publication of our article, we have started recruiting patients for a prospective validation of our algorithm.
Regarding using a larger network such as ResNet50 or DenseNet201, we believe that the use of larger models increases the likelihood of overfitting (greater parameters in an underdetermined system). We used the pretrained VGG19 (a smaller, “outdated” model) to only extract features from the images and only trained a support vector machine classifier. Although the ImageNet images have different characteristics from medical images, as the commentary states, additional training of the VGG19 directly with our data set would have only increased the risk of overfitting, and thus we chose transfer learning. Furthermore, we could have used a larger model as a feature extractor but would have needed to extract a larger number of features to respect shallow and deep Convolutional Neural Network features. This large number of features would also increase the risk of overfitting. Note that there is potential for the use of other medical imaging data sets tangential to our classification task for additional training, but those are outside the scope of our preliminary investigation.
The commentary seems to misunderstand that we used the VGG19 only to extract image feature information, not as a full deep learning classification model. The only model training that was done was the development of the support vector machine classifier, which used the feature extracted from the ImageNet pretrained Visula Geometry Group19 architecture. The Support Vector Model is, in general, a more robust classification approach than a deep learning classifier when considering a limited data set; this was also adopted to reduce the risk of overfitting. We acknowledge that we should probably have been clearer on the description of the model training in the original article. Bayesian optimization for SVM training with a maximum of 30 objective evaluations was used. We agree that nonlinear dimension reduction techniques such as kernel Principal Component Analysis may have better conserved data structure and improved results, but we felt this was unnecessary given the strong performance of the current method. We will reevaluate this, should the current model not maintain strong performance in our prospective validation cohort.
Lastly, we also agree that the use of explainable Artificial Intelligence tools such as heatmaps may be useful to identify specific image features that the model focuses on and understands the classification performance of. As noted in our article [1], we suspect the early signs of Hypoxic-Ischemic Brain Injury may be due to subtle changes in gray–white matter differentiation or brain edema that evade the human eye. Therefore, the output from explainable artificial intelligence tools may not necessarily be easy to interpret to a human reader. Our future plans, once we have a collected a larger prospective data set, do include investigating visualization methods, such as Gradient-Weighted Class Activation Mapping, to observe when the deep learning indicates subtle brain changes.
Reference
Mansour A, Fuhrman JD, Ammar FE, Loggini A, Davis J, Lazaridis C, et al. Machine learning for early detection of hypoxic-ischemic brain injury after cardiac arrest. Neurocrit Care. 2021. https://doi.org/10.1007/s12028-021-01405-y.
Funding
Partial funding for this work was provided by the National Institutes of Health S10-OD025081 and S10-RR021039 awards, and by the National Center for Advancing Translational Sciences of the National Institutes of Health through grant number 5UL1TR002389-02 that funds the Institute for Translational Medicine.
Author information
Authors and Affiliations
Contributions
Ali Mansour: draft and revision of the article for content. Jordan D. Fuhrman: draft and revision of the article for content. Fernando D. Goldenberg: draft and revision of the article for content. Maryellen L. Giger: draft and revision of the article for content. The final manuscript was approved by all authors.
Corresponding author
Ethics declarations
Conflicts of interest
Both Jordan Fuhrman and Maryellen Giger are funded in part by the C3.AI Digital Transformation Institute, the University of Chicago Department of Radiology Pilot Fund, and NIBIB COVID-19 Contract 75N92020D00021. They are also supported, in part, by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under grant number T32 EB002103. All other authors reported no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is a response to the Letter to the Editor available at https://doi.org/10.1007/s12028-022-01526-y. This article is related to the original article available at https://doi.org/10.1007/s12028-021-01405-y.
Rights and permissions
About this article
Cite this article
Mansour, A., Fuhrman, J.D., Goldenberg, F.D. et al. Comment on “Machine Learning for Early Detection of Hypoxic-Ischemic Brain Injury After Cardiac Arrest” Submitted by Noah Salomon Molinski et al.. Neurocrit Care 37, 365–366 (2022). https://doi.org/10.1007/s12028-022-01527-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12028-022-01527-x