Abstract
The cited article reports on a convolutional neural network trained to predict response to neoadjuvant chemotherapy from pre-treatment breast MRI scans. The proposed algorithm attains impressive performance on the test dataset with a mean Area Under the Receiver-Operating Characteristic curve of 0.98 and a mean accuracy of 88%. In this letter, I raise concerns that the reported results can be explained by inadvertent data leakage between training and test datasets. More precisely, I conjecture that the random split of the full dataset in training and test sets did not occur on a patient level, but rather on the level of 2D MRI slices. This allows the neural network to “memorize” a patient’s anatomy and their treatment outcome, as opposed to discovering useful features for treatment response prediction. To provide evidence for these claims, I present results of similar experiments I conducted on a public breast MRI dataset, where I demonstrate that the suspected data leakage mechanism closely reproduces the results reported on in the cited work.
Reference
Saha, A. et al. A machine learning approach to radiogenomics of breast cancer: a study of 922 subjects and 529 DCE-MRI features. Br. J. Cancer 119, 508–516 (2018).
Funding
The author declares that no funds, grants, or other support were received during the preparation of this manuscript.
Author information
Authors and Affiliations
Contributions
All work involved in the preparation of this manuscript was carried out by the author.
Corresponding author
Ethics declarations
Ethics Approval
No ethics approval is required for a letter of this kind.
Consent to Participate
Not applicable.
Consent to Publish
Not applicable.
Competing Interests
The author has no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Brunekreef, J. Letter to the Editor Regarding Article “Prior to Initiation of Chemotherapy, Can We Predict Breast Tumor Response? Deep Learning Convolutional Neural Networks Approach Using a Breast MRI Tumor Dataset”. J Digit Imaging. Inform. med. (2024). https://doi.org/10.1007/s10278-024-01129-3
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10278-024-01129-3