Abstract
Collecting and curating large medical-image datasets for deep neural network (DNN) algorithm development is typically difficult and resource-intensive. While transfer learning (TL) decreases reliance on large data collections, current TL implementations are tailored to two-dimensional (2D) datasets, limiting applicability to volumetric imaging (e.g., computed tomography). Targeting performance enhancement of a DNN algorithm based on a small image dataset, we assessed incremental impact of 3D-to-2D projection methods, one supporting novel data augmentation (DA); photometric grayscale-to-color conversion (GCC); and/or TL on training of an algorithm from a small coronary computed tomography angiography (CCTA) dataset (200 examinations, 50% with atherosclerosis and 50% atherosclerosis-free) producing 245 diseased and 1127 normal coronary arteries/branches. Volumetric CCTA data was converted to a 2D format creating both an Aggregate Projection View (APV) and a Mosaic Projection View (MPV), supporting DA per vessel; both grayscale and color-mapped versions of each view were also obtained. Training was performed both without and with TL, and algorithm performance of all permutations was compared using area under the receiver operating characteristics curve. Without TL, APV performance was 0.74 and 0.87 on grayscale and color images, respectively, compared to 0.90 and 0.87 for MPV. With TL, APV performance was 0.78 and 0.88 on grayscale and color images, respectively, compared with 0.93 and 0.91 for MPV. In conclusion, TL enhances performance of a DNN algorithm from a small volumetric dataset after proposed 3D-to-2D reformatting, but additive gain is achieved with application of either GCC to APV or the proposed novel MPV technique for DA.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Krizhevsky A, Sutskever I, Hinton GE: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th international conference on neural information processing systems 1, 2012, pp. 1097–1105
Perez L, Wang J: The effectiveness of data augmentation in image classification using deep learning. CoRR abs, arXiv:1712.04621, 2017.
Prasad S, Kumar P, Sinha KP: Grayscale to color map transformation for efficient image analysis on low processing devices. In: El-Alfy ES., Thampi S., Takagi H., Piramuthu S., Hanne T. (eds) Advances in Intelligent Informatics. Advances in Intelligent Systems and Computing, vol 320. Springer, cham, 2015, https://doi.org/10.1007/978-3-319-11218-3_2
Ng HW, Nguyen VD, Vonikakis V, Winkler S: Deep learning for emotion recognition on small datasets using transfer learning. Proceedings of ACM International conference on multimodal interaction, 443–449, 2015.
Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35:1285–1298, 2016
Su H, Maji S, Kalogerakis E, Learned-Miller E: Multi-view convolutional neural networks for 3D shape recognition. IEEE International Conference on Computer Vision (ICCV), 945–953, 2015.
Kanezaki A, Matsushita Y, Nishida Y: RotationNet: joint object categorization and pose estimation using multi views from unsupervised viewpoints. International Conference on Computer Vision and Pattern Recognition (CVPR), arXiv:1603.06208, 2018.
White RD, Erdal BS, Bigelow MT, Demirer M, Galizia MS, Gupta V, Carpenter JL, Candemir S, Dikici E, O'Donnell TP, Halabi AH, Prevedello LM et al. Augmented intelligence to facilitate exclusion of coronary atherosclerosis on CCTA during emergency department chest-pain presentations: algorithm development. Radiology, Cardiothoracic Imaging (Submitted).
Litt HI, Gatsonis C, Snyder B, Singh H, Miller CD, Entrikin DW, Leaming JM, Gavin LJ, Pacella CB, Hollander JE: CT angiography for safe discharge of patients with possible acute coronary syndromes. N Engl J Med 366:1393–1403, 2012
Ghoshhajra BB, Engel LC, Major GP, Goehler A, Techasith T, Verdini D, Do S, Liu B, Li X, Sala M, Kim MS, Blankstein R, Prakash P, Sidhu MS, Corsini E, Banerji D, Wu D, Abbara S, Truong Q, Brady TJ, Hoffmann U, Kalra M: Evolution of coronary computed tomography radiation dose reduction at a tertiary referral center. Am J Med 125:764–772, 2012
Lubbers MM, Dedic A, Kurata A, Dijkshoorn M, Schaap J, Lammers J, Lamfers EJ, Rensing BJ, Braam RL, Nathoe HM, Post JC, Rood PP, Schultz CJ, Moelker A, Ouhlous M, van Dalen BM, Boersma E, Nieman K: Round-the-clock performance of coronary CT angiography for suspected acute coronary syndrome: results from the BEACON trial. Eur Radiol 28:2169–2175, 2018
Demirer M, Candemir S, Bigelow M, Yu SM, Gupta V, Prevedello LM et al.: A user interface for optimizing radiologist engagement in image-data curation for artificial intelligence. Radiology, 2019 (In Press)
Siemens Healthineers, Erlangen, Germany. Available at https://www.healthcare.siemens.com/computed-tomography/clinical-imaging-solutions/ct-cardio-vascular-engine Accessed 14 April 2018
Siemens Research. Princeton, USA. Available at http://www.usa.siemens.com/en/about_us/research/research_dev/imaging_visualization/imaging_visualization_publications_2009.htm. Accessed 3 Jan 2019
Zheng Y, Tek H, Funka-Lea G: Robust and accurate coronary artery centerline extraction in CTA by combining model-driven and data-driven approaches. Med Image Comput Comput Assist Interv 16:74–81, 2013
Dalrymple NC, Prasad SR, Freckleton MW, Chintapalli KN: Informatics in radiology (infoRAD): Introduction to the language of three-dimensional imaging with multidetector CT. RadioGraphics 25:1409–1428, 2005. https://doi.org/10.1148/rg.255055044
Dobbin KK, Simon RM: Optimally splitting cases for training and testing high dimensional classifiers. BMC Med Genomics 4(31), 2011. https://doi.org/10.1186/1755-8794-4-31
ArcMap: Understanding the mosaicking rules for a mosaic dataset. ESRI. Available at http://desktop.arcgis.com/en/arcmap/latest/manage-data/raster-and-images/understanding-the-mosaicking-rules-for-a-mosaic-dataset.htm. Accessed 18 November 2018.
Zickler T, Mallick SP, Kriegman DJ, Belhumeur PN: Color subspaces as photometric invariants. International Journal of Computer Vision 79:13–30, 2008
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z: Rethinking the inception architecture for computer vision. Proceedings of the IEEE conference on computer vision and pattern recognition. 2818–2826. arXiv:1512.00567, 2016.
Dahl GE, Sainath TN, Hinton GE: Improving deep neural networks for LVCSR using rectified linear units and dropout. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 8609–8613, 2013.
Chollet F: User experience design for APIs. The Keras Blog; Available at https://blog.keras.io/user-experience-design-for-apis.html. Accessed 18 November 2018.
Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray DG, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X: TensorFlow: a system for large-scale machine learning. USENIX symposium on operating systems design and implementation 16:265–283, 2016. arXiv:1605.08695
Bottou L: Large-scale machine learning with stochastic gradient descent. Proceedings of COMPSTAT 177–186, 2010.
Zreik M, van Hamersvelt RW, Wolterink JM, Leiner T, Viergever MA, Isgum I: Automatic detection and characterization of coronary artery plaque and stenosis using a recurrent convolutional neural network in coronary CT angiography. Available at https://openreview.net/forum?id=BJenxxhof. arXiv:1804.04360v1. Accessed 18 November 2018.
Loewke KE, Camarillo DB, Jobst CA, Salisbury JK: Real-time image mosaicing for medical applications. Stud Health Technol Inform 125:304–309, 2007
Funding
This project was partially funded by:
1. Donation from the Edward J. DeBartolo, Jr. Family.
2. Master Research Agreement with Siemens Healthineers.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
All CCTA image-dataset utilization was retrospective, performed locally under Institutional Review Board approval (including HIPAA compliance) with the waiver of patient consent.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gupta, V., Demirer, M., Bigelow, M. et al. Performance of a Deep Neural Network Algorithm Based on a Small Medical Image Dataset: Incremental Impact of 3D-to-2D Reformation Combined with Novel Data Augmentation, Photometric Conversion, or Transfer Learning. J Digit Imaging 33, 431–438 (2020). https://doi.org/10.1007/s10278-019-00267-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-019-00267-3