Skip to main content

A Deep Convolutional Neural Network for Annotation of Magnetic Resonance Imaging Sequence Type


The explosion of medical imaging data along with the advent of big data analytics has launched an exciting era for clinical research. One factor affecting the ability to aggregate large medical image collections for research is the lack of infrastructure for automated data annotation. Among all imaging modalities, annotation of magnetic resonance (MR) images is particularly challenging due to the non-standard labeling of MR image types. In this work, we aimed to train a deep neural network to annotate MR image sequence type for scans of brain tumor patients. We focused on the four most common MR sequence types within neuroimaging: T1-weighted (T1W), T1-weighted post-gadolinium contrast (T1Gd), T2-weighted (T2W), and T2-weighted fluid-attenuated inversion recovery (FLAIR). Our repository contains images acquired using a variety of pulse sequences, sequence parameters, field strengths, and scanner manufacturers. Image selection was agnostic to patient demographics, diagnosis, and the presence of tumor in the imaging field of view. We used a total of 14,400 two-dimensional images, each visualizing a different part of the brain. Data was split into train, validation, and test sets (9600, 2400, and 2400 images, respectively) and sets consisted of equal-sized groups of image types. Overall, the model reached an accuracy of 99% on the test set. Our results showed excellent performance of deep learning techniques in predicting sequence types for brain tumor MR images. We conclude deep learning models can serve as tools to support clinical research and facilitate efficient database management.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. Commission, M. P. A, Book AD, others: Healthcare spending and the Medicare program. Washington, DC: MedPAC, 2012

    Google Scholar 

  2. Smith-Bindman R, Miglioretti DL, Larson EB: Rising use of diagnostic medical imaging in a large integrated health system. Health Aff 27:1491–1502, 2008

    Article  Google Scholar 

  3. Nitz WR: MR imaging: Acronyms and clinical applications. Eur. Radiol. 9:979–997, 1999

    CAS  Article  Google Scholar 

  4. MRI sequences acronyms. IMAIOS Available at: (Accessed: 30th April 2019)

  5. Enlarge, C. T. T. O. & Ge, G. GRE Acronyms.

  6. Swanson KR, Rockne RC, Claridge J, Chaplain MA, Alvord EC, Anderson ARA: Quantifying the role of angiogenesis in malignant progression of gliomas: in silico modeling integrates imaging and histology. Cancer Res. 71:7366–7375, 2011

    CAS  Article  Google Scholar 

  7. Jackson PR, Juliano J, Hawkins-Daarud A, Rockne RC, Swanson KR: Patient-specific mathematical neuro-oncology: Using a simple proliferation and invasion tumor model to inform clinical practice. Bull. Math. Biol. 77:846–856, 2015

    Article  Google Scholar 

  8. Hawkins-Daarud A, Rockne R, Corwin D, Anderson ARA, Kinahan P, Swanson KR: In silico analysis suggests differential response to bevacizumab and radiation combination therapy in newly diagnosed glioblastoma. J. R. Soc. Interface 12:20150388, 2015

    Article  Google Scholar 

  9. Rockne RC. et al.: A patient-specific computational model of hypoxia-modulated radiation resistance in glioblastoma using 18F-FMISO-PET. J. R. Soc. Interface 12, 2015

  10. Alfonso JCL, Talkenberger K, Seifert M, Klink B, Hawkins-Daarud A, Swanson KR, Hatzikirou H, Deutsch A: The biology and mathematical modelling of glioma invasion: a review. J. R. Soc. Interface 14:20170490, 2017

    Article  Google Scholar 

  11. Rayfield CA, Grady F, de Leon G, Rockne R, Carrasco E, Jackson P, Vora M, Johnston SK, Hawkins-Daarud A, Clark-Swanson KR, Whitmire S, Gamez ME, Porter A, Hu L, Gonzalez-Cuyar L, Bendok B, Vora S, Swanson KR: Distinct phenotypic clusters of glioblastoma growth and response kinetics predict survival. JCO Clinical Cancer Informatics 2:1–14, 2018

    Article  Google Scholar 

  12. Randall EC, Emdal KB, Laramy JK, Kim M, Roos A, Calligaris D, Regan MS, Gupta SK, Mladek AC, Carlson BL, Johnson AJ, Lu FK, Xie XS, Joughin BA, Reddy RJ, Peng S, Abdelmoula WM, Jackson PR, Kolluri A, Kellersberger KA, Agar JN, Lauffenburger DA, Swanson KR, Tran NL, Elmquist WF, White FM, Sarkaria JN, Agar NYR: Integrated mapping of pharmacokinetics and pharmacodynamics in a patient-derived xenograft model of glioblastoma. Nat. Commun. 9:4904, 2018

    Article  Google Scholar 

  13. Swanson KR, Rostomily RC, Alvord, Jr EC: A mathematical modelling tool for predicting survival of individual patients following resection of glioblastoma: A proof of principle. Br. J. Cancer 98:113–119, 2008

    CAS  Article  Google Scholar 

  14. Massey SC, Rockne RC, Hawkins-Daarud A, Gallaher J, Anderson ARA, Canoll P, Swanson KR: Simulating PDGF-Driven Glioma Growth and Invasion in an Anatomically Accurate Brain Domain. Bull. Math. Biol. 80:1292–1309, 2018

    CAS  Article  Google Scholar 

  15. Johnston SK. et al.: ENvironmental Dynamics Underlying Responsive Extreme Survivors (ENDURES) of Glioblastoma: a Multi-disciplinary Team-based. Multifactorial Analytical Approach. bioRxiv 461236, 2018. doi:

  16. Barnholtz-Sloan JS, Swanson KR: Sex differences in GBM revealed by analysis of patient imaging, transcriptome, and survival data. Sci. Transl. Med, 2019

  17. Massey SC et al.: Image-based metric of invasiveness predicts response to adjuvant temozolomide for primary glioblastoma. bioRxiv 509281, 2019. doi:

  18. Brown RW et al.: Magnetic Resonance Imaging: Physical Principles and Sequence Design. (John Wiley & Sons, 2014

  19. Cha S: Neuroimaging in neuro-oncology. Neurotherapeutics 6:465–477, 2009

    CAS  Article  Google Scholar 

  20. Armstrong TS, Cohen MZ, Weinberg J, Gilbert MR: Imaging techniques in neuro-oncology. Semin. Oncol. Nurs. 20:231–239, 2004

    Article  Google Scholar 

  21. Krizhevsky A, Sutskever I, Hinton GE: ImageNet classification with deep convolutional neural networks. in Advances in Neural Information Processing Systems 25 (eds. Pereira, F., Burges, C. J. C., Bottou, L. & Weinberger, K. Q.) 1097–1105 (Curran Associates, Inc., 2012

  22. Mohsen H, El-Dahshan E-SA, El-Horbaty E-SM, Salem A-BM: Classification using deep learning neural networks for brain tumors. Future Computing and Informatics Journal 3:68–71, 2018

    Article  Google Scholar 

  23. Huynh BQ, Li H, Giger ML: Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. Journal of Medical Imaging 3:034501, 2016

    Article  Google Scholar 

  24. Chen Y-JY-J, Hua K-L, Hsu C-H, Cheng W-H, Hidayati SC: Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets and Therapy, 2015.

  25. Simonyan K, Zisserman A: Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv [cs.CV], 2014

  26. Chollet F, Others. Keras. 2015

  27. Abadi, M. et al.: Tensorflow: A system for large-scale machine learning. in 12th ${USENIX} Symposium on Operating Systems Design and Implementation ({OSDI}$ 16) 265–283, 2016

  28. Kingma DP, Adam BJ: A method for stochastic optimization. arXiv [cs.LG], 2014

  29. Pedregosa F et al.: Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 12:2825–2830, 2011

    Google Scholar 

  30. Gong, Y., Jia, Y., Leung, T., Toshev, A. & Ioffe, S. Deep convolutional ranking for multilabel image annotation. arXiv [cs.CV], 2013

  31. Wu, F., Wang Z., Zhang Z., Yang Y., Luo J., Zhu W., Zhuang Y. Weakly Semi-Supervised Deep Learning for Multi-Label Image Annotation. IEEE Transactions on Big Data 1, 109–122 (2015), 109, 122.

  32. Wu J, Yu Y, Huang C, Yu K: Deep multiple instance learning for image classification and auto-annotation. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3460–3469, 2015

  33. Murthy VN, Maji S, Manmatha R: Automatic Image Annotation Using Deep Learning Representations. in Proceedings of the 5th ACM on International Conference on Multimedia Retrieval 603–606 (ACM), 2015.

  34. Ojha U, Adhikari U, Singh DK: Image annotation using deep learning: A review. 2017 International Conference on Intelligent Computing and Control (I2C2), 2017.

  35. Hamm CA, Wang CJ, Savic LJ, Ferrante M, Schobert I, Schlachter T, Lin MD, Duncan JS, Weinreb JC, Chapiro J, Letzen B: Deep learning for liver tumor diagnosis part I: development of a convolutional neural network classifier for multi-phasic MRI. Eur. Radiol. 29:3338–3347, 2019.

    Article  PubMed  Google Scholar 

  36. Hermessi, H., Mourali, O. & Zagrouba, E. Deep feature learning for soft tissue sarcoma classification in MR images via transfer learning. Expert Systems with Applications 120, (2019), 116, 127.

  37. Margeta J, Criminisi A, Cabrera Lozoya R, Lee DC, Ayache N: Fine-tuned convolutional neural nets for cardiac MRI acquisition plane recognition. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 5:339–349, 2017

    Google Scholar 

Download references


This study received support from the James S. McDonnell Foundation, the Ivy Foundation, the Mayo Clinic, the Zicarelli Foundation, and the NIH (R01 NS060752, R01 CA164371, U54 CA210180, U54 CA143970, U54 CA193489, U01 CA220378).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Sara Ranjbar.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Statement of Impact

To the best of our knowledge, no previous work has focused on automatic annotation of MR image types. This form of classification can be highly useful for building large-scale imaging repositories that can receive submissions from heterogeneous data sources.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ranjbar, S., Singleton, K.W., Jackson, P.R. et al. A Deep Convolutional Neural Network for Annotation of Magnetic Resonance Imaging Sequence Type. J Digit Imaging 33, 439–446 (2020).

Download citation

  • Published:

  • Issue Date:

  • DOI:


  • Artificial intelligence
  • Deep learning
  • Magnetic resonance imaging
  • Sequence type
  • Automated annotation
  • Image database