Skip to main content

Advertisement

SpringerLink
Log in

Fully automatic cross-modality localization and labeling of vertebral bodies and intervertebral discs in 3D spinal images

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

We present a cross-modality and fully automatic pipeline for labeling of intervertebral discs and vertebrae in volumetric data of the lumbar and thoracolumbar spine. The main goal is to provide an algorithm that is applicable to a wide range of different sequences and acquisition protocols, like T1- and T2- weighted MR scans, MR Dixon data, and CT scans. This requires that the learned models generalize without retraining to modalities and scans with unseen image contrasts.

Methods

We address this challenge by automatically localizing the sacral region combining local entropy-optimized texture models with convolutional neural networks. For subsequent labeling, local three-disc entropy models are matched iteratively to the spinal column. Every model-matched position is further refined by an intensity-based template-matching approach, based solely on the reduced intensity scale provided by the entropy models.

Results

We evaluated our method on 161 publicly available scans, acquired on various scanners. We showed that our method can deal with a wide range of different MR protocols as well as with CT data. We achieved a sacrum detection rate of 93.6%. Mean center accuracies ranged from 2.5 ± 1.5 to 5.7 ± 3.8 mm for the different sets of scans.

Conclusion

We present a novel spine labeling framework that is applicable to a highly heterogeneous set of scans without retraining of the method. Our approach achieves high sacrum localization accuracy and shows promising labeling results. To the best of our knowledge, an algorithm able to deal with such a diverse set of MR and CT scans has not yet been presented in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://spineweb.digitalimaginggroup.ca/spineweb/.

References

  1. Agur AMR, Dalley AF (2012) Grant’s atlas of anatomy, 13th edn. Lippincott Williams & Wilkins, Philadelphia

    Google Scholar 

  2. Alomari R, Corso J, Chaudhary V (2011) Labeling of lumbar discs using both pixel- and object-level features with a two-level probabilistic model. IEEE Trans Med Imaging 30(1):1–10

    Article  PubMed  Google Scholar 

  3. Alomari RS, Ghosh S, Koh J, Chaudhary V (2015) Vertebral column localization, labeling, and segmentation. Spinal Imaging Image Anal Springer 18:193–229

    Google Scholar 

  4. Cai Y, Osman S, Sharma M, Landis M, Li S (2015) Multi-modality vertebra recognition in arbitrary views using 3D deformable hierarchical model. IEEE Trans Med Imaging 34(8):1676–1693

    Article  PubMed  Google Scholar 

  5. Cai Y, Landis M, Laidley DT, Kornecki A, Lum A, Li S (2016) Multi-modal vertebrae recognition using transformed deep convolution network. Comput Med Imaging Graph 51:11–19

    Article  PubMed  Google Scholar 

  6. Chen C, Belavy D, Zheng G (2014) 3D Intervertebral disc localization and segmentation from MR images by data-driven regression and classification. In: Machine learning in medical imaging, vol 8679, Springer, pp 50–58

  7. Chen C, Belavy D, Yu W, Chu C, Armbrecht G, Bansmann M, Felsenberg D, Zheng G (2015a) Localization and segmentation of 3D intervertebral discs in MR images by data driven estimation. IEEE Trans Med Imaging 34(8):1719–1729

    Article  PubMed  Google Scholar 

  8. Chen H, Shen C, Qin J, Ni D, Shi L, Cheng JC, Heng PA (2015b) Automatic localization and identification of vertebrae in spine CT via a joint learning model with deep neural networks. In: Proceedings of MICCAI, Springer, vol 9349, pp 515–522

  9. Chen H, Dou Q, Wang X, Qin J, Cheng JCY, Heng PA (2016) 3D Fully convolutional networks for intervertebral disc localization and segmentation. In: Proceedings of international conference on medical imaging and augmented reality, vol 9805, Springer, pp 375–382

  10. Chu C, Belavy DL, Armbrecht G, Bansmann M, Felsenberg D, Zheng G (2015) Fully automatic localization and segmentation of 3D vertebral bodies from CT/MR images via a learning-based method. PLoS ONE 10(11):e0143,327

    Article  CAS  Google Scholar 

  11. Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models—their training and application. Comput Vis Image Underst 61(1):38–59

    Article  Google Scholar 

  12. Daenzer S, Freitag S, von Sachsen S, Steinke H, Groll M, Meixensberger J, Leimert M (2014) VolHOG: a volumetric object recognition approach based on bivariate histograms of oriented gradients for vertebra detection in cervical spine MRI. Med Phys 41(8):082,305

    Article  Google Scholar 

  13. Dieleman S, Schlüter J, Raffel C, Olson E, Sønderby SK, Nouri D, Maturana D, Thoma M, Battenberg E, Kelly J, Fauw JD, Heilman M, de Almeida DM, McFee B, Weideman H, Takács G, de Rivaz P, Crall J, Sanders G, Rasul K, Liu C, French G, Degrave J (2015) Lasagne: First release. https://doi.org/10.5281/zenodo.27878

  14. Forsberg D, Sjöblom E, Sunshine JL (2017) Detection and labeling of vertebrae in MR images using deep learning with clinical annotations as training data. J Dig Imaging 30(4):406–412

    Article  Google Scholar 

  15. Gilad I, Nissan M (1985) Sagittal evaluation of elemental geometrical dimensions of human vertebrae. J Anat 143:115–120

    CAS  PubMed  PubMed Central  Google Scholar 

  16. Glocker B, Feulner J, Criminisi A, Haynor DR, Konukoglu E (2012) Automatic localization and identification of vertebrae in arbitrary field-of-view CT scans. In: Proceedings of MICCAI, vol 7512, Springer, pp 590–598

  17. Glocker B, Zikic D, Konukoglu E, Haynor DR, Criminisi A (2013) Vertebrae localization in pathological spine CT via dense classification from sparse annotations. In: Proceedings of MICCAI, vol 8150, Springer, pp 262–270

  18. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, http://www.deeplearningbook.org

  19. Heinrich MP, Oktay O (2016) Accurate intervertebral disc localisation and segmentation in MRI using vantage point hough forests and multi-atlas fusion. In: Computational methods and clinical applications for spine imaging, Springer, pp 77–84

  20. Hojjat SP, Ayed I, Garvin GJ, Punithakumar K (2017) Spine labeling in MRI via regularized distribution matching. Int J Comput Assist Radiol Surg 12(11):1911–1922

    Article  PubMed  Google Scholar 

  21. Jamaludin A, Lootus M, Kadir T, Zisserman A (2016) Automatic intervertebral discs localization and segmentation: a vertebral approach. Comput Methods Clin Appl Spine Imaging Springer 9402:97–103

    Article  Google Scholar 

  22. Ji X, Zheng G, Liu L, Ni D (2016) Fully automatic localization and segmentation of intervertebral disc from 3D multi-modality MR images by regression forest and CNN. Comput Methods Clin Appl Spine Imaging Springer 10182:92–101

    Article  Google Scholar 

  23. Kelm B, Wels M, Zhou S, Seifert S, Suehling M, Zheng Y, Comaniciu D (2013) Spine detection in CT and MR using iterated marginal space learning. Med Image Anal 17(8):1283–1292

    Article  Google Scholar 

  24. Krizhevsky A (2009) Learning multiple layers of features from tiny images. Tech. rep, University of Toronto

  25. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  26. Li X, Dou Q, Chen H, Fu CW, Heng PA (2016) Multi-scale and modality dropout learning for intervertebral disc localization and segmentation. Comput Methods Clin Appl Spine Imaging Springer 10182:85–91

    Article  Google Scholar 

  27. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JA, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  PubMed  Google Scholar 

  28. Lootus M, Kadir T, Zisserman A (2014) Vertebrae detection and labelling in lumbar MR images. In: Computational Methods and clinical applications for spine imaging, vol 17, Springer, pp 219–230

  29. Major D, Hladůvka J, Schulze F, Bühler K (2013) Automated landmarking and labeling of fully and partially scanned spinal columns in CT images. Med Image Anal 17(8):1151–1163

    Article  PubMed  Google Scholar 

  30. Oktay AB, Akgul YS (2013) Simultaneous localization of lumbar vertebrae and intervertebral discs with SVM-based MRF. IEEE Trans Biomed Eng 60(9):2375–2383

    Article  PubMed  Google Scholar 

  31. Panjabi MM, Takata K, Goel V, Federico D, Oxland T, Duranceau J, Krag M (1991) Thoracic human vertebrae: quantitative three-dimensional anatomy. Spine 16(8):888–901

    Article  CAS  PubMed  Google Scholar 

  32. Panjabi MM, Goel V, Oxland T, Takata K, Duranceau J, Krag M, Price M (1992) Human lumbar vertebrae: quantitative three-dimensional anatomy. Spine 17(3):299–306

    Article  CAS  PubMed  Google Scholar 

  33. Rak M, Tönnies KD (2016) On computerized methods for spine analysis in MRI: a systematic review. Int J Comput Assist Radiol Surg 11(8):1445–1465

    Article  PubMed  Google Scholar 

  34. Schmidt S, Kappes J, Bergtholdt M, Pekar V, Dries S, Bystrov D, Schnörr C (2007) Spine detection and labeling using a parts-based graphical model. In: Proceedings of international conference on information processing in medical imaging, vol 4584, Springer, pp122–133

  35. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    Google Scholar 

  36. Štern D, Likar B, Pernuš F, Vrtovec T (2009) Automated detection of spinal centrelines, vertebral bodies and intervertebral discs in CT and MR images of lumbar spine. Phys Med Biol 55(1):247–264

    Article  Google Scholar 

  37. Suzani A, Seitel A, Liu Y, Fels S, Rohling RN, Abolmaesumi P (2015) Fast automatic vertebrae detection and localization in pathological CT scans—a deep learning approach. In: Proceedings of MICCAI vol 9351, Springer, pp 678–686

  38. Vrtovec T, Yao J, Glocker B, Klinder T, Frangi A, Zheng G, Li S (eds) (2015) Computational methods and clinical applications for spine imaging, Springer, chap challenge (automatic intervertebral disc localization and segmentation from 3D T2 MRI data), pp 107–158

  39. Wang Z, Zhen X, Tay K, Osman S, Romano W, Li S (2015) Regression segmentation for M3 spinal images. IEEE Trans Med Imaging 34(8):1640–1648

    Article  PubMed  Google Scholar 

  40. Wimmer M, Major D, Novikov AA, Bühler K (2016) Local entropy-optimized texture models for semi-automatic spine labeling in various MRI protocols. In: Proceedings of IEEE 13th international symposium on biomedical imaging, IEEE, pp 155–159

  41. Zambal S, Bühler K, Hladůvka J (2008) Entropy-optimized texture models. In: Proceedings of MICCAI, vol 5242, Springer, pp 213–221

  42. Zhan Y, Jian B, Maneesh D, Zhou XS (2015) Cross-modality vertebrae localization and labeling using learning-based approaches. In: Spinal imaging and image analysis, Springer, pp 301–322

  43. Zheng G, Chu C, Belavỳ DL, Ibragimov B, Korez R, Vrtovec T, Hutt H, Everson R, Meakin J, Andrade IL, Glocker B, Chen H, Dou Q, Heng PA, Wang C, Forsberg D, Neubert A, Fripp J, Urschler M, Štern D, Wimmer M, Novikov AA, Cheng H, Armbrecht G, Felsenberg D, Li S (2017) Evaluation and comparison of 3D intervertebral disc localization and segmentation methods for 3D T2 MR data: a grand challenge. Med Image Anal 35:327–344

    Article  PubMed  Google Scholar 

  44. Zukić D, Vlasák A, Egger J, Hořínek D, Nimsky C, Kolb A (2014) Robust detection and segmentation for diagnosis of vertebral diseases using routine MR images. Comput Graph Forum 33(6):190–204

    Article  Google Scholar 

Download references

Acknowledgements

Thanks go to our project partner AGFA HealthCare for providing data and valuable input.

Funding

VRVis is funded by BMVIT, BMDW, Styria, SFG, and Vienna Business Agency in the scope of COMET—Competence Centers for Excellent Technologies (854174) which is managed by FFG.

Author information

Authors and Affiliations

  1. VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH, Vienna, Austria

    Maria Wimmer, David Major, Alexey A. Novikov & Katja Bühler

Authors
  1. Maria Wimmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Major
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexey A. Novikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Katja Bühler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria Wimmer.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

For this type of study, formal consent is not required.

Informed consent

This article does not contain patient data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wimmer, M., Major, D., Novikov, A.A. et al. Fully automatic cross-modality localization and labeling of vertebral bodies and intervertebral discs in 3D spinal images. Int J CARS 13, 1591–1603 (2018). https://doi.org/10.1007/s11548-018-1818-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-018-1818-3

Keywords

Search

Navigation