Skip to main content
SpringerLink
Log in

MBNet: A Multi-task Deep Neural Network for Semantic Segmentation and Lumbar Vertebra Inspection on X-Ray Images

  • Conference paper
  • First Online:
Computer Vision – ACCV 2020 (ACCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12626))

Included in the following conference series:

Abstract

Deep learning methods, especially multi-task learning with CNNs, have achieved good results in many fields of computer vision. Semantic segmentation and shape detection of lumbar vertebrae, sacrum, and femoral heads from clinical X-ray images are important and challenging tasks. In this paper, we propose a multi-task deep neural network, MBNet. It is developed based on our new multi-path convolutional neural network, BiLuNet, for semantic segmentation on X-ray images. Our MBNet has two branches, one is for semantic segmentation of lumbar vertebrae, sacrum, and femoral heads. It shares the main features with the second branch to learn and classify by supervised learning. The output of the second branch is to predict the inspected values for lumbar vertebra inspection. These networks are capable of performing the two tasks with very limited training data. We collected our dataset and annotated it by doctors for model training and performance evaluation. Compared to the state-of-the-art methods, our BiLuNet model provides better mIoUs with the same training data. The experimental results have demonstrated the feasibility of our MBNet for semantic segmentation of lumbar vertebrae, as well as the parameter prediction for the doctors to perform clinical diagnosis of low back pains. Code is available at: https://github.com/LuanTran07/BiLUnet-Lumbar-Spine.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The full training and testing code is open source at https://github.com/LuanTran07/BiLUnet-Lumbar-Spine.

References

  1. Sekuboyina, A., et al.: Btrfly Net: vertebrae labelling with energy-based adversarial learning of local spine prior. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018, Part IV. LNCS, vol. 11073, pp. 649–657. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_74

    Chapter  Google Scholar 

  2. Han, Z., Wei, B., Leung, S., Chung, J., Li, S.: Towards automatic report generation in spine radiology using weakly supervised framework. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018, Part IV. LNCS, vol. 11073, pp. 185–193. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_22

    Chapter  Google Scholar 

  3. Zhang, Y., Miao, S., Mansi, T., Liao, R.: Task driven generative modeling for unsupervised domain adaptation: application to X-ray image segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018, Part II. LNCS, vol. 11071, pp. 599–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_67

    Chapter  Google Scholar 

  4. Horng, M., Kuok, C., Fu, M., Lin, C., Sun, Y.: Cobb angle measurement of spine from X-ray images using convolutional neural network. Comp. Math. Methods Med. 2019, 6357171:1–6357171:18 (2019)

    MATH  Google Scholar 

  5. Kalichman, L., Kim, D.H., Li, L., Guermazi, A., Hunter, D.J.: Computed tomography-evaluated features of spinal degeneration: prevalence, intercorrelation, and association with self-reported low back pain. Spine J. Off. J. North Am. Spine Soc. 10(3), 200–208 (2010)

    Article  Google Scholar 

  6. Jamaludin, A., et al.: Automation of reading of radiological features from magnetic resonance images (MRIs) of the lumbar spine without human intervention is comparable with an expert radiologist. Eur. Spine J. 26, 1374–1383 (2017)

    Article  Google Scholar 

  7. Fu, M., et al.: Inter-rater and intra-rater agreement of magnetic resonance imaging findings in the lumbar spine: significant variability across degenerative conditions. Spine J. 14, 2442–2448 (2014)

    Article  Google Scholar 

  8. Tang, M., Valipour, S., Zhang, Z., Cobzas, D., Jagersand, M.: A deep level set method for image segmentation. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 126–134. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_15

    Chapter  Google Scholar 

  9. Ebrahimi, S., Angelini, E., Gajny, L., Skalli, W.: Lumbar spine posterior corner detection in X-rays using haar-based features. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 180–183 (2016)

    Google Scholar 

  10. Moeskops, P., et al.: Deep learning for multi-task medical image segmentation in multiple modalities. CoRR abs/1704.03379 (2017)

    Google Scholar 

  11. Ebner, M., et al.: An automated localization, segmentation and reconstruction framework for fetal brain MRI. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018, Part I. LNCS, vol. 11070, pp. 313–320. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_36

    Chapter  Google Scholar 

  12. Al Arif, S.M.M.R., Knapp, K., Slabaugh, G.: SPNet: shape prediction using a fully convolutional neural network. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018, Part I. LNCS, vol. 11070, pp. 430–439. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_49

    Chapter  Google Scholar 

  13. Li, X., et al.: 3D multi-scale FCN with random modality voxel dropout learning for intervertebral disc localization and segmentation from multi-modality MR images. Med. Image Anal. 45, 41–54 (2018)

    Article  Google Scholar 

  14. Liu, C., Zhao, L.: Intervertebral disc segmentation and localization from multi-modality MR images with 2.5D multi-scale fully convolutional network and geometric constraint post-processing. In: Zheng, G., Belavy, D., Cai, Y., Li, S. (eds.) CSI 2018. LNCS, vol. 11397, pp. 144–153. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-13736-6_12

    Chapter  Google Scholar 

  15. Larhmam, M.A., Mahmoudi, S., Benjelloun, M.: Semi-automatic detection of cervical vertebrae in X-ray images using generalized hough transform. In: 2012 3rd International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 396–401 (2012)

    Google Scholar 

  16. Nie, D., Gao, Y., Wang, L., Shen, D.: ASDNet: attention based semi-supervised deep networks for medical image segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018, Part IV. LNCS, vol. 11073, pp. 370–378. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_43

    Chapter  Google Scholar 

  17. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  18. Girard, N., Charpiat, G., Tarabalka, Y.: Aligning and updating cadaster maps with aerial images by multi-task, multi-resolution deep learning. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018, Part V. LNCS, vol. 11365, pp. 675–690. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20873-8_43

    Chapter  Google Scholar 

  19. Khalel, A., Tasar, O., Charpiat, G., Tarabalka, Y.: Multi-task deep learning for satellite image pansharpening and segmentation, pp. 4869–4872 (2019)

    Google Scholar 

  20. Chen, Q., et al.: A multi-task deep learning model for the classification of age-related macular degeneration. CoRR abs/1812.00422 (2018)

    Google Scholar 

  21. Li, X., Hu, X., Yu, L., Zhu, L., Fu, C.W., Heng, P.A.: Canet: Cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading (2019)

    Google Scholar 

  22. Liu, L., Dou, Q., Chen, H., Olatunji, I.E., Qin, J., Heng, P.-A.: MTMR-Net: multi-task deep learning with margin ranking loss for lung nodule analysis. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 74–82. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_9

    Chapter  Google Scholar 

  23. Xiao, L., Stephen, J.M., Wilson, T.W., Calhoun, V.D., Wang, Y.: A manifold regularized multi-task learning model for IQ prediction from two fMRI paradigms. IEEE Trans. Biomed. Eng. 67, 796–806 (2020)

    Article  Google Scholar 

  24. Sohn, S., et al.: Sagittal spinal alignment in asymptomatic patients over 30 years old in the Korean population. Acta Neurochir. 159(6), 1119–1128 (2017). https://doi.org/10.1007/s00701-017-3100-9

    Article  Google Scholar 

  25. Labelle, H., Roussouly, P., Berthonnaud, E., Dimnet, J., O’Brien, M.: The importance of spino-pelvic balance in l5–s1 developmental spondylolisthesis. Spine 30, S27–34 (2005)

    Article  Google Scholar 

  26. Janssen, M., Kouwenhoven, J.W., Castelein, R.: The role of posteriorly directed shear loads acting on a pre-rotated growing spine: a hypothesis on the pathogenesis of idiopathic scoliosis. Stud. Health Technol. Inf. 158, 112–7 (2010)

    Google Scholar 

  27. Weinberg, D., Morris, W., Gebhart, J., Liu, R.: Pelvic incidence: an anatomic investigation of 880 cadaveric specimens. Eur. Spine J. 25, 3589–3595 (2015)

    Article  Google Scholar 

  28. Singh, R., Yadav, S., Sood, S., Yadav, R., Rohilla, R.: Spino-pelvic radiological parameters in normal Indian population. SICOT-J 4, 14 (2018)

    Article  Google Scholar 

  29. Natarajan, R., Andersson, G.: Lumbar disc degeneration is an equally important risk factor as lumbar fusion for causing adjacent segment disc disease: Fusion, degeneration adjacent disc disease. J. Orthop. Res. 35, 123–130 (2016)

    Article  Google Scholar 

  30. Legaye, J. In: Analysis of the Dynamic Sagittal Balance of the Lumbo-Pelvi-Femoral Complex (2011)

    Google Scholar 

  31. Kim, M., Lee, S.H., Kim, E.S., Eoh, W., Chung, S.S., Lee, C.S.: The impact of sagittal balance on clinical results after posterior interbody fusion for patients with degenerative spondylolisthesis: A pilot study. BMC Musculoskelet. Disor 12, 69 (2011). https://doi.org/10.1186/1471-2474-12-69

    Article  Google Scholar 

  32. Yilgor, C., et al.: Relative lumbar lordosis and lordosis distribution index: individualized pelvic incidence-based proportional parameters that quantify lumbar lordosis more precisely than the concept of pelvic incidence minus lumbar lordosis. Neurosurg. Focus 43, E5 (2017)

    Article  Google Scholar 

  33. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: Bisenet: Bilateral segmentation network for real-time semantic segmentation. CoRR abs/1808.00897 (2018)

    Google Scholar 

  34. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. CoRR abs/1807.10165 (2018)

    Google Scholar 

  35. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 6230–6239 (2017)

    Google Scholar 

  36. Zheng, G.Q., et al.: Relationship between postoperative lordosis distribution index and adjacent segment disease following L4–S1 posterior lumbar interbody fusion. J. Orthop. Surg. Res. 15, 129 (2020). https://doi.org/10.1186/s13018-020-01630-9

    Article  Google Scholar 

Download references

Acknowledgment

The authors would like to thank the support of this work in part by the Ministry of Science and Technology of Taiwan under Grant MOST 106-2221-E-194-004 is gratefully acknowledged.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Chung Cheng University, Chiayi, 621, Taiwan

    Van Luan Tran & Huei-Yung Lin

  2. Biomedical Technology and Device Research Laboratories, Industrial Technology Research Institute, Hsinchu, 310, Taiwan

    Hsiao-Wei Liu

Authors
  1. Van Luan Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huei-Yung Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hsiao-Wei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Van Luan Tran .

Editor information

Editors and Affiliations

  1. Waseda University, Tokyo, Japan

    Hiroshi Ishikawa

  2. Institute of Automation of Chinese Academy of Sciences, Beijing, China

    Cheng-Lin Liu

  3. Czech Technical University in Prague, Prague, Czech Republic

    Tomas Pajdla

  4. University of Pennsylvania, Philadelphia, PA, USA

    Jianbo Shi

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 4545 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tran, V.L., Lin, HY., Liu, HW. (2021). MBNet: A Multi-task Deep Neural Network for Semantic Segmentation and Lumbar Vertebra Inspection on X-Ray Images. In: Ishikawa, H., Liu, CL., Pajdla, T., Shi, J. (eds) Computer Vision – ACCV 2020. ACCV 2020. Lecture Notes in Computer Science(), vol 12626. Springer, Cham. https://doi.org/10.1007/978-3-030-69541-5_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69541-5_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69540-8

  • Online ISBN: 978-3-030-69541-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation