Advertisement

Joint Craniomaxillofacial Bone Segmentation and Landmark Digitization by Context-Guided Fully Convolutional Networks

  • Jun Zhang
  • Mingxia Liu
  • Li Wang
  • Si Chen
  • Peng Yuan
  • Jianfu Li
  • Steve Guo-Fang Shen
  • Zhen Tang
  • Ken-Chung Chen
  • James J. XiaEmail author
  • Dinggang ShenEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)

Abstract

Generating accurate 3D models from cone-beam computed tomography (CBCT) images is an important step in developing treatment plans for patients with craniomaxillofacial (CMF) deformities. This process often involves bone segmentation and landmark digitization. Since anatomical landmarks generally lie on the boundaries of segmented bone regions, the tasks of bone segmentation and landmark digitization could be highly correlated. However, most existing methods simply treat them as two standalone tasks, without considering their inherent association. In addition, these methods usually ignore the spatial context information (i.e., displacements from voxels to landmarks) in CBCT images. To this end, we propose a context-guided fully convolutional network (FCN) for joint bone segmentation and landmark digitization. Specifically, we first train an FCN to learn the displacement maps to capture the spatial context information in CBCT images. Using the learned displacement maps as guidance information, we further develop a multi-task FCN to jointly perform bone segmentation and landmark digitization. Our method has been evaluated on 107 subjects from two centers, and the experimental results show that our method is superior to the state-of-the-art methods in both bone segmentation and landmark digitization.

Supplementary material

451304_1_En_81_MOESM1_ESM.mpg (204 kb)
Supplementary material 1 (mpg 204 KB)
451304_1_En_81_MOESM2_ESM.mpg (206 kb)
Supplementary material 2 (mpg 205 KB)
451304_1_En_81_MOESM3_ESM.mpg (205 kb)
Supplementary material 3 (mpg 204 KB)
451304_1_En_81_MOESM4_ESM.mpg (206 kb)
Supplementary material 4 (mpg 206 KB)

References

  1. 1.
    Shahidi, S., Bahrampour, E., Soltanimehr, E., Zamani, A., Oshagh, M., Moattari, M., Mehdizadeh, A.: The accuracy of a designed software for automated localization of craniofacial landmarks on CBCT images. BMC Med. Imaging 14(1), 32 (2014)CrossRefGoogle Scholar
  2. 2.
    Cheng, E., Chen, J., Yang, J., Deng, H., Wu, Y., Megalooikonomou, V., Gable, B., Ling, H.: Automatic dent-landmark detection in 3-D CBCT dental volumes. In: 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC, pp. 6204–6207 (2011)Google Scholar
  3. 3.
    Zhang, J., Liu, M., An, L., Gao, Y., Shen, D.: Alzheimer’s disease diagnosis using landmark-based features from longitudinal structural MR images. IEEE J. Biomed. Health Inform. (2017). doi: 10.1109/JBHI.2017.2704614
  4. 4.
    Cao, X., Yang, J., Gao, Y., Guo, Y., Wu, G., Shen, D.: Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis. Med. Image Anal. (2017). doi: 10.1016/j.media.2017.05.004
  5. 5.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  6. 6.
    Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_27 CrossRefGoogle Scholar
  7. 7.
    Zhang, J., Gao, Y., Wang, L., Tang, Z., Xia, J.J., Shen, D.: Automatic craniomaxillofacial landmark digitization via segmentation-guided partially-joint regression forest model and multiscale statistical features. IEEE Trans. Biomed. Eng. 63(9), 1820–1829 (2016)CrossRefGoogle Scholar
  8. 8.
    Liu, M., Zhang, D., Chen, S., Xue, H.: Joint binary classifier learning for ECOC-based multi-class classification. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2335–2341 (2016)CrossRefGoogle Scholar
  9. 9.
    Liu, M., Zhang, D., Shen, D.: Relationship induced multi-template learning for diagnosis of Alzheimer’s disease and mild cognitive impairment. IEEE Trans. Med. Imaging 35(6), 1463–1474 (2016)CrossRefGoogle Scholar
  10. 10.
    Schroff, F., Criminisi, A., Zisserman, A.: Object class segmentation using random forests. In: BMVC, pp. 1–10 (2008)Google Scholar
  11. 11.
    Criminisi, A., Robertson, D., Konukoglu, E., Shotton, J., Pathak, S., White, S., Siddiqui, K.: Regression forests for efficient anatomy detection and localization in computed tomography scans. Med. Image Anal. 17(8), 1293–1303 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jun Zhang
    • 1
  • Mingxia Liu
    • 1
  • Li Wang
    • 1
  • Si Chen
    • 2
  • Peng Yuan
    • 3
  • Jianfu Li
    • 3
  • Steve Guo-Fang Shen
    • 3
  • Zhen Tang
    • 3
  • Ken-Chung Chen
    • 3
  • James J. Xia
    • 3
    Email author
  • Dinggang Shen
    • 1
    Email author
  1. 1.Department of Radiology and BRICUNC at Chapel HillChapel HillUSA
  2. 2.Peking University School and Hospital of StomatologyBeijingChina
  3. 3.Houston Methodist HospitalHoustonUSA

Personalised recommendations