Skip to main content

Diffusion-Based Hierarchical Multi-label Object Detection to Analyze Panoramic Dental X-Rays

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 (MICCAI 2023)


Due to the necessity for precise treatment planning, the use of panoramic X-rays to identify different dental diseases has tremendously increased. Although numerous ML models have been developed for the interpretation of panoramic X-rays, there has not been an end-to-end model developed that can identify problematic teeth with dental enumeration and associated diagnoses at the same time. To develop such a model, we structure the three distinct types of annotated data hierarchically following the FDI system, the first labeled with only quadrant, the second labeled with quadrant-enumeration, and the third fully labeled with quadrant-enumeration-diagnosis. To learn from all three hierarchies jointly, we introduce a novel diffusion-based hierarchical multi-label object detection framework by adapting a diffusion-based method that formulates object detection as a denoising diffusion process from noisy boxes to object boxes. Specifically, to take advantage of the hierarchically annotated data, our method utilizes a novel noisy box manipulation technique by adapting the denoising process in the diffusion network with the inference from the previously trained model in hierarchical order. We also utilize a multi-label object detection method to learn efficiently from partial annotations and to give all the needed information about each abnormal tooth for treatment planning. Experimental results show that our method significantly outperforms state-of-the-art object detection methods, including RetinaNet, Faster R-CNN, DETR, and DiffusionDet for the analysis of panoramic X-rays, demonstrating the great potential of our method for hierarchically and partially annotated datasets. The code and the datasets are available at

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions


  1. AbuSalim, S., Zakaria, N., Islam, M.R., Kumar, G., Mokhtar, N., Abdulkadir, S.J.: Analysis of deep learning techniques for dental informatics: a systematic literature review. Healthcare (Basel) 10(10), 1892 (2022)

    Article  Google Scholar 

  2. Bruno, M.A., Walker, E.A., Abujudeh, H.H.: Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction. Radiographics 35(6), 1668–1676 (2015)

    Article  Google Scholar 

  3. Bu, X., Peng, J., Yan, J., Tan, T., Zhang, Z.: GAIA: a transfer learning system of object detection that fits your needs. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 274–283 (2021)

    Google Scholar 

  4. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-End object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020).

    Chapter  Google Scholar 

  5. Chen, S., Sun, P., Song, Y., Luo, P.: DiffusionDet: diffusion model for object detection. arXiv preprint arXiv:2211.09788 (2022)

  6. Chung, M., et al.: Individual tooth detection and identification from dental panoramic X-ray images via point-wise localization and distance regularization. Artif. Intell. Med. 111, 101996 (2021)

    Article  Google Scholar 

  7. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, pp. 248–255 (2009)

    Google Scholar 

  8. Glick, M., et al.: FDI vision 2020: shaping the future of oral health. Int. Dent. J. 62(6), 278 (2012)

    Article  Google Scholar 

  9. Hamamci, I.E., et al.: DENTEX: an abnormal tooth detection with dental enumeration and diagnosis benchmark for panoramic X-rays. arXiv preprint arXiv:2305.19112 (2023)

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arxiv 2015. arXiv preprint arXiv:1512.03385 (2015)

  11. Hwang, J.J., Jung, Y.H., Cho, B.H., Heo, M.S.: An overview of deep learning in the field of dentistry. Imaging Sci. Dent. 49(1), 1–7 (2019)

    Article  Google Scholar 

  12. Krois, J.: Deep learning for the radiographic detection of periodontal bone loss. Sci. Rep. 9(1), 8495 (2019)

    Article  Google Scholar 

  13. Kumar, A., Bhadauria, H.S., Singh, A.: Descriptive analysis of dental X-ray images using various practical methods: a review. PeerJ Comput. Sci. 7, e620 (2021)

    Article  Google Scholar 

  14. Lin, S.Y., Chang, H.Y.: Tooth numbering and condition recognition on dental panoramic radiograph images using CNNs. IEEE Access 9, 166008–166026 (2021)

    Article  Google Scholar 

  15. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 2117–2125 (2017)

    Google Scholar 

  16. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)

    Google Scholar 

  17. Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103.14030 (2021)

  18. Panetta, K., Rajendran, R., Ramesh, A., Rao, S.P., Agaian, S.: Tufts dental database: a multimodal panoramic X-ray dataset for benchmarking diagnostic systems. IEEE J. Biomed. Health Inform. 26(4), 1650–1659 (2021)

    Article  Google Scholar 

  19. Pati, S., et al.: GaNDLF: a generally nuanced deep learning framework for scalable end-to-end clinical workflows in medical imaging. arXiv preprint arXiv:2103.01006 (2021)

  20. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol. 28 (2015)

    Google Scholar 

  21. Shin, S.J., Kim, S., Kim, Y., Kim, S.: Hierarchical multi-label object detection framework for remote sensing images. Remote Sens. 12(17), 2734 (2020)

    Article  Google Scholar 

  22. Shin, S.J., Kim, S., Kim, Y., Kim, S.: Hierarchical multi-label object detection framework for remote sensing images. Remote Sens. 12(17), 2734 (2020)

    Article  Google Scholar 

  23. Tuzoff, D.V., et al.: Tooth detection and numbering in panoramic radiographs using convolutional neural networks. Dentomaxillofacial Radiol. 48(4), 20180051 (2019)

    Article  Google Scholar 

  24. Willemink, M.J., et al.: Preparing medical imaging data for machine learning. Radiology 295(1), 4–15 (2020)

    Article  Google Scholar 

  25. Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2 (2019)

    Google Scholar 

  26. Xie, Z., et al.: SimMIM: a simple framework for masked image modeling. arXiv preprint arXiv:2111.09886 (2021)

  27. Yüksel, A.E., et al.: Dental enumeration and multiple treatment detection on panoramic X-rays using deep learning. Sci. Rep. 11(1), 1–10 (2021)

    Article  Google Scholar 

  28. Zhao, X., Schulter, S., Sharma, G., Tsai, Y.-H., Chandraker, M., Wu, Y.: Object detection with a unified label space from multiple datasets. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12359, pp. 178–193. Springer, Cham (2020).

    Chapter  Google Scholar 

  29. Zhao, Y., et al.: TsasNet: tooth segmentation on dental panoramic X-ray images by two-stage attention segmentation network. Knowl.-Based Syst. 206, 106338 (2020)

    Article  Google Scholar 

  30. Zhu, H., Cao, Z., Lian, L., Ye, G., Gao, H., Wu, J.: CariesNet: a deep learning approach for segmentation of multi-stage caries lesion from oral panoramic X-ray image. Neural Comput. Appl. 35(22), 16051–16059 (2023).

    Article  Google Scholar 

Download references


We would like to thank the Helmut Horten Foundation for supporting our research.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Ibrahim Ethem Hamamci .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 9053 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hamamci, I.E. et al. (2023). Diffusion-Based Hierarchical Multi-label Object Detection to Analyze Panoramic Dental X-Rays. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14225. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43986-5

  • Online ISBN: 978-3-031-43987-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics