Advertisement

TANet: Towards Fully Automatic Tooth Arrangement

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12360)

Abstract

Determining optimal target tooth arrangements is a key step of treatment planning in digital orthodontics. Existing practice for specifying the target tooth arrangement involves tedious manual operations with the outcome quality depending heavily on the experience of individual specialists, leading to inefficiency and undesirable variations in treatment results. In this work, we proposed a learning-based method for fast and automatic tooth arrangement. To achieve this, we formulate the tooth arrangement task as a novel structured 6-DOF pose prediction problem and solve it by proposing a new neural network architecture to learn from a large set of clinical data that encode successful orthodontic treatment cases. Our method has been validated with extensive experiments and shows promising results both qualitatively and quantitatively.

Keywords

Deep learning Orthodontics Tooth arrangement 6D pose prediction Structure Graph neural network 

Supplementary material

504470_1_En_29_MOESM1_ESM.pdf (2.7 mb)
Supplementary material 1 (pdf 2784 KB)

References

  1. 1.
    Andrews, L.F.: The six keys to normal occlusion. Am. J. Orthod. 62(3), 296–309 (1972)CrossRefGoogle Scholar
  2. 2.
    Angle, E.H.: Classification of malocclusion. Dent. Cosmos. 41, 350–375 (1899)Google Scholar
  3. 3.
    Aubry, M., Maturana, D., Efros, A.A., Russell, B.C., Sivic, J.: Seeing 3D chairs: exemplar part-based 2D–3D alignment using a large dataset of cad models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3762–3769 (2014)Google Scholar
  4. 4.
    Besl, P.J., McKay, N.D.: Method for registration of 3-D shapes. In: Sensor Fusion IV: Control Paradigms and Data Structures, vol. 1611, pp. 586–606. International Society for Optics and Photonics (1992)Google Scholar
  5. 5.
    Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., Rother, C.: Learning 6D object pose estimation using 3D object coordinates. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 536–551. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10605-2_35CrossRefGoogle Scholar
  6. 6.
    Chang, Y.B., Xia, J.J., Gateno, J., Xiong, Z., Zhou, X., Wong, S.T.: An automatic and robust algorithm of reestablishment of digital dental occlusion. IEEE Trans. Med. Imaging 29(9), 1652–1663 (2010)CrossRefGoogle Scholar
  7. 7.
    Collet, A., Martinez, M., Srinivasa, S.S.: The MOPED framework: object recognition and pose estimation for manipulation. Int. J. Robot. Res. 30(10), 1284–1306 (2011)CrossRefGoogle Scholar
  8. 8.
    Dai, N., Yu, X., Fan, Q., Yuan, F., Liu, L., Sun, Y.: Complete denture tooth arrangement technology driven by a reconfigurable rule. PLoS One 13(6), e0198252 (2018)CrossRefGoogle Scholar
  9. 9.
    Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 605–613 (2017)Google Scholar
  10. 10.
    Fisher, M., Ritchie, D., Savva, M., Funkhouser, T., Hanrahan, P.: Example-based synthesis of 3D object arrangements. ACM Trans. Graph. (TOG) 31(6) (2012). Article no. 135 Google Scholar
  11. 11.
    Gao, L., et al.: SDM-NET: deep generative network for structured deformable mesh. ACM Trans. Graph. (TOG) 38(6) (2019). Article no. 243Google Scholar
  12. 12.
    Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)Google Scholar
  13. 13.
    Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry, M.: AtlasNet: Apapier-Mâché approach to learning 3D surfacegeneration. arXiv preprint arXiv:1802.05384 (2018)
  14. 14.
    Guerrero, P., Jeschke, S., Wimmer, M., Wonka, P.: Learning shape placements by example. ACM Trans. Graph. (TOG) 34(4) (2015). Article no. 108Google Scholar
  15. 15.
    Hinterstoisser, S., et al.: Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7724, pp. 548–562. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-37331-2_42CrossRefGoogle Scholar
  16. 16.
    Hwang, J.J., Azernikov, S., Efros, A.A., Yu, S.X.: Learning beyond human expertise with generative models for dental restorations. arXiv preprint arXiv:1804.00064 (2018)
  17. 17.
    Li, J., Xu, K., Chaudhuri, S., Yumer, E., Zhang, H., Guibas, L.: GRASS: generative recursive autoencoders for shape structures. ACM Trans. Graph. (TOG) 36(4) (2017). Article no. 52Google Scholar
  18. 18.
    Li, Y., Wang, G., Ji, X., Xiang, Y., Fox, D.: DeepIM: deep iterative matching for 6d pose estimation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 683–698 (2018)Google Scholar
  19. 19.
    Li, Y., Tarlow, D., Brockschmidt, M., Zemel, R.: Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493 (2015)
  20. 20.
    Lian, C., et al.: MeshSNet: deep multi-scale mesh feature learning for end-to-end tooth labeling on 3D dental surfaces. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 837–845. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32226-7_93CrossRefGoogle Scholar
  21. 21.
    Majerowicz, L., Shamir, A., Sheffer, A., Hoos, H.H.: Filling your shelves: synthesizing diverse style-preserving artifact arrangements. IEEE Trans. Vis. Comput. Graph. 20(11), 1507–1518 (2013)CrossRefGoogle Scholar
  22. 22.
    Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
  23. 23.
    Mo, K., et al.: StructureNet: hierarchical graph networks for 3D shape generation. arXiv preprint arXiv:1908.00575 (2019)
  24. 24.
    Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. arXiv preprint arXiv:1901.05103 (2019)
  25. 25.
    Peng, S., Liu, Y., Huang, Q., Zhou, X., Bao, H.: PVNet: pixel-wise voting network for 6dof pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4561–4570 (2019)Google Scholar
  26. 26.
    Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum pointnets for 3D object detection from RGB-D data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 918–927 (2018)Google Scholar
  27. 27.
    Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)Google Scholar
  28. 28.
    Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)Google Scholar
  29. 29.
    Song, S., Xiao, J.: Sliding shapes for 3D object detection in depth images. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 634–651. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10599-4_41CrossRefGoogle Scholar
  30. 30.
    Song, S., Xiao, J.: Deep sliding shapes for amodal 3D object detection in RGB-D images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 808–816 (2016)Google Scholar
  31. 31.
    Sung, M., Su, H., Kim, V.G., Chaudhuri, S., Guibas, L.: ComplementMe: weakly-supervised component suggestions for 3D modeling. ACM Trans. Graph. (TOG) 36(6) (2017). Article no. 226Google Scholar
  32. 32.
    Tatarchenko, M., Dosovitskiy, A., Brox, T.: Octree generating networks: efficient convolutional architectures for high-resolution 3D outputs. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2088–2096 (2017)Google Scholar
  33. 33.
    Tekin, B., Sinha, S.N., Fua, P.: Real-time seamless single shot 6D object pose prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 292–301 (2018)Google Scholar
  34. 34.
    Tremblay, J., To, T., Sundaralingam, B., Xiang, Y., Fox, D., Birchfield, S.: Deep object pose estimation for semantic robotic grasping of household objects. arXiv preprint arXiv:1809.10790 (2018)
  35. 35.
    Wang, C., et al.: DenseFusion: 6D object pose estimation by iterative dense fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3343–3352 (2019)Google Scholar
  36. 36.
    Wang, H., Sridhar, S., Huang, J., Valentin, J., Song, S., Guibas, L.J.: Normalized object coordinate space for category-level 6D object pose and size estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2642–2651 (2019)Google Scholar
  37. 37.
    Wang, K., Lin, Y.A., Weissmann, B., Savva, M., Chang, A.X., Ritchie, D.: PlanIT: planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Trans. Graph. (TOG) 38(4) (2019). Article no. 132Google Scholar
  38. 38.
    Wang, K., Savva, M., Chang, A.X., Ritchie, D.: Deep convolutional priors for indoor scene synthesis. ACM Transactions on Graphics (TOG) 37(4) (2018). Article no. 70Google Scholar
  39. 39.
    Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: PoseCNN: a convolutional neural network for 6D object pose estimation in cluttered scenes. arXiv preprint arXiv:1711.00199 (2017)
  40. 40.
    Xu, D., Anguelov, D., Jain, A.: PointFusion: deep sensor fusion for 3D bounding box estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 244–253 (2018)Google Scholar
  41. 41.
    Xu, X., Liu, C., Zheng, Y.: 3D tooth segmentation and labeling using deep convolutional neural networks. IEEE Trans. Vis. Comput. Graph. 25(7), 2336–2348 (2018)CrossRefGoogle Scholar
  42. 42.
    Yu, L.F., Yeung, S.K., Tang, C.K., Terzopoulos, D., Chan, T.F., Osher, S.: Make it home: automatic optimization of furniture arrangement. ACM Trans. Graph. 30(4) (2011). Article no. 86Google Scholar
  43. 43.
    Zanjani, F.G., et al.: Mask-MCNet: instance segmentation in 3D point cloud of intra-oral scans. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 128–136. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32254-0_15CrossRefGoogle Scholar
  44. 44.
    Zhou, Y., Tuzel, O.: VoxelNet: end-to-end learning for point cloud based 3D object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4490–4499 (2018)Google Scholar
  45. 45.
    Zhu, M., et al.: Single image 3D object detection and pose estimation for grasping. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 3936–3943. IEEE (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.South China University of TechnologyGuangzhouChina
  2. 2.The University of Hong KongPok Fu LamHong Kong

Personalised recommendations