Deep Learning 3D Shape Surfaces Using Geometry Images

  • Ayan Sinha
  • Jing Bai
  • Karthik Ramani
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9910)


Surfaces serve as a natural parametrization to 3D shapes. Learning surfaces using convolutional neural networks (CNNs) is a challenging task. Current paradigms to tackle this challenge are to either adapt the convolutional filters to operate on surfaces, learn spectral descriptors defined by the Laplace-Beltrami operator, or to drop surfaces altogether in lieu of voxelized inputs. Here we adopt an approach of converting the 3D shape into a ‘geometry image’ so that standard CNNs can directly be used to learn 3D shapes. We qualitatively and quantitatively validate that creating geometry images using authalic parametrization on a spherical domain is suitable for robust learning of 3D shape surfaces. This spherically parameterized shape is then projected and cut to convert the original 3D shape into a flat and regular geometry image. We propose a way to implicitly learn the topology and structure of 3D shapes using geometry images encoded with suitable features. We show the efficacy of our approach to learn 3D shape surfaces for classification and retrieval tasks on non-rigid and rigid shape datasets.


Deep learning 3D Shape Surfaces CNN Geometry images 



This work was partially supported by the NSF Award No.1235232 from CMMI as well as the Donald W. Feddersen Chaired Professorship from Purdue School of Mechanical Engineering. Dr. Jing Bai was supported by the National Natural Science Foundation of China (No. 61163016) and by the China Scholarship Council. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors.

Supplementary material

419981_1_En_14_MOESM1_ESM.pdf (862 kb)
Supplementary material 1 (pdf 861 KB)


  1. 1.
    Boscaini, D., Masci, J., Melzi, S., Bronstein, M.M., Castellani, U., Vandergheynst, P.: Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks. Comput. Graph. Forum 34, 13–23 (2015)CrossRefGoogle Scholar
  2. 2.
    Bronstein, A.M., Bronstein, M.M., Guibas, L.J., Ovsjanikov, M.: Shape google: geometric words and expressions for invariant shape retrieval. ACM Trans. Graph. (TOG) 30(1), 1 (2011)CrossRefGoogle Scholar
  3. 3.
    Chen, D.-Y., Tian, X.-P., Shen, Y.-T., Ouhyoung, M.: On visual similarity based 3D model retrieval. Comput. Graph. Forum 22(3), 223–232 (2003)CrossRefGoogle Scholar
  4. 4.
    Desbrun, M., Meyer, M., Alliez, P.: Intrinsic parameterizations of surface meshes. Comput. Graph. Forum 21, (2002)Google Scholar
  5. 5.
    Dominitz, A., Tannenbaum, A.: Texture mapping via optimal mass transport. IEEE Trans. Vis. Comput. Graph. 16(3), 419–433 (2010)CrossRefGoogle Scholar
  6. 6.
    Fang, Y., Xie, J., Dai, G., Wang, M., Zhu, F., Xu, T., Wong, E.: 3D deep shape descriptor. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2319–2328 (2015)Google Scholar
  7. 7.
    Floater, M.S., Hormann, K.: Surface parameterization: a tutorial and survey. Advances in Multiresolution for Geometric Modelling, pp. 157–186. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  8. 8.
    Friedel, I., Schröder, P., Desbrun, M.: Unconstrained spherical parameterization. J. Graph. Tools 12(1), 17–26 (2007)CrossRefGoogle Scholar
  9. 9.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580–587. IEEE (2014)Google Scholar
  10. 10.
    Gotsman, C., Gu, X., Sheffer, A.: Fundamentals of spherical parameterization for 3D meshes. In: Proceedings of the 2006 Symposium on Interactive 3D Graphics and Games, 14–17 March 2006, pp. 28–29 (2003)Google Scholar
  11. 11.
    Gu, X., Gortler, S.J., Hoppe, H.: Geometry images. In: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2002, pp. 355–361. ACM, New York (2002)Google Scholar
  12. 12.
    Gu, X., et al.: Genus zero surface conformal mapping and its application to brain surface mapping. IEEE Trans. Medical Imaging (2003)Google Scholar
  13. 13.
    Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: Proceedings of International Computer Vision and Pattern Recognition (CVPR 2014) (2014)Google Scholar
  14. 14.
    Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the Fourth Eurographics Symposium on Geometry Processing, SGP 2006, pp. 61–70. Eurographics Association, Aire-la-Ville (2006)Google Scholar
  15. 15.
    Kazhdan, M., Funkhouser, T., Rusinkiewicz, S.: Rotation invariant spherical harmonic representation of 3D shape descriptors, June 2003Google Scholar
  16. 16.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105 (2012)Google Scholar
  17. 17.
    Laga, H.T., Schreck, A., Ferreira, A., Godil, I.P., Meshes, W., Lian, Z., Godil, A., Bustos, B., Daoudi, M., Hermans, J., Kawamura, S., Kurita, Y., Lavou, G., Nguyen, H.V., Ohbuchi, R., Ohkita, Y., Ohishi, Y., Porikli, F., Reuter, M., Sipiran, I., Smeets, D., Suetens, P., Tabia, H.: SHREC 2011 Track: shape retrieval on non-rigid 3D (2011)Google Scholar
  18. 18.
    Lévy, B., Petitjean, S., Ray, N., Maillot, J.: Least squares conformal maps for automatic texture atlas generation. ACM Trans. Graph. 21(3), 362–371 (2002)CrossRefGoogle Scholar
  19. 19.
    Masci, J., Boscaini, D., Bronstein, M.M., Vandergheynst, P.: Shapenet: Convolutional neural networks on non-euclidean manifolds. arXiv preprint arXiv:1501.06297 (2015)
  20. 20.
    Maturana, D., Scherer, S.: Voxnet: a 3D convolutional neural network for real-time object recognition. In: Signal Processing Letters (2015)Google Scholar
  21. 21.
    Novotni, M., Klein, R.: Shape retrieval using 3D zernike descriptors. Comput. Aided Design 36, 1047–1062 (2004)CrossRefGoogle Scholar
  22. 22.
    Praun, E., Hoppe, H.: Spherical parametrization and remeshing. In: ACM Transactions on Graphics (TOG), vol. 22, pp. 340–349. ACM (2003)Google Scholar
  23. 23.
    Rustamov, R.M.: Laplace-beltrami eigen functions for deformation invariant shape representation. Proceedings of the Fifth Eurographics Symposium on Geometry Processing, SGP 2007, pp. 225–233, Aire-la-Ville (2007)Google Scholar
  24. 24.
    Shen, L., Makedon, F.: Spherical mapping for processing of 3-D closed surfaces. In: Image and Vision Computing (2006)Google Scholar
  25. 25.
    Shen, L., Makedon, F.: Spherical mapping for processing of 3D closed surfaces. Image Vis. Comput. 24(7), 743–761 (2006)CrossRefGoogle Scholar
  26. 26.
    Shi, B., Bai, S., Zhou, Z., Bai, X.: DeepPano: deep panoramic representation for 3-D shape recognition. IEEE Signal Process. Lett. 22(12), 2339–2343 (2015)CrossRefGoogle Scholar
  27. 27.
    Sinha, A., Choi, C., Ramani, K.: DeepHand: robust hand pose estimation by completing a matrix imputed with deep features. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016Google Scholar
  28. 28.
    Sinha, A., Ramani, K.: Multi-scale kernels using random walks. Comput. Graphics Forum 33(1), 164–177 (2014)CrossRefGoogle Scholar
  29. 29.
    Solomon, J., de Goes, F., Studios, P.A., Peyré, G., Cuturi, M., Butscher, A., Nguyen, A., Du, T., Guibas, L.: Convolutional wasserstein distances: efficient optimal transportation on geometric domains. ACM Transactions on Graphics (Proceeding SIGGRAPH 2015) (2015)Google Scholar
  30. 30.
    Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.G.: Multi-view convolutional neural networks for 3D shape recognition. In: Proceeding ICCV (2015)Google Scholar
  31. 31.
    Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. In: Proceedings of the Symposium on Geometry Processing, SGP 2009, pp. 1383–1392, Aire-la-Ville (2009)Google Scholar
  32. 32.
    Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)Google Scholar
  33. 33.
    Yu, Y., Zhou, K., Xu, D., Shi, X., Bao, H., Guo, B., Shum, H.-Y.: Mesh editing with poisson-based gradient field manipulation. ACM Trans. Graph. 23(3), 644–651 (2004)CrossRefGoogle Scholar
  34. 34.
    Zhao, X., Su, Z., Gu, X.D., Kaufman, A., Sun, J., Gao, J., Luo, F.: Area-preservation mapping using optimal mass transport. IEEE Trans. Visual Comput. Graphics 19(12), 2838–2847 (2013)CrossRefGoogle Scholar
  35. 35.
    Zou, G., Hu, J., Gu, X., Hua, J.: Authalic parameterization of general surfaces using lie advection. IEEE Trans. Visual Comput. Graphics 17(12), 2005–2014 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Purdue UniversityWest LafayetteUSA
  2. 2.Beifang University of NationalitiesYinchuanChina

Personalised recommendations