Skip to main content

Deep Patch-Based Human Segmentation

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12532))

Included in the following conference series:

Abstract

3D human segmentation has seen noticeable progress in recent years. It, however, still remains a challenge to date. In this paper, we introduce a deep patch-based method for 3D human segmentation. We first extract a local surface patch for each vertex and then parameterize it into a 2D grid (or image). We then embed identified shape descriptors into the 2D grids which are further fed into the powerful 2D Convolutional Neural Network for regressing corresponding semantic labels (e.g., head, torso). Experiments demonstrate that our method is effective in human segmentation, and achieves state-of-the-art accuracy.

D. Zhang and Z. Fang—Joint first author.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Guo, K., Zou, D., Chen, X.: 3D mesh labeling via deep convolutional neural networks. ACM Trans. Graph. 35(1), 1–12 (2015)

    Article  Google Scholar 

  2. Masci, J., Boscaini, D., Bronstein, M., Vandergheynst, P.: Geodesic convolutional neural networks on Riemannian manifolds. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 37–45 (2015)

    Google Scholar 

  3. Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.: Learning shape correspondence with anisotropic convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 3189–3197 (2016)

    Google Scholar 

  4. Maron, H., et al.: Convolutional neural networks on surfaces via seamless toric covers. ACM Trans. Graph. 36(4), 71–1 (2017)

    Article  Google Scholar 

  5. Haim, N., Segol, N., Ben-Hamu, H., Maron, H., Lipman, Y.: Surface networks via general covers. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 632–641 (2019)

    Google Scholar 

  6. Hanocka, R., Hertz, A., Fish, N., Giryes, R., Fleishman, S., Cohen-Or, D.: MeshCNN: a network with an edge. ACM Trans. Graph. 38(4), 1–12 (2019)

    Article  Google Scholar 

  7. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. In: ACM SIGGRAPH 2005 Papers, pp. 408–416 (2005)

    Google Scholar 

  8. Bogo, F., Romero, J., Loper, M., Black, M.J.: Faust: Dataset and evaluation for 3D mesh registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3794–3801 (2014)

    Google Scholar 

  9. Vlasic, D., Baran, I., Matusik, W., Popovic, J.: Articulated mesh animation from multi-view silhouettes. In: ACM SIGGRAPH 2008 Papers, pp. 1–9 (2008)

    Google Scholar 

  10. Adobe fuse 3D characters. https://www.mixamo.com

  11. Giorgi, D., Biasotti, S., Paraboschi, L.: Shape retrieval contest 2007: watertight models track. SHREC competition 8(7) (2007)

    Google Scholar 

  12. Aubry, M., Schlickewei, U., Cremers, D.: The wave kernel signature: a quantum mechanical approach to shape analysis. In: 2011 IEEE International Conference on Computer Vision Workshops, pp. 1626–1633. IEEE (2011)

    Google Scholar 

  13. Hilaga, M., Shinagawa, Y., Kohmura, T., Kunii, T.L.: Topology matching for fully automatic similarity estimation of 3D shapes. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 203–212 (2001)

    Google Scholar 

  14. Melvær, E.L., Reimers, M.: Geodesic polar coordinates on polygonal meshes. In: Computer Graphics Forum, vol. 31, pp. 2423–2435. Wiley Online Library (2012)

    Google Scholar 

  15. Schmidt, R., Grimm, C., Wyvill, B.: Interactive decal compositing with discrete exponential maps. In: ACM SIGGRAPH 2006 Papers, pp. 605–613 (2006)

    Google Scholar 

  16. Floater, M.S.: Mean value coordinates. Comput. Aided Geometr. Des. 20(1), 19–27 (2003)

    Article  MathSciNet  Google Scholar 

  17. Ju, T., Schaefer, S., Warren, J.: Mean value coordinates for closed triangular meshes. In: ACM SIGGRAPH 2005 Papers, pp. 561–566 (2005)

    Google Scholar 

  18. Surazhsky, V., Surazhsky, T., Kirsanov, D., Gortler, S.J., Hoppe, H.: Fast exact and approximate geodesics on meshes. ACM Trans. Graph. 24(3), 553–560 (2005)

    Article  Google Scholar 

  19. Kokkinos, I., Bronstein, M.M., Litman, R., Bronstein, A.M.: Intrinsic shape context descriptors for deformable shapes. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 159–166. IEEE (2012)

    Google Scholar 

  20. Floater, M.S., Hormann, K.: Surface parameterization: a tutorial and survey. In: Dodgson, N.A., Floater, M.S., Sabin, M.A. (eds.) Advances in Multiresolution for Geometric Modelling, pp. 157–186. Springer, Heidelberg (2005). https://doi.org/10.1007/3-540-26808-1_9

    Chapter  Google Scholar 

  21. Duchamp, T., Certain, A., DeRose, A., Stuetzle, W.: Hierarchical computation of pl harmonic embeddings. preprint (1997)

    Google Scholar 

  22. Floater, M.S.: Parametric tilings and scattered data approximation. Int. J. Shape Model. 4(03n04), 165–182 (1998)

    Article  Google Scholar 

  23. Gu, X., Yau, S.T.: Global conformal surface parameterization. In: Proceedings of the 2003 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, pp. 127–137 (2003)

    Google Scholar 

  24. Haker, S., Angenent, S., Tannenbaum, A., Kikinis, R., Sapiro, G., Halle, M.: Conformal surface parameterization for texture mapping. IEEE Trans. Vis. Comput. Graph. 6(2), 181–189 (2000)

    Article  Google Scholar 

  25. Praun, E., Hoppe, H.: Spherical parametrization and remeshing. ACM Trans. Graph. 22(3), 340–349 (2003)

    Article  Google Scholar 

  26. Sheffer, A., de Sturler, E.: Parameterization of faceted surfaces for meshing using angle-based flattening. Eng. Comput. 17(3), 326–337 (2001)

    Article  Google Scholar 

  27. Sheffer, A., Gotsman, C., Dyn, N.: Robust spherical parameterization of triangular meshes. Computing 72(1–2), 185–193 (2004)

    Article  MathSciNet  Google Scholar 

  28. Khodakovsky, A., Litke, N., Schroder, P.: Globally smooth parameterizations with low distortion. ACM Trans. Graph. 22(3), 350–357 (2003)

    Article  Google Scholar 

  29. Tutte, W.T.: How to draw a graph. Proc. London Math. Soc. 3(1), 743–767 (1963)

    Article  MathSciNet  Google Scholar 

  30. Floater, M.: One-to-one piecewise linear mappings over triangulations. Math. Comput. 72(242), 685–696 (2003)

    Article  MathSciNet  Google Scholar 

  31. Poulenard, A., Ovsjanikov, M.: Multi-directional geodesic neural networks via equivariant convolution. ACM Trans. Graph. 37(6), 1–14 (2018)

    Article  Google Scholar 

  32. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Transa. Graph. 38(5), 1–12 (2019)

    Article  Google Scholar 

  33. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  34. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  35. Xin, S.Q., Wang, G.J.: Improving Chen and Han’s algorithm on the discrete geodesic problem. ACM Trans. Graph. 28(4), 1–8 (2009)

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported in part by the National Key R&D Program of China under Grant No. 2017YFF0106407, Deakin University (Australia) internal grant (CY01-251301-F003-PJ03906-PG00447) and research grant (PJ06625), National Natural Science Foundation of China under Grant No. 61532002, and National Science Foundation of USA under Grant IIS-1715985 and IIS-1812606.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xuequan Lu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, D. et al. (2020). Deep Patch-Based Human Segmentation. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63830-6_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63829-0

  • Online ISBN: 978-3-030-63830-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics