Advertisement

The Visual Computer

, Volume 31, Issue 9, pp 1147–1161 | Cite as

3D shape creation by style transfer

  • Zhizhong Han
  • Zhenbao Liu
  • Junwei Han
  • Shuhui Bu
Original Article

Abstract

In this paper, we propose a new style transfer method for automatic 3D shape creation based on new concepts of style and content of 3D shapes. Our unsupervised style transfer method could plausibly create novel shapes not only by recombining existent styles and contents in a set but also by combining new-coming styles or contents with the existent ones conveniently. This feature provides a better way to increase the diversity of created shapes. The process of shape creation can be summarized as two stages. First, style and content separation is performed to analyzed shapes in a set. Second, novel shapes are created by style transfer. In our setting, contents are first separated via clustering shapes using a new defined global shape distance, and then, style parts are clustered into different style classes. Specifically, style parts are extracted from each pair of intra-content shapes through comparing their multi-scale corresponding patches instead of corresponding parts. This strategy makes the process of extracting style parts become insensitive to slight geometric changes. The multi-scale corresponding patches are obtained via partitioning the two shapes in a consistent way by the proposed correspondence transfer. Meanwhile, to quantify the comparison results for locating style parts, a novel local shape difference function (LSDF) is introduced. Based on LSDF, extracting a style part from each shape is formulated as an optimal LSDF threshold selection problem. In the experiments, we test our method in several sets of man-made 3D shapes and obtain plausible created shapes based on the reasonably separated styles and contents.

Keywords

Style and content separation Style transfer Shape creation Local shape difference function 

Notes

Acknowledgments

This work was supported partly by grants from National Natural Science Foundation of China (61003137, 61202185, 61005018, 91120005), Northwestern Polytechnical University Basic Research Fund (310201401JCQ01009, 310201401JCQ01012), the Fundamental Research Funds for the Central Universities, Shaanxi Natural Science Fund (2012JQ8037), and Open Project Program of the State Key Lab of CAD&CG (A1306), Zhejiang University, Program for New Century Excellent Talents in University under grant NCET-10-0079, and Doctoral Fund of Ministry of Education of China under grant 20136102110037.

References

  1. 1.
    Attene, M., Falcidieno, B.: Remesh: An interactive environment to edit and repair triangle meshes. In: Proceeding of Shape Modeling International, pp. 41–46 (2006)Google Scholar
  2. 2.
    Aubry, M., Schlickewei, U., Cremers, D.: The wave kernel signature: a quantum mechanical approach to shape analysis. In: ICCV Workshops, pp. 1626–1633 (2011)Google Scholar
  3. 3.
    Besl, P.J., McKay, N.D.: A method for registration for 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)CrossRefGoogle Scholar
  4. 4.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of SIGGRAPH, pp. 187–194 (1999)Google Scholar
  5. 5.
    Brand, M., Hertzmann, A.: Style machines. In: Proceedings of SIGGRAPH, pp. 183–192 (2000)Google Scholar
  6. 6.
    Bronstein, A.M., Bronstein, M.M., Kimmel, R., Mahmoudi, M., Sapiro, G.: A gromov-hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. Int. J. Comput. Vision Mmanuscr. 89(2–3), 266–286 (2010)CrossRefGoogle Scholar
  7. 7.
    Chaudhuri, S., Kalogerakis, E., Guibas, L., Koltun, V.: Probabilistic reasoning for assembly-based 3D modeling. ACM Trans. Graphics 30(4), 35:1–35:10 (2011)CrossRefGoogle Scholar
  8. 8.
    Donoho, D.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Elgammal, A.M., Lee, C.S.: Separating style and content on a nonlinear manifold. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 478–485 (2004)Google Scholar
  10. 10.
    Halkidi, M., Batistakis, Y., Vazirgiannis, M.: On clustering validation techniques. J. Intell. Inf. Syst. 17(2–3), 107–145 (2001)CrossRefzbMATHGoogle Scholar
  11. 11.
    Huang, H., Gong, M., Cohen-Or, D., Ouyang, Y., Tan, F., Zhang, H.: Field-guided registration for feature-conforming shape composition. ACM Trans. Graphics 31, 171:1–171:11 (2012)Google Scholar
  12. 12.
    Jain, V., Zhang, H.: Robust 3d shape correspondence in the spectral domain. In: Proceedings of Shape Modeling International, pp. 118–129 (2006)Google Scholar
  13. 13.
    Jain, V., Zhang, H.: A spectral approach to shape-based retrieval of articulated 3D models. Comput. Aided Design 39(5), 398–407 (2007)CrossRefGoogle Scholar
  14. 14.
    Kalogerakis, E., Chaudhuri, S., Koller, D., Koltun, V.: A probabilistic model of component-based shape synthesis. ACM Trans. Graphics 31(4), 55:1–55:11 (2012)CrossRefGoogle Scholar
  15. 15.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 65–81 (2004)CrossRefGoogle Scholar
  16. 16.
    Kortgen, M., Novotni, M., Klein, R.: 3D shape matching with 3D shape contexts. In: Proceedings of Central European on Computer Graphics (2003)Google Scholar
  17. 17.
    Li, H., Zhang, H., Wang, Y., Cao, J., Shamir, A., Cohen-Or, D.: Curve style analysis in a set of shapes. Comput. Graphics Forum 32(6), 77–88 (2013)CrossRefGoogle Scholar
  18. 18.
    Osada, R., Funkhouser, T., Chazelle, B., Dobkin, D.: Shape distributions. ACM Trans. Graphics 21(4), 807–832 (2002)CrossRefGoogle Scholar
  19. 19.
    Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.J.: Functional maps: a flexible representation of maps between shapes. ACM Trans. Graphics 31(4), 30:1–30:11 (2012)CrossRefGoogle Scholar
  21. 21.
    Rustamov, R.M., Ovsjanikov, M., Azencot, O., Ben-Chen, M., Chazal, F., Guibas, L.: Map-based exploration of intrinsic shape differences and variability. ACM Trans. Graphics 32(4), 72:1–72:12 (2013)CrossRefGoogle Scholar
  22. 22.
    Shamir, A.: A formulation of boundary mesh segmentation. In: Proceedings of International Symposium on 3D Data Processing, Visualization and Transmission, pp. 82–89 (2004)Google Scholar
  23. 23.
    Shapira, L., Shamir, A., Cohen-Or, D.: Consistent mesh partitioning and skeletonisation using the shape diameter function. Visual Comput. 24(4), 249–259 (2008)CrossRefGoogle Scholar
  24. 24.
    Shilane, P., Funkhouser, T.: Distinctive regions of 3D surfaces. ACM Trans. Graphics 26(2), 1–15 (2007)CrossRefGoogle Scholar
  25. 25.
    Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton shape benchmark. In: Proceedings of Shape Modeling International, pp. 167–178 (2004)Google Scholar
  26. 26.
    Tanenbaum, J., Freeman, W.: Separating style and content with bilinear models. Neural Comput. 12(6), 1247–1283 (2000)CrossRefGoogle Scholar
  27. 27.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Multifactor gaussian process models for style-content separation. In: Proceedings of International Conference on Machine learning, pp. 975–982 (2007)Google Scholar
  28. 28.
    Xu, K., Li, H., Zhang, H., Cohen-Or, D., Xiong, Y., Cheng, Z.Q.: Style-content separation by anisotropic part scales. ACM Trans. Graph. 29(6), 184:1–184:10 (2010) Google Scholar
  29. 29.
    Xu, K., Zhang, H., Cohen-Or, D., Chen, B.: Fit and diverse: set evolution for inspiring 3D shape galleries. ACM Trans. Graphics 31(4), 57:1–57:10 (2012)CrossRefGoogle Scholar
  30. 30.
    Zheng, Y., Cohen-Or, D., Mitra, N.J.: Smart variations: functional substructures for part compatibility. Comput. Graphics Forum 32(2pt2), 195–204 (2013)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Zhizhong Han
    • 1
  • Zhenbao Liu
    • 1
  • Junwei Han
    • 1
  • Shuhui Bu
    • 1
  1. 1.Xi’anChina

Personalised recommendations