Skip to main content
SpringerLink
Log in

Sketch based modeling and editing via shape space exploration

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a framework that takes a user provided free-hand sketch as input and realistic 3D shape as result. Specifically, the proposed approach retrieves 3D shape based on a sketch, and then a deformation procedure is carried out for a further consistency with the provided sketch. In the retrieval stage, a locality preserving view selection scheme is proposed, which suggests views that are mostly possible to be views when creating a sketch. The proposed criterion predicates the sketch views accurately while significantly reduces the number of views that need to be rendered. In the deformation stage, retrieved shapes are modified according to sketch. However, it is a tremendously difficult job as free-hand sketches always contain various kinds of drawing errors such as stroke jittering and asymmetry. Extracting plausible deformation from sketch while discarding undesirable drawing errors is difficult. To address these issues, we obtain the plausible deformation contained in sketch by exploring a shape space trained by a collection of shapes in the same category. Furthermore, we also develop a user interface with two different intuitive editing modes based on the shape space established. Experimental results show that more consistent 3D shapes with sketches can be obtained by our method. We also illustrate that our proposed method provides easy and smart ways for both a heuristic and a purposeful shape editing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. http://web.siat.ac.cn/∼yunhai/ssl/ssd.htm

References

  1. Allen B, Curless B, Popovi Z, Allen B, Popovi Z (2003) The space of human body shapes. Acm Transactions on Graphics 22(3):587–594

    Google Scholar 

  2. Andriy M, Xubo S (2010) Point set registration: Coherent point drift. IEEE Transactions on Pattern Analysis & Machine Intelligence 32(12):2262–75

    Google Scholar 

  3. Averkiou M, Kim V, Zheng Y, Mitra NJ (2014) Shapesynth: Parameterizing model collections for coupled shape exploration and synthesis. Computer Graphics Forum (Special issue of Eurographics 2014)

  4. Belongie S, Malik J, Puzicha J (2000) Shape context: a new descriptor for shape matching and object recognition, vol 2, p 3

  5. Blanz V, Vetter T (2015) A morphable model for the synthesis of 3d faces. Computer Graphics Proceedings Siggraph’, pp 187–194

  6. Campbell NDF, Kautz J (2014) Learning a manifold of fonts. ACM Trans Graphic (TOG) 33(4):91

    MATH  Google Scholar 

  7. Chaudhuri S, Kalogerakis E, Giguere S, Funkhouser T (2013) Attribit: content creation with semantic attributes. In: ACM Symposium on User Interface Software and Technology, pp 193–202

  8. Chen D-Y, Tian X-P, Shen Y-T, Ouhyoung M (2003) On visual similarity based 3d model retrieval. In: Computer Graphics Forum, vol 22, Wiley online library, pp 223–232

  9. Cook MT, Agah A (2009) A survey of sketch-based 3-d modeling techniques. Interact Comput 21(3):201–211

    Google Scholar 

  10. Delanoy J, Bousseau A, Aubry M, Isola P, Efros AA (2017) What you sketch is what you get: 3d sketching using multi-view deep volumetric prediction. CoRR, arXiv:1707.08390

  11. Ding C, Liu L (2016) A survey of sketch based modeling systems. Front Comput Sci 10(6):985–999

    Google Scholar 

  12. Eggli L, Hsu C-y, Bruederlin BD, Elber G (1997) Inferring 3d models from freehand sketches and constraints. Comput Aided Des 29(2):101–112

    Google Scholar 

  13. Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M (2012) Sketch-based shape retrieval. ACM Trans Graph 31(4):31

    Google Scholar 

  14. Fan L, Wang R, Xu L, Deng J, Liu L (2013) Modeling by drawing with shadow guidance. In: Computer Graphics Forum, vol 32, Wiley Online Library, pp 157–166

  15. Huffman DA (1971) Impossible objects as nonsense sentences. Mach Intell 6 (1):295–323

    Google Scholar 

  16. Igarashi T (2005) As-rigid-as-possible shape manipulation. Acm Trans Graphic 24(3):1134–1141

    Google Scholar 

  17. Igarashi T, Matsuoka S, Tanaka H (2007) Teddy: a sketching interface for 3d freeform design. In: Acm siggraph 2007 Courses, ACM, p 21

  18. Kara LB, Shimada K (2006) Construction and modification of 3d geometry using a sketch-based interface. In: Eurographics Workshop on Sketch-based Interfaces and Modeling, pp 59–66

  19. Kraevoy V, Dan J, Sheffer A (2007) Shuffler: Modeling with interchangeable parts. Cs.ubc.ca

  20. Lee J, Funkhouser TA (2008) Sketch-based search and composition of 3d models. In: SBM, pp 97–104

  21. Li C, Pan H, Liu Y, Tong X, Sheffer A, Wang W (2017) Bendsketch: Modeling freeform surfaces through 2d sketching. ACM Trans Graph 36(4):125:1–125:14

    Google Scholar 

  22. Liu J, Lian Z, Feng J, Zhou B (2015) Sketch based modeling via manifold regularization. In: SIGGRAPH ASIA 2015 Technical Briefs

  23. Masry M, Lipson H (2007) A sketch-based interface for iterative design and analysis of 3d objects. In: ACM SIGGRAPH 2007 Courses, ACM, p 31

  24. Mitani J, Suzuki H, Kimura F (2002) 3d sketch: sketch-based model reconstruction and rendering. In: From Geometric Modeling to Shape Modeling, Springer, pp 85–98

  25. Neil D (2004) Lawrence. Gaussian process latent variable models for visualisation of high dimensional data. Advances in Neural Information Processing Systems 16 (3):329–336

    Google Scholar 

  26. Niyogi X (2004) Locality preserving projections. In: Neural Information Processing Systems, vol 16, MIT, p 153

  27. Pu J, Ramani K (2005) A 3d model retrieval method using 2d freehand sketches. In: Computational Science–ICCS 2005, Springer, pp 343–346

  28. Secord A, Lu J, Finkelstein A, Singh M, Nealen A (2011) Perceptual models of viewpoint preference. ACM Trans Graphic (TOG) 30(5):109

    Google Scholar 

  29. Shilane P, Min P, Kazhdan M, Funkhouser T (2004) The princeton shape benchmark. In: Shape Modeling Applications, 2004. Proceedings, IEEE, pp 167–178

  30. Shin HJ, Igarashi T (2007) Magic canvas: interactive design of a 3-d scene prototype from freehand sketches. In: Proceedings of Graphics Interface 2007, ACM, pp 63–70

  31. Smirnov D, Bessmeltsev M, Solomon J (2019) Deep sketch-based modeling of man-made shapes. CoRR, arXiv:1906.12337

  32. Tipping ME, Bishop CM (1999) Probabilistic principal component analysis. J R Stat Soc 61(3):611–622

    MathSciNet  MATH  Google Scholar 

  33. Wang L, Qian C, Wang J, Yi F (2018) Unsupervised learning of 3d model reconstruction from hand-drawn sketches. In: ACM Multimedia 2018, pp 1820–1828, 10

  34. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: A deep representation for volumetric shapes. 1912–1920, 06

  35. Xie X, Xu K, Mitra NJ, Cohen-Or D, Gong W, Su Q, Chen B (2013) Sketch-to-design: Context-based part assembly. In: Computer Graphics Forum, vol 32, Wiley Online Library, pp 233–245

  36. Xu B, Chang W, Sheffer A, Bousseau A, Mccrae J, Singh K (2014) True2form: 3d curve networks from 2d sketches via selective regularization. ACM Transactions on Graphics, 33(4)

  37. Xu K, Kim VG, Huang Q, Kalogerakis E (2015) Data-driven shape analysis and processing. 1502.06686

  38. Xu K, Zhang H, Cohen-Or D, Chen B (2012) Fit and diverse: Set evolution for inspiring 3d shape galleries. Acm Trans Graphic 31(4):4

    MathSciNet  Google Scholar 

  39. Xu K, Zheng H, Zhang H, Cohen-Or D, Liu L, Xiong Y (2011) Photo-inspired model-driven 3d object modeling. ACM Trans on Graphic (TOG) 30 (4):80

    Google Scholar 

  40. Yang C, Sharon D, van de Panne M (2005) Sketchbased modeling of parameterized objects. In: EG Workshop on Sketch-based Interfaces and Modeling, pp 63–72

  41. Yoon SM, Scherer M, Schreck T, Kuijper A (2010) Sketch-based 3d model retrieval using diffusion tensor fields of suggestive contours. In: Proceedings of the International Conference on Multimedia, ACM, pp 193–200

  42. Yumer ME, Chaudhuri S, Hodgins JK, Kara LB (2015) Semantic shape editing using deformation handles. Acm Trans Graphic 34(4):1–12

    Google Scholar 

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (Grant No.: 61672043 and 61672056 and Key Laboratory of Science, Technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology).

Author information

Authors and Affiliations

  1. Wangxuan Institute of Computer Technology, Peking University, Beijing, China

    Juncheng Liu, Zhouhui Lian & Jianguo Xiao

Authors
  1. Juncheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhouhui Lian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianguo Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhouhui Lian.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 58.0 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Lian, Z. & Xiao, J. Sketch based modeling and editing via shape space exploration. Multimed Tools Appl 79, 18121–18142 (2020). https://doi.org/10.1007/s11042-020-08677-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08677-0

Keywords

Search

Navigation