Skip to main content
SpringerLink
Log in

Sketch-based 3D model retrieval by incorporating 2D-3D alignment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Sketch-based 3D model retrieval is very important for applications such as 3D modeling and recognition. In this paper, a sketch-based retrieval algorithm is proposed based on a 3D model feature named View Context and 2D relative shape context matching. To enhance the accuracy of 2D sketch-3D model correspondence as well as the retrieval performance, we propose to align a 3D model with a query 2D sketch before measuring their distance. First, we efficiently select some candidate views from a set of densely sampled views of the 3D model to align the sketch and the model based on their View Context similarities. Then, we compute the more accurate relative shape context distance between the sketch and every candidate view, and regard the minimum one as the sketch-model distance. To speed up retrieval, we precompute the View Context and relative shape context features of the sample views of all the 3D models in the database. Comparative and evaluative experiments based on hand-drawn and standard line drawing sketches demonstrate the effectiveness and robustness of our approach and it significantly outperforms several latest sketch-based retrieval algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Adamek T, O’Connor NE (2004) A multiscale representation method for nonrigid shapes with a single closed contour. IEEE Trans Circuits Syst Video Technol 14(5):742–753

    Article  Google Scholar 

  2. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522

    Article  Google Scholar 

  3. Cao L, Liu J, Tang X (2006) 3D object retrieval using 2D line drawing and graph based relevance reedback. In: Proc. of the 14th annual ACM international conference on Multimedia, MULTIMEDIA ’06, pp 105–108

  4. Chalechale A, Mertins A, Naghdy G (2004) Edge image description using angular radial partitioning. IEE Proc, Vis Image Signal Process 151(2):28–41

    Article  Google Scholar 

  5. Chen DY, Tian XP, Shen YT, Ouhyoung M (2003) On visual similarity based 3D model retrieval. Comput Graph Forum 22(3):223–232

    Article  Google Scholar 

  6. Cole F, Golovinskiy A, Limpaecher A, Barros HS, Finkelstein A, Funkhouser T, Rusinkiewicz S (2008) Where do people draw lines? ACM Trans Graph 27(3):1–11

    Article  Google Scholar 

  7. Doug D, Adam F, Szymon R, Anthony S (2003) Suggestive contours for conveying shape. ACM Trans Graph (Proc. SIGGRAPH 2003) 22(3):848–855

    Article  Google Scholar 

  8. Fang R, Godil A, Li X, Wagan A (2008) A new shape benchmark for 3D object retrieval. In: Proc. of the 4th international symposium on advances in visual computing (ISVC), pp 381–392

  9. Feldmar J, Ayache N, Betting F (1997) 3D-2D projective registration of free-form curves and surfaces. Comput Vis Image Underst 65(3):403–424

    Article  Google Scholar 

  10. Fonseca MJ, Ferreira A, Jorge JA (2004) Towards 3D modeling using sketches and retrieval. In: Sketch-based interfaces and modeling, pp 127–136

  11. Funkhouser TA, Min P, Kazhdan MM, Chen J, Halderman JA, Dobkin DP, Jacobs DP (2003) A search engine for 3D models. ACM Trans Graph 22(1):83–105

    Article  Google Scholar 

  12. Hou S, Ramani K (2006) Sketch-based 3D engineering part class browsing and retrieval. In: Proc. of EUROGRAPHICS workshop on sketch-based interfaces and modeling, pp 131–138

  13. Hou S, Ramani K (2007) Classifier combination for sketch-based 3D part retrieval. Comput Graph 31(4):598–609

    Article  Google Scholar 

  14. Huttenlocher D, Ullman S (1987) Object recognition using alignment. In: Proc. of IEEE international conference on computer vision (ICCV), pp 102–111

  15. Jonker R, Volgenant A (1987) A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing 38(4):325–340

    Article  MathSciNet  MATH  Google Scholar 

  16. Kanai S (2008) Content-based 3D mesh model retrieval from hand-written sketch. Int J Interact Des Manuf 2(2):87–98

    Article  Google Scholar 

  17. Kazhdan M, Funkhouser T, Rusinkiewicz S (2003) Rotation invariant spherical harmonic representation of 3D shape descriptors. In: Proc. of the symposium on geometry processing, pp 156–164

  18. Lee J, Funkhouser T (2008) Sketch-based search and composition of 3D models. In: Proc. of EUROGRAPHICS workshop on sketch-based interfaces and modeling, pp 97–104

  19. Li B, Johan H (2010) View context: a 3D model feature for retrieval. In: Boll S et al (eds) MMM 2010. LNCS, vol 5916. Springer, Heidelberg, pp 185–195

    Google Scholar 

  20. Li B, Johan H (2011) View context based 2D sketch-3D model alignment. In: 2011 IEEE workshop on applications of computer vision (WACV), pp 45–50

  21. Li B, Johan H (2011) 3D model retrieval using hybrid features and class information. Multimedia tools and applications, pp 1–26 (online first version)

  22. Loop C (1987) Smooth subdivision surfaces based on triangles. Master thesis, University of Utah

  23. Mori G, Belongie S, Malik J (2005) Efficient shape matching using shape contexts. IEEE Trans Pattern Anal Mach Intell 27(11):1832–1837

    Article  Google Scholar 

  24. Napoléon T, Sahbi H (2010) From 2D silhouettes to 3D object retrieval: contributions and benchmarking. EURASIP J Image Video Process 2010:1–17

    Article  Google Scholar 

  25. Napoléon T, Sahbi H (2010) Sketch-driven mental 3D object retrieval. SPIE, p 75260L

  26. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29(1):51–59

    Article  Google Scholar 

  27. Olsen L, Samavati FF, Sousa MC, Jorge JA (2009) Sketch-based modeling: a survey. Comput Graph 33(1):85–103

    Article  Google Scholar 

  28. Osada R, Funkhouser T, Chazelle B, Dobkin D (2001) Matching 3D models with shape distributions. In: Proc. of shape modeling and applications, pp 154–166

  29. Pu J, Ramani K (2006) On visual similarity based 2D drawing retrieval. Comput-aided Des 38(3):249–259

    Article  Google Scholar 

  30. Pu J, Ramani K (2006) ShapeLab: a unified framework for 2D & 3D shape retrieval. In: Proc. of the 3rd international symposium on 3D data processing, visualization and transmission (3DPVT 2006), pp 1072–1079

  31. Buss SR (2003) 3D computer graphics: a mathematical introduction with OpenGL. Cambridge University Press, Cambridge, MA

    Book  Google Scholar 

  32. Saavedra J, Bustos B, Scherer M, Schreck T (2011) STELA: sketch-based 3D model retrieval using a structure-based local approach. In: Proc. of the first ACM international conference on multimedia retrieval (ICMR’11), p 26

  33. Saavedra JM, Bustos B (2010) An improved histogram of edge local orientations for sketch-based image retrieval. In: Goesele M, Roth S, Kuijper A, Schiele B, Schindler K (eds) DAGM-symposium. Lecture notes in computer science, vol 6376. Springer, pp 432–441

  34. Shilane P, Min P, Kazhdan M, Funkhouser T (2004) The Princeton shape benchmark. In: Proc. of shape modeling and applications, pp 167–178

  35. Snodgrass JG, Vanderwart M (1980) A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. J Exp Psychol Hum Learn Mem 6(2):174–215

    Article  Google Scholar 

  36. Veltkamp RC, ter Haar FB (2007) SHREC 2007 3D retrieval contest. Technical report UU-CS-2007-015, Department of Information and Computing Sciences, Utrecht University

  37. Vranic DV (2004) 3D model retrieval. PhD thesis, University of Leipzig

  38. Wang J, He Y, Tian H, Cai H (2008) Retrieving 3D CAD model by freehand sketches for design reuse. Adv Eng Inf 22(3):385–392

    Article  Google Scholar 

  39. Yoon SM, Scherer M, Schreck T, Kuijper A (2010) Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In: ACM multimedia, pp 193–200

  40. Zamora S, Sherwood T (2010) Sketch-based recognition system for general articulated skeletal figures. In: Proc. of EUROGRAPHICS workshop on sketch-based interfaces and modeling, pp 119–126

  41. Zhang D, Lu G (2002) Shape-based image retrieval using generic Fourier descriptor. Signal Process: Image Commun 17(10):825–848

    Article  Google Scholar 

  42. Zhang D, Luo G (2002) An integrated approach to shape based image retrieval. In: Proc. of the 5th Asian conference on computer vision (ACCV 2002), pp 652–657

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore

    Bo Li & Henry Johan

Authors
  1. Bo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Henry Johan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Li.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, B., Johan, H. Sketch-based 3D model retrieval by incorporating 2D-3D alignment. Multimed Tools Appl 65, 363–385 (2013). https://doi.org/10.1007/s11042-012-1009-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-012-1009-0

Keywords

Search

Navigation