Skip to main content
SpringerLink
Log in

Sketch-based image retrieval using keyshapes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Although sketch based image retrieval (SBIR) is still a young research area, there are many applications capable of exploiting this retrieval paradigm, such as web searching and pattern detection. Moreover, nowadays drawing a simple sketch query turns very simple since touch screen based technology is being expanded. In this work, we propose a novel local approach for SBIR based on detecting simple shapes which are named keyshapes. Our method works as a local strategy, but instead of detecting keypoints, it detects keyshapes over which local descriptors are computed. Our proposal based on keyshapes allow us to represent the structure of the objects in an image which could be used to increase the effectiveness in the retrieval task. Indeed, our results show an improvement in the retrieval effectiveness with respect to the state of the art. Furthermore, we demonstrate that combining our keyshape approach with a Bag of Feature approach allows us to achieve significant improvement with respect to the effectiveness of the retrieval task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24:509–522

    Article  Google Scholar 

  2. Borgefors G (1988) Hierarchical chamfer matching: a parametric edge matching algorithm. IEEE Trans Pattern Anal Mach Intell 10(6):849–865

    Article  Google Scholar 

  3. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698

    Article  Google Scholar 

  4. Cao Y, Wang C, Zhang L, Zhang L (2011) Edgel index for large-scale sketch-based image search. In: Proceedings of the 2011 IEEE conference on computer vision and pattern recognition. IEEE Computer Society, pp 761–768

  5. Chalechale A, Naghdy G, Mertins A (2005) Sketch-based image matching using angular partitioning. IEEE Trans Syst Man Cybern Syst Hum 35(1):28–41

    Article  Google Scholar 

  6. Chen T, Cheng MM, Tan P, Shamir A, Hu SM (2009) Sketch2photo: internet image montage. ACM Trans Graph 28(5):124:1–124:10

    Google Scholar 

  7. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1. IEEE Computer Society, pp 886–893

  8. Del Bimbo A, Pala P (1997) Visual image retrieval by elastic matching of user sketches. IEEE Trans Pattern Anal Mach Intell 19(2):121–132

    Article  Google Scholar 

  9. Eitz M, Hildebrand K, Boubekeur T, Alexa M (2009) A descriptor for large scale image retrieval based on sketched feature lines. In: Proc. of the 6th eurographics symposium on sketch-based interfaces and modeling, pp 29–36

  10. Eitz M, Hildebrand K, Boubekeur T, Alexa M (2009) Photosketch: a sketch based image query and compositing system. In: SIGGRAPH 2009: Talks, SIGGRAPH ’09, pp 60:1–60:1

  11. Eitz M, Hildebrand K, Boubekeur T, Alexa M (2011) Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE Trans Vis Comput Graph 17(11):1624–1636

    Article  Google Scholar 

  12. Funkhouser T, Min P, Kazhdan M, Chen J, Halderman A, Dobkin D, Jacobs D (2003) A search engine for 3d models. ACM Trans Graph 22(1):83–105

    Article  Google Scholar 

  13. Gonzalez R, Woods R (2008) Digital image processing, 3rd edn. Pearson Prentice Hall, New Jersey

    Google Scholar 

  14. Guo Z, Hall RW (1989) Parallel thinning with two-subiteration algorithms. Commun ACM 32(3):359–373

    Article  MathSciNet  Google Scholar 

  15. Hu R, Barnard M, Collomosse J (2010) Gradient field descriptor for sketch based retrieval and localization. In: 17th IEEE International conference on image processing (ICIP), 2010, pp 1025–1028

  16. Hu R, Wang T, Collomosse J (2011) A bag-of-regions approach to sketch-based image retrieval. In: 18th IEEE International conference on image processing (ICIP), pp 3661–3664

  17. Kovesi PD (2000) MATLAB and octave functions for computer vision and image processing. School of Computer Science & Software Engineering, The University of Western Australia. Available from: http://www.csse.uwa.edu.au/~pk/research/matlabfns/

  18. Kuhn HW (2010) The hungarian method for the assignment problem. In: 50 Years of integer programming 1958–2008, pp 29–47

  19. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  20. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: 8th International conference on computer vision, vol 2, pp 416–423

  21. Martínez JM (2002) MPEG-7: overview of MPEG-7 description tools, part 2. IEEE Multimedia 9(3):83–93

    Article  Google Scholar 

  22. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630

    Article  Google Scholar 

  23. Saavedra J, Bustos B (2010) An improved histogram of edge local orientations for sketch-based image retrieval. In: 32nd annual symposium of the German association for pattern recognition (DAGM). no. 6376 in lecture notes in computer science. Springer-Verlag, pp 432–441

  24. Saavedra JM, Bustos B, Scherer M, Schreck T (2011) Stela: sketch-based 3d model retrieval using a structure-based local approach. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR ’11. ACM, pp 26:1–26:8

  25. Stenger B, Thayananthan A, Torr PHS, Cipolla R (2006) Model-based hand tracking using a hierarchical bayesian filter. IEEE Trans Pattern Anal Mach Intell 28(9):1372–1384

    Article  Google Scholar 

  26. Sun Won C, Kwon Park D, Park SJ (2002) Efficient use of MPEG-7 edge histogram descriptor. ETRI 24:23–30

    Article  Google Scholar 

  27. Tuytelaars T, Mikolajczyk K (2008) Local invariant feature detectors: a survey. Now Publishers Inc., Hanover, MA

    Google Scholar 

  28. Wong J (1979) A new implementation of an algorithm for the optimal assignment problem: an improved version of munkres’ algorithm. BIT 19:418–424

    Article  MATH  MathSciNet  Google Scholar 

  29. Xu L, Oja E, Kultanen P (1990) A new curve detection method: Randomized Hough Transform (RHT). Pattern Recogn Lett 11:331–338

    Article  MATH  Google Scholar 

  30. Yao J, Kharma N, Grogono P (2005) A multi-population genetic algorithm for robust and fast ellipse detection. Pattern Anal Applic 8:149–162

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgement

We thank CONICYT-CHILE for supporting this work through the doctoral scholarship number 63080026.

Author information

Authors and Affiliations

  1. PRISMA Research Group, Department of Computer Science, University of Chile, Av. Blanco Encalada 2120, Santiago, Chile

    Jose M. Saavedra & Benjamin Bustos

  2. ORAND S.A., Santiago, Chile

    Jose M. Saavedra

Authors
  1. Jose M. Saavedra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin Bustos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jose M. Saavedra.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Saavedra, J.M., Bustos, B. Sketch-based image retrieval using keyshapes. Multimed Tools Appl 73, 2033–2062 (2014). https://doi.org/10.1007/s11042-013-1689-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1689-0

Keywords

Search

Navigation