Multimedia Tools and Applications

, Volume 73, Issue 3, pp 2033–2062 | Cite as

Sketch-based image retrieval using keyshapes



Although sketch based image retrieval (SBIR) is still a young research area, there are many applications capable of exploiting this retrieval paradigm, such as web searching and pattern detection. Moreover, nowadays drawing a simple sketch query turns very simple since touch screen based technology is being expanded. In this work, we propose a novel local approach for SBIR based on detecting simple shapes which are named keyshapes. Our method works as a local strategy, but instead of detecting keypoints, it detects keyshapes over which local descriptors are computed. Our proposal based on keyshapes allow us to represent the structure of the objects in an image which could be used to increase the effectiveness in the retrieval task. Indeed, our results show an improvement in the retrieval effectiveness with respect to the state of the art. Furthermore, we demonstrate that combining our keyshape approach with a Bag of Feature approach allows us to achieve significant improvement with respect to the effectiveness of the retrieval task.


Sketch-based image retrieval Content-based image retrieval Local descriptors Local matching 



We thank CONICYT-CHILE for supporting this work through the doctoral scholarship number 63080026.


  1. 1.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24:509–522CrossRefGoogle Scholar
  2. 2.
    Borgefors G (1988) Hierarchical chamfer matching: a parametric edge matching algorithm. IEEE Trans Pattern Anal Mach Intell 10(6):849–865CrossRefGoogle Scholar
  3. 3.
    Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698CrossRefGoogle Scholar
  4. 4.
    Cao Y, Wang C, Zhang L, Zhang L (2011) Edgel index for large-scale sketch-based image search. In: Proceedings of the 2011 IEEE conference on computer vision and pattern recognition. IEEE Computer Society, pp 761–768Google Scholar
  5. 5.
    Chalechale A, Naghdy G, Mertins A (2005) Sketch-based image matching using angular partitioning. IEEE Trans Syst Man Cybern Syst Hum 35(1):28–41CrossRefGoogle Scholar
  6. 6.
    Chen T, Cheng MM, Tan P, Shamir A, Hu SM (2009) Sketch2photo: internet image montage. ACM Trans Graph 28(5):124:1–124:10Google Scholar
  7. 7.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1. IEEE Computer Society, pp 886–893Google Scholar
  8. 8.
    Del Bimbo A, Pala P (1997) Visual image retrieval by elastic matching of user sketches. IEEE Trans Pattern Anal Mach Intell 19(2):121–132CrossRefGoogle Scholar
  9. 9.
    Eitz M, Hildebrand K, Boubekeur T, Alexa M (2009) A descriptor for large scale image retrieval based on sketched feature lines. In: Proc. of the 6th eurographics symposium on sketch-based interfaces and modeling, pp 29–36Google Scholar
  10. 10.
    Eitz M, Hildebrand K, Boubekeur T, Alexa M (2009) Photosketch: a sketch based image query and compositing system. In: SIGGRAPH 2009: Talks, SIGGRAPH ’09, pp 60:1–60:1Google Scholar
  11. 11.
    Eitz M, Hildebrand K, Boubekeur T, Alexa M (2011) Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE Trans Vis Comput Graph 17(11):1624–1636CrossRefGoogle Scholar
  12. 12.
    Funkhouser T, Min P, Kazhdan M, Chen J, Halderman A, Dobkin D, Jacobs D (2003) A search engine for 3d models. ACM Trans Graph 22(1):83–105CrossRefGoogle Scholar
  13. 13.
    Gonzalez R, Woods R (2008) Digital image processing, 3rd edn. Pearson Prentice Hall, New JerseyGoogle Scholar
  14. 14.
    Guo Z, Hall RW (1989) Parallel thinning with two-subiteration algorithms. Commun ACM 32(3):359–373CrossRefMathSciNetGoogle Scholar
  15. 15.
    Hu R, Barnard M, Collomosse J (2010) Gradient field descriptor for sketch based retrieval and localization. In: 17th IEEE International conference on image processing (ICIP), 2010, pp 1025–1028Google Scholar
  16. 16.
    Hu R, Wang T, Collomosse J (2011) A bag-of-regions approach to sketch-based image retrieval. In: 18th IEEE International conference on image processing (ICIP), pp 3661–3664Google Scholar
  17. 17.
    Kovesi PD (2000) MATLAB and octave functions for computer vision and image processing. School of Computer Science & Software Engineering, The University of Western Australia. Available from:
  18. 18.
    Kuhn HW (2010) The hungarian method for the assignment problem. In: 50 Years of integer programming 1958–2008, pp 29–47Google Scholar
  19. 19.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110CrossRefGoogle Scholar
  20. 20.
    Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: 8th International conference on computer vision, vol 2, pp 416–423Google Scholar
  21. 21.
    Martínez JM (2002) MPEG-7: overview of MPEG-7 description tools, part 2. IEEE Multimedia 9(3):83–93CrossRefGoogle Scholar
  22. 22.
    Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630CrossRefGoogle Scholar
  23. 23.
    Saavedra J, Bustos B (2010) An improved histogram of edge local orientations for sketch-based image retrieval. In: 32nd annual symposium of the German association for pattern recognition (DAGM). no. 6376 in lecture notes in computer science. Springer-Verlag, pp 432–441Google Scholar
  24. 24.
    Saavedra JM, Bustos B, Scherer M, Schreck T (2011) Stela: sketch-based 3d model retrieval using a structure-based local approach. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR ’11. ACM, pp 26:1–26:8Google Scholar
  25. 25.
    Stenger B, Thayananthan A, Torr PHS, Cipolla R (2006) Model-based hand tracking using a hierarchical bayesian filter. IEEE Trans Pattern Anal Mach Intell 28(9):1372–1384CrossRefGoogle Scholar
  26. 26.
    Sun Won C, Kwon Park D, Park SJ (2002) Efficient use of MPEG-7 edge histogram descriptor. ETRI 24:23–30CrossRefGoogle Scholar
  27. 27.
    Tuytelaars T, Mikolajczyk K (2008) Local invariant feature detectors: a survey. Now Publishers Inc., Hanover, MAGoogle Scholar
  28. 28.
    Wong J (1979) A new implementation of an algorithm for the optimal assignment problem: an improved version of munkres’ algorithm. BIT 19:418–424CrossRefMATHMathSciNetGoogle Scholar
  29. 29.
    Xu L, Oja E, Kultanen P (1990) A new curve detection method: Randomized Hough Transform (RHT). Pattern Recogn Lett 11:331–338CrossRefMATHGoogle Scholar
  30. 30.
    Yao J, Kharma N, Grogono P (2005) A multi-population genetic algorithm for robust and fast ellipse detection. Pattern Anal Applic 8:149–162CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.PRISMA Research Group, Department of Computer ScienceUniversity of ChileSantiagoChile
  2. 2.ORAND S.A.SantiagoChile

Personalised recommendations