Sketch-Based Retrieval Using Content-Aware Hashing

  • Shuang Liang
  • Long Zhao
  • Yichen Wei
  • Jinyuan Jia
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8879)


In this paper, we introduce a generic hashing-based approach. It aims to facilitate sketch-based retrieval on large datasets of visual shapes. Unlike previous methods where visual descriptors are extracted from overlapping grids, a content-aware selection scheme is proposed to generate candidate patches instead. Meanwhile, the saliency of each patch is efficiently estimated. Locality-sensitive hashing (LSH) is employed to integrate and capture both the content and saliency of patches, as well as the spatial information of visual shapes. Furthermore, hash codes are indexed so that a query can be processed in sub-linear time. Experiments on three standard datasets in terms of hand drawn shapes, images and 3D models demonstrate the superiority of our approach.


sketch-based retrieval LSH content-aware windows 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bay, H., Tuytelaars, T., Van Gool, L.: Surf: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  2. 2.
    Bozas, K., Izquierdo, E.: Large Scale Sketch Based Image Retrieval Using Patch Hashing. In: Bebis, G., et al. (eds.) ISVC 2012, Part I. LNCS, vol. 7431, pp. 210–219. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  3. 3.
    Canny, J.: A Computational Approach to Edge Detection. IEEE Transactions on PAMI Pattern Analysis and Machine Intelligence 8(6), 679–698 (1986)CrossRefGoogle Scholar
  4. 4.
    Charikar, M.S.: Similarity estimation techniques from rounding algorithms. In: STOC (2002)Google Scholar
  5. 5.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2005)Google Scholar
  6. 6.
    Eitz, M., Hays, J., Alexa, M.: How Do Humans Sketch Objects? In: SIGGRAPH (2012)Google Scholar
  7. 7.
    Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors. IEEE Transactions on Visualization and Computer Graphics 17(11), 1624–1636 (2011)CrossRefGoogle Scholar
  8. 8.
    Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., Alexa, M.: Sketch-Based Shape Retrieval. In: SIGGRAPH (2012)Google Scholar
  9. 9.
    Fonseca, M., Ferreira, A., Jorge, J.: Sketch-based Retrieval of Vector Drawings. In: Sketch-based Interfaces and Modeling, pp. 181–201. Springer, London (2011)CrossRefGoogle Scholar
  10. 10.
    Furuya, T., Ohbuchi, R.: Visual saliency weighting and cross-domain manifold ranking for sketch-based image retrieval. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014, Part I. LNCS, vol. 8325, pp. 37–49. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  11. 11.
    Liang, S., Sun, Z.: Sketch retrieval and relevance feedback with biased SVM classification. Pattern Recognition Letters 29(12), 1733–1741 (2008)CrossRefGoogle Scholar
  12. 12.
    Liang, S., Sun, Z., Li, B.: Sketch Retrieval Based on Spatial Relations. In: CGIV (2005)Google Scholar
  13. 13.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 42(3), 145–175 (2001)CrossRefGoogle Scholar
  14. 14.
    Ma, C., Yang, X., Zhang, C., Ruan, X., Yang, M.H.: Sketch Retrieval via Dense Stroke Features. In: BMVC (2013)Google Scholar
  15. 15.
    Mikolajczyk, K., Schmid, C.: Scale and Affine Invariant Interest Point Detectors. IJCV 60(1), 63–86 (2004)CrossRefGoogle Scholar
  16. 16.
    Norouzi, M., Punjani, A., Fleet, D.J.: Fast Search in Hamming Space with Multi-Index Hashing. In: CVPR (2012)Google Scholar
  17. 17.
    Perazzi, F., Kr, P., Krahenbuhl, P., Pritch, Y., Hornung, A.: Saliency Filters: Contrast Based Filtering for Salient Region Detection. In: CVPR (2012)Google Scholar
  18. 18.
    Saavedra, J.M., Bustos, B.: Sketch-based image retrieval using keyshapes. In: Multimedia Tools and Applications, pp. 1–30. Springer, US (2013)Google Scholar
  19. 19.
    Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton Shape Benchmark. In: Shape Modeling International (2004)Google Scholar
  20. 20.
    Sun, X., Wang, C., Xu, C., Zhang, L.: Indexing Billions of Images for Sketch-based Retrieval. In: ACM Multimedia (2013)Google Scholar
  21. 21.
    Wang, C., Cao, Y., Zhang, L.: MindFinder: A Sketch-based Image Search Engine based on Edgel Index. In: CVPR (2011)Google Scholar
  22. 22.
    Yoon, S.M., Scherer, M., Schreck, T., Kuijper, A.: Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In: ACM Multimedia (2010)Google Scholar
  23. 23.
    Zhang, L., Zhang, Y., Tang, J., Lu, K., Tian, Q.: Binary Code Ranking with Weighted Hamming Distance. In: CVPR (2013)Google Scholar
  24. 24.
    Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency Optimization from Robust Background Detection. In: CVPR (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Shuang Liang
    • 1
  • Long Zhao
    • 1
  • Yichen Wei
    • 2
  • Jinyuan Jia
    • 1
  1. 1.Tongji UniversityShanghaiChina
  2. 2.Microsoft Research AsiaBeijingChina

Personalised recommendations