Sketch-based Retrieval of Vector Drawings

  • Manuel J. Fonseca
  • Alfredo Ferreira
  • Joaquim A. Jorge


Most drawing (and image) retrieval systems typically use query-by-example mechanisms or textual queries. In this chapter, we use sketches to retrieve complex vector drawings, as a solution to supplement and overcome limitations of conventional query methods.

We start by describing topological and geometric information extracted from drawings and mechanisms to convert them into feature vectors. However, comparing simple sketches to complex drawings requires new and innovative techniques to be successful. We describe a new multilevel approach, which allows for a coarse-to-detailed comparison between sketched queries and indexed drawings.

We present two application examples, one to retrieve clip art figures and another for CAD drawings, which illustrate this sketch-based retrieval technique. Then we describe a new paradigm of implicit retrieval, which combines sketch-based modeling techniques with 3D retrieval mechanisms to enrich the modeling process. Finally, we discuss the implications of our techniques to sketch-based interfaces in general.


Topological Information Topology Graph Retrieval Mechanism Simple Sketch Visual Entity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Berchtold, S., Kriegel, H.P.: S3: Similarity in CAD database systems. In: Proceedings of the International Conference on Management of Data (SIGMOD’97), pp. 564–567. ACM Press, Tucson (1997) Google Scholar
  2. 2.
    Cvetković, D., Rowlinson, P., Simic, S.: Eigenspaces of Graphs. Cambridge University Press, Cambridge (1997) zbMATHCrossRefGoogle Scholar
  3. 3.
    Egenhofer, M.J.: A formal definition of binary topological relationships. In: Litwin, W., Schek, H. (eds.) Third International Conference on Foundations of Data Organization and Algorithms (FODO’89). Lecture Notes in Computer Science, vol. 367, pp. 457–472. Springer, Berlin (1989) CrossRefGoogle Scholar
  4. 4.
    Egenhofer, M.J., Al-Taha, K.K.: Reasoning about gradual changes of topological relationships. In: Frank, A., Campari, I., Formentini, U. (eds.) Theory and Methods of Spatio-Temporal Reasoning in Geographic Space. Lecture Notes in Computer Science, vol. 639, pp. 196–219. Springer, Berlin (1992) CrossRefGoogle Scholar
  5. 5.
    Fonseca, M.J.: Sketch-based retrieval in large sets of drawings. Ph.D. thesis, Instituto Superior Técnico/Technical University of Lisbon (2004) Google Scholar
  6. 6.
    Fonseca, M.J., Barroso, B., Ribeiro, P., Jorge, J.A.: Retrieving ClipArt images by content. In: Proceedings of the 3rd International Conference on Image and Video Retrieval (CIVR’04). Lecture Notes in Computer Science, vol. 3115, pp. 500–507. Springer, Berlin (2004) CrossRefGoogle Scholar
  7. 7.
    Fonseca, M.J., Jorge, J.A.: Using fuzzy logic to recognize geometric shapes interactively. In: Proceedings of the 9th International Conference on Fuzzy Systems (FUZZ-IEEE’00), vol. 1, pp. 291–296. San Antonio, USA (2000) Google Scholar
  8. 8.
    Fonseca, M.J., Jorge, J.A.: Experimental evaluation of an on-line scribble recognizer. Pattern Recognition Letters 22(12), 1311–1319 (2001) zbMATHCrossRefGoogle Scholar
  9. 9.
    Fonseca, M.J., Pimentel, C., Jorge, J.A.: CALI: an online scribble recognizer for calligraphic interfaces. In: Proceedings of the 2002 AAAI Spring Symposium—Sketch Understanding, pp. 51–58. Palo Alto, USA (2002) Google Scholar
  10. 10.
    Fonseca, M.J., Ferreira, A., Jorge, J.A.: Towards 3D modeling using sketches and retrieval. In: Proceedings of the first Eurographics Workshop on Sketch–Based Interfaces and Modeling, pp. 127–136. EG, Grenoble (2004) Google Scholar
  11. 11.
    Fonseca, M.J., Ferreira, A., Jorge, J.A.: Content-based retrieval of technical drawings. International Journal of Computer Applications in Technology (IJCAT) 23(2–4), 86–100 (2005) CrossRefGoogle Scholar
  12. 12.
    Fonseca, M.J., Ferreira, A., Jorge, J.A.: Generic shape classification for retrieval. In: Proceedings of the 6th IAPR International Workshop on Graphics Recognition (GREC’05), pp. 291–299 (2005) Google Scholar
  13. 13.
    Gross, M., Do, E.: Demonstrating the electronic cocktail napkin: a paper-like interface for early design. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI’96), pp. 5–6 (1996) Google Scholar
  14. 14.
    Hou, S., Ramani, K.: Structure-oriented contour representation and matching for engineering shapes. Computer Aided Design 40(1), 94–108 (2008) CrossRefGoogle Scholar
  15. 15.
    Leung, H.W.H.: Representations, feature extraction, matching and relevance feedback for sketch retrieval. PhD Thesis, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA (2003) Google Scholar
  16. 16.
    Liang, S., Sun, Z.X., Li, B., Feng, G.H.: Effective sketch retrieval based on its contents. In: Proceedings of Machine Learning and Cybernetics, vol. 9, pp. 5266–5272 (2005) Google Scholar
  17. 17.
    Lu, G.J., Sajjanhar, A.: Region-based shape representation and similarity measure suitable for content-based image retrieval. Multimedia Systems 7, 165–174 (1999) CrossRefGoogle Scholar
  18. 18.
    Mehtre, B.M., Kankanhali, M.S., Lee, W.F.: Shape measures for content based image retrieval: a comparison. Information Processing and Management 33(3), 319–337 (1997) CrossRefGoogle Scholar
  19. 19.
    Namboodiri, A.M., Jain, A.K.: Retrieval of on-line hand-drawn sketches. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04), vol. 2, pp. 642–645. IEEE Computer Society, Washington (2004) Google Scholar
  20. 20.
    Park, J., Um, B.: A new approach to similarity retrieval of 2D graphic objects based on dominant shapes. Pattern Recognition Letters 20, 591–616 (1999) CrossRefGoogle Scholar
  21. 21.
    Pereira, J.P., Jorge, J.A., Branco, V.A., Ferreira, F.N.: Calligraphic interfaces: mixed metaphors for design. In: 10th International Workshop on the Design, Specification and Verification of Interactive Systems (DSV-IS’03). Funchal, Madeira, Portugal (2003) Google Scholar
  22. 22.
    Persoon, E., Fu, K.S.: Shape discrimination using Fourier descriptors. IEEE Transactions on Systems, Man and Cybernetics 7(3), 170–179 (1977) MathSciNetCrossRefGoogle Scholar
  23. 23.
    Pu, J., Ramani, K.: On visual similarity based 2d drawing retrieval. Journal of Computer Aided Design 38(3), 249–259 (2006) CrossRefGoogle Scholar
  24. 24.
    Rui, Y., Huang, T.S., Chang, S.F.: Image retrieval: current techniques, promising directions, and open issues. Journal of Visual Communication and Image Representation 10(1), 39–62 (1999) CrossRefGoogle Scholar
  25. 25.
    Santos, T., Ferreira, A., Dias, F., Fonseca, M.J.: Using sketches and retrieval to create LEGO models. In: Proceedings of the Eurographics Workshop on Sketch-Based Interfaces and Modeling 2008 (SBIM’08), pp. 89–96. EG, Annecy (2008) Google Scholar
  26. 26.
    Sarkar, S., Boyer, K.: Quantitative measures of change based on feature organization: eigenvalues and eigenvectors. Tech. rep., Image Analysis Research Lab, University of South Florida (1996) Google Scholar
  27. 27.
    Shokoufandeh, A., Dickson, S., Siddiqi, K., Zucker, S.: Indexing using a spectral encoding of topological structure. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’99), pp. 2491–2497. IEEE Computer Society, Los Alamitos (1999) Google Scholar
  28. 28.
    Sousa, P., Fonseca, M.J.: Sketch-based retrieval of drawings using topological proximity. In: Proceedings of the 14th International Conference on Distributed Multimedia Systems, Special Track on Sketch Computing (DMS’08), pp. 276–281. Boston, USA (2008) Google Scholar
  29. 29.
    Sousa, P., Fonseca, M.J.: Geometric matching for clip-art drawing retrieval. Journal of Visual Communication and Image Representation (JVCI) 20(2), 71–83 (2009) CrossRefGoogle Scholar
  30. 30.
    Ullmann, J.R.: An algorithm for subgraph isomorphism. Journal of the ACM (JACM) 23(1), 31–42 (1976) MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Manuel J. Fonseca
    • Alfredo Ferreira
      • Joaquim A. Jorge

        There are no affiliations available

        Personalised recommendations