Free Hand-Drawn Sketch Segmentation

  • Zhenbang Sun
  • Changhu Wang
  • Liqing Zhang
  • Lei Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7572)


In this paper, we study the problem of how to segment a freehand sketch at the object level. By carefully considering the basic principles of human perceptual organization, a real-time solution is presented to automatically segment a user’s sketch during his/her drawing. First, a graph-based sketch segmentation algorithm is proposed to segment a cluttered sketch into multiple parts based on the factor of proximity. Then, to improve the ability of detecting semantically meaningful objects, a semantic-based approach is introduced to simulate the past experience in the perceptual system by leveraging a web-scale clipart database. Finally, other important factors learnt from past experience, such as similarity, symmetry, direction, and closure, are also taken into account to make the approach more robust and practical. The proposed sketch segmentation framework has ability to handle complex sketches with overlapped objects. Extensive experimental results show the effectiveness of the proposed framework and algorithms.


Segmentation Algorithm Perceptual Grouping Meaningful Object Word Distribution Short Stroke 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Marr, D.: Early processing of visual information. Philosophical Transactions of the Royal Society of London. B, Biological Sciences (1976)Google Scholar
  2. 2.
    Cao, Y., Wang, C., Zhang, L., Zhang, L.: Edgel index for large-scale sketch-based image search. In: CVPR (2011)Google Scholar
  3. 3.
    Bronstein, A., Bronstein, M., Guibas, L., Ovsjanikov, M.: Shape google: Geometric words and expressions for invariant shape retrieval. TOG (2011)Google Scholar
  4. 4.
    Sun, Z., Wang, C., Zhang, L., Zhang, L.: Query-adaptive shape topic mining for hand-drawn sketch recognition. In: ACM Multimedia (2012)Google Scholar
  5. 5.
    Temlyakov, A., Munsell, B., Waggoner, J., Wang, S.: Two perceptually motivated strategies for shape classification. In: CVPR (2010)Google Scholar
  6. 6.
    Pu, J., Gur, D.: Automated freehand sketch segmentation using radial basis functions. In: Computer-Aided Design (2009)Google Scholar
  7. 7.
    Sezgin, T., Stahovich, T., Davis, R.: Sketch based interfaces: Early processing for sketch understanding. In: ACM SIGGRAPH (2006)Google Scholar
  8. 8.
    Sezgin, T.: Feature point detection and curve approximation for early processing of freehand sketches (Masters thesis, Massachusetts Institute of Technology)Google Scholar
  9. 9.
    Hammond, T., Davis, R.: Ladder, a sketching language for user interface developers. Computers & Graphics (2005)Google Scholar
  10. 10.
    Sezgin, T., Davis, R.: Hmm-based efficient sketch recognition. In: ACM IUI (2005)Google Scholar
  11. 11.
    Wertheimer, M.: Laws of organization in perceptual forms. A source book of Gestalt psychology (1938)Google Scholar
  12. 12.
    Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. IJCV (2004)Google Scholar
  13. 13.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. PAMI (2000)Google Scholar
  14. 14.
    Boykov, Y., Funka-Lea, G.: Graph cuts and efficient nd image segmentation. IJCV (2006)Google Scholar
  15. 15.
    Carreira, J., Sminchisescu, C.: Constrained parametric min-cuts for automatic object segmentation. In: CVPR (2010)Google Scholar
  16. 16.
    Cao, Y., Wang, H., Wang, C., Li, Z., Zhang, L., Zhang, L.: Mindfinder: Interactive sketch-based image search on millions of images. In: ACM Multimedia (2010)Google Scholar
  17. 17.
    Wang, C., Zhang, J., Yang, B., Zhang, L.: Sketch2cartoon: composing cartoon images by sketching. In: ACM Multimedia (2011)Google Scholar
  18. 18.
    Borgefors, G.: Hierarchical chamfer matching: A parametric edge matching algorithm. PAMI (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Zhenbang Sun
    • 1
  • Changhu Wang
    • 2
  • Liqing Zhang
    • 1
  • Lei Zhang
    • 2
  1. 1.Brain-Like Computing LabShanghai Jiao Tong UniversityP.R. China
  2. 2.Microsoft Research AsiaChina

Personalised recommendations