A Sketch-Based User Interface for Image Search Using Sample Photos

  • Hitoshi Sugimura
  • Hayato Tsukiji
  • Mizuki Kumada
  • Toshiya Iiba
  • Kosuke TakanoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9734)


In this paper, we present a sketch-based user interface for a content-based image search using sample photos. Our sketch-based user interface diverts a part of the images and their color extracted from sample photos, which are obtained from image resources, such as an image search result, to a sketch, and supports a user to draw a more practical sketch as a query. This feature of the proposed sketch-based user interface allows a user to make a sketch-based query according to the thought and image of the user by painting and decorating the sketch using fragments of images snipped from sample photos and their colors. Thus, the proposed sketch-based user interface can help a user to make a sketch for the image retrieval by leveraging objects and colors that actually appear in images and photos. In the experiment using our prototype, we evaluated the feasibility of our approach.


Content-based image retrieval Interaction design Query by sketch User interface Sample photos 


  1. 1.
    Swain, M.J., Ballard, D.H.: Color indexing. Int. J. Comput. Vis. 7(1), 11–32 (1991)CrossRefGoogle Scholar
  2. 2.
    Ohashi, G., Hisamori, T., Mochizuki, K.: Query-by-sketch image retrieval system using relevance feedback. J. Jpn. Soc. Fuzzy Theory Intell. Inform. 19(5), 537–545 (2007)CrossRefGoogle Scholar
  3. 3.
    Klinger, E., Starkweather, D.: The open source perceptual hash library. (Accessed Dec 2015)
  4. 4.
    Ashley, J., Flickner, M., Hafner, J., Lee, D., Niblack, W., Petkovic, D.: The query by image content (QBIC) system. In: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data (SIGMOD 1995), p. 475 (1995)Google Scholar
  5. 5.
    Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: state of the art and challenges. ACM Trans. Multimedia Comput. Commun. Appl. 2(1), 1–19 (2006)CrossRefGoogle Scholar
  6. 6.
    Springmann, M., Kopp, D., Schuldt, H.: QbS: searching for known images using user-drawn sketches. In: Proceedings of the International Conference on Multimedia Information Retrieval (MIR 2010), pp. 417–420 (2010)Google Scholar
  7. 7.
    Shen, H.T., Jiang, S., Tan, K.-L., Huang, Z., Zhou, X.: Speed up interactive image retrieval. Int. J. Very Large Data Bases Arch. (The VLDB J.) 18(1), 329–343 (2009)CrossRefGoogle Scholar
  8. 8.
    He, X., Ma, W.-Y., King, O., Li, M., Zhang, H.: Learning and inferring a semantic space from user’s relevance feedback for image retrieval. In: Proceedings of the Tenth ACM International Conference on Multimedia (MULTIMEDIA 2002), pp. 343–346 (2002)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Hitoshi Sugimura
    • 1
  • Hayato Tsukiji
    • 1
  • Mizuki Kumada
    • 1
  • Toshiya Iiba
    • 1
  • Kosuke Takano
    • 1
    Email author
  1. 1.Department of Information and Computer SciencesKanagawa Institute of TechnologyAtsugiJapan

Personalised recommendations