MARQS: retrieving sketches learned from a single example using a dual-classifier
- 59 Downloads
Mouse and keyboard interfaces handle traditional text-based queries, and standard search engines provide for effective text-based search. However, everyday documents are filled with not only text, but photos, cartoons, diagrams, and sketches. These images can often be easier to recall than the surrounding text. In an effort to make human computer interaction handle more forms of human-human interaction, sketching has recently become an important means of interacting with computer systems. We propose extending the traditional monomodal model of text-based search to include the capabilities of sketch-based search. Our goal is to create a sketch-based search that can find documents from a single query sketch. We imagine an important use for this technology would be to allow users to search a computerized laboratory notebook for a previously drawn sketch. Because such as sketch will have initially been drawn only a single time, it is important that the search-by-sketch system (1) recognize a wide range of shapes that are not necessarily geometric nor drawn in the same way each time, (2) recognize a query example from only one initial training example, and (3) learn from successful queries to improve accuracy over time. We present here such an algorithm. To test the algorithm, we implemented a proof-of-concept-system: MARQS, a system that uses sketches to query existing media albums. Preliminary results show that the system yielded an average search rank of 1.51, indicating that the correct sketch is presented as either the top or second search result on average.
KeywordsSketch recognition Sketch-based interfaces Search by sketch Feature-based recognition
Unable to display preview. Download preview PDF.
- 8.Fonseca MJ, Pimentel C, Jorge JA (2002) Cali: an online scribble recognizer for calligraphic interfaces, In: AAAI spring symposium on sketch understanding, pp 51–58 Google Scholar
- 9.Kara LB, Stahovich TF (2004) An image-based trainable symbol recognizer for sketch-based interfaces. In: AAAI fall symposium series 2004: making pen-based interaction intelligent and natural, pp 99–105 Google Scholar
- 10.Sun Z, Jiang E, Sun J (2005) Adaptive online multi-stroke sketch recognition based on hidden Markov model. In: Lecture notes in artificial intelligences, vol 3784. Springer, Berlin, pp 948–957 Google Scholar
- 14.Paulson B, Hammond T (2008) Paleosketch: Accurate primitive sketch recognition and beautification. In: IUI ’08: proceedings of the 2008 international conference on intelligent user interfaces. pp 1–8 Google Scholar
- 15.Kato T, Kurita T, Otsu N, Hirata K (1992) A sketch retrieval method for full color image databases—query by visual example. In: 11th IAPA international conference on pattern recognition, pp 530–533 Google Scholar
- 16.Gennari LM, Kara LB, Stahovich TF (2004) Combining geometry and domain knowledge to interpret hand-drawn diagrams. In: AAAI fall symposium series 2004: making pen-based interaction intelligent and natural, pp 547–562 Google Scholar
- 17.Leung WH, Chen T (2002) Retrieval of sketches based on spatial relation between strokes. In: Proceedings of the 2002 international conference on image processing, pp I-908–I-911 Google Scholar