Trainable Sketch Recognizer for Graphical User Interface Design

  • Adrien Coyette
  • Sascha Schimke
  • Jean Vanderdonckt
  • Claus Vielhauer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4662)


In this paper we present a new algorithm for automatic recognition of hand drawn sketches based on the Levenshtein distance. The purpose for drawing sketches in our application is to create graphical user interfaces in a similar manner as the well established paper sketching. The new algorithm is trainable by every user and improves the recognition performance of the techniques which were used before for widget recognition. In addition, this algorithm ay serve for recognizing other types of sketches, such as letters, figures, and commands. In this way, there is no modality disruption at sketching time.


Grid Node Edit Distance Interface Design Shape Recognition Levenshtein Distance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bailey, B.P., Konstan, J.A.: Are Informal Tools Better? Comparing DEMAIS, Pencil and Paper, and Authorware for Early Multimedia Design. In: CHI 2003. Proc. of the ACM Conf. on Human Factors in Computing Systems, pp. 313–320. ACM Press, New York (2003)Google Scholar
  2. 2.
    Bresenham, J.E.: Algorithm for Computer Control of a Digital Plotter. IBM Systems Journal 4(1), 25–30 (1965)CrossRefGoogle Scholar
  3. 3.
    Caetano, A., Goulart, N., Fonseca, M., Jorge, J.: JavaSketchIt: Issues in Sketching the Look of User Interfaces. In: Proc. of the 2002 AAAI Spring Symposium - Sketch Understanding, Palo Alto, March 2002, pp. 9–14. AAAI Press, California, USA (2002)Google Scholar
  4. 4.
    Coyette, A., Vanderdonckt, J.: A Sketching Tool for Designing Anyuser, Anyplatform, Anywhere User Interfaces. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 550–564. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Do, T.T.: A Social Patterns Framework for Designing Multiagent Architectures, Ph.D. thesis, Université catholique de Louvain, IAG, Louvain-la-Neuve (June 2005)Google Scholar
  6. 6.
    Fonseca, M.J., Jorge, J.A.: Using Fuzzy Logic to Recognize Geometric Shapes Interactively. In: Proc. of the 9th Int. Conf. on Fuzzy Systems FUZZ-IEEE 2000, San Antonio, pp. 191–196. IEEE Computer Society Press, Los Alamitos (2000)Google Scholar
  7. 7.
    Freeman, H.: Computer Processing of Line-Drawing Images. ACM Computing Surveys 6(1), 57–97 (1974)CrossRefMATHGoogle Scholar
  8. 8.
    Hong, J.I., Li, F.C., Lin, J., Landay, J.A.: End-User Perceptions of Formal and Informal Representations of Web Sites. In: CHI 2001. Proc. of ACM Conf. on Human Aspects in Computing Systems, Extended Abstracts, pp. 385–386. ACM Press, New York (2001)Google Scholar
  9. 9.
    Kolp, M., Giorgini, P., Mylopoulos, J.: An Organizational Perspective on Multi-agent Architectures. In: Meyer, J.-J.C., Tambe, M. (eds.) ATAL 2001. LNCS (LNAI), vol. 2333, Springer, Heidelberg (2002)CrossRefGoogle Scholar
  10. 10.
    Landay, J.A., Myers, B.A.: Sketching Interfaces: Toward More Human Interface Design. IEEE Computer 34(3), 56–64 (2001)CrossRefGoogle Scholar
  11. 11.
    Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Doklady Akademii Nauk SSSR 163(4), 845–848 (1965) [in Russian]. English translation in Soviet Physics Doklady 10(8) , 707–710 (1966)Google Scholar
  12. 12.
    Long, A.C., Landay, J.A., Rowe, L.A.: Implications For a Gesture Design Tool. In: CHI 2001. Proc. of ACM Conf. on Human Factors in Computing Systems, Seattle, pp. 40–47. ACM Press, New York (2001)Google Scholar
  13. 13.
    Lin, J., Newman, M.W., Hong, J.I., Landay, J.A.: Denim: Finding a Tighter Fit Between Tools and Practice for Web Site Design. In: CHI 2000. Proc. of ACM Conf. on Human Factors in Computing Systems, The Hague, April 2000, pp. 510–517. ACM Press, New York (2000)Google Scholar
  14. 14.
    Newman, M.W., Lin, J., Hong, J.I., Landay, J.A.: Denim: An Informal Web Site Design Tool Inspired by Observations of Practice. Human-Computer Interaction 18, 259–324 (2003)CrossRefGoogle Scholar
  15. 15.
    Plimmer, B.E., Apperley, M.: Software for Students to Sketch Interface Designs. In: Proc. of Interact 2003, pp. 73–80. IOS Press, Amsterdam (2003)Google Scholar
  16. 16.
    Plimmer, B.E., Apperley, M.: Interacting with Sketched Interface Designs: An Evaluation Study. In: Proc. of CHI 2004, pp. 1337–1340. ACM Press, New York (2004)Google Scholar
  17. 17.
    Rubine, D.: Specifying Gestures by Example. Computer Graphics 25(3), 329–337 (1991)CrossRefGoogle Scholar
  18. 18.
    Schimke, S., Vielhauer, C., Dittmann, J.: Using Adapted Levenshtein Distance for On-Line Signature Authentication. In: Proc. of ICPR 2004 (2004)Google Scholar
  19. 19.
    Sumner, T., Bonnardel, N., Kallag-Harstad, B.: The Cognitive Ergonomics of Knowledge-based Design Support Systems. In: CHI 1997. Proc. of ACM Conf. on Human Aspects in Computing Systems, Atlanta, April 1997, pp. 83–90. ACM Press, New York (1997)Google Scholar
  20. 20.
    van Duyne, D.K., Landay, J.A., Hong, J.I.: The Design of Sites: Patterns, Principles, and Processes for Crafting a Customer-Centered Web Experience. Addison-Wesley, New York (2002)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2007

Authors and Affiliations

  • Adrien Coyette
    • 1
  • Sascha Schimke
    • 2
  • Jean Vanderdonckt
    • 1
  • Claus Vielhauer
    • 2
  1. 1.Belgian Lab. of Computer-Human Interaction (BCHI), Information Systems Unit (ISYS), Louvain School of Management, Université catholique de Louvain, Place des Doyens 1, B(1348 Louvain-la-Neuve (Belgium)Belgium
  2. 2.Department of Computer Science/ITI,University Otto von Guericke, Universitätsplatz 2, – D-39106 Magdeburg (Germany)Germany

Personalised recommendations