Representing and Parsing Sketched Symbols Using Adjacency Grammars and a Grid-Directed Parser

  • Joan Mas
  • Joaquim A. Jorge
  • Gemma Sanchez
  • Josep Llados
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5046)


While much work has been done in Structural and Syntactical Pattern Recognition applied to drawings, most approaches are non-interactive. However, the recent emergence of viable pen-computers makes it desirable to handle pen-input such as sketches and drawings interactively. This paper presents a syntax-directed approach to parse sketches based on Relational Adjacency Grammars, which describe spatial and topological relations among parts of a sketch. Our approach uses a 2D grid to avoid re-scanning all the previous input whenever new strokes entered into the system, thus speeding up parsing considerably. To evaluate the performance of our approach we have tested the system using non-trivial inputs analyzed with two different grammars, one to design user interfaces and the other to describe floor-plans. The results clearly show the effectiveness of our approach and demonstrate good scalability to larger drawings.


Parse Tree Graph Grammar Visual Language Grammatical Formalism Syntactic Approach 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jorge, J.A.P.: Parsing Adjacency Grammars For Calligraphic Interfaces. Rensselaer Polytechnic Institute, New York (1995)Google Scholar
  2. 2.
    Jorge, J.A.P., Glinert, E.P.: Online Parsing of Visual Languages Using Adjacency Grammars. In: 11th International IEEE Symposium on Visual Languages, pp. 250–257 (1995)Google Scholar
  3. 3.
    Lladós, J., Valveny, E., Sánchez, G., Martí, E.: Symbol Recognition: Current Advances and Perspectives. In: Blostein, D., Kwon, Y.B. (eds.) GREC 2001. LNCS, vol. 2390, pp. 104–127. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Wenyin, L.: On-line Graphics Recognition: State-of-the-Art. In: 5th IAPR Workshop on Graphics Recognition, Barcelona, pp. 291–304 (2003)Google Scholar
  5. 5.
    Mas Romeu, J., Lamiroy, B., Sanchez, G., Llados, J.: Automatic Adjacency Grammar Generator from User Drawn Sketches. In: 18th International Conference on Pattern Recognition, Hong-Kong, pp. 1026–1029 (2006)Google Scholar
  6. 6.
    Wittenburg, K., Weitzman, L.: Relational Grammars: Theory and Practice in a Visual Language Interface for Process Modelling. In: International Workshop on Theory of Visual Languages, Italy (1996)Google Scholar
  7. 7.
    Bunke, H.: Hybrid Pattern Recognition Methods. In: Bunke, H., Sanfeliu, A. (eds.) Syntactic and Structural Pattern Recognition.Theory and Applications, pp. 307–347. World Scientific Publishing Company, Singapore (1990)Google Scholar
  8. 8.
    Hammond, T., Davis, R.: LADDER: A Language to Describe Drawing, Display and Editing in Sketch Recognition. In: Internaltional Joint Conference on Artificial Intelligence, Acapulco, pp. 461–467 (2003)Google Scholar
  9. 9.
    Bresenham, J.: Algorithm for Computer Control of a Digital Plotter. IBM System Journal 4(1), 25–30 (1965)CrossRefGoogle Scholar
  10. 10.
    Alvarado, C., Davis, R.: Dinamically Constructed Bayes Nets for Multi-Domain Sketch Undrestanding. In: Internaltional Joint Conference on Artificial Intelligence, San Francisco, pp. 1407–1412 (2004)Google Scholar
  11. 11.
    Kara, L.B., Stahovich, T.F.: Hierarchical Parsing and recognition of hand-sketched diagrams. In: 17th Annual ACM Symposium on User Interface Software and Technology, pp. 13–22 (2004)Google Scholar
  12. 12.
    Costagliola, G., Deufemia, V., Risi, M.: A Multi-layer Parsing Strategy for On-line Recognition of Hand-Drawn Diagrams. In: IEEE Symposium on Visual Languages and Human-Centric Computing, pp. 103–110 (2006)Google Scholar
  13. 13.
    Landay, J.A., Myers, B.A.: Sketching Interfaces: Toward More Human Interface Design. IEEE Computer 34(3), 56–64 (2001)Google Scholar
  14. 14.
    Rubine, D.: Specifying Gestures by Example. In: 18th Annual Conference on Computer Graphics and Interactive Techniques, pp. 329–337 (2001)Google Scholar
  15. 15.
    Hammond, T., Davis, R.: Tahuti: A Geometrical Sketch Recognition System for UML Class Diagrams. In: AAAI Spring Symposium on Sketch Understanding, pp. 59–68. Palo Alto, Menlo Park (2002)Google Scholar
  16. 16.
    Marriot, K.: Constraint Multiset Grammars. In: IEEE Symposium on Visual Languages, St. Louis, pp. 118–125 (1994)Google Scholar
  17. 17.
    Crimi, C., Guercio, A., Nota, G., Pacini, G., Tortora, G., Tucci, M.: Relation grammars and their application to multi-dimensional languages. Journal of Visual Languages and Computing 2(4), 333–346 (1991)CrossRefGoogle Scholar
  18. 18.
    Golin, E.J.: Parsing Visual Languages with Picture Layout Grammars. Journal of Visual Languages and Computing 2(4), 371–394 (1991)CrossRefGoogle Scholar
  19. 19.
    Rekers, J., Schurr, A.: Defining and Parsing Visual Languages with Layered Graph Grammars. Journal of Visual Languages and Computing 8(1), 27–55 (1997)CrossRefGoogle Scholar
  20. 20.
    Coüasnon, B.: DMOS, a generic document recognition method: application to table structure analysis in a general and in a specific way. International Journal on Document Analysis and Recognition 8(2-3), 111–122 (2006)CrossRefGoogle Scholar
  21. 21.
    Costagliola, G., Deufemia, V., Risi, M.: Sketch Grammars: A Formalism for Describing and Recognizing Diagrammatic Sketch Languages. In: International Conference on Document Analysis and Recognition, Hong-Kong, pp. 1226–1230 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Joan Mas
    • 1
  • Joaquim A. Jorge
    • 2
  • Gemma Sanchez
    • 1
  • Josep Llados
    • 1
  1. 1.Computer Science DepartmentComputer Vision CenterBellaterraSpain
  2. 2.Departamento de Engenharia InformáticaINESCLisboaPortugal

Personalised recommendations