Performance Evaluation of Symbol Recognition

  • Ernest Valveny
  • Philippe Dosch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3163)

Abstract

Symbol recognition is one of the central problems in the field of graphics recognition. Many methods and approaches have been developed in the context of several application domains. In the last years, the need for generic methods, able to perform well on large sets of symbols in different domains, has become clear. Thus, standard evaluation datasets and protocols have to be built up in order to be able to evaluate the performance of all these methods. In this paper we discuss several points which should be taken into account in the design of such evaluation framework, raising a number of open questions for further discussion. These issues were the starting point of the organization of the contest on symbol recognition held during the last Workshop on Graphics Recognition (GREC’03). We also summarize the main features of the dataset and the protocol of evaluation used in the contest, as a first step to define a general evaluation framework, giving answer to these open questions.

Keywords

Application Domain Recognition Method Document Image Vectorial Image Symbol Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full conference paper text

References

  1. 1.
    Haralick, R.: Performance characterization in image analysis: Tninning, a case in point. Pattern Recognition Letters 13, 5–12 (1992)CrossRefGoogle Scholar
  2. 2.
    Guyon, I., Haralick, R., Hull, J., Phipliops, I.: Data sets for ocr and document image understanding research. In: Bunke, H., Wang, P. (eds.) Handbook of Character Recognition and Document Image Analysis, pp. 779–800. World Scientific, Singapore (1997)Google Scholar
  3. 3.
    Kong, B., Phillips, I.T., Haralick, R.M., Prasad, A., Kasturi, R.: A benchmark: Performance evaluation of dashed-line detection algorithms. In: Kasturi, R., Tombre, K. (eds.) Graphics Recognition 1995. LNCS, vol. 1072, pp. 270–285. Springer, Heidelberg (1996)Google Scholar
  4. 4.
    Chhabra, A., Phillips, I.: The second international graphics recognition contest - raster to vector conversion: A report. In: [20], pp. 390–410Google Scholar
  5. 5.
    Chhabra, A., Philips, I.: Performance evaluation of line drawing recognition systems. In: Proceedings of 15th. International Conference on Pattern Recognition, Barcelona, Spain, vol. 4, pp. 864–869 (2000)Google Scholar
  6. 6.
    Wenyin, L., Zhai, J., Dori, D.: Extended summary of the arc segmentation contest. In: Blostein, D., Kwon, Y.-B. (eds.) GREC 2001. LNCS, vol. 2390, pp. 343–349. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Wenyin, L., Dori, D.: A protocol for performance evaluation of line detection algorithms. Machine Vision and Applications 9, 240–250 (1997)CrossRefGoogle Scholar
  8. 8.
    Phillips, I., Chhabra, A.: Empirical performance evaluation of graphics recognition systems. IEEE Transactions on Pattern Analyisis and Machine Intelligence 21, 849–870 (1999)CrossRefGoogle Scholar
  9. 9.
    Hilaire, X.: A matching scheme to enhance performance evaluation of rasterto- vector conversion algorithms. In: Proceedings of 7th International Conference on Document Analysis and Recognition, Edinburgh, Scotland, vol. 1, pp. 629–633 (2003)Google Scholar
  10. 10.
    Aksoy, S., Ye, M., Schauf, M., Song, M., Wang, Y., Haralick, R., Parker, J., Pivovarov, J., Royko, D., Sun, C., Farneboock, G.: Algorithm performance contest. In: Proceedings of 15th. International Conference on Pattern Recognition, Barcelona, Spain, vol. 4, pp. 870–876 (2000)Google Scholar
  11. 11.
    Valveny, E., Dosch, P.: Symbol recognition contest: A synthesis. In: Selected Papers from Fith International Workshop on Graphics Recognition, GREC 2003 (2004) (to appear) Google Scholar
  12. 12.
    Chhabra, A.K.: Graphic Symbol Recognition: An Overview. In: [20], pp. 68–79 Google Scholar
  13. 13.
    Cordella, L.P., Vento, M.: Symbol recognition in documents: a collection of techniques? International Journal on Document Analysis and Recognition 3, 73–88 (2000)CrossRefGoogle Scholar
  14. 14.
    Lladós, J., Valveny, E., Sánchez, G., Martí, E.: Symbol Recognition: Current Advances and Perspectives. In: Blostein, D., Kwon, Y.-B. (eds.) GREC 2001. LNCS, vol. 2390, pp. 104–127. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    Tombre, K., Ah-Soon, C., Dosch, P., Masini, G., Tabbone, S.: Stable and Robust Vectorization: How to Make the Right Choices. In: Chhabra, A.K., Dori, D. (eds.) GREC 1999. LNCS, vol. 1941, pp. 3–18. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  16. 16.
    Baird, H.S.: The State of the Art of Document Image Degradation Modeling. In: Proceedings of 4th IAPR International Workshop on Document Analysis Systems, Rio de Janeiro (Brazil) (2000) Google Scholar
  17. 17.
    Kanungo, T., Haralick, R.M., Baird, H.S., Stuetzle, W., Madigan, D.: Document Degradation Models: Parameter Estimation and Model Validation. In: Proceedings of IAPR Workshop on Machine Vision Applications, Kawasaki (Japan), pp. 552–557 (1994) Google Scholar
  18. 18.
    Tombre, K., Chhabra, A.K.: General Conclusions. In: [20], pp. 411–420 Google Scholar
  19. 19.
    Cootes, T., Taylor, C., Cooper, D., Graham, J.: Active shape models: Their training and application. Computer Vision and Image Understanding 61, 38–59 (1995)CrossRefGoogle Scholar
  20. 20.
    Chhabra, A.K., Tombre, K. (eds.): GREC 1997. LNCS, vol. 1389. Springer, Heidelberg (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Ernest Valveny
    • 1
  • Philippe Dosch
    • 2
  1. 1.Centre de Visió per Computador, Edifici O, Campus UABBellaterra (Cerdanyola), BarcelonaSpain
  2. 2.LoriaVillers-lès-Nancy CedexFrance

Personalised recommendations