Smaller Representation of Finite State Automata

  • Jan Daciuk
  • Dawid Weiss
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6807)

Abstract

This paper is a follow-up to Jan Daciuk’s experiments on space-efficient finite state automata representation that can be used directly for traversals in main memory [4]. We investigate several techniques of reducing the memory footprint of minimal automata, mainly exploiting the fact that transition labels and transition pointer offset values are not evenly distributed and so are suitable for compression. We achieve a size gain of around 20–30% compared to the original representation given in [4]. This result is comparable to the state-of-the-art dictionary compression techniques like the LZ-trie [10] method, but remains memory and CPU efficient during construction.

Keywords

Univer 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Budiscak, I., Piskorski, J., Ristov, S.: Compressing Gazetteers Revisited. In: Yli-Jyrä, A., Kornai, A., Sakarovitch, J., Watson, B. (eds.) FSMNLP 2009. LNCS, vol. 6062, Springer, Heidelberg (2010)Google Scholar
  2. 2.
    Ciura, M., Deorowicz, S.: How to squeeze a lexicon. Software – Practice and Experience 31(11), 1077–1090 (2001)CrossRefMATHGoogle Scholar
  3. 3.
    Daciuk, J.: Incremental Construction of Finite-State Automata and Transducers, and their Use in the Natural Language Processing. Ph.D. thesis, Technical University of Gdańsk (1998)Google Scholar
  4. 4.
    Daciuk, J.: Experiments with automata compression. In: Yu, S., Păun, A. (eds.) CIAA 2000. LNCS, vol. 2088, pp. 105–119. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  5. 5.
    Daciuk, J., Piskorski, J.: Gazetteer Compression Technique Based on Substructure Recognition. In: Proceedings of the International Conference on Intelligent Information Systems, Ustroń, Poland, pp. 87–95 (2006)Google Scholar
  6. 6.
    Daciuk, J., Piskorski, J., Ristov, S.: Scientific Applications of Language Methods. In: Mathematics, Computing, Language, and Life: Frontiers in Mathematical Linguistics and Language Theory, pp. 133–204. World Scientific Publishing, Singapore (2010)Google Scholar
  7. 7.
    Kowaltowski, T., Lucchesi, C.L., Stolfi, J.: Minimization of binary automata. In: 1st South American String Processing Workshop, Belo Horizonte, Brasil (1993)Google Scholar
  8. 8.
    Liang, F.M.: Word Hyphenation by Computer. Ph.D. thesis, Stanford University (1983)Google Scholar
  9. 9.
    Lucchiesi, C., Kowaltowski, T.: Applications of finite automata representing large vocabularies. Software Practice and Experience 23(1), 15–30 (1993)CrossRefGoogle Scholar
  10. 10.
    Ristov, S., Laporte, É.: Ziv Lempel Compression of Huge Natural Language Data Tries Using Suffix Arrays. In: Crochemore, M., Paterson, M. (eds.) CPM 1999. LNCS, vol. 1645, pp. 196–211. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  11. 11.
    Tounsi, L.: Sous-automates à nombre fini d’états. Application à la compression de dictionnaires électroniques. Ph.D. thesis, Université François Rabelais Tours (2008)Google Scholar
  12. 12.
    Tounsi, L., Bouchou, B., Maurel, D.: A Compression Method for Natural Language Automata. In: Proceedings of the 7th International Workshop on Finite-State Methods and Natural Language Processing, Ispra, Italy, pp. 146–157 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jan Daciuk
    • 1
  • Dawid Weiss
    • 2
  1. 1.Knowledge Engineering DepartmentGdańsk University of TechnologyPoland
  2. 2.Institute of Computing SciencePoznan University of TechnologyPoland

Personalised recommendations