Per-node Optimization of Finite-State Mechanisms for Natural Language Processing

  • Alexander Troussov
  • Brian O’Donovan
  • Seppo Koskenniemi
  • Nikolay Glushnev
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2588)

Abstract

Finite-state processing is typically based on structures that allow for efficient indexing and sequential search. However, this “rigid” framework has several disadvantages when used in natural language processing, especially for non-alphabetical languages. The solution is to systematically introduce polymorphic programming techniques that are adapted to particular cases. In this paper we describe the structure of a morphological dictionary implemented with finite-state automata using variable or polymorphic node formats. Each node is assigned a format from a predefined set reflecting its utility in corpora processing as measured by a number of graph theoretic metrics and statistics. Experimental results demonstrate that this approach permits a 52% increase in the performance of dictionary look-up.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Alexander Troussov
    • 1
  • Brian O’Donovan
    • 1
  • Seppo Koskenniemi
    • 2
  • Nikolay Glushnev
    • 1
  1. 1.IBM Dublin Software LabAirways Ind. Est.Dublin 17Ireland
  2. 2.Oy IBM Ab.HelsinkiFinland

Personalised recommendations