Skip to main content

Improving Bilingual Search Performance Using Compact Full-Text Indices

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 9041)

Abstract

Machine Translation tasks must tackle the ever-increasing sizes of parallel corpora, requiring space and time efficient solutions to support them. Several approaches were developed based on full-text indices, such as suffix arrays, with important time and space achievements. However, for supporting bilingual tasks, the search time efficiency of such indices can be improved using an extra layer for the text alignment. Additionally, their space requirements can be significantly reduced using more compact indices. We propose a search procedure on top of a compact bilingual framework that improves bilingual search response time, while having a space efficient representation of aligned parallel corpora.

Keywords

  • Machine Translation
  • Translation Model
  • Statistical Machine Translation
  • Computational Linguistics
  • Parallel Corpus

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-18111-0_44
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-18111-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bourdaillet, J., Huet, S., Langlais, P., Lapalme, G.: Transsearch: from a bilingual concordancer to a translation finder. Machine Translation 24(3-4), 241–271 (2010)

    CrossRef  Google Scholar 

  2. Brisaboa, N.R., Fariña, A., Navarro, G., Places, Á.S., Rodríguez, E.: Self-indexing natural language. In: Amir, A., Turpin, A., Moffat, A. (eds.) SPIRE 2008. LNCS, vol. 5280, pp. 121–132. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  3. Brisaboa, N.R., Fariña, A., Ladra, S., Navarro, G.: Reorganizing compressed text. In: Proceedings of the 31th Annual International ACM SIGIR Conference, pp. 139–146 (2008)

    Google Scholar 

  4. Callison-Burch, C., Bannard, C., Schroeder, J.: A compact data structure for searchable translation memories. In: European Association for Machine Translation (2005)

    Google Scholar 

  5. Callison-Burch, C., Bannard, C., Schroeder, J.: Scaling phrase-based statistical machine translation to larger corpora and longer phrases. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp. 255–262 (2005)

    Google Scholar 

  6. Chiang, D.: Hierarchical phrase-based translation. Computational Linguistics 33(2) (2007)

    Google Scholar 

  7. Ferragina, P., Fischer, J.: Suffix arrays on words. In: Ma, B., Zhang, K. (eds.) CPM 2007. LNCS, vol. 4580, pp. 328–339. Springer, Heidelberg (2007)

    CrossRef  Google Scholar 

  8. Gomes, L., Aires, J., Lopes, G.P.: Parallel Texts Alignment. In: New Trends in Artificial Intelligence: 14th Portuguese Conference on Artificial Intelligence, pp. 513–524 (2009)

    Google Scholar 

  9. Grossi, R., Gupta, A., Vitter, J.S.: High-order entropy-compressed text indexes. In: Proceedings of the 14th ACM-SIAM Symposium on Discrete Algorithms, pp. 841–850 (2003)

    Google Scholar 

  10. Koehn, P.: Europarl: A parallel corpus for statistical machine translation. In: Machine Translation Summit X, vol. 5 (2005)

    Google Scholar 

  11. Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based translation. In: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, vol. 1, pp. 48–54 (2003)

    Google Scholar 

  12. Lopez, A.: Hierarchical phrase-based translation with suffix arrays. In: Proceedings of the Joint Meeting EMNLP-CoNLL, pp. 976–985 (2007)

    Google Scholar 

  13. Lopez, A.: Tera-scale translation models via pattern matching. In: Proceedings of the 22nd International Conference on Computational Linguistics, vol. 1, pp. 505–512 (2008)

    Google Scholar 

  14. Manber, U., Myers, G.: Suffix arrays: a new method for on-line string searches. SIAM Journal on Computing 22(5), 935–948 (1993)

    CrossRef  MATH  MathSciNet  Google Scholar 

  15. Navarro, G., Mäkinen, V.: Compressed full-text indexes. ACM Computing Surveys 39(1) (2007)

    Google Scholar 

  16. Ribeiro, A., Dias, G., Lopes, G., Mexia, J.: Cognates alignment. In: Machine Translation Summit VIII, Machine Translation in The Information Age, pp. 287–292 (2001)

    Google Scholar 

  17. Tiedemann, J.: News from opus-a collection of multilingual parallel corpora with tools and interfaces. In: Recent Advances in Natural Language Processing, vol. 5, pp. 237–248 (2009)

    Google Scholar 

  18. Yamamoto, M., Church, K.W.: Using suffix arrays to compute term frequency and document frequency for all substrings in a corpus. Computational Linguistics 27(1), 1–30 (2001)

    CrossRef  Google Scholar 

  19. Zhang, Y., Vogel, S.: An efficient phrase-to-phrase alignment model for arbitrarily long phrase and large corpora. In: Proceedings of the 10th European Association for Machine Translation Conference, pp. 294–301 (2005)

    Google Scholar 

  20. Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Computing Surveys 38(2),  6 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jorge Costa .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Costa, J., Gomes, L., Lopes, G.P., Russo, L.M.S. (2015). Improving Bilingual Search Performance Using Compact Full-Text Indices. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2015. Lecture Notes in Computer Science(), vol 9041. Springer, Cham. https://doi.org/10.1007/978-3-319-18111-0_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-18111-0_44

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-18110-3

  • Online ISBN: 978-3-319-18111-0

  • eBook Packages: Computer ScienceComputer Science (R0)