Efficient Search in Hidden Text of Large DjVu Documents

  • Janusz S. Bień
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6699)

Abstract

The paper describes an open-source tool which allows to present end-users with results of advanced language technologies. It relies on the DjVu format, which for some applications is still superior to other modern formats including PDF/A. The DjVu GPLed tools are not limited just to the DjVuLibre library, but are being supplemented by various new programs, such as pdf2djvu developed by Jakub Wilk. It allows in particular to convert to DjVu the PDF output of popular OCR programs like FineReader preserving the hidden text layer and some other features.

The tool in question has been conceived by the present author and consist of a modification of the Poliqarp corpus query tool, used for National Corpus of Polish; his ideas have been very succesfully implemented by Jakub Wilk. The new system, called here simply Poliqarp for DjVu, inherits from its origin not only the powerfull search facilities based two-level regular expressions, but also the ability to represent low-level ambiguities and other linguistic phenomena. Although at present the tool is used mainly to facilitate access to the results of dirty OCR, it is ready to handle also more sophisticated output of linguistic technologies.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bień, J.S.: Digitalizing dictionaries of polish. In: Bogacki, K., Cholewa, J., Rozumko, A. (eds.) Methods of Lexical Analysis: Theoretical Assumption and Practical Applications, Wydawnictwo Uniwersytetu w Białymstoku, Białystok, pp. 37–45 (2009), http://bc.klf.uw.edu.pl/71/
  2. 2.
    Bień, J.S.: Facilitating access to digitalized dictionaries in djvu format. Studia Kognitywne - Études Cognitives 9, 161–170 (2009), http://bc.klf.uw.edu.pl/160/ Google Scholar
  3. 3.
    Breuel, T.: The hOCR microformat for OCR workflow and results. In: Proceedings of the Ninth International Conference on Document Analysis and Recognition, pp. 1063–1067. IEEE Computer Society, Los Alamitos (2007), http://madm.dfki.de/publication&pubid=4373 Google Scholar
  4. 4.
    Le Cun, Y., Bottou, L., Erofeev, A., Haffner, P., Riemers, B.W.: DjVu document browsing with on-demand loading and rendering of image components. In: Internet Imaging, San Jose (January 2001), http://leon.bottou.org/papers/lecun-2001
  5. 5.
    Piotrowski, T.: Digitization of Polish historic(al) dictionaries. Review of the National Center for Digitization 6, 95–102 (2005), http://elib.mi.sanu.ac.rs/files/journals/ncd/6/d009download.pdf Google Scholar
  6. 6.
    Pletschacher, S., Antonacopoulos, A.: The PAGE (Page Analysis and Ground-Truth Elements) format framework. In: International Conference on Pattern Recognition, pp. 257–260. IEEE Computer Society, Los Alamitos (2010), http://www.cse.salford.ac.uk/prima/papers/ICPR2010_Pletschacher_PAGE.pdf Google Scholar
  7. 7.
    Przepiórkowski, A.: The IPI PAN Corpus: Preliminary version. Institute of Computer Science, Polish Academy of Sciences, Warsaw (2004), http://nlp.ipipan.waw.pl/~adamp/Papers/2004-corpus/
  8. 8.
    Przepiórkowski, A.: TEI P5 as an XML standard for treebank encoding. In: Proceedings of the Eighth International Workshop on Treebanks and Linguistic Theories (TLT,8), pp. 149–160 (2009), http://nlp.ipipan.waw.pl/~adamp/Papers/2009-tlt-tei/
  9. 9.
    Przepiórkowski, A., Buczyński, A., Wilk, J.: The National Corpus of Polish Cheatsheet (2006), http://nkjp.pl/poliqarp/help/en.html (accessed 2011-02-08)

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Janusz S. Bień
    • 1
  1. 1.Formal Linguistics DepartmentUniversity of WarsawWarszawaPoland

Personalised recommendations