Probabilistic Automaton Model for Fuzzy English-Text Retrieval

  • Manabu Ohta
  • Atsuhiro Takasu
  • Jun Adachi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1923)

Abstract

Optical character reader (OCR) misrecognition is a serious problem when searching against OCR-scanned documents in databases such as digital libraries. This paper proposes fuzzy retrieval methods for English text that contains errors in the recognized text without cor- recting the errors manually. Costs are thereby reduced. The proposed methods generate multiple search terms for each input query term based on probabilistic automata reflecting both error-occurrence probabilities and character-connection probabilities. Experimental results of test-set retrieval indicate that one of the proposed methods improves the recall rate from 95.56% to 97.88% at the cost of a decrease in precision rate from 100.00% to 95.52% with 20 expanded search terms.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Eugene Charniak. Statistical Language Learning. The MIT Press, 1993.Google Scholar
  2. [2]
    W. B. Croft, S. M. Harding, K. Taghva, and J. Borsack. An evaluation of information retrieval accuracy with simulated OCR output. In Proc. of SDAIR’94 3rd Annual Symposium on Document Analysis and Information Retrieval, pages 115–126, Las Vegas, NV, April 1994.Google Scholar
  3. [3]
    Daniel Lopresti and Jiangying Zhou. Retrieval strategies for noisy text. In Proc. of SDAIR’96 5th Annual Symposium on Document Analysis and Information Retrieval, pages 255–269, Las Vegas, NV, April 1996.Google Scholar
  4. [4]
    Daniel P. Lopresti. Robust retrieval of noisy text. In Proc. of ADL’96 Forum on Research and Technology Advances in Digital Libraries, pages 76–85, Library of Congress, Washington, D. C., May 1996. URL http://dlt.gsfc.nasa.gov/adl96/.
  5. [5]
    Manabu Ohta, Atsuhiro Takasu, and Jun Adachi. Reduction of expanded search terms for fuzzy English-text retrieval. In Proc. of ECDL’98, LNCS 1513, pages 619–633, Crete, Greece, September 1998. Springer.Google Scholar
  6. [6]
    Kazem Taghva, Julie Borsack, and Allen Condit. An expert system for automatically correcting OCR output. In Proc. of the IS&T/SPIE 1994 International Symposium on Electronic Imaging Science and Technology, pages 270–278, San Jose, CA, February 1994.Google Scholar
  7. [7]
    Kazem Taghva, Julie Borsack, and Allen Condit. Results of applying probabilistic IR to OCR text. In Proc. of the 17th Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, pages 202–211, Dublin, Ireland, July 1994.Google Scholar
  8. [8]
    Kazem Taghva, Julie Borsack, and Allen Condit. Effects of OCR errors on ranking and feedback using the vector space model. Information Processing & Management, 32(3):317–327, 1996.CrossRefGoogle Scholar
  9. [9]
    Kazem Taghva, Julie Borsack, and Allen Condit. Evaluation of model-based retrieval effectiveness with OCR text. ACM Trans. on Information Systems, 14(1):64–93, January 1996.CrossRefGoogle Scholar
  10. [10]
    Kazem Taghva, Allen Condit, and Julie Borsack. An evaluation of an automatic markup system. In Proc. of the IS&T/SPIE 1995 International Symposium on Electronic Imaging Science and Technology, pages 317–327, San Jose, CA, February 1995.Google Scholar
  11. [11]
    Kazem Taghva, Allen Condit, Julie Borsack, John Kilburg, Changshi Wu, and Jeff Gilbreth. The MANICURE document processing system. Technical Report 95–02, Information Science Research Institute, University of Nevada, Las Vegas, NV, March 1995.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Manabu Ohta
    • 1
  • Atsuhiro Takasu
    • 1
  • Jun Adachi
    • 1
  1. 1.National Institute of Informatics (NII)TokyoJapan

Personalised recommendations