Automated Processing of Digitized Historical Newspapers: Identification of Segments and Genres

  • Robert B. Allen
  • Ilya Waldstein
  • Weizhong Zhu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5362)

Abstract

Many historical newspapers are being digitized. We aim to support access to them via text analysis of the OCRd content. However, the OCR includes many errors; so extracting meaningful content from it is difficult. A pipeline of processing steps is proposed. Here, we describe the first two steps: segmentation and genre identification. The segmentation procedure based on headings was quite successful. Genre identification worked well for easily defined genre categories such as weather reports. We also propose additional techniques which may improve the accuracy still farther.

Keywords

Categorization Genres Historical Newspapers OCR Segmentation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Allen, R.B.: A Focus-Context Timeline for Browsing Historical Newspapers. In: ACM/IEEE Joint Conference on Digital Libraries, pp. 260–261 (2005)Google Scholar
  2. 2.
    Allen, R.B., Japzon, A., Achananuparp, P., Lee, K.-J.: A Framework for Text Processing and Supporting Access to Collections of Digitized Historical Newspapers. In: HCI International Conf. (2007)Google Scholar
  3. 3.
    Allen, R.B., Schalow, J.: Metadata and Data Structures for the Historical Newspaper Digital Library Project. In: ACM CIKM, Kansas City, November, pp. 147–153 (1999)Google Scholar
  4. 4.
    Choi, Y.Y.: Advances in domain independent linear text segmentation. In: Proceedings of NAACL, Seattle, USA (2000)Google Scholar
  5. 5.
    Gatos, B., Gouraros, N., Mantzaris, S., Perantonis, S., Tsigris, A., Tzavelis, P., Vassilas, N.: A New Method for Segmenting Newspaper Articles. In: SIGIR, p. 389 (1998)Google Scholar
  6. 6.
    Kanungo, T., Allen, R.B.: Full-Text Access to Historical Newspapers. Technical Report: LAMP-TR-033/CAR-TR-915/CS-TR-4014, University of Maryland, College Park (April 1999)Google Scholar
  7. 7.
    Murray, R.: Towards a Metadata Standard for Digitized Historical Newspapers. JCDL, 330–331 (2005)Google Scholar
  8. 8.
    von Ahn, L., Maurer, B., McMillen, C., Abraham, D., Blum, M.: ReCAPTCHA: Human-Based Character Recognition via Web Security Measures. Science 321, 1465–1468 (2008)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Zhu, W., Allen, R.B.: Topic and Event Categorization of Historical Newspapers (in preparation)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Robert B. Allen
    • 1
  • Ilya Waldstein
    • 1
  • Weizhong Zhu
    • 1
  1. 1.College of Information Science and TechnologyDrexel UniversityUSA

Personalised recommendations