Impact Analysis of OCR Quality on Research Tasks in Digital Archives

  • Myriam C. Traub
  • Jacco van Ossenbruggen
  • Lynda Hardman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9316)


Humanities scholars increasingly rely on digital archives for their research instead of time-consuming visits to physical archives. This shift in research method has the hidden cost of working with digitally processed historical documents: how much trust can a scholar place in noisy representations of source texts? In a series of interviews with historians about their use of digital archives, we found that scholars are aware that optical character recognition (OCR) errors may bias their results. They were, however, unable to quantify this bias or to indicate what information they would need to estimate it. This, however, would be important to assess whether the results are publishable. Based on the interviews and a literature study, we provide a classification of scholarly research tasks that gives account of their susceptibility to specific OCR-induced biases and the data required for uncertainty estimations. We conducted a use case study on a national newspaper archive with example research tasks. From this we learned what data is typically available in digital archives and how it could be used to reduce and/or assess the uncertainty in result sets. We conclude that the current knowledge situation on the users’ side as well as on the tool makers’ and data providers’ side is insufficient and needs to be improved.


OCR quality Digital libraries Digital humanities 



We would like to thank our interviewees for their contributions, the National Library of The Netherlands for their support and the reviewers for their helpful feedback. This research is funded by the Dutch COMMIT/ program.


  1. 1.
    Acerbi, A., Lampos, V., Garnett, P., Bentley, R.A.: The expression of emotions in 20th century books. PLoS ONE 8(3), e59030 (2013)CrossRefGoogle Scholar
  2. 2.
    Alex, B., Grover, C., Klein, E., Tobin, R.: Digitised historical text: does it have to be mediOCRe? In: Jancsary, J. (ed.) Proceedings of KONVENS 2012, LThist 2012 Workshop, pp. 401–409. ÖGAI, September 2012Google Scholar
  3. 3.
    Bingham, A.: The digitization of newspaper archives: opportunities and challenges for historians. Twentieth Century Br. Hist. 21(2), 225–231 (2010)CrossRefGoogle Scholar
  4. 4.
    Bron, M.; Exploration and contextualization through interaction and concepts. Ph.D. Thesis (2013)Google Scholar
  5. 5.
    Brown, C.D.: Straddling the humanities and social sciences: the research process of music scholars. Libr. Inf. Sci. Res. 24(1), 73–94 (2002)CrossRefGoogle Scholar
  6. 6.
    Cohen, D.J., Rosenzweig, R.: Digital History: A Guide to Gathering, Preserving, and Presenting the Past on the Web, vol. 28. University of Pennsylvania Press, Philadelphia (2006)Google Scholar
  7. 7.
    Croft, W.B., Harding, S., Taghva, K., Borsack, J.: An evaluation of information retrieval accuracy with simulated OCR output. Technical report, Amherst, MA, USA (1993)Google Scholar
  8. 8.
    Fuhr, N., Hansen, P., Mabe, M., Micsik, A., Sølvberg, I.T.: Digital libraries: a generic classification and evaluation scheme. In: Constantopoulos, P., Sølvberg, I.T. (eds.) ECDL 2001. LNCS, vol. 2163, pp. 187–199. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  9. 9.
    Holley, R.: How good can it get? Analysing and improving OCR accuracy in large scale historic newspaper digitisation programs. D-Lib Mag. 15(3/4) (2009)Google Scholar
  10. 10.
    Holley, R.: Many hands make light work: public collaborative OCR text correction in Australian Historic Newspapers. Technical report, National Library of Australia, March 2009Google Scholar
  11. 11.
    Kettunen, K., Honkela, T., Lindén, K., Kauppinen, P., Pääkkönen, T., Kervinen, J. et al.: Analyzing and improving the quality of a historical news collection using language technology and statistical machine learning methods. In: Proceedings of the 80th IFLA General Conference and Assembly, IFLA World Library and Information Congress (2014)Google Scholar
  12. 12.
    Klijn, E.: The current state-of-art in newspaper digitization a market perspective. D-Lib Mag. 14, January 2008Google Scholar
  13. 13.
    Mittendorf, E., Schäuble, P.: Information retrieval can cope with many errors. Inf. Retr. 3(3), 189–216 (2000)CrossRefzbMATHGoogle Scholar
  14. 14.
    Nicholson, B.: Counting culture; or, how to read Victorian newspapers from a distance. J. Victorian Cult. 17(2), 238–246 (2012)CrossRefGoogle Scholar
  15. 15.
    Strange, C., McNamara, D., Wodak, J., Wood, I.: Mining for the meanings of a murder: the impact of OCR quality on the use of digitized historical newspapers. Digital Humanit. Q. 8(1) (2014)Google Scholar
  16. 16.
    Taghva, K., Beckley, R., Coombs, J.: The effects of OCR error on the extraction of private information. In: Bunke, H., Spitz, A.L. (eds.) DAS 2006. LNCS, vol. 3872, pp. 348–357. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  17. 17.
    Taghva, K., Borsack, J., Condit, A., Erva, S.: The effects of noisy data on text retrieval. J. Am. Soc. Inf. Sci. 45(1), 50–58 (1994)CrossRefGoogle Scholar
  18. 18.
    Tanner, S., Muñoz, T., Ros, P.H.: Measuring mass text digitization quality and usefulness. D-Lib Mag. 15(7/8), 1082–9873 (2009)Google Scholar
  19. 19.
    Weymann, A., Luna Orozco, R.A., Mueller, C., Nickolay, B., Schneider, J., Barzik, K.: Einführung in die Digitalisierung von gedrucktem Kulturgut - Ein Handbuch für Einsteiger. Ibero-American Institute (Berlin) (2010)Google Scholar
  20. 20.
    Xie, H.I.: Evaluation of digital libraries: criteria and problems from users’ perspectives. Libr. Inf. Sci. Res. 28(3), 433–452 (2006)CrossRefGoogle Scholar
  21. 21.
    Xie, H.I.: Users’ evaluation of digital libraries (DLs): Their uses, their criteria, and their assessment. Inf. Process. Manage. 44(3), 1346–1373 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Myriam C. Traub
    • 1
  • Jacco van Ossenbruggen
    • 1
  • Lynda Hardman
    • 1
  1. 1.Centrum Wiskunde & InformaticaAmsterdamThe Netherlands

Personalised recommendations