Impact Analysis of OCR Quality on Research Tasks in Digital Archives
Humanities scholars increasingly rely on digital archives for their research instead of time-consuming visits to physical archives. This shift in research method has the hidden cost of working with digitally processed historical documents: how much trust can a scholar place in noisy representations of source texts? In a series of interviews with historians about their use of digital archives, we found that scholars are aware that optical character recognition (OCR) errors may bias their results. They were, however, unable to quantify this bias or to indicate what information they would need to estimate it. This, however, would be important to assess whether the results are publishable. Based on the interviews and a literature study, we provide a classification of scholarly research tasks that gives account of their susceptibility to specific OCR-induced biases and the data required for uncertainty estimations. We conducted a use case study on a national newspaper archive with example research tasks. From this we learned what data is typically available in digital archives and how it could be used to reduce and/or assess the uncertainty in result sets. We conclude that the current knowledge situation on the users’ side as well as on the tool makers’ and data providers’ side is insufficient and needs to be improved.
KeywordsOCR quality Digital libraries Digital humanities
We would like to thank our interviewees for their contributions, the National Library of The Netherlands for their support and the reviewers for their helpful feedback. This research is funded by the Dutch COMMIT/ program.
- 2.Alex, B., Grover, C., Klein, E., Tobin, R.: Digitised historical text: does it have to be mediOCRe? In: Jancsary, J. (ed.) Proceedings of KONVENS 2012, LThist 2012 Workshop, pp. 401–409. ÖGAI, September 2012Google Scholar
- 4.Bron, M.; Exploration and contextualization through interaction and concepts. Ph.D. Thesis (2013)Google Scholar
- 6.Cohen, D.J., Rosenzweig, R.: Digital History: A Guide to Gathering, Preserving, and Presenting the Past on the Web, vol. 28. University of Pennsylvania Press, Philadelphia (2006)Google Scholar
- 7.Croft, W.B., Harding, S., Taghva, K., Borsack, J.: An evaluation of information retrieval accuracy with simulated OCR output. Technical report, Amherst, MA, USA (1993)Google Scholar
- 9.Holley, R.: How good can it get? Analysing and improving OCR accuracy in large scale historic newspaper digitisation programs. D-Lib Mag. 15(3/4) (2009)Google Scholar
- 10.Holley, R.: Many hands make light work: public collaborative OCR text correction in Australian Historic Newspapers. Technical report, National Library of Australia, March 2009Google Scholar
- 11.Kettunen, K., Honkela, T., Lindén, K., Kauppinen, P., Pääkkönen, T., Kervinen, J. et al.: Analyzing and improving the quality of a historical news collection using language technology and statistical machine learning methods. In: Proceedings of the 80th IFLA General Conference and Assembly, IFLA World Library and Information Congress (2014)Google Scholar
- 12.Klijn, E.: The current state-of-art in newspaper digitization a market perspective. D-Lib Mag. 14, January 2008Google Scholar
- 15.Strange, C., McNamara, D., Wodak, J., Wood, I.: Mining for the meanings of a murder: the impact of OCR quality on the use of digitized historical newspapers. Digital Humanit. Q. 8(1) (2014)Google Scholar
- 18.Tanner, S., Muñoz, T., Ros, P.H.: Measuring mass text digitization quality and usefulness. D-Lib Mag. 15(7/8), 1082–9873 (2009)Google Scholar
- 19.Weymann, A., Luna Orozco, R.A., Mueller, C., Nickolay, B., Schneider, J., Barzik, K.: Einführung in die Digitalisierung von gedrucktem Kulturgut - Ein Handbuch für Einsteiger. Ibero-American Institute (Berlin) (2010)Google Scholar