HistorEx: Exploring Historical Text Corpora Using Word and Document Embeddings
Written text can be understood as a means to acquire insights into the nature of past and present cultures and societies. Numerous projects have been devoted to digitizing and publishing historical textual documents in digital libraries which scientists can utilize as valuable resources for research. However, the extent of textual data available exceeds humans’ abilities to explore the data efficiently. In this paper, a framework is presented which combines unsupervised machine learning techniques and natural language processing on the example of historical text documents on the 19th century of the USA. Named entities are extracted from semi-structured text, which is enriched with complementary information from Wikidata. Word embeddings are leveraged to enable further analysis of the text corpus, which is visualized in a web-based application.
KeywordsWord embeddings Document vectors Wikidata Cultural heritage Visualization Recommender system
This paper is motivated by the the first German open cultural data hackathon, Coding da V1nc1. It supports interdisciplinary work on cultural heritage data by bringing together GLAM institutions, programmers and designers to develop ideas and prototypes for the cultural sector and for the public.
- 2.Jänicke, S., Franzini, G., Cheema, M.F., Scheuermann, G.: Visual text analysis in digital humanities. In: Computer Graphics Forum, vol. 36, pp. 226–250. Wiley Online Library (2017)Google Scholar
- 3.Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. CoRR abs/1405.4053 (2014). http://arxiv.org/abs/1405.4053