Abstract
With the rapidly growing progress of science and technology, a large quantity of data and information is generated utilizing computers. Traditional relational databases, such as MySQL, are becoming increasingly unable to meet the user demands for rapid retrieval. However, Elasticsearch compensates for the delayed retrieval by offering users a fast search compatibility while ensuring high quality. The paper outlines the design and development of a system for indexing, clustering, and searching scientific documents. A Java Spring web server with Bootstrap, jQuery, and Foamtree was developed, visualizing the data with Kibana, allowing an orchestration of used technologies, enabling an efficient search and analysis function. The clustering of the documents is based on the metadata of Zotero and accessed via Carrot\(^2\).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bulk API: Elasticsearch Guide [7.16]. Elastic. https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html
Crontab Generator: Generate crontab syntax. https://crontab-generator.org/
Bai, J.: Feasibility analysis of big log data real time search based on Hbase and ElasticSearch. In: 2013 Ninth International Conference on Natural Computation (ICNC), pp. 1166–1170. IEEE (2013)
Baidu, Inc: Baidu, you will know. https://www.baidu.com/
Botev, C., Ameryahia, S., Shanmugasundaram, J.: Expressiveness and performance of full-text search languages. In: International Conference on Advances in Database Technology—EDBT. DBLP (2005)
Corporation for Digital Scholarship: Quick Start guide [Zotero Documentation]. https://www.zotero.org/support/quick_start_guide (2021)
Dwivedi, K., Dubey, S.K.: Analytical review on Hadoop distributed file system. In: 2014 5th International Conference-Confluence the Next Generation Information Technology Summit (Confluence), pp. 174–181. IEEE (2014)
Elasticsearch B.V.: Elasticsearch: The Official Distributed Search & Analytics Engine. https://www.elastic.co/elasticsearch (2021)
Gupta, S., Rani, R.: A comparative study of ElasticSearch and CouchDB document oriented databases. In: 2016 International Conference on Inventive Computation Technologies (ICICT). vol. 1, pp. 1–4 (2016). https://doi.org/10.1109/INVENTIVE.2016.7823252
Holst, A.: Total data volume worldwide 2010–2025. https://www.statista.com/statistics/871513/worldwide-data-created/ (2021)
Horky, V.: Command-line client for Zotero. https://github.com/vhotspur/cli-zotero. original-date: 2015-06-30T14:36:12Z
VMware, Inc.: Java Spring Web MVC Framework—Introduction. https://docs.spring.io/spring-framework/docs/3.2.x/spring-framework-reference/html/mvc.html (2021)
Walter-Tscharf, V.: DBPRO-DokCluster. https://github.com/FranzTscharf/DBPRO-DokCluster, original-date: 2017-11-02T11:17:00Z (2020)
Walter-Tscharf, V.: Practical approach to monitor runtime engine statistics for Apache Spark and Docker Swarm using graphite, grafana, and CollectD. In: 2021 International Conference on Converging Technology in Electrical and Information Engineering (ICCTEIE). pp. 91–96 (2021). https://doi.org/10.1109/ICCTEIE54047.2021.9650649
Wei, X.: Research on website search engine optimization based on mining of visiting behavior rules. In: Hubei University of Technology (2017)
Wu, S., Bao, L., Zhu, Z., Yi, F., Chen, W.: Storage and retrieval of massive heterogeneous iot data based on hybrid storage. In: 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). pp. 2982–2987. IEEE (2017)
Yin, H.: Deng fengdong: design and implementation of meteorological big data platform based on hadoop and ElasticSearch. In: IEEE 4th International Conference on Cloud Computing and Big Data Analysis (ICCCBDA). pp. 705–710. IEEE Press. IEEE, Chengdu (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Walter-Tscharf, F.F.W.V. (2022). Indexing, Clustering, and Search Engine for Documents Utilizing Elasticsearch and Kibana. In: Shakya, S., Ntalianis, K., Kamel, K.A. (eds) Mobile Computing and Sustainable Informatics. Lecture Notes on Data Engineering and Communications Technologies, vol 126. Springer, Singapore. https://doi.org/10.1007/978-981-19-2069-1_62
Download citation
DOI: https://doi.org/10.1007/978-981-19-2069-1_62
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-2068-4
Online ISBN: 978-981-19-2069-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)