Skip to main content
SpringerLink
Log in

Redis-based full-text search extensions for relational databases

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In order to overcome the inefficiency and resource consumption of full-text search in relational databases, a light full-text search model with auxiliary cache is developed. Specially, we utilize the MySQL as the data storage layer and the Redis as the index cache layer. We first design a full-index cache mechanism by the Redis-based inverted indexes construction methods to augment the efficient memory processing capability of relational databases. In addition, an increment-index synchronization mechanism is implemented to fit the dynamic update of relation database. For hot data, an index update optimization mechanism is provided to guarantee the fast response and accuracy of full-text search. The proposed Redis-based auxiliary cache method has also been put into practical industrial applications and achieved promising results. Finally, we evaluate our method from index space occupation, time consumption and the accuracy of retrieval results. The experimental results show that the proposed model outperforms MySQL Full-Text method 2–3 times and surpasses ElasticSearch 12 times in space resource consumption.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Jatana N, Puri S, Ahuja M, Kathuria I, Gosain D (2012) A survey and comparison of relational and non-relational database. Int J Eng Res Technol 1(6):1–5

    Google Scholar 

  2. Qtaish A, Alshammari MT (2019) A narrative review of storing and querying xml documents using relational database. J Inf Knowl Manag 18(04):1950048

    Article  Google Scholar 

  3. Kadilierakis G Nikas C, Fafalios P, Papadakos P, Tzitzikas Y (2020) Elas4rdf: Multi-perspective triple-centered keyword search over rdf using elasticsearch. In: Semantic Web: ESWC 2020 Satellite, pp. 122–128

  4. Čerešňák R, Kvet M (2019) Comparison of query performance in relational a non-relation databases. Transp Res Procedia 40:170–177

    Article  Google Scholar 

  5. Jin D, Chen G, Hao W, Bin L (2020) Whole database retrieval method of general relational database based on lucene. In: Proceedings of the International Conference on Artificial Intelligence and Computer Applications, pp. 1277–1279

  6. Safder I, Hassan S-U, Visvizi A, Noraset T, Nawaz R, Tuarob S (2020) Deep learning-based extraction of algorithmic metadata in full-text scholarly documents. Inf Process Manag 57(6):102269

    Article  Google Scholar 

  7. Roy I, Velugoti VSBR, Chakrabarti S, De A (2022) Interpretable neural subgraph matching for graph retrieval. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 8115–8123

  8. Wu X (2020) Analysis of data fuzzy retrieval method in embedded database (mysql). Comput Eng Softw 41(9):115–118

    Google Scholar 

  9. Akram S (2021) Exploiting intel optane persistent memory for full text search. In: Proceedings of the ACM SIGPLAN International Symposium on Memory Management, pp. 80–93

  10. Chaitanya BSSK, Reddy DAK, Chandra BPSE, Krishna AB, Menon RR (2019) Full-text search using database index. In: Proceedings of the International Conference on Computing, Communication, Control and Automation, pp. 1–5

  11. Gyorodi C, Gyorodi R, Pecherle G, Cornea GM (2010) Full-text search engine using mysql. Int J Comput Commun Control 5(5):735–743

    Article  Google Scholar 

  12. Bast H, Buchhold B (2013) An index for efficient semantic full-text search. In: Proceedings of the ACM International Conference on Information & Knowledge Management, pp. 369–378

  13. Iqbal W, Malik WI, Bukhari F, Almustafa KM, Nawaz Z (2021) Big data full-text search index minimization using text summarization. Inf Technol Control 50(2):375–389

    Article  Google Scholar 

  14. Zamfir V-A, Carabas M, Carabas C, Tapus N (2019) Systems monitoring and big data analysis using the elasticsearch system. In: Proceedings of the International Conference on Control Systems and Computer Science, pp. 188–193

  15. Sun J, Nie P, Xu L, Zhang H (2022) Design and implementation of analyzer management system based on elasticsearch. In: Proceedings of the International Conference on Web Information Systems and Applications, pp. 254–266

  16. Zhan J, Mao J, Liu Y, Guo J, Zhang M, Ma S (2021) Jointly optimizing query encoder and product quantization to improve retrieval performance. In: Proceedings of the ACM International Conference on Information & Knowledge Management, pp. 2487–2496

  17. Devins J, Tibshirani J, Lin J (2022) Aligning the research and practice of building search applications: Elasticsearch and pyserini. In: Proceedings of the ACM International Conference on Web Search and Data Mining, pp. 1573–1576

  18. Amato G, Bolettieri P, Carrara F, Falchi F, Gennaro C (2018) Large-scale image retrieval with elasticsearch. In: Proceedings of the International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 925–928

  19. Shah N, Willick D, Mago V (2022) A framework for social media data analytics using Elasticsearch and Kibana. Wireless Netw 28:1179–1187

    Article  Google Scholar 

  20. Mackenzie J, Dai Z, Gallagher L, Callan J (2020) Efficiency implications of term weighting for passage retrieval. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1821–1824

  21. Mallia A, Khattab O, Suel T, Tonellotto N (2021) Learning passage impacts for inverted indexes. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1723–1727

  22. Shin Y, Ahn J, Im D (2022) Join optimization for inverted index technique on relational database management systems. Expert Syst Appl 198:116956

    Article  Google Scholar 

  23. Zheng Q, Yang T, Kan Y, Tan X, Yang J, Jiang X (2021) On the analysis of cache invalidation with lru replacement. IEEE Trans Parallel Distrib Syst 33(3):654–666

    Article  Google Scholar 

  24. Feng B, Tian A, Yu S, Li J, Zhou H, Zhang H (2022) Efficient cache consistency management for transient iot data in content-centric networking. IEEE Internet Things J 9(15):12931–12944

    Article  Google Scholar 

  25. Pan C, Wang X, Luo Y, Wang Z (2021) Penalty-and locality-aware memory allocation in redis using enhanced aet. ACM Trans Storage 17(2):1–45

    Article  Google Scholar 

  26. Su Q, Gao X, Zhang X, Wang Z (2022) A novel cache strategy leveraging redis with filters to speed up queries. In: Proceedings of the International Conference on High Performance Computing and Communication, vol. 12162, pp. 150–154

  27. Matallah H, Belalem G, Bouamrane K (2020) Evaluation of nosql databases: Mongodb, cassandra, hbase, redis, couchbase, orientdb. Int J Softw Sci Comput Intell 12(4):71–91

    Article  Google Scholar 

  28. Chen X, Wang F, Xu J, Zhu D, Tan P, Ma J (2020) A distributed cache system based on redis for high-speed railway catenary monitoring system. In: Proceedings of the Chinese Automation Congress, pp. 2048–2053

  29. Zhang X, Wu P, Cai J, Wang K (2019) A contrastive study of Chinese text segmentation tools in marketing notification texts. In: Proceedings of the Journal of Physics: Conference Series, vol. 1302, p. 022010

  30. Mallia A, Khattab O, Suel T, Tonellotto N (2021) Learning passage impacts for inverted indexes. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1723–1727

  31. Li P, Luo B, Zhu W, Xu H (2020) Cluster-based distributed dynamic cuckoo filter system for redis. Int J Parallel Emerg Distrib Syst 35(3):340–353

    Article  Google Scholar 

  32. Gonçalves J, Matos M, Rodrigues R (2022) Sconekv: a scalable, strongly consistent key-value store. IEEE Trans Parallel Distrib Syst 33(12):4164–4175

    Article  Google Scholar 

  33. Eddelbuettel D (2022) A brief introduction to redis. arXiv preprint arXiv:2203.06559

  34. Siddiqui IF, Lee SU-J, Abbas A (2020) A novel knowledge-based battery drain reducer for smart meters. Intell Autom Soft Comput 26(1):107–119

    Google Scholar 

  35. Melnik S, Gubarev A, Long JJ, Romer G, Shivakumar S, Tolton M, Vassilakis T (2010) Dremel: interactive analysis of web-scale datasets. In: Proceedings of the VLDB Endowment, pp. 330–339

  36. He J (2020) Research on personalized search based on elasticsearch. In: Proceedings of the International Conference on Civil Aviation Safety and Information Technology, pp. 572–575

Download references

Acknowledgements

This work was partially supported by grants from the National Science Foundation of China (No. 62176221), the National Social Science Fund of China (No. 20BMZ092).

Author information

Author notes

  1. Xuehua Liao, Lilan Peng, Ting Yang and Zhousen Zhu have contributed equally to this work.

Authors and Affiliations

  1. School of Computer Science, Sichuan Normal University, Chengdu, 610101, Sichuan, China

    Xuehua Liao

  2. School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, 611756, Sichuan, China

    Lilan Peng & Tianrui Li

  3. School of Physics and Electronic Engineering, Sichuan Normal University, Chengdu, 610101, Sichuan, China

    Ting Yang & Zhousen Zhu

Authors
  1. Xuehua Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lilan Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ting Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianrui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhousen Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianrui Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liao, X., Peng, L., Yang, T. et al. Redis-based full-text search extensions for relational databases. Int. J. Mach. Learn. & Cyber. (2024). https://doi.org/10.1007/s13042-024-02160-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13042-024-02160-0

Keywords

Search

Navigation