Skip to main content
SpringerLink
Log in

Approach to Relevance Based Data Filtering in Data Retrieval Tasks

  • Conference paper
  • First Online:
Cyber-Physical Systems and Control II (CPS&C 2021)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 460))

Included in the following conference series:

  • 403 Accesses

Abstract

The research is devoted to solving the problem of conjugation of the virtual information space and the physical world in terms of data retrieval. Wherein the methods for solving the problem of extracting data from the virtual information space are determined by these data themselves (data-driven). The paper discusses ways to solve the problem of thematic content obtaining (data retrieval) from an unstructured set of information resources or news feeds. The problems of the “growing bubble” of unprocessed documents that arise during the “blind” collection of documents are discussed and ways to solve these problems are proposed in the paper. To reduce the resource consumption of the problem of forming a periodically updated search base, three approaches to the automatic collection of “raw data” are proposed. The proposed approaches to developing the sub-search systems are the part of a large class of modern methods and algorithms of adaptive heterogeneous data filtering for content retrieval and aggregation in subject oriented knowledge bases formation. A possible field of application and relevance are determined by the fact that for the closed cyber-physical systems the use of sub-search systems is proposed with unlimited and unstructured information space as an input that is being processed in real time. One of the possible implementations of proposed methods is in the development of the knowledge bases for science-technical documentation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alexandrov, V.V., Kuleshov, S.V.: Analytical monitoring of internet content infological approach. Quality. Innovation. Educ. 3(34), 68–70 (2008). (In Russ)

    Google Scholar 

  2. Borrero, J.D., Gualda, E.: Crawling big data in a new frontier for socioeconomic research: testing with social tagging. J. Spat. Organ. Dyn. 1(1), 1–22 (2013)

    Google Scholar 

  3. Ciolac, C.E.: A benchmark for banks’ strategy in online presence−an innovative approach based on elements of search engine optimization (SEO) and machine learning techniques. Econ. Ser. Manag. [Economy-Management series] 14(1), 91−105 (2011)

    Google Scholar 

  4. Xian-Yi, C., Ling-ling, Z., Qian, Z., Jin, W.: The framework of network public opinion monitoring and analyzing system based on semantic content identification. J. Converg. Inf. Technol. 5(10), 1−5 (2010)

    Google Scholar 

  5. Moran, A., Hampton, S., Dowson, S., Dagdelen, J., et al.: Online interactive platform for Covid-19 literature visual analytics: platform development study. J. Med. Internet Res. 23(7), e26995 (2021)

    Article  Google Scholar 

  6. AI-powered content research, intelligence, and writing. https://www.marketmuse.com/. Accessed 30 Aug 2021

  7. Content Curation−Curata, https://www.curata.com/. Accessed 30 Aug 2021

  8. Antiplagiat Homepage. https://www.antiplagiat.ru/. Accessed 30 Aug 2021 (In Russ)

  9. Olston, C., Najork, M.: Web Crawling. Found. Trends Inf. Retr. 4(3), 175–246 (2010)

    Article  MATH  Google Scholar 

  10. Najork, M., Heydon, A.: High-performance web crawling. In: Abello, J., Pardalos, P.M., Resende, M.G.C. (eds.) Handbook of Massive Data Sets. MC, vol. 4, pp. 25–45. Springer, Boston, MA (2002). https://doi.org/10.1007/978-1-4615-0005-6_2

    Chapter  Google Scholar 

  11. Laliwala, Z., Shaikh, A.: Web Crawling and Data Mining with Apache Nutch. Packt Publishing (2013)

    Google Scholar 

  12. Nasraoui, O.: Web data mining: exploring hyperlinks, contents, and usage data. In: ACM SIGKDD Explorations Newsletter (2008)

    Google Scholar 

  13. Chakrabarti, S.: Mining the Web: Discovering Knowledge from Hypertext Data. Elsevier (2003)

    Google Scholar 

  14. Vanden Broucke, S., Baesens, B.: From web scraping to web crawling. In: Practical Web Scraping for Data Science, pp. 155–172. Apress, Berkeley, CA (2018). https://doi.org/10.1007/978-1-4842-3582-9_6

  15. Manning, Ch.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval Cambridge. Cambridge University Press (2008)

    Google Scholar 

  16. Markov, Z., Larose, D.T.: Data Mining the Web: Uncovering Patterns in Web Content, Structure, and Usage. John Wiley & Sons (2007)

    Google Scholar 

  17. Liu, B.: Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data. Springer Science & Business Media (2007)

    Google Scholar 

  18. Pugh, W. Henzinger, M.H.: U.S. Patent Application No. 13/313, 913p (2012)

    Google Scholar 

  19. Enge, E., Spencer, S., Stricchiola, J., Fishkin, R.: The Art of SEO. O'Reilly Media, Inc (2012)

    Google Scholar 

  20. Aksenov, A.Y., Zaytseva, A.A., Kuleshov, S.V.: The features of internet content formation and the problem of it's automatic analysis. In: Proceedings of 2-d International Scientific Conference “Technological perspective within the framework of the Eurasian space: new markets and points of economic growth”, SPb: Publishing “Asterion”, p. 309−311 (2016)

    Google Scholar 

  21. Tsai, C.-F., Chen, Z.-Y.: Towards high dimensional instance selection: an evolutionary approach. Decis. Support. Syst. 61, 79–92 (2014)

    Article  Google Scholar 

  22. Derrac, J., García, S., Herrera, F.: A survey on evolutionary instance selection and generation. Int. J. Appl. Metaheuristic Comput. 1(1), 60–92 (2010)

    Article  Google Scholar 

  23. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. In: NIPS Proceedings. Preprint at https://arxiv.org/pdf/1503.02531.pdf (2015). Accessed 30 Aug 2021

  24. Fiori, A. (ed.): Innovative Document Summarization Techniques: Revolutionizing Knowledge Understanding. IGI Global (2014)

    Google Scholar 

  25. Kuleshov, S.V., Mikhailov, S.N.: The variant of the subsearch engine architecture for analytical monitoring. SPIIRAS Proc. 8(31), 247–254 (2013). (In Russian)

    Article  Google Scholar 

  26. Zaytseva, A.A., Kuleshov, S.V., Mikhailov, S.N.: The method for the text quality estimation in the task of analytical monitoring of information resources. SPIIRAS Proc. 6(37), 144–155 (2014). (In Russian)

    Article  Google Scholar 

  27. Wang, T., Zhu, J., Torralba, A., Efros, A.: Dataset distillation. Preprint at arXiv:1811.10959 (2018). Accessed 30 Aug 2021

  28. Medvedev D., D’yakonov A.: New properties of the data distillation method when working with tabular data. Preprint at arXiv:2010.09839 (2020). Accessed 30 Aug 2021

  29. Chekhovich Y.: Plagiarist's honest eyes, or another look at the future of borrowing detection systems [Chestnyye glaza plagiatora, ili yeshche odin vzglyad na budushcheye sistem obnaruzheniya zaimstvovaniy]. https://habr.com/ru/company/antiplagiat/blog/549982/. Accessed 30 Aug 2021 (In Russian)

  30. This is how the search for borrowings in Antiplagiat works [Tak ustroyen poisk zaimstvovaniy v Antiplagiate]. https://habr.com/ru/company/antiplagiat/blog/429634/. Accessed 30 Aug 2021 (In Russian)

  31. Osipov, V., Kuleshov, S., Zaytseva, A., Levonevskiy, D., Miloserdov, D.: Neural network forecasting of news feeds. Expert. Syst. Appl. 169, 114521 (2021)

    Article  Google Scholar 

Download references

Acknowledgments

The reported study was partly funded by RFBR, project number 20–04-60455 and budget, project No FFZF-2022–0005.

Author information

Authors and Affiliations

  1. St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences (SPIIRAS), St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 39, 14th Line V.O. (Vasilyevsky Island), 199178, St. Petersburg, Russia

    Sergey V. Kuleshov, Alexandra A. Zaytseva & Alexey Aksenov

Authors
  1. Sergey V. Kuleshov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexandra A. Zaytseva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexey Aksenov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexandra A. Zaytseva .

Editor information

Editors and Affiliations

  1. Peter the Great St. Petersburg Polytechnic University, St. Petersburg, Russia

    Dmitry G. Arseniev

  2. School of Science and Technology, City, University of London, London, UK

    Nabil Aouf

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kuleshov, S.V., Zaytseva, A.A., Aksenov, A. (2023). Approach to Relevance Based Data Filtering in Data Retrieval Tasks. In: Arseniev, D.G., Aouf, N. (eds) Cyber-Physical Systems and Control II. CPS&C 2021. Lecture Notes in Networks and Systems, vol 460. Springer, Cham. https://doi.org/10.1007/978-3-031-20875-1_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20875-1_47

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20874-4

  • Online ISBN: 978-3-031-20875-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation