Skip to main content
SpringerLink
Log in

Using Open Source Intelligence as a Tool for Reliable Web Searching

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Open source intelligence (OSINT) refers to the gathering and retrieval of data and information from online servers or sources that are freely accessible or are open-source. OSINT has been prominent over decades, under one name or the other. With the advent of instant communication and rapid knowledge exchange, much actionable and informative knowledge can now be accessed from unclassified, public sources. OSINT can help to collect data from the web and use it according to a model’s requirements and needs. A web search tool has been proposed here to look for similar advantages, explaining the tool’s functionalities and development strategy, which aims to minimize target search time and use minimum input data for the same. In addition, to provide a different means for searching and data filtering so that only the best information is provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. George RZ, Kline RD, Lowenthal MM. Intelligence and the national security strategist: enduring issues and challenges, vol. 58. Rowman and Littlefield; 2005. p. 273–84 (ISBN 9780742540392).

    Google Scholar 

  2. Pastor-Galindo J, Nespoli P, Gómez Mármol F, Martínez PG. The not yet exploited goldmine of OSINT: opportunities, open challenges and future trends. IEEE Access. 2020;8:10282–304.

    Article  Google Scholar 

  3. Rai BK, Srivastava AK. Pseudonymization techniques for providing privacy and security in EHR. Int J Emerg Trends Technol Comput Sci (IJETTCS). 2016;5(4)

  4. Rai BK, Srivastava AK. Patient controlled Pseudonym-based mechanism suitable for privacy and security of electronic health record. Int J Res Eng IT Soc Sci (IJREISS). 2017;588:26–30

  5. Rai BK, Srivastava AK. Prototype Implementation of patient controlled pseudonym-based mechanism for electronic health record (PcPbEHR). Int J Res Eng IT Soc Sci (IJREISS). 2017;7(7)

  6. Aliprandi C, et al CAPER: crawling and analysing Facebook for intelligence purposes. In: 2014 IEEE/ACM International Conference on advances in social networks analysis and mining (ASONAM 2014), Beijing, 2014; p. 665–669.

  7. Schaurer F, Störger J. Guide to the study of intelligence. The evolution of open source intelligence (OSINT). Intell J US Intell Stud. 2010;19(3):53–6.

    Google Scholar 

  8. Best C. Web mining for open source intelligence. In: 2008 12th International Conference information visualisation, London, 2008; p. 321–25.

  9. Yang H-C, Lee C-H. Mining open source text documents for intelligence gathering. In: 2012 International Symposium on information technologies in medicine and education, Hokodate, Hokkaido, 2012; p. 969–73.

  10. Best C. Challenges in open source intelligence. In: 2011 European Intelligence and Security Informatics Conference, Athens, 2011; p. 58–62.

  11. Shree R, Choudhury T, Gupta SC, Kumar P. KAFKA: the modern platform for data management and analysis in big data domain. In: 2nd International Conference on telecommunication and networks (TEL-NET), 2017; p. 1–5.

  12. Bell J. Machine learning streaming with Kafka. O’Reilly; 2020. p. 239–303 (ch12).

    Google Scholar 

  13. Wang X, Loguinov D. Load-balancing performance of consistent hashing: asymptotic analysis of random node join. IEEE/ACM Trans Netw. 2007;15(4):892–905.

    Article  Google Scholar 

  14. Zi-qiong Z, Qiang Y, Yi-jun L. Using Naïve Bayes classifier to distinguish reviews from non-review documents in Chinese. In: 2007 International Conference on management science and engineering, Harbin, 2007; p. 115–21.

  15. Francesco PD, Malavolta I, Lago P. Research on architecting microservices: trends, focus, and potential for industrial adoption. In: 2017 IEEE International Conference on software architecture (ICSA), Gothenburg, 2017; p. 21–30.

  16. Chen R, Miao H. A Selenium based approach to automatic test script generation for refactoring JavaScript code. In: 2013 IEEE/ACIS 12th International Conference on computer and information science (ICIS), IEEE, Niigata, Japan, p. 341–46.

  17. Cosmina I. Building reactive applications using spring. In: Pivotal Certified Professional Core Spring 5 Developer Exam, 2020; p. 903–55.

  18. Prakash SW, Deepalakshmi P. Server-based dynamic load balancing. In: 2017 International Conference on networks & advances in computational technologies (NetACT), Thiruvanthapuram, 2017; p. 25–28.

  19. Wehner P, Piberger C, Göhringer D. Using JSON to manage communication between services in the Internet of Things. In: 2014 9th International Symposium on reconfigurable and communication-centric systems-on-Chip (ReCoSoC), Montpellier, 2014; p. 1–4.

  20. Mishra A. Amazon rekognition—machine learning in the AWS cloud. Wiley; 2019. p. 421–44 (ch18).

    Book  Google Scholar 

  21. Tiwari S, Verma R, Jaiswal J, Rai BK. Open source intelligence initiating efficient investigation and reliable web searching. In: ICACDS 2020, Springer CCIS, 2020.

  22. Saini C, Arora V. Information retrieval in web crawling: a survey. In: 2016 International Conference on advances in computing, communications and informatics (ICACCI), Jaipur, 2016; p. 2635–643.

  23. Kamanwar NV, Kale SG. Web data extraction techniques: a review. In: 2016 World Conference on futuristic trends in research and innovation for social welfare (Startup Conclave), Coimbatore, 2016; p. 1–5.

  24. Qiang W, Zhongli Z. Reinforcement learning model, algorithms and its application. In: 2011 International Conference on mechatronic science, electric engineering and computer (MEC), Jilin, 2011; p. 1143–146.

Download references

Author information

Authors and Affiliations

  1. Department of IT, ABES Institute of Technology, Ghaziabad, Uttar Pradesh, India

    Bipin Kumar Rai, Ravi Verma & Shiva Tiwari

Authors
  1. Bipin Kumar Rai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ravi Verma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiva Tiwari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bipin Kumar Rai.

Ethics declarations

Conflict of Interest

On behalf of all the authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Advanced Computing and Data Sciences” guest edited by Mayank Singh, Vipin Tyagi and P.K. Gupta.

Appendix

Appendix

f :

Frequency of letter in the document

d :

JSONF document

D :

Total number of JSONF documents

N :

Number of d in which t occurs

pi:

Ith person

cw:

Crawler

S :

Set of links to be searched

lst:

Local storage of each crawl result

G_Stack:

Global stack

JSONF:

Final JSON

t :

Triggers

ß :

Learning rate

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rai, B.K., Verma, R. & Tiwari, S. Using Open Source Intelligence as a Tool for Reliable Web Searching. SN COMPUT. SCI. 2, 402 (2021). https://doi.org/10.1007/s42979-021-00777-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-021-00777-4

Keywords

Search

Navigation