OSINT and the Dark Web

Part of the Advanced Sciences and Technologies for Security Applications book series (ASTSA)


The Dark Web, a part of the Deep Web that consists of several darknets (e.g. Tor, I2P, and Freenet), provides users with the opportunity of hiding their identity when surfing or publishing information. This anonymity facilitates the communication of sensitive data for legitimate purposes, but also provides the ideal environment for transferring information, goods, and services with potentially illegal intentions. Therefore, Law Enforcement Agencies (LEAs) are very much interested in gathering OSINT on the Dark Web that would allow them to successfully prosecute individuals involved in criminal and terrorist activities. To this end, LEAs need appropriate technologies that would allow them to discover darknet sites that facilitate such activities and identify the users involved. This chapter presents current efforts in this direction by first providing an overview of the most prevalent darknets, their underlying technologies, their size, and the type of information they contain. This is followed by a discussion of the LEAs’ perspective on OSINT on the Dark Web and the challenges they face towards discovering and de-anonymizing such information and by a review of the currently available techniques to this end. Finally, a case study on discovering terrorist-related information, such as home made explosive recipes, on the Dark Web is presented.


Search Engine Relay Node Money Laundering Child Pornography Silk Road 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Bartlett J (2014) The Dark Net. Random House, LondonGoogle Scholar
  2. Bergman MK (2001) White paper: the deep web: surfacing hidden value. J Electron Pub 7(1)Google Scholar
  3. Biryukov A, Pustogarov I, Thill F, Weinmann RP (2014) Content and popularity analysis of Tor hidden services. In: 2014 IEEE 34th International conference on distributed computing systems workshops (ICDCSW). IEEE, pp 188–193Google Scholar
  4. Chen H (2011) Dark web: exploring and data mining the dark side of the web (vol 30). Springer Science and Business Media, BerlinGoogle Scholar
  5. Christin N (2013) Traveling the silk road: a measurement analysis of a large anonymous online marketplace. In: Proceedings of the 22nd international conference on world wide web. ACM, pp 213–224Google Scholar
  6. Clarke I, Sandberg O, Toseland M, Verendel V (2010) Private communication through a network of trusted connections: the dark Freenet. Available at:
  7. Dingledine R, Mathewson N, Syverson P (2004) Tor: the second-generation onion router. Naval Research Lab Washington DCGoogle Scholar
  8. Erdin E, Zachor C, Gunes MH (2015) How to find hidden users: a survey of attacks on anonymity networks. IEEE Commun Surv Tutorials 17(4):2296–2316CrossRefGoogle Scholar
  9. Huber M, Mulazzani M, Weippl E (2010) Tor HTTP usage and information leakage. In: IFIP international conference on communications and multimedia security. Springer, Berlin, pp 245–255Google Scholar
  10. I2P (n.d.) I2P: a scalable framework for anonymous communication—I2P, from
  11. Kalpakis G, Tsikrika T, Iliou C, Mironidis T, Vrochidis S, Middleton J, Kompatsiaris I (2016) Interactive discovery and retrieval of web resources containing home made explosive recipes. In: International conference on human aspects of information security, privacy, and trust. Springer International Publishing, Berlin, pp 221–233Google Scholar
  12. Manikonda L, Hu Y, Kambhampati S (2014) Analyzing user activities, demographics, social network structure and user-generated content on instagram. arXiv preprint arXiv:1410.8099
  13. Memex Project (Domain-Specific Search) Open Catalog. Available at:
  14. Mittal P, Khurshid A, Juen J, Caesar M, Borisov N (2011) Stealthy traffic analysis of low-latency anonymous communication using throughput fingerprinting. In: Proceedings of the 18th ACM conference on computer and communications security. ACM, pp 215–226Google Scholar
  15. Moore D, Rid T (2016) Cryptopolitik and the darknet. Survival 58(1):7–38Google Scholar
  16. Murdock V (2011) Your mileage may vary: on the limits of social media. SIGSPATIAL Spec 3(2):62–66CrossRefGoogle Scholar
  17. Olston C, Najork M (2010) Web crawling: foundations and trends in information retrievalGoogle Scholar
  18. Owen G, Savage N (2015) The Tor dark net.’ global commission on internet governance (No. 20)Google Scholar
  19. Paganini P (2015) PunkSPIDER, the crawler that scanned the Dark Web. Retrieved 27 Jul 2016, from
  20. Pant G, Srinivasan P (2005) Learning to crawl: comparing classification schemes. ACM Trans Inform Syst (TOIS) 23(4):430–462Google Scholar
  21. Project Artemis—OSINT activities on Deep Web, (2013, July). Available at
  22. Ricardo BY, Berthier RN (2011) Modern information retrieval: the concepts and technology behind search second edition. Addision Wesley, 84, 2Google Scholar
  23. Sherman C, Price G (2003) The invisible web: uncovering sources search engines can’t see. Libr Trends 52(2):282–298Google Scholar
  24. Thomee B, Shamma DA, Friedland G, Elizalde B, Ni K, Poland D, Li LJ (2016) YFCC100M: the new data in multimedia research. Commun ACM 59(2):64–73Google Scholar
  25. Tian G, Duan Z, Baumeister T, Dong Y (2013) A traceback attack on freenet. In: INFOCOM, 2013 Proceedings IEEE. IEEE, pp 1797–1805Google Scholar
  26. Tsikrika T, Moumtzidou A, Vrochidis S, Kompatsiaris I (2016) Focussed crawling of environmental web resources based on the combination of multimedia evidence. Multimedia Tools Appl 75(3):1563–1587CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Centre for Research and Technology Hellas, Information Technologies Institute (CERTH-ITI)Thermi-ThessalonikiGreece
  2. 2.Police Service Northern IrelandBelfastIreland

Personalised recommendations