Abstract
The Dark Web, a part of the Deep Web that consists of several darknets (e.g. Tor, I2P, and Freenet), provides users with the opportunity of hiding their identity when surfing or publishing information. This anonymity facilitates the communication of sensitive data for legitimate purposes, but also provides the ideal environment for transferring information, goods, and services with potentially illegal intentions. Therefore, Law Enforcement Agencies (LEAs) are very much interested in gathering OSINT on the Dark Web that would allow them to successfully prosecute individuals involved in criminal and terrorist activities. To this end, LEAs need appropriate technologies that would allow them to discover darknet sites that facilitate such activities and identify the users involved. This chapter presents current efforts in this direction by first providing an overview of the most prevalent darknets, their underlying technologies, their size, and the type of information they contain. This is followed by a discussion of the LEAs’ perspective on OSINT on the Dark Web and the challenges they face towards discovering and de-anonymizing such information and by a review of the currently available techniques to this end. Finally, a case study on discovering terrorist-related information, such as home made explosive recipes, on the Dark Web is presented.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
The term Dark Web is often confused with Deep Web, especially in media reporting. However, it should be clear that the two terms are distinguished, and Dark Web constitutes a subset of Deep Web exhibiting specific properties. Moreover, the term Dark Web has also been used to refer to the specific content generated by international terrorist groups and made available either on the Surface Web (including on Web sites, forums, chat rooms, blogs, and social networking sites) or on the Deep Web (Chen 2011); this definition is different to the one employed in this book.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
As estimated by the Tor Project https://metrics.torproject.org/ on April 27, 2016.
- 14.
As estimated by http://stats.i2p/ on April 27, 2016.
- 15.
As estimated by http://asksteved.com/stats/ on April 27, 2016.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
HOMER (Home Made Explosives and Recipes characterization—http://www.homer-project.eu/) is an EU funded project that aims to expand the knowledge of European bodies about HMEs and to improve the capacity of security and law enforcement agencies to cope with current and anticipated threats so as to reduce the probability that HMEs will be used by terrorists.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
References
Bartlett J (2014) The Dark Net. Random House, London
Bergman MK (2001) White paper: the deep web: surfacing hidden value. J Electron Pub 7(1)
Biryukov A, Pustogarov I, Thill F, Weinmann RP (2014) Content and popularity analysis of Tor hidden services. In: 2014 IEEE 34th International conference on distributed computing systems workshops (ICDCSW). IEEE, pp 188–193
Chen H (2011) Dark web: exploring and data mining the dark side of the web (vol 30). Springer Science and Business Media, Berlin
Christin N (2013) Traveling the silk road: a measurement analysis of a large anonymous online marketplace. In: Proceedings of the 22nd international conference on world wide web. ACM, pp 213–224
Clarke I, Sandberg O, Toseland M, Verendel V (2010) Private communication through a network of trusted connections: the dark Freenet. Available at: https://freenetproject.org/assets/papers/freenet-0.7.5-paper.pdf
Dingledine R, Mathewson N, Syverson P (2004) Tor: the second-generation onion router. Naval Research Lab Washington DC
Erdin E, Zachor C, Gunes MH (2015) How to find hidden users: a survey of attacks on anonymity networks. IEEE Commun Surv Tutorials 17(4):2296–2316
Huber M, Mulazzani M, Weippl E (2010) Tor HTTP usage and information leakage. In: IFIP international conference on communications and multimedia security. Springer, Berlin, pp 245–255
I2P (n.d.) I2P: a scalable framework for anonymous communication—I2P, from https://geti2p.net/en/docs/how/tech-intro
Kalpakis G, Tsikrika T, Iliou C, Mironidis T, Vrochidis S, Middleton J, Kompatsiaris I (2016) Interactive discovery and retrieval of web resources containing home made explosive recipes. In: International conference on human aspects of information security, privacy, and trust. Springer International Publishing, Berlin, pp 221–233
Manikonda L, Hu Y, Kambhampati S (2014) Analyzing user activities, demographics, social network structure and user-generated content on instagram. arXiv preprint arXiv:1410.8099
Memex Project (Domain-Specific Search) Open Catalog. Available at: http://opencatalog.darpa.mil/MEMEX.html
Mittal P, Khurshid A, Juen J, Caesar M, Borisov N (2011) Stealthy traffic analysis of low-latency anonymous communication using throughput fingerprinting. In: Proceedings of the 18th ACM conference on computer and communications security. ACM, pp 215–226
Moore D, Rid T (2016) Cryptopolitik and the darknet. Survival 58(1):7–38
Murdock V (2011) Your mileage may vary: on the limits of social media. SIGSPATIAL Spec 3(2):62–66
Olston C, Najork M (2010) Web crawling: foundations and trends in information retrieval
Owen G, Savage N (2015) The Tor dark net.’ global commission on internet governance (No. 20)
Paganini P (2015) PunkSPIDER, the crawler that scanned the Dark Web. Retrieved 27 Jul 2016, from http://securityaffairs.co/wordpress/37632/hacking/punkspider-scanned-tor.html
Pant G, Srinivasan P (2005) Learning to crawl: comparing classification schemes. ACM Trans Inform Syst (TOIS) 23(4):430–462
Project Artemis—OSINT activities on Deep Web, infosecinstitute.com (2013, July). Available at http://resources.infosecinstitute.com/project-artemis-osint-activities-on-deep-web/
Ricardo BY, Berthier RN (2011) Modern information retrieval: the concepts and technology behind search second edition. Addision Wesley, 84, 2
Sherman C, Price G (2003) The invisible web: uncovering sources search engines can’t see. Libr Trends 52(2):282–298
Thomee B, Shamma DA, Friedland G, Elizalde B, Ni K, Poland D, Li LJ (2016) YFCC100M: the new data in multimedia research. Commun ACM 59(2):64–73
Tian G, Duan Z, Baumeister T, Dong Y (2013) A traceback attack on freenet. In: INFOCOM, 2013 Proceedings IEEE. IEEE, pp 1797–1805
Tsikrika T, Moumtzidou A, Vrochidis S, Kompatsiaris I (2016) Focussed crawling of environmental web resources based on the combination of multimedia evidence. Multimedia Tools Appl 75(3):1563–1587
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this chapter
Cite this chapter
Kalpakis, G. et al. (2016). OSINT and the Dark Web. In: Akhgar, B., Bayerl, P., Sampson, F. (eds) Open Source Intelligence Investigation. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-47671-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-47671-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47670-4
Online ISBN: 978-3-319-47671-1
eBook Packages: Computer ScienceComputer Science (R0)