Abstract
Distributing data has always been a fundamental Internet use case. Today, the Internet is the primary channel for distributing media content, and many other types of data are increasingly managed online. This tendency has led to growing requirements regarding the quantity and quality of Internet technologies with which to manage distributed data. In addition, certain fundamental characteristics of Internet computing, such as location-based communication and centralized service provisioning, are being challenged. This chapter focuses on next-generation peer-to-peer (P2P) data networks that seek to solve some of these challenges and integrate well with other modern Internet technologies, including cloud computing, blockchains, and the Internet of Things. Beyond introducing the family of modern P2P data networks, this chapter presents a prime example of such networks—the InterPlanetary File System (IPFS)—in detail. It illustrates the technical realization of main IPFS characteristics like content addressing and decentralization. It also explains the Filecoin system, which extends the IPFS by means of a storage and retrieval market. The chapter covers a broad view of P2P data networks by outlining and comparing alternative approaches, listing open challenges, and explaining current research areas.
Chapter co-author: Christian Zirpins, Hochschule Karlsruhe, Institut für Datenzentrierte Software-Systeme (DSS), Karlsruhe, Germany
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The regulations of BitTorrent foster Pareto efficiency, a state where no peer might increase its download rate without decreasing the download rate of another peer (Cohen 2003).
- 3.
Off-chaining is a DLT strategy, which includes storing parts of transaction data outside the resource-constrained ledger. The latter contains a secure reference to the externally stored data.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
Information can be found in the Filecoin online documentation (https://docs.filecoin.io/).
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
Filecoin requires storage miners to dedicate (storage) resources and to provide upfront token collaterals proportional to the storage hardware committed (Protocol-Labs 2017).
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
References
Ahlgren B, Dannewitz C, Imbrenda C, Kutscher D, Ohlman B (2012) A survey of information-centric networking. IEEE Commun Mag 50(7):26–36
Alexander PJ (2002) Peer-to-peer file sharing: the case of the music recording industry. Rev Ind Organ 20:151–161
Ali M, Nelson J, Shea R, Freedman MJ (2016, June 22–24) Blockstack: a global naming and storage system secured by Blockchains. In: Paper presented at the 2016 USENIX annual technical conference (USENIX ATC 16), Denver, CO, USA
Ali MS, Dolui K, Antonelli F (2017, October 22–25) IoT data privacy via Blockchains and IPFS. In: Paper presented at the proceedings of the seventh international conference on the Internet of things, Linz, Austria
Androutsellis-Theotokis S, Spinellis D (2004) A survey of peer-to-peer content distribution technologies. ACM Comput Surv 36(4):335–371
Ascigil O, Reñé S, Król M, Pavlou G, Zhang L, Hasegawa T, Koizumi Y, Kita K (2019, September 24–26) Towards peer-to-peer content retrieval markets: enhancing IPFS with ICN. In: Paper presented at the proceedings of the 6th ACM conference on information-centric networking, Macao China
Balakrishnan H, Kaashoek MF, Karger D, Morris R, Stoica I (2003) Looking up data in P2p systems. Commun ACM 46(2):43–48
Balduf L, Henningsen S, Florian M, Rust S, Scheuermann B (2022, July 10–13) Monitoring data requests in decentralized data storage systems: a case study of IPFS. In: Paper presented at the International Conference on Distributed Computing Systems
Battah AA, Madine MM, Alzaabi H, Yaqoob I, Salah K, Jayaraman R (2020) Blockchain-based multi-party authorization for accessing IPFS encrypted data. IEEE Access 8:196813–196825
Benet J (2014) IPFS-content addressed, versioned, P2p file system. arXiv preprint arXiv:14073561
Benisi NZ, Aminian M, Javadi B (2020) Blockchain-based decentralized storage networks: a survey. J Netw Comput Appl 162:102656
Bharambe AR, Herley C, Padmanabhan VN (2006, April 23–29) Analyzing and improving a BitTorrent networks performance mechanisms. In: Paper presented at the proceedings IEEE INFOCOM 2006. 25th IEEE international conference on computer communications, Barcelona, Spain
Chai WK, Wang N, Psaras I, Pavlou G, Wang C, de Blas GG, Ramon-Salguero FJ, Liang L, Spirou S, Beben A (2011) Curling: content-ubiquitous resolution and delivery infrastructure for next-generation services. IEEE Commun Mag 49(3):112–120
Chothia T, Chatzikokolakis K (2005, December 6–9) A survey of anonymous peer-to-peer file-sharing. In: Paper presented at the international conference on embedded and ubiquitous computing, Nagasaki, Japan
Clarke I, Sandberg O, Wiley B, Hong TW (2001) Freenet: a distributed anonymous information storage and retrieval system. In: Paper presented at the Designing privacy enhancing technologies: international workshop on design issues in anonymity and unobservability Berkeley, CA, USA, July 25–26, 2000 Proceedings, Berkeley, CA, USA
Clarke I, Miller SG, Hong TW, Sandberg O, Wiley B (2002) Protecting free expression online with Freenet. IEEE Internet Comput 6(1):40–49
Cohen B (2003, June 5–6) Incentives build robustness in BitTorrent. In: Paper presented at the workshop on economics of peer-to-peer systems, Berkeley, CA, USA
Confais B, Lebre A, Parrein B (2017, May 14–15) An object store service for a fog/edge computing infrastructure based on IPFS and a scale-out NAS. In: Paper presented at the 2017 IEEE 1st international conference on fog and edge computing (ICFEC), Madrid, Spain
Dale B (2017) Turkey can’t block this copy of Wikipedia. https://observer.com/2017/05/turkey-wikipedia-ipfs/. Accessed 7 December 2023
Daniel E, Tschorsch F (2022) IPFS and friends: a qualitative comparison of next generation peer-to-peer data networks. IEEE Commun Surv Tutor 24(1):31–52
Dannewitz C, Kutscher D, Ohlman B, Farrell S, Ahlgren B, Karl H (2013) Network of information (NetInf) – an information-centric networking architecture. Comput Commun 36(7):721–735
De la Rocha A, Dias D, Psaras Y (2021) Accelerating content routing with BitSwap: a multi-path file transfer protocol in IPFS and Filecoin. https://research.protocol.ai/publications/accelerating-content-routing-with-bitswap-a-multi-path-file-transfer-protocol-in-ipfs-and-filecoin/. Accessed 7 December 2023
De la Rocha A, Gillis A, Dryjanski D, Schasny I, Munizaga M, Derkani M, Scott W (2023) IPNI: interplanetary network indexer. https://github.com/ipni/specs/blob/main/IPNI.md. Accessed 10 November 2023
Doan TV, Psaras Y, Ott J, Bajpai V (2022) Towards decentralised cloud storage with IPFS: opportunities, challenges, and future considerations. IEEE Internet Comput 26(6)
Dworkin MJ (2015) SHA-3 standard: permutation-based hash and extendable-output functions
El Ioini N, Pahl C (2018, October 22–26) A review of distributed ledger technologies. In: Paper presented at the on the move to meaningful Internet systems. OTM 2018 conferences: confederated international conferences: CoopIS, C&TC, and ODBASE Valletta, Malta
Feldman M, Chuang J (2005) Overcoming free-riding behavior in peer-to-peer systems. ACM SIGecom Exchanges 5(4):41–50
Fisch B, Bonneau J, Greco N, Benet J (2018) Scaling proof-of-replication for Filecoin mining. https://research.protocol.ai/publications/accelerating-content-routing-with-bitswap-a-multi-path-file-transfer-protocol-in-ipfs-and-filecoin/. Accessed 7 December 2023
Guidi B, Michienzi A, Ricci L (2022, November 7) Evaluating the decentralisation of Filecoin. In: Paper presented at the proceedings of the 3rd international workshop on distributed infrastructure for the common good, Quebec, Quebec City, Canada
Hao J, Sun Y, Luo H (2018) A safe and efficient storage scheme based on Blockchain and IPFS for agricultural products tracking. J Comput 29(6):158–167
Hasan R, Anwar Z, Yurcik W, Brumbaugh L, Campbell R (2005, April 4–6) A survey of peer-to-peer storage techniques for distributed file systems. In: Paper presented at the international conference on information technology: coding and computing (ITCC’05)-volume II, Las Vegas, NV, USA
Heinisuo O-P, Lenarduzzi V, Taibi D (2019, April 4–9) Asterism: decentralized file sharing application for mobile devices. In: Paper presented at the 2019 7th IEEE international conference on mobile cloud computing, services, and engineering (MobileCloud), Newark, CA, USA
Henningsen S, Florian M, Rust S, Scheuermann B (2020a, June 22–26) Mapping the interplanetary filesystem. In: Paper presented at the 2020 IFIP networking conference (networking), Paris, France
Henningsen S, Rust S, Florian M, Scheuermann B (2020b, June 22–26) Crawling the IPFS network. In: Paper presented at the 2020 IFIP networking conference (networking), Paris, France
Herrera O, Znati T (2007, March 26–28) Modeling churn in P2P networks. In: Paper presented at the 40th annual simulation symposium (ANSS’07), Norfolk, VA, USA
Hildrum K, Kubiatowicz JD, Rao S, Zhao BY (2002, September 24–26) Distributed object location in a dynamic network. In: Paper presented at the proceedings of the fourteenth annual ACM symposium on parallel algorithms and architectures, Winnipeg, Manitoba, Canada
Hoang V-H, Lehtihet E, Ghamri-Doudane Y (2020, June 22–26) Privacy-preserving Blockchain-based data sharing platform for decentralized storage systems. In: Paper presented at the 2020 IFIP networking conference (networking), Paris, France
Ismail A, Toohey M, Lee YC, Dong Z, Zomaya AY (2022, August 22–25) Cost and performance analysis on decentralized file systems for blockchain-based applications: state-of-the-art report. In: Paper presented at the 2022 IEEE international conference on Blockchain (Blockchain), Espoo, Finland
Jacobson V, Smetters DK, Thornton JD, Plass MF, Briggs NH, Braynard RL (2009, December 1–4) Networking named content. In: Paper presented at the proceedings of the 5th international conference on emerging networking experiments and technologies, New York, NY, USA
Kannengießer N, Lins S, Dehling T, Sunyaev A (2020) Trade-offs between distributed ledger technology characteristics. ACM Comput Surv (CSUR) 53(2):1–37
Kelly M, Alam S, Nelson ML, Weigle MC (2016) InterPlanetary Wayback: peer-to-peer permanence of web archives. In: Fuhr N, Kovács L, Risse T, Nejdl W (eds) Research and advanced technology for digital libraries, vol 9819. Springer International Publishing, pp 411–416. https://doi.org/10.1007/978-3-319-43997-6_35
Khalid MI, Ehsan I, Al-Ani AK, Iqbal J, Hussain S, Ullah SS (2023) A comprehensive survey on blockchain-based decentralized storage networks. IEEE Access 11
Khatal S, Rane J, Patel D, Patel P, Busnel Y (2021, July 18–24) Fileshare: a blockchain and IPFS framework for secure file sharing and data provenance. In: Paper presented at the advances in machine learning and computational intelligence: proceedings of ICMLCI 2019, Singapore
Kumar R, Marchang N, Tripathi R (2020, January 5–9) Distributed off-chain storage of patient diagnostic reports in healthcare system using IPFS and blockchain. In: Paper presented at the 2020 international conference on communication systems & networks (COMSNETS), Bengaluru, India
Kutscher D, Eum S, Pentikousis K, Psaras I, Corujo D, Saucez D, Schmidt TC, Waehlisch M (2016) Information-centric networking (ICN) research challenges. Internet Res Task Force (IRTF) 2016:1–38
Lambert N, Bollen B (2014) The safe network: a new. Internet, Decentralised
Ma Z, Jiang M, Gao H, Wang Z (2018) Blockchain for digital rights management. Futur Gener Comput Syst 89:746–764
Mannes E, Maziero C (2019) Naming content on the network layer: a security analysis of the information-centric network model. ACM Comput Surv (CSUR) 52(3):1–28
Mastorakis S, Afanasyev A, Yu Y, Zhang L (2017, July 31–August 3) Ntorrent: peer-to-peer file sharing in named data networking. In: Paper presented at the 2017 26th international conference on computer communication and networks (ICCCN), Vancouver, BC, Canada
Maymounkov P, Mazieres D (2002, March 7–8) Kademlia: a peer-to-peer information system based on the XOR metric. In: Paper presented at the international workshop on peer-to-peer systems, Cambridge, MA, USA
Muralidharan S, Ko H (2019, January 11–13) An Interplanetary file system (IPFS) based IoT framework. In: Paper presented at the 2019 IEEE international conference on consumer electronics (ICCE), Las Vegas, NV, USA
Nakamoto S (2008) Bitcoin: a peer-to-peer electronic cash system.
Narayanan A, Clark J (2017) Bitcoin’s academic pedigree. Commun ACM 60(12):36–45
Norvill R, Pontiveros BBF, State R, Cullen A (2018, July 30–August 3) IPFS for reduction of chain size in ethereum. In: Paper presented at the 2018 IEEE international conference on Internet of things (iThings) and IEEE green computing and communications (GreenCom) and IEEE cyber, physical and social computing (CPSCom) and IEEE smart data (SmartData), Halifax, NS, Canada
Nyaletey E, Parizi RM, Zhang Q, Choo K-KR (2019, July 14–17) BlockIPFS-Blockchain-enabled interplanetary file system for forensic and trusted data traceability. In: Paper presented at the 2019 IEEE international conference on Blockchain (Blockchain), Atlanta, GA, USA
Ogden M, McKelvey K, Madsen MB (2017) Dat-distributed dataset synchronization and versioning. https://osf.io/nsv2c. Accessed 7 December 2023
Pentikousis K, Ohlman B, Davies E, Spirou S, Boggia G (2016) Information-centric networking: evaluation and security considerations. https://www.rfc-editor.org/rfc/rfc7945. Accessed 7 December 2023
Pouwelse J, Garbacki P, Epema D, Sips H (2005, February 24–25) The BitTorrent P2P file-sharing system: measurements and analysis. In: Paper presented at the peer-to-peer systems IV: 4th international workshop, IPTPS 2005, Ithaca, NY, USA, Ithaca, NY, USA
Protocol-Labs (2017) Filecoin: a decentralized storage network. https://research.protocol.ai/publications/filecoin-a-decentralized-storage-network/. Accessed 7 December 2023
Protocol-Labs (2023a) Interplanetary Name System (IPNS). https://docs.ipfs.tech/concepts/ipns/. Accessed 17 November 2023
Protocol-Labs (2023b) IPFS documentation. https://docs.ipfs.tech. Accessed 13 November 2023
Psaras Y, Soares JM, Dias D (2022) To the interplanetary file system–and beyond!: peer-to-peer file sharing would make the Internet far more efficient. IEEE Spectr 59(11):34–39
Ratnasamy S, Francis P, Handley M, Karp R, Shenker S (2001, August 27) A scalable content-addressable network. In: Paper presented at the proceedings of the 2001 conference on applications, technologies, architectures, and protocols for computer communications, San Diego, CA, USA
Ray PP (2023) Web3: a comprehensive review on background, technologies, applications, zero-trust architectures, challenges and future directions. In: Internet of things cyber-physical systems, vol 3. Elsevier, pp 213–248
Ripeanu M (2001, August 27–29) Peer-to-peer architecture case study: Gnutella network. In: Paper presented at the proceedings first international conference on peer-to-peer computing, Linkoping, Sweden
Rodrigues R, Druschel P (2010) Peer-to-peer systems. Commun ACM 53(10):72–82
Rowstron A, Druschel P (2001, November 12–16) Pastry: scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In: Paper presented at the middleware 2001: IFIP/ACM international conference on distributed systems platforms, Heidelberg, Germany
Rumburg R, Sethi S, Nagaraj H (2020) Audius: a decentralized protocol for audio content. Whitepaper Audius. https://whitepaper.audius.co/AudiusWhitepaper.pdf. Accessed 7 December 2023
Sandvine (2023) Global Internet phenomena report. https://www.sandvine.com/phenomena. Accessed December 7 2023
Sanjuan H, Poyhtari S, Teixeira P, Psaras I (2020) Merkle-Crdts: Merkle-Dags Meet Crdts. arXiv preprint arXiv:200400107
Saroiu S, Gummadi PK, Gribble SD (2001, December 10) Measurement study of peer-to-peer file sharing systems. In: Paper presented at the multimedia computing and networking 2002, San Jose, CA, USA
Shen J, Li Y, Zhou Y, Wang X (2019, June 24–25) Understanding I/O performance of IPFS storage: a client’s perspective. In: Paper presented at the proceedings of the international symposium on quality of service, Phoenix, AZ, USA
Stoica I, Morris R, Karger D, Kaashoek MF, Balakrishnan H (2001) Chord: a scalable peer-to-peer lookup service for Internet applications. ACM SIGCOMM Comput Commun Rev 31(4):149–160
Storj Labs I (2018) Storj: a decentralized cloud storage network framework. https://www.storj.io/. Accessed 7 December 2023
Swarm (2021) Swarm-storage and communication infrastructure for a self-sovereign digital society. https://ethersphere.github.io/swarm-home/. Accessed 7 December 2023
Tan CB, Hijazi MHA, Lim Y, Gani A (2018) A survey on proof of retrievability for cloud data integrity and availability: cloud storage state-of-the-art, issues, solutions and future trends. J Netw Comput Appl 110:75–86
Tenorio-Fornés A, Jacynycz V, Llop-Vila D, Sánchez-Ruiz A, Hassan S (2019, January 8–11) Towards a decentralized process for scientific publication and peer review using blockchain and IPFS. In: Paper presented at the proceedings of the 52nd Hawaii international conference on system sciences, Grand Wailea, Hawaii, USA
Tourani R, Misra S, Mick T, Panwar G (2017) Security, privacy, and access control in information-centric networking: a survey. IEEE Commun Surv Tutor 20(1):566–600
Trautwein D, Raman A, Tyson G, Castro I, Scott W, Schubotz M, Gipp B, Psaras Y (2022, August 22–26) Design and evaluation of IPFS: a storage layer for the decentralized Web. In: Paper presented at the proceedings of the ACM SIGCOMM 2022 conference, Amsterdam, The Netherlands
Tschorsch F, Scheuermann B (2016) Bitcoin and beyond: a technical survey on decentralized digital currencies. IEEE Commun Surv Tutor 18(3):2084–2123
Van Steen M, Tanenbaum AS (2023) Distributed systems, 4th edn. Prentice Hall, Upper Saddle River, NJ
Wang L, Kangasharju J (2013, September 9–11) Measuring large-scale distributed systems: case of BitTorrent Mainline DHT. In: Paper presented at the IEEE P2P 2013 proceedings, Trento, Italy
Wang S, Zhang Y, Zhang Y (2018) A blockchain-based framework for data sharing with fine-grained access control in decentralized storage systems. IEEE Access 6:38437–38450
Wang Q, Li R, Wang Q, Chen S (2021) Non-fungible token (NFT): overview, evaluation, opportunities and challenges. arXiv preprint arXiv:210507447
Williams S, Diordiiev V, Berman L, Uemlianin I (2019) Arweave: a protocol for economically sustainable information permanence. https://www.semanticscholar.org/paper/Arweave%3A-A-Protocol-for-Economically-Sustainable-Williams-Diordiiev/c7c62789ca397106e3df50b300bcdd494fecd27b. Accessed 7 December 2023
Wojciechowski M, Capotă M, Pouwelse J, Iosup A (2010, June 21–25) BTWorld: towards observing the global BitTorrent file-sharing network. In: Paper presented at the proceedings of the 19th ACM international symposium on high performance distributed computing, New York, NY, USA
Xia RL, Muppala JK (2010) A survey of BitTorrent performance. IEEE Commun Surv Tutor 12(2):140–158
Xu J, Xue K, Li S, Tian H, Hong J, Hong P, Yu N (2019) Healthchain: a blockchain-based privacy preserving scheme for large-scale health data. IEEE Internet Things J 6(5):8770–8781
Zhang L, Afanasyev A, Burke J, Jacobson V, Claffy K, Crowley P, Papadopoulos C, Wang L, Zhang B (2014) Named data networking. ACM SIGCOMM Comput Commun Rev 44(3):66–73
Further Reading
Benet J (2014) IPFS - content addressed, versioned, P2P file system
Daniel E, Tschorsch F (2022) IPFS and friends: a qualitative comparison of next generation peer-to-peer data networks. IEEE Commun Surv Tutor 24:31–52
Maymounkov P, Mazières D (2002) Kademlia: a peer-to-peer information system based on the XOR metric. In: Druschel P, Kaashoek F, Rowstron A (eds) Peer-to-peer systems. Springer, Berlin, pp 53–65
Psaras Y, Soares JM, Dias D (2022) To the InterPlanetary file system–and beyond!: peer-to-peer file sharing would make the Internet far more efficient. IEEE Spectr 59:34–39
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Sunyaev, A., Zirpins, C. (2024). Peer-to-Peer Data Networks and the InterPlanetary File System. In: Internet Computing. Springer, Cham. https://doi.org/10.1007/978-3-031-61014-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-61014-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-61013-4
Online ISBN: 978-3-031-61014-1
eBook Packages: Computer ScienceComputer Science (R0)