Advertisement

BitTorrent for Storage and File Transfer in Grid Environments

  • Nikolas Kasioumis
  • Constantinos Kotsokalis
  • Pavlos Kranas
  • Panayiotis Tsanakas
Chapter

Abstract

Grid Computing is a computing model that treats all resources as a collection of manageable entities with common interfaces to such functionalities as lifetime management, discoverable properties and accessibility via open protocols. It also provides a robust infrastructure for distributed computing, and is widely used in large-scale scientific applications that generate large volumes of data. One of the key factors to those applications is the ability to access very large files in a short -and predictable, if possible– time. This chapter introduces a new architecture, developed to achieve better throughput, more reliable file transfers and better file resilience. The suggested system combines the secure resource sharing paradigm of grid technologies with the qualitative characteristics of peer-to-peer systems for efficiency and resilience. We essentially propose a Grid filesystem and the relevant transport layer, using a slightly modified version of the BitTorrent peer-to-peer protocol, and describe the additional components required to implement it, adhering to de facto Grid Computing best practices. We experimentally find that BitTorrent, compared to GridFTP, is more reliable with regard to completion times and, as expected, more resilient to networking failures.

Keywords

Grid Computing Storage Element Grid Network Grid Desktop Network Interface Card 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    B. Cohen. Incentives build robustness in BitTorrent. Workshop on Economics of Peer-to-Peer Systems, 2003Google Scholar
  2. 2.
    Globus GridFTP: http://www.globus.org/grid_software/data/gridftp.php, accessed December 2008
  3. 3.
    A.L. Chervenak, N. Palavalli, S. Bharathi, C. Kesselman, R. Schwartzkopf. Performance and scalability of a replica location service. 13th IEEE International Symposium on High Performance Distributed Computing, June 4–6, Honolulu, Hawaii, USA, 182–191, (2004)Google Scholar
  4. 4.
    A. Bharambe, C. Herley, V. Padmanabhan. Analyzing and improving bittorrent performance. Microsoft research technical report msr-tr-2005-03, Carnegie Mellon University and Microsoft Research, 2005Google Scholar
  5. 5.
    C. Gkantsidis, P.R.: Rodriguez. Network coding for large scale content distribution. INFOCOM 2005. 24th Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings IEEE 4, 2235–2245, 2005Google Scholar
  6. 6.
    M. Piatek, T. Isdal, T. Anderson, A. Krishnamurthy, A. Venkataramani. Do incentives build robustness in BitTorrent? Technical report, University of Washington Computer Science 8 Engineering, 2006Google Scholar
  7. 7.
    D. Arthur, R. Panigrahy. Analyzing BitTorrent and related peer-to-peer networks. SODA ’06: Proceedings of the 17 annual ACM-SIAM symposium on Discrete algorithm, New York, NY, ACM, 961–969, (2006)Google Scholar
  8. 8.
    M.R. Ceballos, J.L. Gorricho. P2P file sharing analysis for a better performance. ICSE ’06: Proceeding of the 28th International Conference on Software Engineering, New York, NY, ACM 941–944, (2006)Google Scholar
  9. 9.
    D. Qiu, R. Srikant. Modeling and performance analysis of BitTorrent-like peer-to-peer networks. SIGCOMM ’04: Proceedings of the 2004 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, ACM New York, NY, 367–378, (2004)Google Scholar
  10. 10.
    A.A. Hamra, P.A. Felber. Design choices for content distribution in P2P networks. SIGCOMM Computer Communication Review, 35(5), 29–40, (2005)CrossRefGoogle Scholar
  11. 11.
    J.A. Pouwelse, P. Garbacki, D.H.J. Epema, H.J. Sips. The Bittorrent P2P file-sharing system: Measurements and analysis. 4th International Workshop on Peer-to-Peer Systems (IPTPS), February 24–25, Ithaca, NY, USA, 2005Google Scholar
  12. 12.
    P. Rizk, C. Kiddle, R. Simmonds. Improving gridFTP performance with split TCP connections. E-SCIENCE ’05: Proceedings of the 1st International Conference on e-Science and Grid Computing, Washington, DC, IEEE Computer Society 263–270, (2005)Google Scholar
  13. 13.
    T. Ito. Understanding and optimizing gridFTP: Leveraging parallel data transfer for long-fat networks. Master’s thesis, Graduate School of Information Science and Technology, Osaka University, 2006Google Scholar
  14. 14.
    R.S. Bhuvaneswaran, Y. Katayama, N. Takahashi. Redundant parallel data transfer schemes for the grid environment. ACSW Frontiers ’06: Proceedings of the 2006 Australasian Workshops on Grid Computing and e-Research, Darlinghurst, NSW, Australian Computer Society, 71–78, (2006)Google Scholar
  15. 15.
    L. Caviglione, C. Cervellera. Design of a peer-to-peer system for optimized content replication. Computer Communications, 30(16), 3107–3116 Special Issue: Advances in Communication Networking, (2007)CrossRefGoogle Scholar
  16. 16.
    B. Wei, G. Fedak, F. Cappello. Collaborative data distribution with BitTorrent for computational desktop grids. ISPDC ’05: Proceedings of the The 4th International Symposium on Parallel and Distributed Computing (ISPDC’05), Washington, DC, IEEE Computer Society, 250–257, (2005)Google Scholar
  17. 17.
    Planet Lab: An Open Platform for Developing, Deploying and Accessing Planetary-Scale Services: http://https://www.planet-lab.org/, accessed December 2008
  18. 18.
    Fedora Core Linux: http://fedoraproject.org/, accessed December 2008
  19. 19.
    Azureus BitTorrent client: http://azureus.sourceforge.net/, accessed December 2008
  20. 20.
    Globus Toolkit: http://www.globus.org/toolkit/, accessed December 2008
  21. 21.
    B. Kónya, O. Smirnova. Performance Evaluation of the GridFTP wihtin the NorduGrid Project. Technical report, NorduGrid, 2001Google Scholar
  22. 22.
    A. Bharambe, C. Herley, V. Padmanabhan. Analyzing and Improving a BitTorrent network’s performance mechanisms. IEEE Conference on Computer Communications (INFOCOM), April 23–29, Barcelona, Catalonia, Spain, 2006Google Scholar
  23. 23.
    R. Wolski, N.T. Spring, J. Hayes. The network weather service: A distributed resource performance forecasting service for metacomputing. Future Generation Computer Systems, 15(5–6), 757–768, (1999)CrossRefGoogle Scholar
  24. 24.
    G. Neglia, G. Reina, H. Zhang, D. Towsley, A. Venkataramani, J. Danaher. Availability in BitTorrent systems. IEEE Conference on Computer Communications (INFOCOM), May 6–12, Anchorage, Alaska, USA 2007Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Nikolas Kasioumis
    • 1
  • Constantinos Kotsokalis
    • 2
  • Pavlos Kranas
    • 1
  • Panayiotis Tsanakas
    • 1
  1. 1.National Technical University of AthensAthensGreece
  2. 2.Dortmund University of TechnologyDortmundGermany

Personalised recommendations