Abstract.
The transfer of large volume data over computer network is important and unavoidable operation in the past, now and in any feasible future. There are a number of methods/tools to transfer the data over computer global network (Internet). In this paper the transfer of data over Internet is discussed. Several free of charge utilities to transfer the data are analyzed here. The most important architecture features are emphasized and suggested idea to add SDN Openflow protocol technique for fine tuning the data transfer over several parallel data links.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Big Data. http://en.wikipedia.org/wiki/Big_data
Information Revolution: Big Data Has Arrived at an Almost Unimaginable Scale. http://www.wired.com/magazine/2013/04/bigdata/
Square Kilometer Array. http://skatelescope.org/
Large Synoptic Survey Telescope. http://www.lsst.org/lsst/
Facility for Antiproton and Ion Research. http://www.fair-center.eu/
International Thermonuclear Experimental Reactor. http://www.iter.org/
CERN. http://www.cern.ch/
Borovick, L., Villars, R.L.: White paper. The Critical Role of the Network in Big Data Applications. http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/critical_big_data_applications.pdf
The Center for Large-scale Data Systems Research at the San Diego Supercomputer Center. http://clds.sdsc.edu/
Johnston, W. E., Dart, E., Ernst, M., Tierney, B.: Enabling high throughput in widely distributed data management and analysis systems: Lessons from the LHC. https://tnc2013.terena.org/getfile/402
Ah Nam, H. et al: The Practical Obstacles of Data Transfer: Why researchers still love scp. http://dl.acm.org/citation.cfm?id=2534695.2534703&coll=DL&dl=ACM&CFID=563485433&CFTOKEN=25267057
Gunter, D., et al.: Exploiting Network Parallelism for Improving Data Transfer Performance. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6496123
UDT: Breaking the Data Transfer Bottleneck. http://udt.sourceforge.net/
MutiPath TCP – Linux Kernel Implementation. http://multipath-tcp.org/
OpenSSH. http://openssh.org/
Patched OpenSSH. http://sourceforge.net/projects/hpnssh/
BBCP – utility to trasnfer the data over network. http://www.slac.stanford.edu/~abh/bbcp/
BBFTP – utility for bulk data transfer. http://doc.in2p3.fr/bbftp/
Hodson, S.W., Poole, S.W., Ruwart, T. M., Settlemyer, B. W.: Moving Large Data Sets Over High-Performance Long Distance Networks. http://info.ornl.gov/sites/publications/files/Pub28508.pdf
Fast Data Transfer. http://monalisa.cern.ch/FDT/
Grid/Globus data transfer tool. Client part is known as globus-url-copy. http://toolkit.globus.org/toolkit/data/gridftp/
File Transfer Service. http://www.eu-emi.eu/products/-/asset_publisher/1gkD/content/fts3
Data Transfer Tools. http://fasterdata.es.net/data-transfer-tools/
The CMS Collaboration: The CMS experiment at the CERN LHC. http://iopscience.iop.org/1748-0221/3/08/S08004/
Kaselis, R., Piperov, S., Magini, N., Flix, J., Gutsche, O., Kreuzer, P., Yang, M., Liu, S., Ratnikova, N., Sartirana, A., Bonacorsi, D., Letts, J.: CMS data transfer operations after the first years of LHC collisions. In: International Conference on Computing in High Energy and Nuclear Physics 2012. IOP Publishing Journal of Physics: Conference Series, vol. 396 (2012)
PhEDEx, CMS Data Transfers. https://cmsweb.cern.ch/phedex
PHEDEX data transfer system. https://twiki.cern.ch/twiki/bin/view/CMSPublic/PhedexAdminDocsInstallation
Laboratory of the Network Technology. http://sdn.ifmo.ru/
The Rucio project is the new version of ATLAS Distributed Data Management (DDM) system services. http://rucio.cern.ch/
Open Networking Foundation White Paper Software-Defined Networking: The New Norm for Networks. https://www.opennetworking.org/images/stories/downloads/white-papers/wp-sdn-newnorm.pdf
Nunes, B. A., Mendonca, M., Nguyen, X., Obraczka, K., Turletti, T.: A Survey of Software-Defined Networking: Past, Present, and Future of Programmable Networks. http://hal.inria.fr/index.php?halsid=ig58511e1q1ekqq75uud43dn66&view_this_doc=hal-00825087&version=5
Zurawski, J., Balasubramanian, S., Brown, A., Kissel, E., Lake, A., Swany, M., Tierney, B., Zekauskas, M.: perfSONAR: On-board Diagnostics for Big Data. https://www.es.net/assets/pubs_presos/20130910-IEEE-BigData-perfSONAR2.pdf
Tierney, B., Kissel, E., Swany, M., Pouyoul, E.: Efficient Data Transfer Protocol for BigData. http://www.es.net/assets/pubs_presos/eScience-networks.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Khoruzhnikov, S.E., Grudinin, V.A., Sadov, O.L., Shevel, A.Y., Kairkanov, A.B. (2015). Transfer of Large Volume Data over Internet with Parallel Data Links and SDN. In: Tan, Y., Shi, Y., Buarque, F., Gelbukh, A., Das, S., Engelbrecht, A. (eds) Advances in Swarm and Computational Intelligence. ICSI 2015. Lecture Notes in Computer Science(), vol 9142. Springer, Cham. https://doi.org/10.1007/978-3-319-20469-7_49
Download citation
DOI: https://doi.org/10.1007/978-3-319-20469-7_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20468-0
Online ISBN: 978-3-319-20469-7
eBook Packages: Computer ScienceComputer Science (R0)