Environment for Data Transfer Measurement

  • Sergey Khoruzhnikov
  • Vladimir Grudinin
  • Oleg Sadov
  • Andrey Shevel
  • Stefanos Georgiou
  • Arsen Kairkanov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9714)

Abstract

Measurement of the data transfer considered as often task when regular transfer over long distant network of large volume data is required. The data transfer over network depends on many conditions and parameters. Quite often measurements have to be repeated several times. The paper describes procedure how to implement appropriate measurements to store automatically all the parameters and messages during the data transfer. The saved history tracking permits to do detailed comparison of measurement results.

Keywords

Big data Linux Data transfer Measurement Internet 

Notes

Acknowledgements

The research has been carried out with the financial support of the Ministry of Education and Science of the Russian Federation under grant agreement #14.575.21.0058.

References

  1. 1.
    ISO/IEC JTC 1 - Big Data. Preliminary report (2014). http://www.iso.org/iso/home/store/publication_item.htm?pid=PUB100361
  2. 2.
    Khoruzhnikov, S.E., Grudinin, V.A., Sadov, O.L., Shevel, A.Y., Kairkanov, A.B.: Transfer of large volume data over internet with parallel data links and SDN. In: Tan, Y., Shi, Y., Buarque, F., Gelbukh, A., Das, S., Engelbrecht, A. (eds.) ICSI-CCI 2015. LNCS, vol. 9142, pp. 463–471. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  3. 3.
  4. 4.
    NIST Big Data public working group. http://bigdatawg.nist.gov
  5. 5.
    Johnston, W.E., Dart, E., Ernst, M., Tierney, B.: Enabling high throughput in widely distributed data management and analysis systems: lessons from the LHC (2013). https://tnc2013.terena.org/getfile/402
  6. 6.
  7. 7.
    Ah Nam, H. et al: The practical obstacles of data transfer: why researchers still love scp. In: NDM 2013 Proceedings of the Third International Workshop on Network-Aware Data Management, Article No. 7 (2013). doi: 10.1145/2534695.2534703
  8. 8.
  9. 9.
    BBCP – utility to transfer the data over network. http://www.slac.stanford.edu/~abh/bbcp
  10. 10.
    BBFTP – utility for bulk data transfer. http://doc.in2p3.fr/bbftp
  11. 11.
    Fast Data Transfer. http://monalisa.cern.ch/FDT
  12. 12.
    Grid/Globus data transfer tool. Client part is known as globus-url-copy. http://toolkit.globus.org/toolkit/data/gridftp
  13. 13.
  14. 14.
    Generate debug information for the system. http://linux.die.net/man/1/sosreport
  15. 15.
  16. 16.
    Laboratory of the Network Technology. http://sdn.ifmo.ru

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Sergey Khoruzhnikov
    • 1
  • Vladimir Grudinin
    • 1
  • Oleg Sadov
    • 1
  • Andrey Shevel
    • 1
    • 2
  • Stefanos Georgiou
    • 1
    • 3
  • Arsen Kairkanov
    • 1
  1. 1.ITMO UniversitySt. PetersburgRussian Federation
  2. 2.National Research Centre “Kurchatov Institute”B.P. Konstantinov, Petersburg Nuclear Physics InstituteGatchinaRussian Federation
  3. 3.Athens University of Economics and BusinessAthensGreece

Personalised recommendations