Advertisement

Evaluation of OrangeFS as a Tool to Achieve a High-Performance Storage and Massively Parallel Processing in HPC Cluster

  • Hugo Eduardo Camacho CruzEmail author
  • Jesús Humberto Foullon Peña
  • Julio Cesar González Mariño
  • Ma. de Lourdes Cantú Gallegos
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 948)

Abstract

Nowadays, the requirements of modern software demand a greater computing power; numerous scientific and engineering applications request an increase in data storage capacity, be able to exchange of information at high speeds, as well as a faster data processing and better memory management. The implementation of personal computers interconnected to form a cluster and the use of distributed/parallel file systems are presented as a highly suitable alternative in the solution of complex problems that require these resources as their needs grow. The present work shows the evaluation of OrangeFS as a tool to achieve high performance storage and massive parallel processing. It takes advantage of the capacity of the hard drives included in each node of the cluster through the virtual file system and the network bandwidth, instead of having to add a more expensive type of storage. The tests carried out in a cluster with CentOS show that stripping a large file into small objects and distributed in parallel to the I/O servers provides that upcoming read/write operations runs faster; In addition, the use of the message passing interface in the development and execution of applications allows to increase the parallelism of the data in terms of processing due to the intervention of the multicore processor in each of the clients.

Keywords

OrangeFS MPI HPC cluster Parallel file system 

Notes

Acknowledgments

We thank the Programa para el Desarrollo Profesional Docente (PRODEP) for the support granted mentioned in the Official Letter No. 511-6/17/8212, and the Universidad Autónoma de Tamaulipas - Facultad de Medicina e Ingeniería en Sistemas Computacionales de Matamoros, all of them for providing the means to carry out this work.

References

  1. 1.
    Abacus-I Supercomputer. http://www.abacus.cinvestav.mx/caracteristicas. Accessed 03 Dec 2017
  2. 2.
    Carns, P.H., Ligon III, W.B., Ross, R.B., Thakur, R.: PVFS: a parallel file system for Linux clusters. In: Proceedings of the Extreme Linux Track: 4th Annual Linux Showcase and Conference (2000)Google Scholar
  3. 3.
    Dickens, P.M., Logan, J.: A high performance implementation of MPI-IO for a Lustre file system environment. Concurrency Comput.: Pract. Exper. 22, 1433–1449 (2010).  https://doi.org/10.1002/cpe.1491CrossRefGoogle Scholar
  4. 4.
    Riahi, H., et al.: J. Phys.: Conf. Ser. 396, 042050 (2012).  https://doi.org/10.1088/1742-6596/396/4/042050CrossRefGoogle Scholar
  5. 5.
    Hua, X., Wu, H., Li, Z., Ren, S.: Enhancing throughput of the Hadoop Distributed File System for interaction-intensive tasks. J. Parallel Distrib. Comput. 74(8), 2770–2779 (2014).  https://doi.org/10.1016/j.jpdc.2014.03.010. http://www.elsevier.com/inca/publications/store/6/2/2/8/9/5/index.httCrossRefGoogle Scholar
  6. 6.
  7. 7.
  8. 8.
    Message Passing Interface Forum. MPI: A Message-Passing Interface Standard, Version 2.2. University of Tennessee (2009)Google Scholar
  9. 9.
    Miztli Architecture. http://www.super.unam.mx/index.php/home/acerca-de?start=2. Accessed 02 Jan 2018
  10. 10.
    MPICH is a high performance and widely portable implementation of the Message Passing Interface (MPI) standard, mpich.org. http://www.mpich.org/documentation/guides/. Accessed 20 Dec 2017
  11. 11.
    PVFS2 Team: “Parallel Virtual File System, Version 2”, September 2003. http://www.pvfs.org/pvfs2-guide.html
  12. 12.
    Sampath, S., Sagar, B.B., Subbaraya, C.K., Nanjesh, B.R.: Performance evaluation of parallel applications using MPI in cluster based parallel computing architecture. In: Proceeding of International Conference on “Emerging Research in Computing, Information, Communication and Applications” (2013). ISBN 9789351071020Google Scholar
  13. 13.
    Top500 Homepage. https://www.top500.org/. Accessed 02 Jan 2018
  14. 14.
    The OrangeFS Project, OrangeFS 2.9 Documentation, Orangefs.org. http://docs.orangefs.com/v_2_9/index.htm. Accessed 09 Feb 2018
  15. 15.
    William Gropp, Ewing Lusk and Anthony Skjellum: Using MPI, 3rd Edition (2014)Google Scholar
  16. 16.
    William Gropp, Torsten Hoefler, Rajeev Thakur and Ewing Lusk: Using Advance MPI, 1st Edition (2014)Google Scholar
  17. 17.
    Wu, Y., et al.: J. Phys.: Conf. Ser. 219, 062068 (2010).  https://doi.org/10.1088/1742-6596/219/6/062068Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Universidad Autónoma de Tamaulipas - FMeISC de MatamorosTamaulipasMexico

Personalised recommendations