Advertisement

An Efficient Grid Data Access with StoRM

  • Riccardo Zappi
  • Elisabetta Ronchieri
  • Alberto Forti
  • Antonia Ghiselli
Conference paper

Abstract

In production data Grids, high performance disk storage solutions using parallel file systems are becoming increasingly important to provide reliability and high speed I/O operations needed by High Energy Physics analysis farms. Today, Storage Area Network solutions are commonly deployed at Large Hadron Collider data centres, and parallel file systems such as GPFS and Lustre provide reliable, high speed native POSIX I/O operations in parallel fashion. In this paper, we describe StoRM, a Grid middleware component, implementing the standard Storage Resource Manager v2.2 interface. Its architecture fully exploits the potentiality offered by the underlying cluster file system. Indeed, it enables and encourages the use of the native POSIX file protocol (i.e. ”file://”) allowing managed Storage Element to improve job efficiency in data accessing. The job running on the worker node can perform a direct access to the Storage Element managed by StoRM as if it were a local disk, instead of transferring data from Storage Elements to the local disk.

Keywords

File System Storage Resource Storage Element Local Disk File Transfer Protocol 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Allcock, W., Bester, J., Bresnahan, J., Chervenak, A., Foster, I., Kesselman, C., Meder, S., Nefedova, V., Quesnel, D., Tuecke, S.: Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing. In: IEEE Mass Storage Conference (2001)Google Scholar
  2. 2.
    Foster, I.: The Grid: A New Infrastructure for 21st Century Science. Physics Today. 54:2 (2002)Google Scholar
  3. 3.
    Stevens, R., Woodward, P., De Fanti, T., Catlett, C.: From the I-WAY to the National Technology Grid. Communications of the ACM. 40:11, 50–61 (1997)Google Scholar
  4. 4.
    De Riese, M., Fuhrmann, P., Mkrtchyan, T., Ernst, M., Kulyavtsev, A., Podstavkov, V., Radicke, M., Sharma, M., Litvintsev, M., Perelmutov, T.: dCache, the Book. Nov. (2006) http://www.dcache.org/manuals/Book/
  5. 5.
    Schmuck, F. and Haskin, R.: GPFS: A Shared-Disk File System for Large Computing Clusters. USENIX FAST 2002 Conference on File and Storage Technologies. (200) http://www.usenix.org/events/fast02/full_papers/schmuck/ schmuck.pdf
  6. 6.
    Shoshani, A., Sim, A., Gu, J.: Storage Resource Managers: Essential Components for the Grid. chapter in book: Grid Resource Management: State of the Art and Future Trends. Edited by Jarek Nabrzyski, Jennifer M. Schopf, Jan weglarz, Kluwer Academic Publishers (2003)Google Scholar
  7. 7.
    Allcock, W.: GridFTP: Protocol extensions to FTP for the Grid. Global Grid Forum GFD-R P.020 (2003) http://www.ggf.org/documents/GWD-R/GFD-R.020.pdf
  8. 8.
    Borland, M.: A self-describing file protocol for simulation integration and shared postprocessors. In: Particle Accelerator Conference. (Dallas,TX) 4, 2184–2186 (1995)Google Scholar
  9. 9.
    Barring, O., Baud, J., Durand, J.: CASTOR Project Status. In: Computing in High Energy Physics. Padua, Italy (2000)Google Scholar
  10. 10.
    Donno, F., Abadie, L., Badino, P., Baud, J.-P., Corso, E., De Witt, S., Fuhrman, P., Gu, J., Koblitz, B., Lemaitre, S., Litmaath, M., Litvintsev, D., Lo Presti, G., Magnoni, L., McCance, G., Mkrtchan, T., Mollan, R., Natarajan, V., Perelmutov, T., Petravick, D., Shoshani, A., Sim, A., Smith, D., Tedesco, P., and Zappi, R.: Storage Resource Manager Version 2.2: design, implementation, and testing experience. J. Physics: Conference Series. 119 (2008)Google Scholar
  11. 11.
    Sim, A., Shoshani, A., Donno, F., and Jensen, J.: Storage Resource Manager Interface Specifiation v2.2 Implementation Experience Report. Global Grid Forum GFD-E.154 (2009) http://www.ogf.org/documents/GFD.154.pdf
  12. 12.
    Foster, I., and Kesselman, C. and Tuecke, S.: A Security Architecture for Computation Grid. In: ACM Conference on Computer and Communications Security. (San Francisco, CA, USA) 83–92 (1998)Google Scholar
  13. 13.
    Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., Berners-Lee, T.: Hypertext Transfer Protocol – HTTP/1.1. June (1999)http://www.w3.org/Protocols/HTTP/1.1/rfc2616.pdf
  14. 14.
    Ernst, M., Fuhrmann, P., Mkrtchyan, T., Bakken, J., Fisk, I., Perelmutov, T., and Petravick, D.: Managed Data Storage and Data Access Services for Data Grids. In: Computing in High Energy Physics and Nuclear Physics 2004. Interlaken, Switzerland, 27 Sep -1 Oct (2004)Google Scholar
  15. 15.
    Heiss, A. and Schwickerath, U.: RFIO/IB Project.http://hikwww2.fzk.de/hik/orga/ges/infiniband/rfioib.html
  16. 16.
    Postel, J., and Reynolds, J.: RFC959, File Transfer Protocol. The Internet Society. (1995)http://www.faqs.org/ftp/rfc/pdf/rfc959.txt.pdf
  17. 17.
    Horowitz, M., and Lunt, S.: RFC2228, FTP Security Extensions. The Internet Society. (1997) http://www.faqs.org/ftp/rfc/pdf/rfc2228.txt.pdf
  18. 18.
    Hethmon, P., and Elz, R.: Feature negotiation mechanism for the File Transfer Protocol. (1998)Google Scholar
  19. 19.
    Ernst, M., Fuhrmann, P., Gasthuber, M., Mkrtchyan, T., and Waldman, C.: dCache, a Distributed Storage Data Caching System. In: Computing in High Energy Physics. Beijing, China (2001)Google Scholar
  20. 20.
    Linn, J.: Generic Security Service Application Program Interface, Version 2, Update 1. Network Working Group. (2000) http://www.ietf.org/rfc/rfc2743.txt
  21. 21.
    Ghiselli A., Magnoni L., and Zappi R.: StoRM: a Flexible Solution for Storage Resource Manager in Grid. In: Nuclear Science Symposium Conference Record, 2008 (NSS ’08) IEEE. Dresden, Germany (2008)Google Scholar
  22. 22.
    Grnbacher, A.: POSIX Access Control Lists on Linux. In: USENIX 2003 Annual Technical Conference, FREENIX Track. 259–272 (2003)http://www.usenix.org/event/usenix03/tech/freenix03/full_ papers/gruenbacher/gruenbacher.pdf
  23. 23.
    E.Corso, A.Forti, A.Ghiselli, L.Magnoni, R.Zappi: An effective XML based name mapping mechanism within StoRM. In: International Conference on Computing in High Energy and Nuclear Physics (CHEP’07) IOP Publishing, Journal of Physics: Conference Series 119 (2008)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Riccardo Zappi
    • 1
  • Elisabetta Ronchieri
    • 1
  • Alberto Forti
    • 1
  • Antonia Ghiselli
    • 1
  1. 1.INFN-CNAFBolognaItaly

Personalised recommendations