Advertisement

The Cost of Transparency: Grid-Based File Access on the Avaki Data Grid

  • H. Howie Huang
  • Andrew S. Grimshaw
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4330)

Abstract

Grid computing has been a hot topic for a decade. Several systems have been developed. Despite almost a decade of research and tens of millions of dollars spent, uptake of grid technology has been slow. Most deployed grids are based on a toolkit approach that requires significant software modification or development. An operating system technique used for over 30 years has been to reduce application complexity by providing transparency, e.g. file systems mask details of devices, virtual machines mask finite memory, etc. It has been argued that providing transparency in a grid environment is too costly in terms of performance. This paper examines that question in the context of data grids by measuring the performance of a commercially available data grid product – the Avaki Data Grid (ADG). We present the architecture of the ADG, describe our experimental setup, and provide performance results, comparing the ADG to a native NFS V3 implementation for both local and wide area access cases. The results were mixed, though encouraging. For single client local file operations, native NFS outperformed the ADG by 15% to 45% for smaller files, though for files larger than 32 MB ADG outperformed native NFS.

Keywords

File System Share Server Local File Aggregate Bandwidth Network File System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Berman, F., Fox, G.C., Hey, A.J.G.: Grid Computing: Making the Global Infrastructure a Reality. Wiley Series in Communication Networking & Distributed Systems. John Wiley & Sons, Chichester (2003)Google Scholar
  2. 2.
    Abbas, A.: Grid Computing: A Practical Guide to Technology and Applications, Charles River Media (2004)Google Scholar
  3. 3.
    Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufman Publishers, San Francisco (1999)Google Scholar
  4. 4.
    Foster, I., et al.: The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration (2002)Google Scholar
  5. 5.
    Grimshaw, A.S., et al.: From Legion to Avaki: The Persistence of Vision. In: Berman, F., Fox, G., Hey, T. (eds.) Grid Computing: Making the Global Infrastructure a Reality (2003)Google Scholar
  6. 6.
    Grimshaw, A., Natrajan, A.: Legion: Lessons Learned Building a Grid Operating System. Transactions of the IEEE (2005)Google Scholar
  7. 7.
    Lewis, M.J., et al.: Support for Extensibility and Site Autonomy in the Legion Grid System Object Model. Journal of Parallel and Distributed Computing 63, 525–538 (2003)MATHCrossRefGoogle Scholar
  8. 8.
    Frey, J., et al.: Condor-G: A Computation Management Agent for Multi-Institutional Grids. Journal of Cluster Computing 5, 237–246 (2002)CrossRefGoogle Scholar
  9. 9.
    Thain, D., Tannenbaum, T., Livny, M.: Condor and the Grid. In: Berman, F., Hey, A.J.G., Fox, G. (eds.) Grid Computing: Making The Global Infrastructure a Reality. John Wiley, Chichester (2003)Google Scholar
  10. 10.
    Foster, I., Kesselman, C.: Computational Grids, in The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco (1999)Google Scholar
  11. 11.
    Grimshaw, A.S.: The Legion Vision of a Worldwide Virtual Computer. Communications of the ACM 40(1), 39–45 (1997)CrossRefGoogle Scholar
  12. 12.
    Kola, G., et al.: DISC: A System for Distributed Data Intensive Scientific Computing. In: Proceedings of the First Workshop on Real, Large Distributed Systems (WORLDS 2004), San Francisco, CA (2004)Google Scholar
  13. 13.
    White, B., Grimshaw, A., Nguyen-Tuong, A.: Grid Based File Access: The Legion I/ O Model. In: Proceedings of the Symposium on High Performance Distributed Computing (HPDC-9), Pittsburgh, PA (2000)Google Scholar
  14. 14.
    Grimshaw, A.: Avaki Data Grid - Secure Transparent Access to Data. In: Abbas, A. (ed.) Grid Computing: A Practical Guide To Technology And Applications, Charles River Media (2003)Google Scholar
  15. 15.
    OGSA-DAI project home page, Available from:http://www.ogsadai.org.uk/
  16. 16.
    White, B., et al.: LegionFS: A Secure and Scalable File System Supporting Cross-Domain High-Performance Applications. In: Proceedings SC 2001, Denver, CO (January 2001)Google Scholar
  17. 17.
    DAIS Working Group, Available from: https://forge.gridforum.org/projects/dais-wg
  18. 18.
    OGSA Byte I/O Working Group, Available from: https://forge.gridforum.org/projects/byteio-wg
  19. 19.
    Allcock, W.: GridFTP Protocol Specification (Global Grid Forum Recommendation GFD.20) (2003), Available from: http://www.globus.org/alliance/publications/papers.php#GridftpSpec02
  20. 20.
    Bester, J., et al.: GASS: A Data Movement and Access Service for Wide Area Computing Systems. In: Sixth Workshop on I/O in Parallel and Distributed Systems (1999)Google Scholar
  21. 21.
    Postel, J., Reynolds, J.: RFC 959 - File Transfer Protocol (1985)Google Scholar
  22. 22.
    Rajasekar, A., et al.: Storage Resource Broker - Managing Distributed Data in a Grid. Computer Society of India Journal, Special Issue on SAN 33(4), 42–54 (2003)Google Scholar
  23. 23.
    GGF, Simple API for Grid Applications (SAGA), Global Grid Forum Google Scholar
  24. 24.
    Satyanarayanan, M.: Scalable, Secure, and Highly Available Distributed File Access. Computer 23, 9–18, 20–21 (1990)CrossRefGoogle Scholar
  25. 25.
  26. 26.
    Avaki, Avaki Data Grid Administration Guide, Release 5.1 (May 2004)Google Scholar
  27. 27.

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • H. Howie Huang
    • 1
  • Andrew S. Grimshaw
    • 1
  1. 1.Department of Computer ScienceUniversity of VirginiaCharlottesville

Personalised recommendations