Abstract
Grid computing has been a hot topic for a decade. Several systems have been developed. Despite almost a decade of research and tens of millions of dollars spent, uptake of grid technology has been slow. Most deployed grids are based on a toolkit approach that requires significant software modification or development. An operating system technique used for over 30 years has been to reduce application complexity by providing transparency, e.g. file systems mask details of devices, virtual machines mask finite memory, etc. It has been argued that providing transparency in a grid environment is too costly in terms of performance. This paper examines that question in the context of data grids by measuring the performance of a commercially available data grid product – the Avaki Data Grid (ADG). We present the architecture of the ADG, describe our experimental setup, and provide performance results, comparing the ADG to a native NFS V3 implementation for both local and wide area access cases. The results were mixed, though encouraging. For single client local file operations, native NFS outperformed the ADG by 15% to 45% for smaller files, though for files larger than 32 MB ADG outperformed native NFS.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Berman, F., Fox, G.C., Hey, A.J.G.: Grid Computing: Making the Global Infrastructure a Reality. Wiley Series in Communication Networking & Distributed Systems. John Wiley & Sons, Chichester (2003)
Abbas, A.: Grid Computing: A Practical Guide to Technology and Applications, Charles River Media (2004)
Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufman Publishers, San Francisco (1999)
Foster, I., et al.: The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration (2002)
Grimshaw, A.S., et al.: From Legion to Avaki: The Persistence of Vision. In: Berman, F., Fox, G., Hey, T. (eds.) Grid Computing: Making the Global Infrastructure a Reality (2003)
Grimshaw, A., Natrajan, A.: Legion: Lessons Learned Building a Grid Operating System. Transactions of the IEEE (2005)
Lewis, M.J., et al.: Support for Extensibility and Site Autonomy in the Legion Grid System Object Model. Journal of Parallel and Distributed Computing 63, 525–538 (2003)
Frey, J., et al.: Condor-G: A Computation Management Agent for Multi-Institutional Grids. Journal of Cluster Computing 5, 237–246 (2002)
Thain, D., Tannenbaum, T., Livny, M.: Condor and the Grid. In: Berman, F., Hey, A.J.G., Fox, G. (eds.) Grid Computing: Making The Global Infrastructure a Reality. John Wiley, Chichester (2003)
Foster, I., Kesselman, C.: Computational Grids, in The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco (1999)
Grimshaw, A.S.: The Legion Vision of a Worldwide Virtual Computer. Communications of the ACM 40(1), 39–45 (1997)
Kola, G., et al.: DISC: A System for Distributed Data Intensive Scientific Computing. In: Proceedings of the First Workshop on Real, Large Distributed Systems (WORLDS 2004), San Francisco, CA (2004)
White, B., Grimshaw, A., Nguyen-Tuong, A.: Grid Based File Access: The Legion I/ O Model. In: Proceedings of the Symposium on High Performance Distributed Computing (HPDC-9), Pittsburgh, PA (2000)
Grimshaw, A.: Avaki Data Grid - Secure Transparent Access to Data. In: Abbas, A. (ed.) Grid Computing: A Practical Guide To Technology And Applications, Charles River Media (2003)
OGSA-DAI project home page, Available from: http://www.ogsadai.org.uk/
White, B., et al.: LegionFS: A Secure and Scalable File System Supporting Cross-Domain High-Performance Applications. In: Proceedings SC 2001, Denver, CO (January 2001)
DAIS Working Group, Available from: https://forge.gridforum.org/projects/dais-wg
OGSA Byte I/O Working Group, Available from: https://forge.gridforum.org/projects/byteio-wg
Allcock, W.: GridFTP Protocol Specification (Global Grid Forum Recommendation GFD.20) (2003), Available from: http://www.globus.org/alliance/publications/papers.php#GridftpSpec02
Bester, J., et al.: GASS: A Data Movement and Access Service for Wide Area Computing Systems. In: Sixth Workshop on I/O in Parallel and Distributed Systems (1999)
Postel, J., Reynolds, J.: RFC 959 - File Transfer Protocol (1985)
Rajasekar, A., et al.: Storage Resource Broker - Managing Distributed Data in a Grid. Computer Society of India Journal, Special Issue on SAN 33(4), 42–54 (2003)
GGF, Simple API for Grid Applications (SAGA), Global Grid Forum
Satyanarayanan, M.: Scalable, Secure, and Highly Available Distributed File Access. Computer 23, 9–18, 20–21 (1990)
Avaki, http://www.avaki.com/
Avaki, Avaki Data Grid Administration Guide, Release 5.1 (May 2004)
Globus, http://www.globus.org/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, H.H., Grimshaw, A.S. (2006). The Cost of Transparency: Grid-Based File Access on the Avaki Data Grid. In: Guo, M., Yang, L.T., Di Martino, B., Zima, H.P., Dongarra, J., Tang, F. (eds) Parallel and Distributed Processing and Applications. ISPA 2006. Lecture Notes in Computer Science, vol 4330. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11946441_60
Download citation
DOI: https://doi.org/10.1007/11946441_60
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68067-3
Online ISBN: 978-3-540-68070-3
eBook Packages: Computer ScienceComputer Science (R0)