Abstract
In this paper, we propose a dynamic file transfer scheme with co-allocation architecture, called Dynamic Adjustment Strategy, a dynamic file transfer scheme with co-allocation architecture that reduce the file transfer times and improves the performance in Data Grid environments. Our approach reduces the idle time faster servers spend waiting for the slowest server, and decreases file transfer completion time. We also present a new toolkit, called Cyber-Transformer, with a friendly graphical user interface interface running on the client side that makes it easy for inexperienced users to manage replicas and download the files in Data Grid environments. We also provide an effective scheme for reducing the cost of reassembling data blocks.
This work is supported in part by the National Science Council, Taiwan R.O.C., under grants no. NSC95-2221-E-029-004 and NSC95-2218-E-007-025.
Chapter PDF
Similar content being viewed by others
References
Allcock, B., Bester, J., Bresnahan, J., Chervenak, A., Foster, I., Kesselman, C., Meder, S., Nefedova, V., Quesnel, D., Tuecke, S.: Data Management and Transfer in High-Performance Computational Grid Environments. Parallel Computing 28(5), 749–771 (2002)
Allcock, B., Bester, J., Bresnahan, J., Chervenak, A., Foster, I., Kesselman, C., Meder, S., Nefedova, V., Quesnel, D., Tuecke, S.: Secure, efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing. In: Proceedings of the Eighteenth IEEE Symposium on Mass Storage Systems and Technologies, pp. 13–28 (2001)
Allcock, B., Tuecke, S., Foster, I., Chervenak, A., Kesselman, C.: Protocols and Services for Distributed Data-Intensive Science. In: ACAT2000 Proceedings, pp. 161–163 (2000)
Chervenak, A., Deelman, E., Foster, I., Guy, L., Hoschek, W., Iamnitchi, A., Kesselman, C., Kunszt, P., Ripeanu, M.: Giggle: A Framework for Constructing Scalable Replica Location Services. In: Proceedings of Supercomputing 2002, Baltimore, MD (2002)
Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.: The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. Journal of Network and Computer Applications 23, 187–200 (2001)
Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid Information Services for Distributed Resource Sharing. In: HPDC-10 2001. Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing, August 2001, pp. 181–194 (2001)
Czajkowski, K., Foster, I., Kesselman, C.: Resource Co-Allocation in Computational Grids. In: HPDC-8 1999. Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing (August 1999)
Donno, F., Gaido, L., Ghiselli, A., Prelz, F., Sgaravatto, M.: DataGrid Prototype 1. In: Proceedings of the TERENA Networking Conference (June 2002), http://www.terena.nl/conferences/tnc2002/Papers/p5a2-ghiselli.pdf
Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International Journal of Supercomputer Applications and High Performance Computing 15(3), 200–222 (2001)
Foster, I., Kesselman, C.: Globus: A Metacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications and High Performance Computing 11(2), 115–128 (1997)
Global Grid Forum, http://www.ggf.org/
Hoschek, W., Jaen-Martinez, J., Samar, A., Stockinger, H., Stockinger, K.: Data Management in an International Data Grid Project. In: Buyya, R., Baker, M. (eds.) GRID 2000. LNCS, vol. 1971, Springer, Heidelberg (2000)
IBM Red Books, Introduction to Grid Computing with Globus, IBM Press, http://www.redbooks.ibm.com/redbooks/pdfs/sg246895.pdf
Stockinger, H., Samar, A., Allcock, B., Foster, I., Holtman, K., Tierney, B.: File and Object Replication in Data Grids. Journal of Cluster Computing 5(3), 305–314 (2002)
SYSSTAT utilities home page, http://perso.wanadoo.fr/sebastien.godard/
The Globus Alliance, http://www.globus.org/
Vazhkudai, S.: Enabling the Co-Allocation of Grid Data Transfers. In: Proceedings of Fourth International Workshop on Grid Computing, November 2003, pp. 41–51 (2003)
Vazhkudai, S., Schopf, J.: Using Regression Techniques to Predict Large Data Transfers. International Journal of High Performance Computing Applications (IJHPCA) 17, 249–268 (2003)
Vazhkudai, S., Schopf, J.: Predicting Sporadic Grid Data Transfers. In: HPDC-11 2002. Proceedings of 11th IEEE International Symposium on High Performance Distributed Computing, July 2002, pp. 188–196 (2002)
Vazhkudai, S., Schopf, J., Foster, I.: Predicting the Performance of Wide Area Data Transfers. In: IPDPS 2002. Proceedings of the 16th International Parallel and Distributed Processing Symposium, April 2002, pp. 34–43 (2002)
Vazhkudai, S., Tuecke, S., Foster, I.: Replica Selection in the Globus Data Grid. In: CCGRID 2001. Proceedings of the 1st International Symposium on Cluster Computing and the Grid, May 2001, pp. 106–113 (2001)
Wolski, R., Spring, N., Hayes, J.: The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing. Future Generation Computer Systems 15(5-6), 757–768 (1999)
Yang, C.T., Chen, C.H., Li, K.C., Hsu, C.H.: Performance Analysis of Applying Replica Selection Technology for Data Grid Environments. In: Malyshkin, V. (ed.) PaCT 2005. LNCS, vol. 3606, pp. 278–287. Springer, Heidelberg (2005)
Yang, C.T., Shih, P.C., Chen, S.Y.: A Domain-based Model for Efficient Network Information on Grid Computing Environments. IEICE Trans. Information and Systems, Special Issue on Parallel/Distributed Computing and Networking E89-D(2), 738–742 (accepted and to appear)
Yang, C.T., Wang, S.Y., Lin, C.H., Lee, M.H., Wu, T.Y.: Cyber-Transformer: A Toolkit for Files Transfer with Replica Management in Data Grid Environments. In: WoGTA 2005. Proceedings of the Second Workshop on Grid Technologies and Applications, December 2005, pp. 73–80 (2005)
Yang, C.T., Yang, I.H., Chen, C.H., Wang, S.Y.: Implementation of a Dynamic Adjustment Mechanism with Efficient Replica Selection in Co-Allocation Data Grid Environments. In: SAC 2006. Proceedings of the 21st Annual ACM Symposium on Applied Computing - Distributed Systems and Grid Computing Track, April 23-27, 2006, pp. 797–804 (2006)
Yang, C.T., Yang, I.H., Li, K.C., Hsu, C.H.: A Recursive-Adjustment Co-Allocation Scheme in Data Grid Environments. In: Hobbs, M., Goscinski, A.M., Zhou, W. (eds.) Distributed and Parallel Computing. LNCS, vol. 3719, pp. 40–49. Springer, Heidelberg (2005)
Yang, C.T., Yang, I.H., Li, K.C., Wang, S.Y.: Improvements on Dynamic Adjustment Mechanism in Co-Allocation Data Grid Environments. The Journal of Supercomputing, December 2006 (accepted and to appear)
Zhang, X., Freschl, J., Schopf, J.: A Performance Study of Monitoring and Information Services for Distributed Systems. In: HPDC-12 2003. Proceedings of 12th IEEE International Symposium on High Performance Distributed Computing, August 2003, pp. 270–282 (2003)
Venugopal, S., Buyya, R., Ramamohanarao, K.: A Taxonomy of Data Grids for Distributed Data Sharing, Management, and Processing. ACM Computing Surveys 38(1), 1–53 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 IFIP International Federation for Information Processing
About this paper
Cite this paper
Yang, CT., Wang, SY., Fu, CP. (2007). A Dynamic Adjustment Strategy for File Transformation in Data Grids. In: Li, K., Jesshope, C., Jin, H., Gaudiot, JL. (eds) Network and Parallel Computing. NPC 2007. Lecture Notes in Computer Science, vol 4672. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74784-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-74784-0_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74783-3
Online ISBN: 978-3-540-74784-0
eBook Packages: Computer ScienceComputer Science (R0)