Abstract
As storage system scales to thousands of disks, data distribution, load balance and the support for heterogeneous disks become increasingly important. In this paper, we present a new data-placement method named Weighted Interval Algorithm (WIA) for heterogeneous disks. Through it is not optimal in some circumstances, the difference between WIA and the optimal algorithm is trivial. Combined with replication, WIA can nearly balance access load and space utilization and improve reliability simultaneously. For the first time, we implement a data-placement system with high scalability, reliability and performance. The experimental results show that WIA reduces the average response time by 14.8% and decreases coefficient of relative load from 78.09% to 47.46% while the difference of the ratio of space utilization between disks is not more than 0.79%.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Honicky, R.J., Miller, E.L.: A fast algorithm for online placement and reorganization of replicated data. In: Proceedings of the 17th International Parallel & Distributed Processing Symposium, Nice, France (April 2003)
Brinkmann, A., Salzwedel, K., Scheideler, C.: Compact, Adaptive Placement Schemes for Non-Union Capacities. In: Proceedings of the 14th ACM Symposium on Parallel Algorithms and Architectures (SPAA), Winnipeg, Manitoba, Canada, August 2002, pp. 53–62 (2002)
Seo, B.: Survey on Data Placement and Migration Algorithms in Distributed Disk Systems. In: Proceedings of 2004 International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 2004), Las Vegas, Nevada, USA, June 21-24 (2004)
Jose, R.B.: Comparing Random Data Allocation and Data Striping in Multimedia Servers. In: Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, Santa Clara, California, United States, pp. 44–55 (2000)
Zimmermann, R., Ghandeharizadeh, S.: Continuous Display Using Heterogeneous Disk-Subsystems. In: Proceedings of the fifth ACM international conference on Multimedia Seattle, Washington, United States, pp. 227–238
Ruemmler, C., Wilkes, J.: UNIX disk access patterns. In: Proceedings of the Winter 1993 USENIX Conference, January 1993, pp. 405–420 (1993)
Wilkes, J., Golding, R., Staelin, C., Sullivan, T.: The HP AutoRAID hierarchical storage system. ACM Transactions on Computer Systems 14(1), 108–136
Watanabe, A., Yokota, H.: Adaptive Overlapped Declustering: A Highly Available Data-Placement Method Balancing Access Load and Space Utilization. In: 21st International Conference on Data Engineering (ICDE 2005) (2005)
Litwin, W., Menon, J., Risch, T.: LH* schemes with scalable availability. Technical Report RJ 10121 (91937), IBM Research, Almaden Center (May 1998)
Litwin, W., Neimat, M., Levy, G., Ndiaye, S., Seek, T., Schwarz, T.: LH*s: a high-availability and high-security scalable distributed data structure. In: Proceeding of the 7th International Workshop on Research Issue in Data Engineering 1997, Birmingham, UK, pp. 141–150. IEEE, Los Alamitos (April 1997)
Litwin, W., Neimat, M.-A.: High-availability LH* schemes with mirroring. In: Proceeding of the Conference on Cooperative Information Systems, pp. 196–205 (1996)
Litwin, W., Neimat, M.-A., Schneider, D.A.: LH*-a scalable, distributed data structure. ACM Transactions on Database Systems 21(4), 480–525 (1996)
Park, Y.-S., Kim, J.-W., Chung, K.-D.: A Continuous Media Placement using B-ZBSR on Heterogeneous MZR Disk Array. In: Proceedings of International Workshops on Parallel Processing, pp. 482–487 (1999)
Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google File System. In: Proceedings of the nineteenth ACM symposium on Operating systems principles Bolton Landing, NY, USA, pp. 29–43
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, K., Xue, W., Wang, D., Shu, J. (2006). Design and Implementation of a Random Data-Placement System with High Scalability, Reliability and Performance. In: Alexandrov, V.N., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds) Computational Science – ICCS 2006. ICCS 2006. Lecture Notes in Computer Science, vol 3993. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11758532_91
Download citation
DOI: https://doi.org/10.1007/11758532_91
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34383-7
Online ISBN: 978-3-540-34384-4
eBook Packages: Computer ScienceComputer Science (R0)