Advertisement

Design and Implementation of a Random Data-Placement System with High Scalability, Reliability and Performance

  • Kun Liu
  • Wei Xue
  • Di Wang
  • Jiwu Shu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3993)

Abstract

As storage system scales to thousands of disks, data distribution, load balance and the support for heterogeneous disks become increasingly important. In this paper, we present a new data-placement method named Weighted Interval Algorithm (WIA) for heterogeneous disks. Through it is not optimal in some circumstances, the difference between WIA and the optimal algorithm is trivial. Combined with replication, WIA can nearly balance access load and space utilization and improve reliability simultaneously. For the first time, we implement a data-placement system with high scalability, reliability and performance. The experimental results show that WIA reduces the average response time by 14.8% and decreases coefficient of relative load from 78.09% to 47.46% while the difference of the ratio of space utilization between disks is not more than 0.79%.

Keywords

Average Response Time Data Placement Space Utilization Disk Access Relative Load 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Honicky, R.J., Miller, E.L.: A fast algorithm for online placement and reorganization of replicated data. In: Proceedings of the 17th International Parallel & Distributed Processing Symposium, Nice, France (April 2003)Google Scholar
  2. 2.
    Brinkmann, A., Salzwedel, K., Scheideler, C.: Compact, Adaptive Placement Schemes for Non-Union Capacities. In: Proceedings of the 14th ACM Symposium on Parallel Algorithms and Architectures (SPAA), Winnipeg, Manitoba, Canada, August 2002, pp. 53–62 (2002)Google Scholar
  3. 3.
    Seo, B.: Survey on Data Placement and Migration Algorithms in Distributed Disk Systems. In: Proceedings of 2004 International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 2004), Las Vegas, Nevada, USA, June 21-24 (2004)Google Scholar
  4. 4.
    Jose, R.B.: Comparing Random Data Allocation and Data Striping in Multimedia Servers. In: Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, Santa Clara, California, United States, pp. 44–55 (2000)Google Scholar
  5. 5.
    Zimmermann, R., Ghandeharizadeh, S.: Continuous Display Using Heterogeneous Disk-Subsystems. In: Proceedings of the fifth ACM international conference on Multimedia Seattle, Washington, United States, pp. 227–238Google Scholar
  6. 6.
    Ruemmler, C., Wilkes, J.: UNIX disk access patterns. In: Proceedings of the Winter 1993 USENIX Conference, January 1993, pp. 405–420 (1993)Google Scholar
  7. 7.
    Wilkes, J., Golding, R., Staelin, C., Sullivan, T.: The HP AutoRAID hierarchical storage system. ACM Transactions on Computer Systems 14(1), 108–136Google Scholar
  8. 8.
    Watanabe, A., Yokota, H.: Adaptive Overlapped Declustering: A Highly Available Data-Placement Method Balancing Access Load and Space Utilization. In: 21st International Conference on Data Engineering (ICDE 2005) (2005)Google Scholar
  9. 9.
    Litwin, W., Menon, J., Risch, T.: LH* schemes with scalable availability. Technical Report RJ 10121 (91937), IBM Research, Almaden Center (May 1998)Google Scholar
  10. 10.
    Litwin, W., Neimat, M., Levy, G., Ndiaye, S., Seek, T., Schwarz, T.: LH*s: a high-availability and high-security scalable distributed data structure. In: Proceeding of the 7th International Workshop on Research Issue in Data Engineering 1997, Birmingham, UK, pp. 141–150. IEEE, Los Alamitos (April 1997)Google Scholar
  11. 11.
    Litwin, W., Neimat, M.-A.: High-availability LH* schemes with mirroring. In: Proceeding of the Conference on Cooperative Information Systems, pp. 196–205 (1996)Google Scholar
  12. 12.
    Litwin, W., Neimat, M.-A., Schneider, D.A.: LH*-a scalable, distributed data structure. ACM Transactions on Database Systems 21(4), 480–525 (1996)CrossRefGoogle Scholar
  13. 13.
    Park, Y.-S., Kim, J.-W., Chung, K.-D.: A Continuous Media Placement using B-ZBSR on Heterogeneous MZR Disk Array. In: Proceedings of International Workshops on Parallel Processing, pp. 482–487 (1999)Google Scholar
  14. 14.
    Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google File System. In: Proceedings of the nineteenth ACM symposium on Operating systems principles Bolton Landing, NY, USA, pp. 29–43Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Kun Liu
    • 1
  • Wei Xue
    • 1
  • Di Wang
    • 1
  • Jiwu Shu
    • 1
  1. 1.Dept. of Computer Science and TechnologyTsinghua UniversityBeijing

Personalised recommendations