Advertisement

A Hybrid Scheme for Object Allocation in a Distributed Object-Storage System

  • Fang Wang
  • Shunda Zhang
  • Dan Feng
  • Hong Jiang
  • Lingfang Zeng
  • Song Lv
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3994)

Abstract

The object-based storage system, in which files are mapped onto one or more data objects stored on Object-Based Storage Devices (OSDs), has distributed storage system architecture. In such a system, the policy for object allocation is a critical aspect affecting the overall systems performance. Hashing and fragment-strip are two common techniques used for managing objects, but both have their disadvantages, and advantages, e.g. hashing achieves good workload balance and provide rather high effectiveness in allocating data, but it can not provide readily available parallelism for large file; fragment-strip takes full advantage device parallelism, simplifies the clients’ operations, but this policy is not fit for small file. In this paper, we present an efficient algorithm that combines the advantages of these two approaches while avoiding their shortcomings. The key factors which can impact the performance of the whole system in the objects allocation are also be discussed.

Keywords

File System Hybrid Scheme Lawrence Livermore National Laboratory Message Size Large File 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Wang, F., Brandt, S.A., Miller, E.L., Long, D.D.E.: OBFS: A File System for Object-based Storage Devices. In: 21st IEEE / 12th NASA Goddard Conference on Mass Storage Systems and Technologies (MSST 2004), College Park, MD (April 2004)Google Scholar
  2. 2.
    Xue, L., Liu, Y.: MDS Functionality Analysis, December 4 (2001)Google Scholar
  3. 3.
    Morris, J.H., Satyanarayanan, M., Conner, M.H., Howard, J.H., Rosenthal, D.S.H., Smith, F.D.: Andrew: A distributed personal computing environment. Communications of the ACM 29(3), 184–201 (1986)CrossRefGoogle Scholar
  4. 4.
    McKusick, M.K., Joy, W.N., Leffler, S.J., Fabry, R.S.: A fast file system for UNIX. ACM Transactions on Computer Systems 2(3), 181–197 (1984)CrossRefGoogle Scholar
  5. 5.
    Gibson, G.A., Meter, R.V.: Network attached storage architecture. Communications of the ACM 43(11), 37–45 (2000)CrossRefGoogle Scholar
  6. 6.
    Spasojevic, M., Satyanarayanan, M.: An Empirical Study of a Wide-Area Distributed File System. ACM Transactions on Computer Systems 14(2) (May 1996)Google Scholar
  7. 7.
    Roselli, D., Lorch, J., Anderson, T.: A comparison of file system workloads. In: Proceedings of the 2000 USENIX Annual Technical Conference (June 2000)Google Scholar
  8. 8.
    Braam, P.J., et al.: The Lustre Storage Architecture. Cluster File Systems, Inc. (August 2003), http://www.clusterfs.com
  9. 9.
    Moase, J.R.: Panasas Storage Cluster & Object Storage Overview (October 2004), http://www.panasas.com
  10. 10.
    Hennessy, J.L., Patterson, D.A.: Computer Architecture A Quantitative Approach, 3rd edn.Google Scholar
  11. 11.
    Soltis, S.R., Ruwart, T.M., O’Keefe, M.T.: The Global File System. In: Proceedings of the 5th NASA Goddard Conference on Mass Storage Systems and Technologies, College Park, MD (1996)Google Scholar
  12. 12.
    Satyanarayanan, M., Kistler, J.J., Kumar, P., Okasaki, M.E., Siegel, E.H., Steere, D.C.: Coda: A highly available file system for a distributed workstation environment. IEEE Transactions on Computers 39(4), 447–459 (1990)CrossRefGoogle Scholar
  13. 13.
    Schmuck, F., Haskin, R.: GPFS: A shared-disk file-system for large computing clusters. In: Proceedings of the 2002 Conference on File and Storage Technologies (FAST), USENIX (January 2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Fang Wang
    • 1
  • Shunda Zhang
    • 1
  • Dan Feng
    • 1
  • Hong Jiang
    • 1
  • Lingfang Zeng
    • 1
  • Song Lv
    • 1
  1. 1.Key Laboratory of Data Storage System, Ministry of Education, School of ComputerHuazhong University of Science and TechnologyWuhanChina

Personalised recommendations