Advertisement

Hashing Computation for Scalable Metadata

  • Yu HuaEmail author
  • Xue Liu
Chapter
First Online:

Abstract

This section presents a scalable and adaptive decentralized metadata lookup scheme for ultra large-scale file systems (more than Petabytes or even Exabytes). Our scheme logically organizes metadata servers (MDSs) into a multilayered query hierarchy and exploits grouped Bloom filters to efficiently route metadata requests to desired MDSs through the hierarchy. This metadata lookup scheme can be executed at the network or memory speed, without being bounded by the performance of slow disks. An effective workload balance method is also developed for server reconfigurations. This scheme is evaluated through extensive trace-driven simulations and a prototype implementation in Linux. Experimental results show that this scheme can significantly improve metadata management scalability and query efficiency in ultra large-scale storage systems (©{2011}IEEE. Reprinted, with permission, from Ref. [1].).

This is a preview of subscription content, log in to check access.

References

  1. 1.
    Y. Hua, Y. Zhu, H. Jiang, D. Feng, L. Tian, Supporting scalable and adaptive metadata management in ultra large-scale file systems. IEEE Trans. Parallel Distrib. Syst. (TPDS) 22, 580–593 (2011)CrossRefGoogle Scholar
  2. 2.
    J. Piernas, The design of new journaling file systems: the DualFS case. IEEE Trans. Comput. 56(2), 267–281 (2007)MathSciNetCrossRefGoogle Scholar
  3. 3.
    S.A. Brandt, E.L. Miller, D.D.E. Long, L. Xue, Efficient metadata management in large distributed storage systems, in Proceedings of the MSST (2003)Google Scholar
  4. 4.
    D. Roselli, J.R. Lorch, T.E. Anderson, A comparison of file system workloads, in Proceedings of the Annual USENIX Technical Conference (2000)Google Scholar
  5. 5.
    L. Guy, P. Kunszt, E. Laure, H. Stockinger, K. Stockinger, Replica management in data grids, in Global Grid Forum, vol. 5 (2002)Google Scholar
  6. 6.
    S. Moon, T. Roscoe, Metadata management of terabyte datasets from an IP backbone network: experience and challenges, in Proceedings of the NRDM (2001)Google Scholar
  7. 7.
    M. Cai, M. Frank, B. Yan, R. MacGregor, A subscribable peer-to-peer RDF repository for distributed metadata management. J. Web Semant. Sci. Serv. Agents World Wide Web 2(2) (2005)CrossRefGoogle Scholar
  8. 8.
    C. Lukas, M. Roszkowski, The Isaac network: LDAP and distributed metadata for resource discovery, in Internet Scout Project (2001), http://scout.cs.wisc.edu/research/isaac/ldap.html
  9. 9.
    D. Fisher, J. Sobolewski, T. Tyler, Distributed metadata management in the high performance storage system, in Proceedings of the IEEE Metadata Conference (1996)Google Scholar
  10. 10.
    A. Foster, C. Salisbury, S. Tuecke, The data grid: towards an architecture for the distributed management and analysis of large scientific datasets. J. Netw. Comput. Appl. 23, 187–200 (2001)Google Scholar
  11. 11.
    M. Zingler, Architectural components for metadata management in earth observation, in Proceedings of the IEEE Metadata Conference (1996)Google Scholar
  12. 12.
    B. Bloom, Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422–426 (1970)CrossRefGoogle Scholar
  13. 13.
    P.J. Braam, Lustre whitepaper (2005), http://www.lustre.org
  14. 14.
    P.F. Corbett, D.G. Feitelson, The vesta parallel file system. ACM Trans. Comput. Syst. 14(3), 225–264 (1996)CrossRefGoogle Scholar
  15. 15.
    P.J. Braam, P.A. Nelson, Removing bottlenecks in distributed file systems: Coda & intermezzo as examples, in Proceedings of the Linux Expo (1999)Google Scholar
  16. 16.
    T.E. Anderson, M.D. Dahlin, J.M. Neefe, D.A. Patterson, D.S. Roselli, R.Y. Wang, Serverless network file systems. ACM Trans. Comput. Syst. 14(1), 41–79 (1996)CrossRefGoogle Scholar
  17. 17.
    O. Rodeh, A. Teperman, zFS-a scalable distributed file system using object disks, in Proceedings of the MSST (2003), pp. 207–218Google Scholar
  18. 18.
    B. Pawlowski, C. Juszczak, P. Staubach, C. Smith, D. Lebel, D. Hitz, NFS version3: design and implementation, in Proceedings of the USENIX Technical Conference (1994), pp. 137–151Google Scholar
  19. 19.
    J.H. Morris, M. Satyanarayanan, M.H. Conner, J.H. Howard, D.S. Rosenthal, F.D. Smith, Andrew: a distributed personal computing environment. Commun. ACM 29(3), 184–201 (1986)CrossRefGoogle Scholar
  20. 20.
    M. Satyanarayanan, J.J. Kistler, P. Kumar, M.E. Okasaki, E.H. Siegel, D.C. Steere, Coda: a highly available file system for a distributed workstation environment. IEEE Trans. Comput. 39(4), 447–459 (1990)CrossRefGoogle Scholar
  21. 21.
    M.N. Nelson, B.B. Welch, J.K. Ousterhout, Caching in the sprite network file system. ACM Trans. Comput. Syst. 6(1), 134–154 (1988)CrossRefGoogle Scholar
  22. 22.
    A. Adya, R. Wattenhofer, W. Bolosky, M. Castro, G. Cermak, R. Chaiken, J. Douceur, J. Howell, J. Lorch, M. Theimer, Farsite: federated, available, and reliable storage for an incompletely trusted environment. ACM SIGOPS Oper. Syst. Rev. 36, 1–14 (2002)CrossRefGoogle Scholar
  23. 23.
    V. Cate, T. Gross, Combining the concepts of compression and caching for a two-level filesystem. ACM SIGARCH Comput. Archit. News 19(2), 200–211 (1991)CrossRefGoogle Scholar
  24. 24.
    S. Weil, K. Pollack, S.A. Brandt, E.L. Miller, Dynamic metadata management for petabyte-scale file systems, in Proceedings of the ACM/IEEE Supercomputing (2004)Google Scholar
  25. 25.
    S. Weil, S.A. Brandt, E.L. Miller, D.D.E. Long, C. Maltzahn, Ceph: a scalable, high-performance distributed file system, in Proceedings of the OSDI (2006)Google Scholar
  26. 26.
    S. Weil, S.A. Brandt, E.L. Miller, C. Maltzahn, Crush: controlled, scalable, decentralized placement of replicated data, in Proceedings of the ACM/IEEE Supercomputing (2006)Google Scholar
  27. 27.
    R.J. Honicky, E.L. Miller, Replication under scalable hashing: a family of algorithms for scalable decentralized data distribution, in Proceedings of the IEEE IPDPS (2004)Google Scholar
  28. 28.
    L. Fan, P. Cao, J. Almeida, A.Z. Broder, Summary cache: a scalable wide area web cache sharing protocol. IEEE/ACM Trans. Netw. 8(3), 281–293 (2000)CrossRefGoogle Scholar
  29. 29.
    A. Chervenak, N. Palavalli, S. Bharathi, C. Kesselman, R. Schwartzkopf, Performance and scalability of a replica location service, in Proceedings of the HPDC (2004)Google Scholar
  30. 30.
    Y. Zhu, H. Jiang, J. Wang, F. Xian, HBA: distributed metadata management for large cluster-based storage systems. IEEE Trans. Parallel Distrib. Syst. 19(4), 1–14 (2008)CrossRefGoogle Scholar
  31. 31.
    A. Broder, M. Mitzenmacher, Network applications of Bloom filters: a survey. Internet Math. 1, 485–509 (2005)MathSciNetCrossRefGoogle Scholar
  32. 32.
    E. Riedel, M. Kallahalla, R. Swaminathan, A framework for evaluating storage system security, in Proceedings of the FAST (2002), pp. 15–30Google Scholar
  33. 33.
    Y. Zhu, H. Jiang, False rate analysis of Bloom filter replicas in distributed systems, in Proceedings of the ICPP (2006), pp. 255–262Google Scholar
  34. 34.
    D. Ellard, J. Ledlie, P. Malkani, M. Seltzer, Passive NFS tracing of email and research workloads, in Proceedings of the FAST (2003), pp. 203–216Google Scholar
  35. 35.
    J. Kubiatowicz, D. Bindel, Y. Chen, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, C. Wells, B. Zhao, Oceanstore: an architecture for global-scale persistent storage, in Proceedings of the ACM ASPLOS (2000)Google Scholar
  36. 36.
    A. Azagury, V. Dreizin, M. Factor, E. Henis, D. Naor, N. Rinetzky, O. Rodeh, J. Satran, A. Tavory, L. Yerushalmi, Towards an object store, in Proceedings of the MSST, pp. 165–176, Apr 2003Google Scholar
  37. 37.
    B. Welch, G. Gibson, Managing scalability in object storage systems for HPC Linux clusters, in Proceedings of the MSST, pp. 433–445, Apr 2004Google Scholar
  38. 38.
    E.L. Miller, R.H. Katz, RAMA: an easy-to-use, high-performance parallel file system, in Parallel Computing, vol. 23 (1997)CrossRefGoogle Scholar
  39. 39.
    P. Carns, W. Ligon III, R. Ross, R. Thakur, PVFS: a parallel file system for Linux clusters, in Proceedings of the Annual Linux Showcase and Conference (2000), pp. 317–327Google Scholar
  40. 40.
    N. Nieuwejaar, D. Kotz, The Galley Parallel File System (ACM Press, New York, NY, USA, 1996)CrossRefGoogle Scholar
  41. 41.
    A.W. Leung, M. Shao, T. Bisson, S. Pasupathy, E.L. Miller, Spyglass: fast, scalable metadata search for large-scale storage systems. Technical Report UCSC-SSRC-08-01 (2008)Google Scholar
  42. 42.
    A. Sweeney, D. Doucette, W. Hu, C. Anderson, M. Nishimoto, G. Peck, Scalability in the XFS file system, in Proceedings of the USENIX Technical Conference (1996), pp. 1–14Google Scholar
  43. 43.
    M. Mitzenmacher, Compressed Bloom filters. IEEE/ACM Trans. Netw. 10(5), 604–612 (2002)CrossRefGoogle Scholar
  44. 44.
    A. Kumar, J. Xu, E.W. Zegura, Efficient and scalable query routing for unstructured peer-to-peer networks, in Proceedings of the INFOCOM (2005)Google Scholar
  45. 45.
    C. Saar, M. Yossi, Spectral Bloom filters, in Proceedings of the SIGMOD (2003)Google Scholar
  46. 46.
    Y. Zhang, D. Li, L. Chen, X. Lu, Collaborative search in large-scale unstructured peer-to-peer networks, in Proceedings of the ICPP (2007)Google Scholar
  47. 47.
    F. Bonomi, M. Mitzenmacher, R. Panigrahy, S. Singh, G. Varghese, Beyond Bloom filters: from approximate membership checks to approximate state machines, in Proceedings of the SIGCOMM (2006)Google Scholar
  48. 48.
    D. Guo, J. Wu, H. Chen, X. Luo, Theory and network application of dynamic Bloom filters, in Proceedings of the INFOCOM (2006)Google Scholar
  49. 49.
    B. Xiao, Y. Hua, Using parallel Bloom filters for multi-attribute representation on network services. IEEE Trans. Parallel Distrib. Syst. 21, 20–32 (2010)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Huazhong University of Science and TechnologyWuhanChina
  2. 2.McGill UniversityMontrealCanada

Personalised recommendations

Cite chapter

Buy options