Distributed Range-Based Meta-Data Management for an In-Memory Storage

Conference paper

DOI: 10.1007/978-3-319-27308-2_1

Part of the Lecture Notes in Computer Science book series (LNCS, volume 9523)
Cite this paper as:
Klein F., Beineke K., Schöttner M. (2015) Distributed Range-Based Meta-Data Management for an In-Memory Storage. In: Hunold S. et al. (eds) Euro-Par 2015: Parallel Processing Workshops. Euro-Par 2015. Lecture Notes in Computer Science, vol 9523. Springer, Cham

Abstract

Large-scale interactive applications and online graph processing require fast data access to billions of small data objects. DXRAM addresses this challenge by keeping all data always in RAM of potentially many nodes aggregated in a data center. Such storage clusters need a space-efficient and fast meta-data management. In this paper we propose a range-based meta-data management allowing fast node lookups while being space efficient by combining data object IDs in ranges. A super-peer overlay network is used to manage these ranges together with backup-node information allowing parallel and fast recovery of meta data and data of failed peers. Furthermore, the same concept can also be used for client-side caching. The measurement results show the benefits of the proposed concepts compared to other meta-data management strategies as well as its very good overall performance evaluated using the social network benchmark BG.

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Florian Klein
    • 1
  • Kevin Beineke
    • 1
  • Michael Schöttner
    • 1
  1. 1.Institut für InformatikHeinrich-Heine-Universität DüsseldorfDüsseldorfGermany

Personalised recommendations