The VLDB Journal

, Volume 25, Issue 2, pp 151–169 | Cite as

RailwayDB: adaptive storage of interaction graphs

Regular Paper

Abstract

We are living in an ever more connected world, where data recording the interactions between people, software systems, and the physical world is becoming increasingly prevalent. These data often take the form of a temporally evolving graph, where entities are the vertices and the interactions between them are the edges. We call such graphs interaction graphs. Various domains, including telecommunications, transportation, and social media, depend on analytics performed on interaction graphs. The ability to efficiently support historical analysis over interaction graphs requires effective solutions for the problem of data layout on disk. This paper presents an adaptive disk layout called the railway layout for optimizing disk block storage for interaction graphs. The key idea is to divide blocks into one or more sub-blocks. Each sub-block contains the entire graph structure, but only a subset of the attributes. This improves query I/O, at the cost of increased storage overhead. We introduce optimal integer linear program (ILP) formulations for partitioning disk blocks into sub-blocks with overlapping and nonoverlapping attributes. Additionally, we present greedy heuristics that can scale better compared to the ILP alternatives, yet achieve close to optimal query I/O. We provide an implementation of the railway layout as part of RailwayDB—an open-source graph database we have developed. To demonstrate the benefits of the railway layout, we provide an extensive experimental evaluation, including model-based as well as empirical results comparing our approach to baseline alternatives.

Keywords

Interaction graphs Adaptive storage I/O optimization 

References

  1. 1.
    Hoque, I., Gupta, I.: Disk layout techniques for online social network data. Internet Comput. 16(3), 24–36 (2012)CrossRefGoogle Scholar
  2. 2.
    Steinhaus, R.: G-Store: a storage manager for graph data. Master’s thesis, University of Oxford (2011)Google Scholar
  3. 3.
    Gedik, B., Bordawekar, R.: Disk-based management of interaction graphs. IEEE Trans. Knowl. Data. Eng. (TKDE) 26(11), 650–665 (2014)CrossRefGoogle Scholar
  4. 4.
    Alagiannis, I., Idreos, S., Ailamaki, A.: H2O: a hands-free adaptive store. In: ACM International Conference on Management of Data (SIGMOD), pp. 1103–1114 (2014)Google Scholar
  5. 5.
    Grund, M., Zeier, A., Krüger, J., Plattner, H., Madden, S.: HYRISE: a main memory hybrid storage engine. VLDB J. (VLDBJ) 4(2), 105–116 (2010)Google Scholar
  6. 6.
    Stonebraker, M., Abadi, D.J., Batkin, A., Chen, X., Cherniack, M., Ferreira, M., Lau, E., Lin, A., Madden, S., O’Neil, E., O’Neil, P., Rasin, A., Tran, N., Zdonik, S.: C-store: a column-oriented DBMS. In: International Conference on Very Large Data Bases (VLDB), pp. 553–564 (2005)Google Scholar
  7. 7.
    Hellerstein, J.M., Stonebraker, M., Hamilton, J.: Architecture of a database system. Found. Trends Databases 1(2), 141–259 (2007). doi:10.1561/1900000002 CrossRefMATHGoogle Scholar
  8. 8.
    Abadi, D., Boncz, P., Harizopoulos, S., Idreos, S., Madden, S.: The design and implementation of modern column-oriented database systems. Found. Trends Databases 5(3), 197–280 (2013)CrossRefGoogle Scholar
  9. 9.
    Wilson, C., Boe, B., Sala, A., Puttaswamy, K.P., Zhao, B.Y.: User interactions in social networks and their implications. In: European Conference on Computer Systems (EUROSYS), pp. 205–218 (2009)Google Scholar
  10. 10.
    Wilson, C., Sala, A., Puttaswamy, K.P.N., Zhao, B.Y.: Beyond social graphs: user interactions in online social networks and their implications. ACM Trans. Web (TWEB) 6(4), 17:1–17:31 (2012)Google Scholar
  11. 11.
    Ghemawat, S.: LevelDB - a fast and lightweight key/value database library by Google. Available at https://github.com/google/leveldb, retrieved March, 2015
  12. 12.
    Hadjieleftheriou, M., Hoel, E.G., Tsotras, V.J.: SaIL: a spatial index library for efficient application integration. GeoInformatica 9(4), 367–389 (2005)CrossRefGoogle Scholar
  13. 13.
    Gurobi Optimization Inc.: The Gurobi Optimizer. Available at http://www.gurobi.com
  14. 14.
    Turkey Protests Spread from Istanbul to Ankara, Euronews. http://www.euronews.com/2013/05/31/turkey-protests-spread-from-istanbul-to-ankara/
  15. 15.
    Malewicz, G., Austern, M.H., Bik, A.J., Dehnert, J.C., Horn, I., Leiser, N., Czajkowski, G.: Pregel: a system for large-scale graph processing. In: ACM International Conference on Management of Data (SIGMOD), pp. 135–146 (2010)Google Scholar
  16. 16.
    Apache giraph. http://giraph.apache.org
  17. 17.
    Gonzalez, J.E., Low, Y., Gu, H., Bickson, D., Guestrin, C.: PowerGraph: distributed graph-parallel computation on natural graphs. In: USENIX Symposium on Operating Systems Design and Implementation (OSDI), pp. 17–30 (2012)Google Scholar
  18. 18.
    Kyrola, A., Blelloch, G., Guestrin, C.: GraphChi: large-scale graph computation on just a PC. In: USENIX Symposium on Operating Systems Design and Implementation (OSDI), pp. 31–46 (2012)Google Scholar
  19. 19.
    Kang, U., Tsourakakis, C.E., Faloutsos, C.: PEGASUS: A peta-scale graph mining system implementation and observations. In: IEEE International Conference on Data Mining (ICDM), pp. 229–238 (2009)Google Scholar
  20. 20.
    Idreos, S., Kersten, M.L., Manegold, S.: Self-organizing tuple reconstruction in column-stores. In: ACM International Conference on Management of Data (SIGMOD), pp. 297–308 (2009)Google Scholar
  21. 21.
    Idreos, S., Kersten, M.L., Manegold, S.: Database cracking. In: Conference on Innovative Data Systems Research (CIDR), pp. 68–78 (2007)Google Scholar
  22. 22.
    Graefe, G., Kuno, H.: Self-selecting, self-tuning, incrementally optimized indexes. In: International Conference on Extending Database Technology (EDBT), pp. 371–381 (2010)Google Scholar
  23. 23.
    Graefe, G., Kuno, H.A.: Adaptive indexing for relational keys. In: Workshop on Self-Managing Database Systems (SMDB), pp. 69–74 (2010)Google Scholar
  24. 24.
    Idreos, S., Manegold, S., Kuno, H., Graefe, G.: Merging what’s cracked, cracking what’s merged: adaptive indexing in main-memory column-stores. In: International Conference on Very Large Data Bases (VLDB), pp. 586–597 (2014)Google Scholar
  25. 25.
    Wu, E., Madden, S.: Partitioning techniques for fine-grained indexing. In: IEEE International Conference on Data Engineering (ICDE), pp. 1127–1138 (2011)Google Scholar
  26. 26.
    Neo4j Graph Database. http://www.neo4j.org
  27. 27.
    Kang, U., Tong, H., Sun, J., Lin, C.Y., Faloutsos, C.: GBASE: A scalable and general graph management system. In: ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD), pp. 1091–1099 (2011)Google Scholar
  28. 28.
    Khurana, U., Deshpande, A.: Efficient snapshot retrieval over historical graph data. In: IEEE International Conference on Data Engineering (ICDE), pp. 997–1008 (2013)Google Scholar
  29. 29.
    Bornea, M.A., Dolby, J., Kementsietsidis, A., Srinivas, K., Dantressangle, P., Udrea, O., Bhattacharjee, B.: Building an efficient rdf store over a relational database. In: ACM International Conference on Management of Data (SIGMOD), pp. 121–132 (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Faculty of InformaticsUniversità della Svizzera italianaLuganoSwitzerland
  2. 2.Department of Computer EngineeringBilkent UniversityAnkaraTurkey

Personalised recommendations