The Equi-Join Processing and Optimization on Ring Architecture Key/Value Database

  • Xite Wang
  • Derong Shen
  • Tiezheng Nie
  • Yue Kou
  • Ge Yu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7235)

Abstract

Huge data analysis (e.g. equi-join) on key/value data storage system is a very novel and necessary issue. For the master-slave architecture distributed system(MSADS) (e.g. Hadoop), equi-join can be implemented by utilizing MapReduce framework which is designed for performing scalable parallel data analysis on MSADS. However, for the ring architecture distributed system (RADS), there has no general method for processing data analysis task (e.g. equi-join). Hence in this paper, a novel approach is proposed for processing equi-join on RADS. Firstly, by making in-depth analysis of RADS, we propose a new type of index, ColumnValue index(CVI), based on ColumnFamily data model. Then, by utilizing CVI, an efficient algorithm, called pre-join table generation algorithm (PJTG), is proposed to process the equi-join query on RADS, and a memory index(MI) is utilized to further improve the performance of PJTG. In addition, the update method for the join result is present , and the update is caused by the alteration of original data. Finally, the validity of PJTG is verified through plenty of simulation experiments. Experimental results show that the proposed algorithm is an effective way to solve the equi-join query problem on RADS and could meet the requirements of practical applications.

Keywords

Master Node Slave Node MapReduce Framework Data Storage System Memory Index 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
    Avinash, L., Prashant, M.: Cassandra: a decentralized structured storage system. ACM SIGOPS Operating Systems Review 44(2) (April 2010)Google Scholar
  3. 3.
    Giuseppe, D., Deniz, H., Madan, J.: Dynamo: Amazon’s Highly Available Key-value Store. In: SOSP, Washington, USA, October 14-17, pp. 205–220 (2007)Google Scholar
  4. 4.
    Jeffrey, D., Sanjay, G.: MapReduce: simplified data processing on large clusters. In: OSDI, California, USA, December 6-8 (2004)Google Scholar
  5. 5.
    Foto, A., Jeffrey, U.: Optimizing Joins in a Map-Reduce Environment. In: EDBT, Lausanne, Switzerland, March 22-26 (2010)Google Scholar
  6. 6.
    Kamil, P., Daniel, A., Avi, S.: Efficient Processing of Data Warehousing Queries in a Split Execution Environment. In: SIGMOD, Athens, Greece, June 12-16, pp. 1165–1176 (2011)Google Scholar
  7. 7.
  8. 8.
  9. 9.
    Yuting, L., Divyakant, A., Chun, C.: Llama: Leveraging Columnar Storage for Scalable Join Processing in the MapReduce Framework. In: SIGMOD, Athens, Greece, June 12-16, pp. 961–972 (2011)Google Scholar
  10. 10.
    Alper, O., Mirek, R.: Processing Theta-Joins using MapReduce. In: SIGMOD, Athens, Greece, June 12-16, pp. 949–960 (2011)Google Scholar
  11. 11.
    Karger, D., Lehman, E., Leighton, T.: Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spotson the World Wide Web. In: STOC, USA, pp. 654–663 (1997)Google Scholar
  12. 12.
    Fay, C., Jeffrey, D., Sanjay, G.: Bigtable: A Distributed Storage System for Structured Data. In: OSDI, Seattle, WA, USA, November 6-8 (2006)Google Scholar
  13. 13.

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Xite Wang
    • 1
  • Derong Shen
    • 1
  • Tiezheng Nie
    • 1
  • Yue Kou
    • 1
  • Ge Yu
    • 1
  1. 1.Key Laboratory of Medical Image Computing (NEU), Ministry of Education, College of Information Science and EngineeringNortheastern UniversityP.R. China

Personalised recommendations