ASAWA: An Automatic Partition Key Selection Strategy

  • Xiaoyan Wang
  • Jinchuan Chen
  • Xiaoyong Du
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7808)

Abstract

With the rapid increase of data volume, more and more applications have to be implemented in a distributed environment. In order to obtain high performance, we need to carefully divide the whole dataset into multiple partitions and put them into distributed data nodes. During this process, the selection of partition key would greatly affect the overall performance. Nevertheless, there are few works addressing this topic. Most previous projects on data partitioning either utilize a simple strategy, or rely on a commercial database system, to choose partition keys. In this work, we present an automatic partition key selection strategy called ASAWA. It chooses partition keys according to the analysis on both dataset and workload schemas. In this way, intimate tuples, i.e. co-appearing in queries frequently, would be probably put into the same partition. Hence the cross-node joins could be greatly reduced and the system performance could be improved. We conduct a series of experiments over the TPC-H datasets to illustrate the effectiveness of the ASAWA strategy.

Keywords

partition key selection data partitioning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Zilio, D.C.: Physical Database Design Decision Algorithms and Concurrent Reorganization for Parallel Database Systems. PhD Thesis, Department of Computer Science, University of Toronto (1998)Google Scholar
  2. 2.
    Pavlo, A., Curino, C., Zdonik, S.: Skew-aware Automatic Database Partitioning in Shared-Nothing Parallel OLTP Systems. In: Proc. of the ACM SIGMOD, pp. 61–72 (2012)Google Scholar
  3. 3.
    TPC BenchmarkTM H, http://www.tpc.org/tpch/
  4. 4.
    Stonebraker, M., Cattell, R.: 10 Rules for Scalable Performance in ‘Simple Operation’ Datastores. Communications of the ACM 54, 72–80 (2011)CrossRefGoogle Scholar
  5. 5.
    Ceri, S., Negri, M., Pelagatti, G.: Horizontal Data Partitioning in Database Design. In: Proc. of the ACM SIGMOD, pp. 128–136 (1982)Google Scholar
  6. 6.
    Navathe, S., Ceri, G., Wiederhold, G., Dou, J.: Vertical Partitioning Algorithms for Database Systems. ACM Transactions on Database Systems 9(4), 680–710 (1984)CrossRefGoogle Scholar
  7. 7.
    Agrawal, S., Narasayya, V., Yang, B.: Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design. In: Proc. of the ACM SIGMOD, pp. 359–370 (2004)Google Scholar
  8. 8.
    Curino, C., Jones, E., Zhang, Y., Madden, S.: Schism: a Workload-Driven Approach to Database Replication and Partitioning. Proc. of the VLDB Endowment 3, 48–57 (2010)Google Scholar
  9. 9.
  10. 10.
    Zilio, D.C., Jhingran, A., Padmanabhan, S.: Partition Key Selection for a Shared-nothing Parallel Database System. Technical Report RC 19820(87739) 11/10/94, IBM T. J. Watson Research Center (1994)Google Scholar
  11. 11.
    Eadon, G., Chong, E.I., Shankar, S., Raghavan, A., Srinivasan, J., Das, S.: Supporting Table Partitioning by Reference in Oracle. In: Proc. of the ACM SIGMOD, pp. 1111–1122 (2008)Google Scholar
  12. 12.
    Zilio, D.C., Rao, J., Lightstone, S., Lohman, G., et al.: DB2 Design Advisor: Integrated Automatic Physical Database Design. In: Proceedings of the VLDB, pp. 1087–1097 (2004)Google Scholar
  13. 13.
    Nehme, R., Bruno, N.: Automated Partitioning Design in Parallel Database Systems. In: Proc. of the ACM SIGMOD, pp. 1137–1148 (2011)Google Scholar
  14. 14.
    Özsu, M.T., Valduriez, P.: Principles of Distributed Database Systems, 3rd edn. Springer, New York (2011)Google Scholar
  15. 15.
    Rahimi, S., Haug, F.S.: Distributed Database Management Systems: A Practical Approach. IEEE Computer Society, Hoboken (2010)MATHCrossRefGoogle Scholar
  16. 16.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Xiaoyan Wang
    • 1
    • 2
    • 3
  • Jinchuan Chen
    • 3
  • Xiaoyong Du
    • 1
    • 3
  1. 1.School of InformationRenmin University of ChinaChina
  2. 2.School of Information and Electrical EngineeringLudong UniversityChina
  3. 3.Key Laboratory of Data Engineering and Knowledge EngineeringMOEChina

Personalised recommendations