Advertisement

Dynamic Data Migration Policies for Query-Intensive Distributed Data Environments

  • Tengjiao Wang
  • Bishan Yang
  • Allen Huang
  • Qi Zhang
  • Jun Gao
  • Dongqing Yang
  • Shiwei Tang
  • Jinzhong Niu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5446)

Abstract

Modern large distributed applications, such as telecommunication and banking services, need to respond instantly to a huge number of queries within a short period of time. The data-intensive, query-intensive nature makes it necessary to build these applications in a distributed data environment that involves a number of data servers sharing service load. How data is distributed among the servers has a crucial impact on the system response time. This paper introduces two policies that dynamically migrate data in such an environment as the pattern of queries on data changes, and achieve query load balance. One policy is based on a central controller that periodically collects the query load information on all data servers and regulates data migration across the whole system. The other policy lets individual server dynamically selects a partner to migrate data and balance query load in between. Experimental results show that both policies significantly improve system performance in terms of average query response time and fairness, and communication overhead incurred is marginal.

Keywords

Source Node Destination Node Load Level Average Response Time Server Node 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Narendran, B., Rangarajan, S., Yajnik, S.: Data distribution algorithms for load balanced fault-tolerant Webaccess. In: Proceedings of The Sixteenth Symposium on Reliable Distributed Systems, pp. 97–106 (1997)Google Scholar
  2. 2.
    Savio, S.: Approximate Algorithms for Document Placement in Distributed Web Servers. IEEE Transactions on Software Engineering, 100–106 (2004)Google Scholar
  3. 3.
    Yokota, H., Kanemasa, Y., Miyazaki, J.: Fat-Btree: An Update-Conscious Parallel Directory Structure. In: International Conference on Data Engineering (ICDE), pp. 448–457 (1999)Google Scholar
  4. 4.
    Lee, M.-L., Kitsuregawa, M., Ooi, B.-C., Tan, K.-L., Mondal, A.: Towards Self-Tuning Data Placement in Parallel Database Systems. In: International Conference on Management of Data (SIGMOD), pp. 225–236 (2000)Google Scholar
  5. 5.
    Feelifl, H., Kitsuregawa, M., Ooi, B.-C.: A fast convergence technique for online heat-balancing of btree indexed database over shared-nothing parallel systems. In: Ibrahim, M., Küng, J., Revell, N. (eds.) DEXA 2000. LNCS, vol. 1873, pp. 846–858. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  6. 6.
    Watanabe, A., Yokota, H.: Adaptive Lapped Declustering: A Highly Available Data-Placement Method Balancing Access Load and Space Utilization. In: International Conference on Data Engineering (ICDE), pp. 828–839 (2005)Google Scholar
  7. 7.
    Feldmann, M., Rissen, J.P.: GSM Network Systems and Overall System Integration. Electrical Communication (1993)Google Scholar
  8. 8.
    Jain, R.: The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. Wiley Interscience, Hoboken (1991)zbMATHGoogle Scholar
  9. 9.
    Wang, T., Yang, B., Gao, J., Yang, D.: Effective data distribution and reallocation strategies for fast query response in distributed query-intensive data environments. In: Zhang, Y., Yu, G., Bertino, E., Xu, G. (eds.) APWeb 2008. LNCS, vol. 4976, pp. 548–559. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Tengjiao Wang
    • 1
  • Bishan Yang
    • 1
  • Allen Huang
    • 2
  • Qi Zhang
    • 1
  • Jun Gao
    • 1
  • Dongqing Yang
    • 1
  • Shiwei Tang
    • 1
  • Jinzhong Niu
    • 3
  1. 1.Key Laboratory of High Confidence Software Technologies, Peking University,Ministry of Education, China School of Electronics Engineering and Computer SciencePeking UniversityBeijingChina
  2. 2.Microsoft SQL China R&D CenterChina
  3. 3.Computer Science, Graduate School and University CenterCity University of New YorkNew YorkUSA

Personalised recommendations