High-throughput query scheduling with spatial clustering based on distributed exponential moving average
In distributed scientific query processing systems, leveraging distributed cached data is becoming more important. In such systems, a front-end query scheduler distributes queries among many application servers rather than processing queries in a few high-performance workstations. Although many query scheduling policies exist such as round-robin and load-monitoring, they are not sophisticated enough to exploit cached results as well as balance the workload. Efforts were made to improve the query processing performance using statistical methods such as exponential moving average. However, existing methods have limitations for certain query patterns: queries with hotspots, or dynamic query distributions. In this paper, we propose novel query scheduling policies that take into account both the contents of distributed caching infrastructure and the load balance among the servers. Our experiments show that the proposed query scheduling policies outperform existing policies by producing better query plans in terms of load balance and cache-hit ratio.
KeywordsDistributed query scheduling Multiple query optimization Spatial clustering Cache aware load balancing
This research was supported by PLSI resources, 1.100027.01 Research Fund of the UNIST (Ulsan National Institute of Science and Technology), and 2.110147.01 National Research Foundation of Korea. This work was also supported by 2011 Research Fund of Myongji University.
- 1.Andrade, H., Kurc, T., Sussman, A., Saltz, J.: Multiple query optimization for data analysis applications on clusters of SMPs. In: Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid). IEEE Comput. Soc., Los Alamitos (2002) Google Scholar
- 2.Aron, M., Sanders, D., Druschel, P., Zwaenepoel, W.: Scalable content-aware request distribution in cluster-based network servers. In: Proceedings of Usenix Annual Technical Conference (2000) Google Scholar
- 4.de Berg, M., Cheong, O., van Kreveld, M., Overmars, M.: Computational Geometry, Algorithms and Applications. Springer, Berlin (1998) Google Scholar
- 5.Chen, F.C.F., Dunham, M.H.: Common subexpression processing in multiple-query processing. Transactions on Knowledge and Data Engineering 10(5), 493–499 (199) Google Scholar
- 6.Chou, Y.l.: Statistical Analysis. Holt International (1975) Google Scholar
- 8.Menasce, D.A., Almeida, V.A.F.: Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning. Prentice Hall, New York (2000) Google Scholar
- 10.Pai, V., Aron, M., Banga, G., Svendsen, M., Druschel, P., Zwaenepoel, W., Nahum, E.: Locality-aware request distribution in cluster-based network servers. In: Proceedings of ACM ASPLOS (1998) Google Scholar
- 14.Xiong, X., Mokbel, M.F., Aref, W.G., Hambrusch, S.E., Prabhakar, S.: Scalable spatio-temporal continuous query processing for location-aware services. In: Proceedings of 16th International Conference on Scientific and Statistical Database Management (SSDBM) (2004) Google Scholar
- 15.Zhang, K., Andrade, H., Raschid, L., Sussman, A.: Query planning for the Grid: adapting to dynamic resource availability. In: Proceedings of the 5th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid), Cardiff, UK (2005) Google Scholar