Abstract
Query processing on cloud database systems is a challenging problem due to the dynamic cloud environment. The configuration and utilization of the distributed hardware used to process queries change continuously. A query optimizer aims to generate query execution plans (QEPs) that are optimal meet user requirements. In order to achieve such QEPs under dynamic environments, performing query re-optimizations during query execution has been proposed in the literature. In cloud database systems, besides query execution time, users also consider the monetary cost to be paid to the cloud provider for executing queries. Thus, such query re-optimizations are multi-objective optimizations which take both time and monetary costs into consideration. However, traditional re-optimization requires accurate cost estimations, and obtaining these estimations adds overhead to the system, and thus causes negative impacts on query performance. To fill this gap, in this paper, we introduce ReOptRL, a novel query processing algorithm based on deep reinforcement learning. It bootstraps a QEP generated by an existing query optimizer and dynamically changes the QEP during the query execution. It also keeps learning from incoming queries to build a more accurate optimization model. In this algorithm, the QEP of a query is adjusted based on the recent performance of the same query so that the algorithm does not rely on cost estimations. Our experiments show that the proposed algorithm performs better than existing query optimization algorithms in terms of query execution time and query execution monetary costs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Barata, M., Bernardino, J., Furtado, P.: An overview of decision support benchmarks: TPC-DS, TPC-H and SSB. In: Rocha, A., Correia, A.M., Costanzo, S., Reis, L.P. (eds.) New Contributions in Information Systems and Technologies. AISC, vol. 353, pp. 619–628. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16486-1_61
Bruno, N., Jain, S., Zhou, J.: Continuous cloud-scale query optimization and processing. Proc. VLDB Endow. 6(11), 961–972 (2013)
Helff, F., Gruenwald, L., d’Orazio, L.: Weighted sum model for multi-objective query optimization for mobile-cloud database environments. In: Proceedings of the Workshops of the EDBT/ICDT 2016 Joint Conference, vol. 1558 (2016)
Kabra, N., DeWitt, D.J.: Efficient mid-query re-optimization of sub-optimal query execution plans. In: SIGMOD 1998, pp.106–117 (1998)
Kipf, A., Kipf, T., Radke, B., Leis, V., Boncz, P.A., Kemper, A.: Learned cardinalities: estimating correlated joins with deep learning. In: 9th Biennial Conference on Innovative Data Systems Research, CIDR 2019 (2019)
Liu, H., Xu, M., Yu, Z., Corvinelli, V., Zuzarte, C.: Cardinality estimation using neural networks. In: CASCON 2015, pp. 53–59. IBM Corp. (2015)
Marcus, R., et al.: Neo: a learned query optimizer. Proc. VLDB Endow. 12(11), 1705–1718 (2019)
Marcus, R., Papaemmanouil, O.: Deep reinforcement learning for join order enumeration. In: aiDM 2018 (2018)
Markl, V., Raman, V., Simmen, D., Lohman, G., Pirahesh, H., Cilimdzic, M.: Robust query processing through progressive optimization. In: Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data, SIGMOD 2004, pp. 659–670 (2004)
Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press (2012)
Ohnishi, S., Uchibe, E., Yamaguchi, Y., Nakanishi, K., Yasui, Y., Ishii, S.: Constrained deep q-learning gradually approaching ordinary q-learning. Front. Neurorobot. 13, 103 (2019)
Ortiz, J., Balazinska, M., Gehrke, J., Keerthi, S.S.: Learning state representations for query optimization with deep reinforcement learning. In: Proceedings of the 2nd Workshop on Data Management for End-To-End Machine Learning, DEEM 2018 (2018)
Park, Y., Tajik, A.S., Cafarella, M., Mozafari, B.: Database learning: toward a database that becomes smarter every time. In: Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD 2017, pp. 587–602 (2017)
PostgreSQL (2021). https://www.postgresql.org/
Stillger, M., Lohman, G.M., Markl, V., Kandil, M.: LEO, DB2’s learning optimizer. In: Proceedings of the 27th International Conference on Very Large Data Bases, VLDB 2001, pp. 19–28 (2001)
Trummer, I., Moseley, S., Maram, D., Jo, S., Antonakakis, J.: SkinnerDB: regret-bounded query evaluation via reinforcement learning. Proc. VLDB Endow. 11(12), 2074–2077 (2018)
Wang, C., Arani, Z., Gruenwald, L., d’Orazio, L.: Adaptive time, monetary cost aware query optimization on cloud database systems. In: IEEE International Conference on Big Data, Big Data 2018, pp. 3374–3382. IEEE (2018)
Wang, W., Zhang, M., Chen, G., Jagadish, H.V., Ooi, B.C., Tan, K.L.: Database meets deep learning: Challenges and opportunities. SIGMOD Rec. 45(2), 17–22 (2016)
Wiering, M., van Otterlo, M.: Reinforcement Learning: State-of-the-Art. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-27645-3
Wu, C., et al.: Towards a learning optimizer for shared clouds. Proc. VLDB Endow. 12(3), 210–222 (2018)
Wu, W., Naughton, J.F., Singh, H.: Sampling-based query re-optimization. In: SIGMOD 2016, pp. 1721–1736. Association for Computing Machinery (2016)
Acknowledgements
This work is partially supported by the National Science Foundation Award No. 1349285.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, C., Gruenwald, L., d’Orazio, L., Leal, E. (2021). Cloud Query Processing with Reinforcement Learning-Based Multi-objective Re-optimization. In: Attiogbé, C., Ben Yahia, S. (eds) Model and Data Engineering. MEDI 2021. Lecture Notes in Computer Science(), vol 12732. Springer, Cham. https://doi.org/10.1007/978-3-030-78428-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-78428-7_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78427-0
Online ISBN: 978-3-030-78428-7
eBook Packages: Computer ScienceComputer Science (R0)