Abstract
Indexes are crucial for the efficient processing of database workloads and an appropriately selected set of indexes can drastically improve query processing performance. However, the selection of beneficial indexes is a non-trivial problem and still challenging. Recent work in deep reinforcement learning (DRL) may bring a new perspective on this problem. In this paper, we studied the index selection problem in the context of reinforcement learning and proposed an end-to-end DRL-based index selection framework. The framework poses the index selection problem as a series of 1-step single index recommendation tasks and can learn from data. Unlike most existing DRL-based index selection solutions that focus on selecting single-column indexes, our framework can recommend both single-column and multi-column indexes for the database. A set of comparative experiments with existing solutions was conducted to demonstrate the effectiveness of our proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chaudhuri, S., Datar, M., Narasayya, V.: Index selection for databases: a hardness study and a principled heuristic solution. IEEE Trans. Knowl. Data Eng. 16, 1313–1323 (2004). https://doi.org/10.1109/TKDE.2004.75
Schnaitter, K., Polyzotis, N., Getoor, L.: Index interactions in physical design tuning: modeling, analysis, and applications. Proc. VLDB Endow. 2, 1234–124512 (2009)
Lan, H., Bao, Z., Peng, Y.: A survey on advancing the DBMS query optimizer: cardinality estimation, cost model, and plan enumeration. Data Sci. Eng. 6(1), 86–101 (2021). https://doi.org/10.1007/s41019-020-00149-7
Ding, B., Das, S., Marcus, R., Wu, W., Chaudhuri, S., Narasayya, V.R.: AI meets AI: leveraging query executions to improve index recommendations. In: Proceedings of the 2019 International Conference on Management of Data - SIGMOD 2019, pp. 1241–1258. ACM Press, Amsterdam (2019). https://doi.org/10.1145/3299869.3324957
Chaudhuri, S., Narasayya, V.R.: An efficient cost-driven index selection tool for Microsoft SQL server. In: Proceedings of 23rd International Conference on Very Large Data Bases, VLDB 1997, August 25–29, 1997, Athens, Greece, pp. 146–155 (1997)
Valentin, G., Zuliani, M., Zilio, D.C., Lohman, G., Skelley, A.: DB2 advisor: an optimizer smart enough to recommend its own indexes. In: Proceedings of 16th International Conference on Data Engineering (Cat. No. 00CB37073), pp. 101–110. IEEE Computer Society, San Diego, CA, USA (2000). https://doi.org/10.1109/ICDE.2000.839397
Bruno, N., Chaudhuri, S.: Automatic physical database tuning: a relaxation-based approach. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, Baltimore, Maryland, USA, June 14–16, 2005, pp. 227–238 (2005). https://doi.org/10.1145/1066157.1066184
Sharma, A., Schuhknecht, F.M., Dittrich, J.: The case for automatic database administration using deep reinforcement learning. arXiv:1801.05643 [cs] (2018)
Paludo Licks, G., Colleoni Couto, J., de Fátima Miehe, P., de Paris, R., Dubugras Ruiz, D., Meneguzzi, F.: SmartIX: a database indexing agent based on reinforcement learning. Appl. Intell. 50(8), 2575–2588 (2020). https://doi.org/10.1007/s10489-020-01674-8
Lan, H., Bao, Z., Peng, Y.: An index advisor using deep reinforcement learning. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2105–2108. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3340531.3412106
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2018)
Knuth, D.E.: Generating all combinations and partitions, vol. 4, fascicle 3 of the art of computer programming (2005)
Irpan, A.: Deep reinforcement learning doesn’t work yet (2018). https://www.alexirpan.com/2018/02/14/rl-hard.html
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv:1707.06347 [cs] (2017)
Chaudhuri, S., Narasayya, V.: AutoAdmin “What-if” index analysis utility. In: Proceedings of the 1998 ACM SIGMOD International Conference on Management of Data, pp. 367–378. ACM, New York (1998). https://doi.org/10.1145/276304.276337
Kossmann, J., Halfpap, S., Jankrift, M., Schlosser, R.: Magic mirror in my hand, which is the best in the land? An experimental evaluation of index selection algorithms, vol. 14 (2020)
Acknowledgments
This work is supported by the National Natural Science Foundation of China (No. U1811263) and CCF-Huawei Database System Innovation Research Plan (CCF-HuaweiDBIR003A).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lai, S., Wu, X., Wang, S., Peng, Y., Peng, Z. (2021). Learning an Index Advisor with Deep Reinforcement Learning. In: U, L.H., Spaniol, M., Sakurai, Y., Chen, J. (eds) Web and Big Data. APWeb-WAIM 2021. Lecture Notes in Computer Science(), vol 12859. Springer, Cham. https://doi.org/10.1007/978-3-030-85899-5_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-85899-5_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85898-8
Online ISBN: 978-3-030-85899-5
eBook Packages: Computer ScienceComputer Science (R0)