Reinforcement-Based Simultaneous Algorithm and Its Hyperparameters Selection

  • Valeria Efimova
  • Andrey FilchenkovEmail author
  • Anatoly Shalyto
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 794)


There exist many algorithms for data analysis, especially for classification problems. To solve data analysis problem, a proper algorithm should be chosen, and also its hyperparameters should be selected. In this paper we present a new method for the simultaneous selection of an algorithm and its hyperparameters. In order to do so, we reduced this problem to the multi-armed bandit problem. We consider an algorithm as an arm and algorithm hyperparameters search during a fixed time as the corresponding arm play. We also suggest a problem-specific reward function. We performed the experiments on 10 real datasets and compare the suggested method with the existing one implemented in Auto-WEKA. The results show that our method is significantly better in most cases and never worse than the Auto-WEKA.


Algorithm selection Hyperparameter optimization Multi-armed bandit Reinforcement learning 


  1. 1.
    Abdulrahman, S.M., Brazdil, P., van Rijn, J.N., Vanschoren, J.: Algorithm selection via meta-learning and sample-based active testing. In: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases; International Workshop on Meta-Learning and Algorithm Selection. University of Porto (2015)Google Scholar
  2. 2.
    Aha, D.W.: Generalizing from case studies: a case study. In: Proceedings of the 9th International Conference on Machine Learning, pp. 1–10 (1992)Google Scholar
  3. 3.
    Ali, S., Smith, K.A.: On learning algorithm selection for classification. Appl. Soft Comput. 6(2), 119–138 (2006). Scholar
  4. 4.
    Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Bergstra, J.S., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: Advances in Neural Information Processing Systems, pp. 2546–2554 (2011)Google Scholar
  6. 6.
    Bottou, L.: Online learning and stochastic approximations. On-line Learn. Neural Netw. 17(9), 142 (1998). Scholar
  7. 7.
    Brazdil, P.B., Soares, C., Da Costa, J.P.: Ranking learning algorithms: using IBL and meta-learning on accuracy and time results. Mach. Learn. 50(3), 251–277 (2003)CrossRefGoogle Scholar
  8. 8.
    Castiello, C., Castellano, G., Fanelli, A.M.: Meta-data: characterization of input features for meta-learning. In: Torra, V., Narukawa, Y., Miyamoto, S. (eds.) MDAI 2005. LNCS (LNAI), vol. 3558, pp. 457–468. Springer, Heidelberg (2005). Scholar
  9. 9.
    Filchenkov, A., Pendryak, A.: Datasets meta-feature description for recommending feature selection algorithm. In: Artificial Intelligence and Natural Language and Information Extraction, Social Media and Web Search FRUCT Conference (AINL-ISMW FRUCT), pp. 11–18. IEEE (2015).
  10. 10.
    Giraud-Carrier, C., Vilalta, R., Brazdil, P.: Introduction to the special issue on meta-learning. Mach. Learn. 54(3), 187–193 (2004). Scholar
  11. 11.
    Hastie, T., Tibshirani, R., Friedman, J., Franklin, J.: The elements of statistical learning: data mining, inference and rediction. Math. Intell. 27(2), 83–85 (2005). Scholar
  12. 12.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). Scholar
  13. 13.
    Hutter, F., Lücke, J., Schmidt-Thieme, L.: Beyond manual tuning of hyperparameters. KI-Künstliche Intell. 29(4), 329–337 (2015). Scholar
  14. 14.
    Jamieson, K., Talwalkar, A.: Non-stochastic best arm identification and hyperparameter optimization. JMLR 41, 240–248 (2015)Google Scholar
  15. 15.
    Leite, R., Brazdil, P., Vanschoren, J.: Selecting classification algorithms with active testing. In: Perner, P. (ed.) MLDM 2012. LNCS (LNAI), vol. 7376, pp. 117–131. Springer, Heidelberg (2012). Scholar
  16. 16.
    Mantovani, R.G., Rossi, A.L., Vanschoren, J., Carvalho, A.C.P.D.L., et al.: Meta-learning recommendation of default hyper-parameter values for SVMs in classifications tasks. In: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases; International Workshop on Meta-Learning and Algorithm Selection. University of Porto (2015)Google Scholar
  17. 17.
    Rodriguez, J.D., Perez, A., Lozano, J.A.: Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 569–575 (2010). Scholar
  18. 18.
    Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, pp. 2951–2959 (2012)Google Scholar
  19. 19.
    Strijov, V., Weber, G.W.: Nonlinear regression model generation using hyperparameter optimization. Comput. Math. Appl. 60(4), 981–988 (2010). Scholar
  20. 20.
    Sun, Q., Pfahringer, B.: Pairwise meta-rules for better meta-learning-based algorithm ranking. Mach. Learn. 93(1), 141–161 (2013). Scholar
  21. 21.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)zbMATHGoogle Scholar
  22. 22.
    Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-WEKA: automated selection and hyper-parameter optimization of classification algorithms. CoRR, abs/1208.3719 (2012).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Valeria Efimova
    • 1
  • Andrey Filchenkov
    • 1
    Email author
  • Anatoly Shalyto
    • 1
  1. 1.ITMO UniversitySt. PetersburgRussia

Personalised recommendations