Abstract
Whether a model is utilizing classical machine learning (ML) algorithms (e.g., Support Vector Machines, KNN, etc.) or neural architectures (e.g., Deep Learning), one must choose a method for selecting optimal hyperparameters. There are both manual and automated methods for hyperparameter optimization. Automated methods are generally referred to as automated machine learning, or AutoML [1]. Several automated selection algorithms have shown similar or improved performance over state-of-the-art methods [2–6]. This breakthrough has led to the development of cloud-based services like Google AutoML [8], which is based on Deep Learning [9] and is widely considered to be the industry leader in AutoML services. We use a fundamentally different type of neural architecture: Extreme Learning Machines (ELMs) [7–13]. ELMs are generally less computationally expensive than Deep Learning. We benchmark our Extreme AutoML technology versus Google’s AutoML using four popular classification datasets from the University of California at Irvine’s (UCI) repository [27]. We observe significant advantages for Extreme AutoML in accuracy, Jaccard Indices [28], the variance of Jaccard Indices across classes (i.e. class variance) and training times.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yao, Q., Wang, M., Chen, Y., Dai, W., Yi-Qi, H., Yu-Feng, L., Wei-Wei, T., Qiang, Y., Yang, Y.: Taking human out of learning applications: a survey on automated machine learning. arXiv preprint arXiv:1810.13306 (2018)
Zoph, B., Quoc, V.L.: Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016)
Pham, H., Guan, M., Zoph, M., Le, Q., Dean, J.: Efficient neural architecture search via parameters sharing. In: International Conference on Machine Learning, pp. 4095–4104. PMLR (2018)
Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, vol. 25 (2012)
He, X., Zhao, K., Chu, X.: AutoML: a survey of the state-of-the-art. Knowl.-Based Syst. 212, 106622 (2021)
Waring, J., Lindvall, C., Umeton, R.: Automated machine learning: review of the state-of-the-art and opportunities for healthcare. Artif. Intell. Med. 104, 101822 (2020)
Komer, B., Bergstra, J., and Eliasmith, C.: Hyperopt-sklearn: automatic hyperparameter configuration for scikit-learn. In: ICML Workshop on AutoML, vol. 9. Citeseer, Austin (2014)
Google AutoML homepage. https://cloud.google.com/automl/. Accessed 14 Feb 2022
Ivakhnenko, A.G., Ivakhnenko, A.G., Lapa, V.G., Lapa, V.G.: Cybernetics and Forecasting Techniques, vol. 8. American Elsevier Publishing Company (1967)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541), pp. 985–990 (2004)
Cambria, E., et al.: Extreme learning machines [trends and controversies]. IEEE Intell. Syst. 28(6), 30–59 (2013)
Miche, Y., Heeswijk, M., Bas, P., Simula, O., Lendasse, A.: TROP-ELM: a double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 74(16), 2413–2421 (2011)
Lendasse, A., Man, V.C., Miche, Y., Huang, G.B.: Advances in extreme learning machines (ELM2014). Neurocomputing 174, 1–3 (2016)
Akusok, A., Miche, Y., Hegedus, J., Nian, R., Lendasse, A.: A two-stage methodology using K-NN and false-positive minimizing ELM for nominal data classification. Cogn. Comput. 6(3), 432–445 (2014)
Lauren, P., Qu, G., Yang, J., Watta, P., Huang, G.B., Lendasse, A.: Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks. Cogn. Comput. 10(4), 625–638 (2018)
Miche, Y., Sorjamaa, A., Bas, P., Simula, O., Jutten, C., Lendasse, A.: OP-ELM: optimally pruned extreme learning machine. IEEE Trans. Neural Netw. 21(1), 158–162 (2009)
Khan, K., Ratner, E., Ludwig, R., Lendasse, A.: feature bagging and extreme learning machines: machine learning with severe memory constraints. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–7 (2020)
Liu, N., Wang, H.: Ensemble based extreme learning machine. Signal Process. Lett. 754–757 (2010). IEEE
Van Heeswijk, M., Miche, Y., Lendasse, A.: GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 74(16), 2430–2437 (2011)
Grigorievskiy, A., Miche, Y., Ventela, A.M., Severin, E., Lendasse, A.: Long-term time series prediction using OP-ELM. Neural Netw. 51, 50–56 (2014)
Yu, Q., Van Heeswijk, M., Miche, Y., Nian, R., Severin, E., Lendasse, A.: Ensemble delta test-extreme learning machine (DT-ELM) for regression. Neurocomputing 129, 153–158 (2014)
Van Heeswijk, M., Miche, Y., Lindh-Knuutila, T., Hilbers, P.A.J., Honkela, T., Oja, R., Lendasse, A.: Adaptive ensemble models of extreme learning machines for time series prediction. In: International Conference on Artificial Neural Networks, pp. 305–314. Springer, Berlin (2009)
Song, Y., et al.: Gaussian derivative models and ensemble extreme learning machine for texture image classification. Neurocomputing 277, 53–64 (2018)
Yu, Q., Miche, Y., Severin, E., Lendasse, A.: Bankruptcy prediction using extreme learning machine and financial expertise. Neurocomputing 128, 296–302 (2014)
Han, B., Miche, Y., Severin, E., Lendasse, A.: LARSEN-ELM: selective ensemble of extreme learning machines using LARS for blended data. Neurocomputing 149, 285–294 (2015)
Lan, Y., Yeng, C.S., Huang, G.B.: Ensemble of online sequential extreme learning machine. Neurocomputing 72(13–15), 3391–3395 (2009)
Dua, D., Graff, C.: UCI machine learning repository. http://archive.ics.uci.edu/ml. University of California, School of Information and Computer Science, Irvine (2019)
Fletcher, S., Islam, Z.: Comparing sets of patterns with the Jaccard index. Australas. J. Inf. Syst. 22 (2018)
Anguita, D., Ghio, A., Oneto, L., Parra Perez, X., Reyes Ortiz, J.L.: A public domain dataset for human activity recognition using smartphones. In: Proceedings of the 21th International European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp. 437–442 (2013)
Sakar, C.O., et al.: A comparative analysis of speech signal processing algorithms for Parkinson’s disease classification and the use of the tunable Q-factor wavelet transform. Appl. Soft Comput. 74, 255–263 (2019)
Ballabio, D., Grisoni, F., Consonni, V., Todeschini, R.: Integrated QSAR models to predict acute oral systemic toxicity. Mol. Inf. 38 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Warner, B., Ratner, E., Lendasse, A. (2023). Edammo’s Extreme AutoML Technology – Benchmarks and Analysis. In: Björk, KM. (eds) Proceedings of ELM 2021. ELM 2021. Proceedings in Adaptation, Learning and Optimization, vol 16. Springer, Cham. https://doi.org/10.1007/978-3-031-21678-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-21678-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21677-0
Online ISBN: 978-3-031-21678-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)