Skip to main content
SpringerLink
Log in

Adaptive Splitting and Selection Algorithm for Regression

  • Published:
New Generation Computing Aims and scope Submit manuscript

Abstract

Developing system for regression tasks like predicting prices, temperature is not a trivial task. There are many of issues which must be addressed such as: selecting appropriate model, eliminating irrelevant inputs, removing noise, etc. Most of them can be solved by application of machine learning methods. Although most of them were developed for classification tasks, they can be successfully applied for regression too. Therefore, in this paper we present Adaptive Splitting and Selection for Regression algorithm, whose predecessor was successfully applied in many classification tasks. The algorithm uses ensemble techniques whose strength comes from exploring local competences of several predictors. This is achieved by decomposing input space into disjointed competence areas and establishing local ensembles for each area respectively. Learning procedure is implemented as a compound optimisation process solved by means of evolutionary algorithm. The performance of the system is evaluated in series of experiments carried on several benchmark datasets. Obtained results show that proposed algorithm is valuable option for those who look for regression method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., and Herrera, F., “KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework,” Multiple-Valued Logic and Soft Computing, 17, 2-3, pp. 255–287, 2011.

  2. Alpaydin, E., Introduction to Machine Learning (Second Edition), The MIT Press, 2010.

  3. Alpaydin E.: “Combined 5 x 2 cv f test for comparing supervised classification learning algorithms”. Neural Computation 11, 1885–1892 (1998)

    Article  Google Scholar 

  4. Assaad, M., Boné, R. and Cardot, H., “A new boosting algorithm for improved time-series forecasting with recurrent neural networks,” Inf. Fusion, 9, 1, pp. 41–55, January 2008.

  5. Avnimelech, R. and Intrator, N., “Boosting regression estimators,” Neural Comput., 11, 2, pp. 499–520, February 1999.

  6. Bache, K. and Lichman, M., UCI machine learning repository, 2013.

  7. Bäck, T., Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms, Oxford University Press, Oxford, UK, 1996.

  8. Basterrech, S., Mohammed, S., Rubino, G. and Soliman, M., “Levenberg-marquardt training algorithms for random neural networks,” The Computer Journal, 54, 1, pp. 125–135, November 2009.

  9. Basterrech, S. and Rubino, G., “Echo State Queueing Network: A new reservoir computing learning tool,” CCNC, pp. 118–123, IEEE, January 2013.

  10. Breiman, L., Friedman, J., Olshen, R. and Stone, C., Classification and Regression Trees, Wadsworth and Brooks, Monterey, CA, 1984.

  11. Breiman, L., “Bagging predictors,” Mach. Learn., 24, 2, pp. 123–140, August 1996.

  12. Brown, G., Wyatt, J. L. and Tiňo, P., “Managing diversity in regression ensembles,” J. Mach. Learn. Res. 6, pp. 1621–1650, December 2005.

  13. Coelho, G. P. and Von Zuben, F. J., “The influence of the pool of candidates on the performance of selection and combination techniques in ensembles,” in IJCNN, pp. 5132–5139. IEEE, 2006.

  14. Cost S., Salzberg S.: “A weighted nearest neighbor algorithm for learning with symbolic features”. Machine Learning 10, 57–78 (1993)

    Google Scholar 

  15. Demšar, J., “Statistical comparisons of classifiers over multiple data sets,” J. Mach. Learn. Res., 7, pp. 1–30, December 2006.

  16. Drucker, H., “Improving regressors using boosting techniques,” in Proc. of the Fourteenth International Conference on Machine Learning, ICML ’97, San Francisco, CA, USA, Morgan Kaufmann Publishers Inc., pp. 107–115, 1997.

  17. García, S., Fernández, A., Luengo, J. and Herrera, F., “Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power,” Inf. Sci., 180, 10, pp. 2044–2064, May 2010.

  18. Graczyk, M., Lasota, T., Telec, Z. and Trawiński, B., “A multi-agent system to assist with property valuation using heterogeneous ensembles of fuzzy models,” in Proc. of the 4th KES International Conference on Agent and Multi-agent Systems: Technologies and Applications, Part I, KES-AMSTA’10, Springer-Verlag, Berlin, Heidelberg, pp. 420–429, 2010.

  19. Hernández-Lobato, D., Martínez-Muñoz, G. and Suárez, A., “Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles,” Neurocomput., 74, 12-13, pp. 2250–2264, June 2011.

  20. Hernández-Lobato, D., Martínez-Muñoz, G. and Suárez, A., “Pruning in ordered regression bagging ensembles,” in Proc. of the International Joint Conference on Neural Networks, IJCNN 2006, part of the IEEE World Congress on Computational Intelligence, WCCI 2006, Vancouver, BC, Canada, 16-21 July 2006, pp. 1266–1273, 2006.

  21. Härdle, W., Applied Nonparametric Regression, Cambridge University Press, Cambridge, UK, 1990.

  22. Jackowski, K., Krawczyk, B. and Wozniak, M., “Application of adaptive splitting and selection classifier to the spam filtering problem,” Cybern. Syst., 44, 6-7, pp. 569–588, October 2013.

  23. Jackowski, K., Platos, J. and Prilepok, M., “Evolutionary weighted ensemble for eeg signal recognition,” Intelligent Data analysis and its Applications, Volume II, Advances in Intelligent Systems and Computing, 298 (Pan, J.-S., Snasel, V., Corchado, E. S., Abraham, A. and Wang, S.-L. eds.), Springer International Publishing, pp. 201–210, 2014.

  24. Jackowski, K. and Wozniak, M., “Adaptive splitting and selection method of classifier ensemble building,” in Hybrid Artificial Intelligence Systems, LNCS, 5572 (Corchado, E., Wu, X., Oja, E., Herrero, Á. and Baruque, B. eds.), Springer Berlin Heidelberg, pp. 525–532, 2009.

  25. Keerthi, S. S., Shevade, S. K., Bhattacharyya, C. and Murthy, K. R. K., “Improvements to platt’s smo algorithm for svm classifier design,” Neural Comput., 13, 3, pp. 637–649, March 2001.

  26. Mitchell, T. M., Machine Learning (1 edition), McGraw-Hill, Inc., New York, NY, USA, 1997.

  27. Rousseeuw, P. J. and Leroy, A. M., Robust regression and outlier detection, 1987.

  28. Shrestha, D. L. and Solomatine, D. P., Experiments with adaboost.rt, an improved boosting scheme for regression,” Neural Comput., 18, 7, pp. 1678–1710, July 2006.

  29. Silipo, R. and Mazanetz, M. P., The KNIME Cookbook, Recipes for the Advanced User, KNIME Press, 2012.

  30. Smola, A. J. and Schölkopf, B., “A tutorial on support vector regression,” Statistics and Computing, 14, 3, pp. 199–222, August 2004.

  31. Tsanas, A., Little, M. A., McSharry, P. E. and Ramig, L.O., “Accurate telemonitoring of parkinson’s disease progression by noninvasive speech tests,” Biomedical Engineering, IEEE Transactions on, 57, 4, pp. 884–893, April 2010.

  32. Tufekci P.: “Prediction of full load electrical power output of a base load operated combined cycle power plant using machine learning methods”. International Journal of Electrical Power and Energy Systems 60, 126–140 (2014)

    Article  Google Scholar 

  33. Tyree, S., Weinberger, K. Q., Agrawal, K. and Paykin, J., “Parallel boosted regression trees for web search ranking,” in Proc. of the 20th International Conference on World Wide Web, WWW ’11, pp. 387–396, ACM, New York, NY, USA, 2011.

  34. Wang, Y. and Witten, I. H., “Modeling for optimal probability prediction,” ICML ’02 Proc. of the Nineteenth International Conference on Machine Learning, pp. 650–657, Sydney, Australia, 2002.

  35. Yeh, I.-C., “Modeling of strength of high performance concrete using artificial neural networks,” Cement and Concrete Research, 28, 12, Elsevier, pp. 1797–1808, 1998.

  36. Zhou, Z.-H., Wu, J. and Tang, W., “Ensembling neural networks: Many could be better than all,” Artif. Intell., 137, 1-2, pp. 239–263, May 2002.

Download references

Author information

Authors and Affiliations

  1. Department of Systems and Computer Networks, Wroclaw University of Technology, Wyb. Wyspianskiego 27, 50-370, Wrocław, Poland

    Konrad Jackowski

Authors
  1. Konrad Jackowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Konrad Jackowski.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jackowski, K. Adaptive Splitting and Selection Algorithm for Regression. New Gener. Comput. 33, 425–448 (2015). https://doi.org/10.1007/s00354-015-0405-1

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00354-015-0405-1

Keywords

Search

Navigation