Evolutionary Optimized Forest of Regression Trees: Application in Metallurgy

  • Mirosław Kordos
  • Jerzy Piotrowski
  • Szymon Bialka
  • Marcin Blachnik
  • Slawomir Golak
  • Tadeusz Wieczorek
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7208)


A forest of regression trees is generated, with each tree using a different randomly chosen subset of data. Then the forest is optimized in two ways. First each tree independently by shifting the split points to the left or to the right to compensate for the fact, that the original split points were set up as being optimal only for the given node and not for the whole tree. Then evolutionary algorithms are used to exchange particular tree subnodes between different trees in the forest. This leads to the best single tree, which although may produce not better results than the forest, but can generate comprehensive logical rules that are very important in some practical applications. The system is currently being applied in the optimization of metallurgical processes.


Decision tree regression evolutionary optimization logical rules 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kordos, M., Blachnik, M., Perzyk, M., Kozłowski, J., Bystrzycki, O., Gródek, M., Byrdziak, A., Motyka, Z.: A Hybrid System with Regression Trees in Steel-Making Process. In: Corchado, E., Kurzyński, M., Woźniak, M. (eds.) HAIS 2011, Part I. LNCS, vol. 6678, pp. 222–230. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Duch, W., Setiono, R., Zurada, J.: Computational intelligence methods for understanding of data. Proceedings of the IEEE 92(5), 771–805 (2008)CrossRefGoogle Scholar
  3. 3.
    Jankowski, N., Grabczewski, K.: Heterogenous Committees with Competence Analysis. In: Fifth International Conference on Hybrid Intelligent Systems, Rio de Janeiro, Brasil, pp. 417–422 (2005)Google Scholar
  4. 4.
    Abraham, A., Corchado, E., Corchado, J.M.: Hybrid learning machines. Neurocomputing 72(13-15), 2729–2730 (2009)CrossRefGoogle Scholar
  5. 5.
    Kordos, M., Blachnik, M., Wieczorek, T., Golak, S.: Neural Network Committees Optimized with Evolutionary Methods for Steel Temperature Control. In: Jędrzejowicz, P., Nguyen, N.T., Hoang, K. (eds.) ICCCI 2011, Part I. LNCS, vol. 6922, pp. 42–51. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Grąbczewski, K., Duch, W.: Heterogeneous Forests of Decision Trees. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, pp. 504–509. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Kordos, M., Blachnik, M., Wieczorek, T.: Evolutionary Optimization of Regression Model Ensembles in Steel-Making Process. In: Yin, H., Wang, W., Rayward-Smith, V. (eds.) IDEAL 2011. LNCS, vol. 6936, pp. 369–376. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  8. 8.
    Setiono, R., Thong, J.: An approach to generate rules from neural networks for regression problems. European Journal of Operational Research 155(1) (2004)Google Scholar
  9. 9.
    Tresp, V.: Committee Machines. Handbook for Neural Network Signal Processing. CRC Press (2001)Google Scholar
  10. 10.
    Breiman, L.: Combining predictors. In: Sharkey, A.J.C. (ed.) Combining Artificial Neural Nets. Springer, Heidelberg (1999)Google Scholar
  11. 11.
    Barbosa, B.H.G., Bui, L.T., Abbass, H.A., Aguirre, L.A., Braga, A.P.: Evolving an Ensemble of Neural Networks Using Artificial Immune Systems. In: Li, X., Kirley, M., Zhang, M., Green, D., Ciesielski, V., Abbass, H.A., Michalewicz, Z., Hendtlass, T., Deb, K., Tan, K.C., Branke, J., Shi, Y. (eds.) SEAL 2008. LNCS, vol. 5361, pp. 121–130. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  12. 12.
    Chen, H., Yao, X.: Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning. IEEE Trans. on Knowledge and Data Engineering 22, 1738–1751 (2010)CrossRefGoogle Scholar
  13. 13.
    Corchado, E., et al.: Hybrid intelligent algorithms and applications. Information Science 180(14), 2633–2634 (2010)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Quinlan, J.: Learning with Continuous Classes. In: Proc. of AI 1992, pp. 343–348. World Scientific, Singapore (1992)Google Scholar
  15. 15.
    Malerba, D., Esposito, F., Ceci, M., Appice, A.: Top-down Induction of Model Trees with Regression and Splitting Nodes. IEEE Transactions on PAMI 26(5), 612–625 (2004)CrossRefGoogle Scholar
  16. 16.
    Czajkowski, M., Kretowski, M.: An Evolutionary Algorithm for Global Induction of Regression Trees with Multivariate Linear Models. In: Kryszkiewicz, M., Rybinski, H., Skowron, A., Raś, Z.W. (eds.) ISMIS 2011. LNCS(LNAI), vol. 6804, pp. 230–239. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  17. 17.
    Potts, D., Sammut, C.: Incremental Learning of Linear Model Trees. Machine Learning 62, 5–48 (2005)CrossRefGoogle Scholar
  18. 18.
    Blachnik, M., Mączka, K., Wieczorek, T.: A Model for Temperature Prediction of Melted Steel in the Electric Arc Furnace (EAF). In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010. LNCS, vol. 6114, pp. 371–378. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  19. 19.
  20. 20.
    Landwehr, N., Hall, M., Frank, E.: Logistic Model Trees. Machine Learning 95 (2005)Google Scholar
  21. 21.
    Zelinka, I., et al.: Evolutionary Algorithms and Chaotic systems. Springer, Heidelberg (2010)zbMATHCrossRefGoogle Scholar
  22. 22.
    Zelinka, I., Senkerik, R., Oplatkova, Z.: Evolutionary Scanning and Neural Network Optimization. In: 19th International Conference on Database and Expert Systems Application, DEXA, pp. 576–582 (2008)Google Scholar
  23. 23.
    Tsutsui, S., et al.: Multi-parent Recombination with Simplex Crossover in Real Coded Genetic Algorithms. In: The 1999 Genetic and Evolutionary Computation Conference, pp. 657–664 (1999)Google Scholar
  24. 24.
    Semya, E., et al.: Multiple crossover genetic algorithm for the multiobjective traveling salesman problem. Electronic Notes in Discrete Mathematics 36, 939–946 (2010)CrossRefGoogle Scholar
  25. 25.
    Kordos, M., Duch, W.: Variable Step Search Algorithm for Feedforward Networks. Neurocomputing 71(13-15), 2470–2480 (2008)CrossRefGoogle Scholar
  26. 26.
    Blake, C., Keogh, E., Merz, C.: UCI Repository of Machine Learning Databases (1998-2011),

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Mirosław Kordos
    • 1
  • Jerzy Piotrowski
    • 1
  • Szymon Bialka
    • 1
  • Marcin Blachnik
    • 2
  • Slawomir Golak
    • 2
  • Tadeusz Wieczorek
    • 2
  1. 1.Department of Mathematics and Computer ScienceUniversity of Bielsko-BialaBielsko-BialaPoland
  2. 2.Department of Management and InformaticsSilesian University of TechnologyKatowicePoland

Personalised recommendations