Utilizing Hierarchies in Tree-Based Online Structured Output Prediction

  • Aljaž OsojnikEmail author
  • Panče Panov
  • Sašo Džeroski
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11828)


Methods for online prediction of structured values are becoming more and more popular. However, hierarchical prediction, which has recently been shown to produce good results in terms of predictive performance in the batch learning setting, has not yet been applied in the online learning setting. We address the recently introduced task of hierarchical multi-target regression. To this end, we propose a hierarchical extension of iSOUP-Tree, which can address online multi-target regression. The extension weighs the split evaluation heuristic according to the location of the targets in the hierarchy. We design the experimental setup to ascertain whether the additional information contained in the hierarchy can be utilized to improve the predictive performance in the leaf targets. The proposed method shows promising results, producing potential improvements that should be investigated further.


Online hierarchical prediction Hierarchical multi-target regression 



This work is supported by grants funded by the Slovenian Research Agency (P2-0103, N2-0056, J2-9230) and grants funded by the European Commission (H2020 785907 HBP SGA2, H2020 635201 LANDMARK, H2020 769661 SAAM, H2020 833671 RESILOC).


  1. 1.
    Barutcuoglu, Z., Schapire, R.E., Troyanskaya, O.G.: Hierarchical multi-label prediction of gene function. Bioinformatics 22(7), 830–836 (2006)CrossRefGoogle Scholar
  2. 2.
    Blockeel, H., Schietgat, L., Struyf, J., Džeroski, S., Clare, A.: Decision trees for hierarchical multilabel classification: a case study in functional genomics. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) PKDD 2006. LNCS (LNAI), vol. 4213, pp. 18–29. Springer, Heidelberg (2006). Scholar
  3. 3.
    Breskvar, M., et al.: Predicting thermal power consumption of the Mars Express satellite with machine learning. In: 6th International Conference on Space Mission Challenges for Information Technology (SMC-IT 2017), pp. 88–93. IEEE (2017)Google Scholar
  4. 4.
    Domingos, P., Hulten, G.: Mining high-speed data streams. In: International Conference on Knowledge Discovery and Data Mining (KDD 2000), pp. 71–80. ACM (2000)Google Scholar
  5. 5.
    Duarte, J., Gama, J.: Multi-target regression from high-speed data streams with adaptive model rules. In: International Conference on Data Science and Advanced Analytics (DSAA 2015), pp. 1–10. IEEE (10 2015)Google Scholar
  6. 6.
    Fanaee-T, H., Gama, J.: Event labeling combining ensemble detectors and background knowledge. Prog. Artif. Intell. 2(2–3), 113–127 (2013)Google Scholar
  7. 7.
    Gama, J.: Knowledge Discovery from Data Streams. CRC Press, Boca Raton (2010)CrossRefGoogle Scholar
  8. 8.
    Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc. 58(301), 13–30 (1963)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Ikonomovska, E., Gama, J., Džeroski, S.: Incremental multi-target model trees for data streams. In: Symposium on Applied Computing (SAC 2011), pp. 988–993. ACM (2011)Google Scholar
  10. 10.
    Ikonomovska, E., Gama, J., Džeroski, S.: Learning model trees from evolving data streams. Data Min. Knowl. Disc. 23(1), 128–168 (2011)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Mileski, V.: Tree methods for hierarchical multi-target regression. Master’s thesis, Jožef Stefan International Postgraduate School, Ljubljana, Slovenia (2017)Google Scholar
  12. 12.
    Mileski, V., Džeroski, S., Kocev, D.: Predictive clustering trees for hierarchical multi-target regression. In: Adams, N., Tucker, A., Weston, D. (eds.) IDA 2017. LNCS, vol. 10584, pp. 223–234. Springer, Cham (2017). Scholar
  13. 13.
    Osojnik, A., Panov, P., Džeroski, S.: Multi-label classification via multi-target regression on data streams. Mach. Learn. 106(6), 745–770 (2017)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Osojnik, A., Panov, P., Džeroski, S.: Tree-based methods for online multi-target regression. J. Intell. Inf. Syst. 50(2), 315–339 (2017)CrossRefGoogle Scholar
  15. 15.
    Rousu, J., Saunders, C., Szedmak, S., Shawe-Taylor, J.: Kernel-based learning of hierarchical multilabel classification models. J. Mach. Learn. Res. 7, 1601–1626 (2006)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Stenger, B., Thayananthan, A., Torr, P.H., Cipolla, R.: Estimating 3D hand pose using hierarchical multi-label classification. Image Vis. Comput. 25(12), 1885–1894 (2007)CrossRefGoogle Scholar
  17. 17.
    Vens, C., Struyf, J., Schietgat, L., Džeroski, S., Blockeel, H.: Decision trees for hierarchical multi-label classification. Mach. Learn. 73(2), 185–214 (2008)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Knowledge TechnologiesJožef Stefan InstituteLjubljanaSlovenia
  2. 2.Jožef Stefan International Postgraduate SchoolLjubljanaSlovenia

Personalised recommendations