Skip to main content
SpringerLink
Log in

Reweighting Forest for Extreme Multi-label Classification

  • Conference paper
  • First Online:
Big Data Analytics and Knowledge Discovery (DaWaK 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10440))

Included in the following conference series:

Abstract

In recent years, data volume is getting larger along with the fast development of Internet technologies. Some datasets contain a huge number of labels, dimensions and data points. As a result, some of them cannot be loaded by typical classifiers, and some of them require very long and unacceptable time for execution. Extreme multi-label classification is designed for these challenges. Extreme multi-label classification differs from traditional multi-label classification in a number of ways including the need for lower execution time, training at an extreme scale with millions of data points, features and labels, etc. In order to enhance the practicality, in this paper, we focus on designing an extreme multi-label classification approach which can be performed on a single person-al computer. We devise a two-phase framework for dealing with the above issues. In the reweighting phase, the prediction precision is improved by paying more attention on hard-to-classify instances and increasing the diversity of the model. In the pretesting phase, trees with lower quality will be removed from the prediction model for reducing the model size and increasing the prediction precision. Experiments on real world datasets will verify that the pro-posed method is able to generate better prediction results and the model size is successfully shrunk down.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Jain, H., Prabhu, Y., Varma, M.: Extreme multi-label loss functions for recommendation, tagging, ranking & other missing label applications. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 935–944. ACM (2016)

    Google Scholar 

  2. Prabhu, Y., Varma, M.: FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 263–272. ACM (2014)

    Google Scholar 

  3. Bhatia, K., Jain, H., Kar, P., Varma, M., Jain, P.: Sparse local embeddings for extreme multi-label classification. In: Advances in Neural Information Processing Systems, pp. 730–738 (2015)

    Google Scholar 

  4. Babbar, R., Shoelkopf, B.: DiSMEC: distributed sparse machines for extreme multi-label classification. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 721–729. ACM (2017)

    Google Scholar 

  5. Yen, I.E., Huang, X., Zhong, K., Ravikumar, P., Dhillon, I.S.: PD-sparse: a primal and dual sparse approach to extreme multiclass and multilabel classification. In: Proceedings of The 33rd International Conference on Machine Learning, pp. 3069–3077. IEEE (2016)

    Google Scholar 

  6. Xu, C., Tao, D., Xu, C.: Robust extreme multi-label learning. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1275–1284. ACM (2016)

    Google Scholar 

  7. Yu, H.F., Jain, P., Kar, P., Dhillon, I.S.: Large-scale multi-label learning with missing labels. In: Proceedings of The 31st International Conference on Machine Learning, pp. 593–601. IEEE (2014)

    Google Scholar 

  8. Adnan, Md.N., Islam, Md.Z.: Forest CERN: a new decision forest building technique. In: Bailey, J., Khan, L., Washio, T., Dobbie, G., Huang, J.Z., Wang, R. (eds.) PAKDD 2016. LNCS, vol. 9651, pp. 304–315. Springer, Cham (2016). doi:10.1007/978-3-319-31753-3_25

    Chapter  Google Scholar 

  9. Adnan, Md N., Islam, Md Z.: On improving random forest for hard-to-classify records. In: Li, Jinyan, Li, Xue, Wang, Shuliang, Li, Jianxin, Sheng, Quan Z. (eds.) ADMA 2016. LNCS, vol. 10086, pp. 558–566. Springer, Cham (2016). doi:10.1007/978-3-319-49586-6_39

    Chapter  Google Scholar 

  10. Snoek, C.G., Worring, M., Van Gemert, J.C., Geusebroek, J.M., Smeulders, A.W.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of the 14th Annual ACM International Conference on Multimedia, pp. 421– 430. ACM (2006)

    Google Scholar 

  11. Katakis, I., Tsoumakas, G., Vlahavas, I: Multilabel text classification for automated tag suggestion. In: ECML/PKDD Discovery Challenge (2008)

    Google Scholar 

  12. Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective and efficient multilabel classification in domains with large number of labels. In: ECML/PKDD 2008 Workshop on Mining Multidimensional Data, pp. 30–44 (2008)

    Google Scholar 

  13. Mencia, E.L., Fürnkranz, J.: Efficient pairwise multilabel classification for large-scale problems in the legal domain. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 50–65 (2008)

    Google Scholar 

  14. McAuley, J., Leskovec, J.: Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM conference on Recommender systems, pp. 165–172. ACM (2013)

    Google Scholar 

  15. Zubiaga, A.: Enhancing navigation on wikipedia with social tags. Preprint (2012)

    Google Scholar 

  16. Partalas, I., Kosmopoulos, A., Baskiotis, N., Artieres, T., Paliouras, G., Gaussier, E., Androutsopoulos, I., Amini, M.-R., Galinari, P.: LSHTC: A benchmark for large-scale text classification. Preprint (2015)

    Google Scholar 

  17. The Extreme Classification Repository. http://research.microsoft.com/en-us/um/people/manik/downloads/XC/XMLRepository.html

  18. John, G.H., Langley, P.: Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338–345 (1995)

    Google Scholar 

  19. Safavian, S.R., Landgrebe, D.: A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybern. 21(3), 660–674 (1991)

    Article  MathSciNet  Google Scholar 

  20. Weston, J., Makadia, A., Yee, H.: Label partitioning for sublinear ranking. In: Proceeding of ICML, pp. 181–189 (2013)

    Google Scholar 

  21. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  22. Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9, 293–300 (1999)

    Article  MATH  Google Scholar 

  23. Al Bataineh, M., Al-Qudah, Z.: A novel gene identification algorithm with Bayesian classification. In: Biomedical Signal Processing and Control, pp. 6–15 (2017)

    Google Scholar 

  24. Goodman, K.E., Lessler, J., Cosgrove, S.E., Harris, A.D., Lautenbach, E., Han, J.H., Tamma, P.D.: A clinical decision tree to predict whether a bacteremic patient is infected with an extended-spectrum β-Lactamase–producing organism. In: Clinical Infectious Diseases, pp. 896–903 (2016)

    Google Scholar 

  25. Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017)

    Article  Google Scholar 

  26. Geng, Y., Chen, J., Fu, R., Bao, G., Pahlavan, K.: Enlighten wearable physiological monitoring systems: on-body RF characteristics based human motion classification using a support vector machine. IEEE Trans. Mobile Comput. 15, 656–671 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, No. 43, Section 4, Keelung Road, Daan District, Taipei, 106, Taiwan, ROC

    Zhun-Zheng Lin & Bi-Ru Dai

Authors
  1. Zhun-Zheng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bi-Ru Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bi-Ru Dai .

Editor information

Editors and Affiliations

  1. LIAS/ISAE-ENSMA, Chasseneuil, France

    Ladjel Bellatreche

  2. University of Texas at Arlington, Arlington, Texas, USA

    Sharma Chakravarthy

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Lin, ZZ., Dai, BR. (2017). Reweighting Forest for Extreme Multi-label Classification. In: Bellatreche, L., Chakravarthy, S. (eds) Big Data Analytics and Knowledge Discovery. DaWaK 2017. Lecture Notes in Computer Science(), vol 10440. Springer, Cham. https://doi.org/10.1007/978-3-319-64283-3_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64283-3_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64282-6

  • Online ISBN: 978-3-319-64283-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation