Combining Dimensionality Reduction with Random Forests for Multi-label Classification Under Interactivity Constraints

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10235)


Learning from multi-label data in an interactive framework is a challenging problem as algorithms must withstand some additional constraints: in particular, learning from few training examples in a limited time. A recent study of multi-label classifier behaviors in this context has identified the potential of the ensemble method “Random Forest of Predictive Clustering Trees” (RF-PCT). However, RF-PCT has shown a degraded performance in terms of computation time for large feature spaces. To overcome this limit, this paper proposes a new hybrid multi-label learning approach IDSR-RF (Independent Dual Space Reduction with RF-PCT) which first reduces the data dimension and then learns a predictive regression model in the reduced spaces with RF-PCT. The feature and the label spaces are independently reduced using the fast matrix factorization algorithm Gravity. The experimental results on nine high-dimensional datasets show that IDSR-RF significantly reduces the computation time without deteriorating the learning performances. To the best of our knowledge, it is currently the most promising learning approach for an interactive multi-label learning system.


Root Mean Square Error Unlabelled Data Interactive Framework Label Space Label Ranking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Amershi, S., Cakmak, M., Knox, W.B., Kulesza, T.: Power to the people: the role of humans in interactive machine learning. AI Mag. 35(4), 105–120 (2014)Google Scholar
  2. 2.
    Amershi, S., Fogarty, J., Weld, D.: Regroup: interactive machine learning for on-demand group creation in social networks. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 21–30. ACM, New York (2012)Google Scholar
  3. 3.
    Bennett, J., Lanning, S.: The netflix prize. In: Proceedings of KDD Cup and Workshop, vol. 2007, p. 35 (2007)Google Scholar
  4. 4.
    Dabrowski, J.R., Munson, E.V.: Is 100 milliseconds too fast? In: CHI 2001 Extended Abstracts on Human Factors in Computing Systems, pp. 317–318. ACM, New York (2001)Google Scholar
  5. 5.
    Drucker, S.M., Fisher, D., Basu, S.: Helping users sort faster with adaptive machine learning recommendations. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6948, pp. 187–203. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23765-2_13 CrossRefGoogle Scholar
  6. 6.
    Fogarty, J., Tan, D., Kapoor, A., Winder, S.: Cueflik: interactive concept learning in image search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 29–38. ACM, New York (2008)Google Scholar
  7. 7.
    Hsu, D., Kakade, S., Langford, J., Zhang, T.: Multi-label prediction via compressed sensing. In: Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009, Proceedings of a Meeting Held 7–10 December, pp. 772–780. Vancouver, British Columbia (2009)Google Scholar
  8. 8.
    Kocev, D., Vens, C., Struyf, J., Džeroski, S.: Ensembles of multi-objective decision trees. In: Kok, J.N., Koronacki, J., Mantaras, R.L., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 624–631. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-74958-5_61 CrossRefGoogle Scholar
  9. 9.
    Madjarov, G., Kocev, D., Gjorgjevikj, D., Dzeroski, S.: An extensive experimental comparison of methods for multi-label learning. Pattern Recogn. 45(9), 3084–3104 (2012)CrossRefGoogle Scholar
  10. 10.
    Nair-Benrekia, N.Y., Kuntz, P., Meyer, F.: Learning from multi-label data with interactivity constraints: an extensive experimental study. Expert Syst. Appl. 42(13), 5723–5736 (2015)CrossRefGoogle Scholar
  11. 11.
    Pacharawongsakda, E., Theeramunkong, T.: A comparative study on single and dual space reduction in multi-label classification. In: Skulimowski, A.M.J., Kacprzyk, J. (eds.) Knowledge, Information and Creativity Support Systems: Recent Trends, Advances and Solutions. AISC, vol. 364, pp. 389–400. Springer, Cham (2016). doi: 10.1007/978-3-319-19090-7_29 CrossRefGoogle Scholar
  12. 12.
    Read, J.: Scalable multi-label classification. Ph.D. thesis. University of Waikato (2010)Google Scholar
  13. 13.
    Shu, X., Lai, D., Xu, H., Tao, L.: Learning shared subspace for multi-label dimensionality reduction via dependence maximization. Neurocomputing 168, 356–364 (2015)CrossRefGoogle Scholar
  14. 14.
    Takacs, G., Pilaszy, I., Nemeth, B., Tikk, D.: On the gravity recommendation system. In: Proceedings of KDD Cup and Workshop, vol. 2007 (2007)Google Scholar
  15. 15.
    Yu, K., Yu, S., Tresp, V.: Multi-label informed latent semantic indexing. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 258–265. ACM, New York (2005)Google Scholar
  16. 16.
    Zhang, M.L., Zhou, Z.H.: Ml-KNN: a lazy learning approach to multi-label learning. Pattern Recogn. 40(7), 2038–2048 (2007)CrossRefzbMATHGoogle Scholar
  17. 17.
    Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)CrossRefGoogle Scholar
  18. 18.
    Zhang, Y., Zhou, Z.H.: Multilabel dimensionality reduction via dependence maximization. ACM Trans. Knowl. Discov. Data (TKDD) 4(3), 1–21 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.TechnicolorCesson-sévigné, RennesFrance
  2. 2.Laboratoire d’Informatique de Nantes AtlantiqueSite Polytech’NantesNantes CedexFrance
  3. 3.Orange LabsLannion CedexFrance

Personalised recommendations