Skip to main content
SpringerLink
Log in

A Fast Feature Selection Method Based on Mutual Information in Multi-label Learning

  • Conference paper
  • First Online:
Computer Supported Cooperative Work and Social Computing (ChineseCSCW 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 917))

Abstract

Recently, multi-label learning is concerned and studied in lots of fields by many researchers. However, multi-label datasets often have noisy, irrelevant and redundant features with high dimensionality. Accompanying with these issues, a critical challenge called “the curse of dimensionality” makes many tasks of multi-label learning very difficult. Therefore, many method such as feature selection to solve this problem has received much attention. Among many feature selection methods, a large number of information-theoretical-based methods are developed to solve the learning issue and the results are very good. Unfortunately, most of existing feature selection methods are either directly transformed from single-label methods or insufficient in light of using heuristic algorithms as the search component. Motivated by this, a novel fast method based on mutual information with no parameter is proposed, which obtains the optimal solution via constrained convex optimization with less time. Specifically, by incorporating the label information into the feature selection process, label-correlation is taken into consideration to generate the generalized model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Schapire, R.E., Singer, Y.: BoosTexter: a boosting-based system for text categorization. Mach. Learn. 39(2/3), 135–168 (2000)

    Article  Google Scholar 

  2. Sebastiani, F.: Machine learning in automated text categorization. ACM Comput. Surv. 34(1), 1–47 (2002)

    Article  MathSciNet  Google Scholar 

  3. Diplaris, S., Tsoumakas, G., Mitkas, P.A., Vlahavas, I.: Protein classification with multiple algorithms. In: Bozanis, P., Houstis, E.N. (eds.) PCI 2005. LNCS, vol. 3746, pp. 448–456. Springer, Heidelberg (2005). https://doi.org/10.1007/11573036_42

    Chapter  Google Scholar 

  4. Liu, G.P., Li, G.Z., Wang, Y.L., Wang, Y.Q.: Modelling of inquiry diagnosis for coronary heart disease in traditional Chinese medicine by using multi-label learning. BMC Complement. Altern. Med. 10(1), 37 (2010)

    Article  Google Scholar 

  5. Naula, P., Airola, A., Salakoski, T., Pahikkala, T.: Multi-label learning under feature extraction budgets. Pattern Recognit. Lett. 40, 56–65 (2014)

    Article  Google Scholar 

  6. Zhang, L., Hu, Q., Duan, J., Wang, X.: Multi-label feature selection with fuzzy rough sets. In: Miao, D., Pedrycz, W., Ślȩzak, D., Peters, G., Hu, Q., Wang, R. (eds.) RSKT 2014. LNCS (LNAI), vol. 8818, pp. 121–128. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11740-9_12

    Chapter  Google Scholar 

  7. Zhang, J., Fang, M., Li, X.: Multi-label learning with discriminative features for each label. Neurocomputing 154, 305–316 (2015)

    Article  Google Scholar 

  8. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  9. Molchanov, V., Linsen, L.: Overcoming the curse of dimensionality when clustering multivariate volume data. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018) - Volume 3: IVAPP, Funchal, Madeira, Portugal, 27–29 January 2018, pp. 29–39 (2018)

    Google Scholar 

  10. Zhang, M.-L., Zhou, Z.-H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)

    Article  Google Scholar 

  11. Kong, X., Yu, P.S.: gMLC: a multi-label feature selection framework for graph classification. Knowl. Inf. Syst. 31(2), 281–305 (2012)

    Article  Google Scholar 

  12. Lee, J.-S., Kim, D.-W.: Memetic feature selection algorithm for multi-label classification. Inf. Sci. 293, 80–96 (2015)

    Article  Google Scholar 

  13. Li, P., Li, H., Min, W.: Multi-label ensemble based on variable pairwise constraint projection. Inf. Sci. 222, 269–281 (2013)

    Article  MathSciNet  Google Scholar 

  14. Zhang, M.-L., Lei, W.: LIFT: multi-label learning with label-specific features. IEEE Trans. Pattern Anal. Mach. Intell. 37(1), 107–120 (2015)

    Article  Google Scholar 

  15. Lin, Y., Qinghua, H., Zhang, J., Xindong, W.: Multi-label feature selection with streaming labels. Inf. Sci. 372, 256–275 (2016)

    Article  Google Scholar 

  16. Liu, J., Lin, Y., Lin, M., Shunxiang, W., Zhang, J.: Feature selection based on quality of information. Neurocomputing 225, 11–22 (2017)

    Article  Google Scholar 

  17. Teisseyre, P.: CCnet: joint multi-label classification and feature selection using classifier chains and elastic net regularization. Neurocomputing 235, 98–111 (2017)

    Article  Google Scholar 

  18. Pudil, P., Novovicová, J., Kittler, J.: Floating search methods in feature selection. Pattern Recognit. Lett. 15(10), 1119–1125 (1994)

    Article  Google Scholar 

  19. Reunanen, J.: Overfitting in making comparisons between variable selection methods. J. Mach. Learn. Res. 3, 1371–1382 (2003)

    MATH  Google Scholar 

  20. Somol, P., Pudil, P., Novovicová, J., Paclík, P.: Adaptive floating search methods in feature selection. Pattern Recognit. Lett. 20(11–13), 1157–1163 (1999)

    Article  Google Scholar 

  21. Blum, A., Langley, P.: Selection of relevant features and examples in machine learning. Artif. Intell. 97(1–2), 245–271 (1997)

    Article  MathSciNet  Google Scholar 

  22. John, G.H., Kohavi, R., Pfleger, K.: Irrelevant features and the subset selection problem. In: Machine Learning, Proceedings of the Eleventh International Conference, Rutgers University, New Brunswick, NJ, USA, 10–13 July 1994, pp. 121–129 (1994)

    Google Scholar 

  23. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1–3), 389–422 (2002)

    Article  Google Scholar 

  24. Law, M.H.C., Figueiredo, M.A.T., Jain, A.K.: Simultaneous feature selection and clustering using mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1154–1166 (2004)

    Article  Google Scholar 

  25. Peng, H., Long, F., Ding, C.H.Q.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)

    Article  Google Scholar 

  26. Xing, E.P., Karp, R.M.: CLIFF: clustering of high-dimensional microarray data via iterative feature filtering using normalized cuts. In: Proceedings of the Ninth International Conference on Intelligent Systems for Molecular Biology, 21–25 July 2001, Copenhagen, Denmark, pp. 306–315 (2001)

    Google Scholar 

  27. Zhao, Z., Liu, H.: Semi-supervised feature selection via spectral analysis. In: Proceedings of the Seventh SIAM International Conference on Data Mining, 26–28 April 2007, Minneapolis, Minnesota, USA, pp. 641–646 (2007)

    Google Scholar 

  28. Sheikhpour, R., Sarram, M.A., Gharaghani, S., Chahooki, M.A.Z.: A survey on semi-supervised feature selection methods. Pattern Recognit. 64, 141–158 (2017)

    Article  Google Scholar 

  29. Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 5(4), 537–550 (1994)

    Article  Google Scholar 

  30. Lin, Y., Qinghua, H., Liu, J., Duan, J.: Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168, 92–103 (2015)

    Article  Google Scholar 

  31. Lee, J.-S., Kim, D.-W.: Feature selection for multi-label classification using multivariate mutual information. Pattern Recognit. Lett. 34(3), 349–357 (2013)

    Article  Google Scholar 

  32. Lim, H., Lee, J.-S., Kim, D.-W.: Optimization approach for feature selection in multi-label classification. Pattern Recognit. Lett. 89, 25–30 (2017)

    Article  Google Scholar 

  33. Zhang, J., et al.: Multi-label learning with label-specific features by resolving label correlations. Knowl.-Based Syst. 159, 148–157 (2018)

    Article  Google Scholar 

  34. Wang, J., Wei, J.-M., Yang, Z., Wang, S.-Q.: Feature selection by maximizing independent classification information. IEEE Trans. Knowl. Data Eng. 29(4), 828–841 (2017)

    Article  Google Scholar 

  35. Brown, G., Pocock, A.C., Zhao, M.-J., Luján, M.: Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. J. Mach. Learn. Res. 13, 27–66 (2012)

    MathSciNet  MATH  Google Scholar 

  36. Wang, J., Wei, J., Yang, Z.: Supervised feature selection by preserving class correlation. In: Proceedings of the 25th ACM International Conference on Information and Knowledge Management, CIKM 2016, Indianapolis, IN, USA, 24–28 October 2016, pp. 1613–1622 (2016)

    Google Scholar 

  37. Wang, X., Chen, R.-C., Hong, C., Zeng, Z., Zhou, Z.: Semi-supervised multi-label feature selection via label correlation analysis with l\({}_{\text{1 }}\)-norm graph embedding. Image Vis. Comput. 63, 10–23 (2017)

    Google Scholar 

  38. Spolaôr, N., Monard, M.C., Tsoumaka, G., Lee, H.D.: A systematic review of multi-label feature selection and a new method based on label construction. Neurocomputing 180, 3–15 (2016)

    Article  Google Scholar 

  39. Shannon, C.E.: A mathematical theory of communication. Mob. Comput. Commun. Rev. 5(1), 3–55 (2001)

    Article  MathSciNet  Google Scholar 

  40. Willems, F.M.J.: Review of ‘elements of information theory’ (Cover, T.M., and Thomas, J.A.; 1991). IEEE Trans. Inf. Theory 39(1), 313 (1993)

    Article  Google Scholar 

  41. Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11, 2837–2854 (2010)

    MathSciNet  MATH  Google Scholar 

  42. Zhang, M.-L., Zhou, Z.-H.: ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn. 40(7), 2038–2048 (2007)

    Article  Google Scholar 

  43. Zhang, M., Ding, C.H.Q., Zhang, Y., Nie, F.: Feature selection at the discrete limit. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, 27–31 July 2014, Québec City, Québec, Canada, pp. 1355–1361 (2014)

    Google Scholar 

  44. Zhang, M.-L., Sánchez, J.M.P., Robles, V.: Feature selection for multi-label naive bayes classification. Inf. Sci. 179(19), 3218–3229 (2009)

    Article  Google Scholar 

  45. Zhang, Y., Zhou, Z.-H.: Multi-label dimensionality reduction via dependence maximization. In: Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, 13–17 July 2008, pp. 1503–1505 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cognitive Science, Xiamen University, Xiamen, 361005, People’s Republic of China

    Zhenqiang Sun, Jia Zhang, Zhiming Luo, Donglin Cao & Shaozi Li

Authors
  1. Zhenqiang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiming Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Donglin Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaozi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaozi Li .

Editor information

Editors and Affiliations

  1. Shandong University, Jinan, China

    Yuqing Sun

  2. Fudan University, Shanghai, China

    Tun Lu

  3. Guilin University of Technology, Guilin, China

    Xiaolan Xie

  4. University of Shanghai for Science and Technology, Shanghai , China

    Liping Gao

  5. Tongji University, Shanghai, China

    Hongfei Fan

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sun, Z., Zhang, J., Luo, Z., Cao, D., Li, S. (2019). A Fast Feature Selection Method Based on Mutual Information in Multi-label Learning. In: Sun, Y., Lu, T., Xie, X., Gao, L., Fan, H. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2018. Communications in Computer and Information Science, vol 917. Springer, Singapore. https://doi.org/10.1007/978-981-13-3044-5_31

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-3044-5_31

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-3043-8

  • Online ISBN: 978-981-13-3044-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation