Abstract
In multi-dimensional classification (MDC), the semantics of objects are characterized by multiple class spaces from different dimensions. Most MDC approaches try to explicitly model the dependencies among class spaces in output space. In contrast, the recently proposed feature augmentation strategy, which aims at manipulating feature space, has also been shown to be an effective solution for MDC. However, existing feature augmentation approaches only focus on designing holistic augmented features to be appended with the original features, while better generalization performance could be achieved by exploiting multiple kinds of augmented features. In this paper, we propose the selective feature augmentation strategy that focuses on synergizing multiple kinds of augmented features. Specifically, by assuming that only part of the augmented features is pertinent and useful for each dimension’s model induction, we derive a classification model which can fully utilize the original features while conduct feature selection for the augmented features. To validate the effectiveness of the proposed strategy, we generate three kinds of simple augmented features based on standard kNN, weighted kNN, and maximum margin techniques, respectively. Comparative studies show that the proposed strategy achieves superior performance against both state-of-the-art MDC approaches and its degenerated versions with either kind of augmented features.
Article PDF
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
References
D. Xu, Y. X. Shi, I. W. Tsang, Y. S. Ong, C. Gong, X. B. Shen. Survey on multi-output learning. IEEE Transactions on Neural Networks and Learning Systems, vol.31, no. 7, pp. 2409–2429, 2020. DOI: https://doi.org/10.1109/TNNLS.2019.2945133.
J. Read, C. Bielza, P. Larrañaga. Multi-dimensional classification with super-classes. IEEE Transactions on Knowledge and Data Engineering, vol.26, no. 7, pp. 1720–1733, 2014. DOI: https://doi.org/10.1109/TKDE.2013.167.
B. B. Jia, M. L. Zhang. Maximum margin multi-dimensional classification. IEEE Transactions on Neural Networks and Learning Systems, published online. DOI: https://doi.org/10.1109/TNNLS.2021.3084373.
J. D. Rodríguez, A. Pérez, D. Arteta, D. Tejedor, J. A. Lozano. Using multidimensional Bayesian network classifiers to assist the treatment of multiple sclerosis. IEEE Transactions on Systems, Man, and Cybernetics — Part C, vol.42, no.6, pp. 1705–1715, 2012. DOI: https://doi.org/10.1109/TSMCC.2012.2217326.
H. Borchani, C. Bielza, C. Toro, P. Larrañaga. Predicting human immunodeficiency virus inhibitors using multi-dimensional Bayesian network classifiers. Artificial Intelligence in Medicine, vol.57, no.3, pp. 219–229, 2013. DOI: https://doi.org/10.1016/j.artmed.2012.12.005.
H. Shatkay, F. X. Pan, A. Rzhetsky, W. J. Wilbur. Multidimensional classification of biomedical text: Toward automated, practical provision of high-utility text to diverse users. Bioinformatics, vol.24, no. 18, pp. 2086–2093, 2008. DOI: https://doi.org/10.1093/bioinformatics/btn381.
F. Serafino, G. Pio, M. Ceci, D. Malerba. Hierarchical multidimensional classification of web documents with multi-webclass. In Proceedings of the 18th International Conference on Discovery Science, Springer, Banff, Canada, pp. 236–250, 2015. DOI: https://doi.org/10.1007/978-3-319-24282-8_20.
Z. Lian, Y. Li, J. H. Tao, J. Huang, M. Y. Niu. Expression analysis based on face regions in real-world conditions. International Journal of Automation and Computing, vol.17, no.1, pp. 96–107, 2020. DOI: https://doi.org/10.1007/s11633-019-1176-9.
Z. W. He, L. Zhang, F. Y. Liu. DiscoStyle: Multi-level logistic ranking for personalized image style preference inference. International Journal of Automation and Computing, vol.17, no. 5, pp. 637–651, 2020. DOI: https://doi.org/10.1007/s11633-020-1244-1.
Y. Zhang, X. Y. Shi, S. Y. Mi, X. Yang. Image captioning with transformer and knowledge graph. Pattern Recognition Letters, vol. 143, pp. 43–49, 2021. DOI: https://doi.org/10.1016/j.patrec.2020.12.020.
A. H. Al Muktadir, T. Miyazawa, P. Martinez-Julia, H. Harai, V. P. Kafle. Multi-target classification based automatic virtual resource allocation scheme. IEICE Transactions on Information and Systems, vol.E102-D, no. 5, pp. 898–909, 2019. DOI: https://doi.org/10.1587/transinf.2018NTP0016.
J. Arias, J. A. Gamez, T. D. Nielsen, J. M. Puerta. A scalable pairwise class interaction framework for multidimensional classification. International Journal of Approximate Reasoning, vol.68, pp. 194–210, 2016. DOI: https://doi.org/10.1016/j.ijar.2015.07.007.
B. B. Jia, M. L. Zhang. Multi-dimensional classification via stacked dependency exploitation. Science China Information Sciences, vol.63, no. 12, Article number 222102, 2020. DOI: https://doi.org/10.1007/s11432-019-2905-3.
B. B. Jia, M. L. Zhang. MD-KNN: An instance-based approach for multi-dimensional classification. In Proceedings of the 25th International Conference on Pattern Recognition, IEEE, Milan, Italy, pp. 126–133, 2021. DOI: https://doi.org/10.1109/ICPR48806.2021.9412974.
J. H. Zaragoza, L. E. Sucar, E. F. Morales, C. Bielza, P. Larranaga. Bayesian chain classifiers for multidimensional classification. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Spain, pp. 2192–2197, 2011. DOI: https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-365.
J. Read, L. Martino, D. Luengo. Efficient Monte Carlo methods for multi-dimensional learning with classifier chains. Pattern Recognition, vol.47, no.3, pp. 1535–1546, 2014. DOI: https://doi.org/10.1016/j.patcog.2013.10.006.
C. Bielza, G. Li, P. Larrañaga. Multi-dimensional classification with Bayesian networks. International Journal of Approximate Reasoning, vol.52, no.6, pp. 705–727, 2011. DOI: https://doi.org/10.1016/j.ijar.2011.01.007.
J. H. Bolt, L. C. Van Der Gaag. Balanced sensitivity functions for tuning multi-dimensional Bayesian network classifiers. International Journal of Approximate Reasoning, vol. 80, pp. 361–376, 2017. DOI: https://doi.org/10.1016/j.ijar.2016.07.011.
M. Benjumeda, C. Bielza, P. Larrañaga. Tractability of most probable explanations in multidimensional Bayesian network classifiers. International Journal of Approximate Reasoning, vol.93, pp. 74–87, 2018. DOI: https://doi.org/10.1016/j.ijar.2017.10.024.
B. B. Jia, M. L. Zhang. Multi-dimensional classification via kNN feature augmentation. Pattern Recognition, vol. 106, Article number 107423, 2020. DOI: https://doi.org/10.1016/j.patcog.2020.107423.
H. B. Wang, C. Chen, W. W. Liu, K. Chen, T. L. Hu, G. Chen. Incorporating label embedding and feature augmentation for multi-dimensional classification. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, AAAI, New York, USA, pp. 6178–6185, 2020. DOI: https://doi.org/10.1609/AAAI.V34I04.6083.
M. L. Zhang, Z. H. Zhou. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, vol.26, no.8, pp. 1819–1837, 2014. DOI: https://doi.org/10.1109/TKDE.2013.39.
E. Gibaja, S. Ventura. A tutorial on multilabel learning. ACM Computing Surveys, vol.47, no.3, Article number 52, 2015. DOI: https://doi.org/10.1145/2716262.
M. L. Zhang, Y. K. Li, X. Y. Liu, X. Geng. Binary relevance for multi-label learning: An overview. Frontiers of Computer Science, vol.12, no. 2, pp. 191–202, 2018. DOI: https://doi.org/10.1007/s11704-017-7031-7.
S. Gil-Begue, C. Bielza, P. Larrañaga. Multi-dimensional Bayesian network classifiers: A survey. Artificial Intelligence Review, vol.54, no. 1, pp.519–559, 2021. DOI: https://doi.org/10.1007/s10462-020-09858-x.
J. Huang, G. R. Li, Q. M. Huang, X. D. Wu. Learning label-specific features and class-dependent labels for multi-label classification. IEEE Transactions on Knowledge and Data Engineering, vol.28, no. 12, pp.3309–3323, 2016. DOI: https://doi.org/10.1109/TKDE.2016.2608339.
H. H. Bauschke, J. Bolte, M. Teboulle. A descent lemma beyond Lipschitz gradient continuity: First-order methods revisited and applications. Mathematics of Operations Research, vol.42, no. 2, pp. 330–348, 2017. DOI: https://doi.org/10.1287/moor.2016.0817.
A. Beck, M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, vol.2, no. 1, pp. 183–202, 2009. DOI: https://doi.org/10.1137/080716542.
Z. C. Ma, S. C. Chen. Multi-dimensional classification via a metric approach. Neurocomputing, vol.275, pp. 1121–1131, 2018. DOI: https://doi.org/10.1016/j.neucom.2017.09.057.
R. E. Fan, K. W. Chang, C. J. Hsieh, X. R. Wang, C. J. Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, vol.9, pp. 1871–1874, 2008.
K. Crammer, Y. Singer. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, vol.2, pp. 265–292, 2001.
J. Demšar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, vol.7, pp. 1–30, 2006.
J. Zhao, X. J. Xie, X. Xu, S. L. Sun. Multi-view learning overview: Recent progress and new challenges. Information Fusion, vol.38, pp. 43–54, 2017. DOI: https://doi.org/10.1016/j.inffus.2017.02.007.
Acknowledgements
This work was supported by National Science Foundation of China (No. 62176055) and China University S&T Innovation Plan Guided by the Ministry of Education. The authors wish to thank the associate editor and anonymous reviewers for their insightful comments and suggestions. The authors thank the Big Data Center of Southeast University for providing the facility support on the numerical calculations in this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Colored figures are available in the online version at https://link.springer.com/journal/11633
Bin-Bin Jia received the B.Sc. degree in electronic information science and technology from North China Electric Power University, China in 2010, and the M.Sc. degree in information and communication engineering from Beihang University, China in 2013. He joined College of Electrical and Information Engineering, Lanzhou University of Technology, China in 2013. Currently, he is a Ph.D. degree candidate in School of Computer Science and Engineering, Southeast University, China.
His research interests include machine learning and data mining, especially in multi-dimensional classification. E-mail: jiabb@seu.edu.cn ORCID iD: 0000-0003-3302-9398
Min-Ling Zhang received the B.Sc., M.Sc., and Ph.D. degrees in computer science from Nanjing University, China in 2001, 2004 and 2007, respectively. Currently, he is a professor at School of Computer Science and Engineering, Southeast University, China. In recent years, Dr. Zhang has served as the General Co-Chairs of ACML’18, Program Co-Chairs of PAK-DD’19, CCF-ICAI’19, ACML’17, CCFAI’17, PRICAI’16, Senior PC member or Area Chair of AAAI 2017–2020, IJCAI 2017–2022, KDD 2021, ICDM 2015–2021, etc. He is also on the editorial board of IEEE Transactions on Pattern Analysis and Machine Intelligence, ACM Transactions on Intelligent Systems and Technology, Neural Networks, Science China Information Sciences, Frontiers of Computer Science, etc. Dr. Zhang is the Steering Committee Member of ACML and PAKDD, Vice Chair of CAAI Machine Learning Society, standing committee member of the CCF Artificial Intelligence & Pattern Recognition Society. He is a Distinguished Member of CCF, CAAI, and Senior Member of ACM, IEEE.
His research interests include machine learning and data mining. E-mail: zhangml@seu.edu.cn (Corresponding author) ORCID iD: 0000-0003-1880-5918
Rights and permissions
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/
About this article
Cite this article
Jia, BB., Zhang, ML. Multi-dimensional Classification via Selective Feature Augmentation. Mach. Intell. Res. 19, 38–51 (2022). https://doi.org/10.1007/s11633-022-1316-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-022-1316-5