Abstract
Multi-label classification is a type of classification where each instance is associated with a set of labels. Many methods such as BP-MLL, rank-SVM, and MLRBF have been proposed for multi-label classification but their learning abilities are too slow. Extreme Learning Machine (ELM) is a well known algorithm for SLFNs that can learn faster than the traditional gradient-base neural networks and it also provides better generalization performance. However, the classification performance of ELM involving multi-label classification may not be good enough despite its advantage in fast training. Therefore, this paper proposes two multi-label classification approaches in ELM. The first approach uses the 1-norm regularized Least-square for Canonical Correlation Analysis (1-norm LSCCA) to obtain the projection vectors, which in turn uses the vectors to provide the new information. Then, ELM is then used to learn this new information in the new space. The second approach applies the ensemble method to the first approach to reduce the random effects of ELM. The experimental results show that the two proposed methods can improve the performance of ELM in multi-label classification and are also faster than the previous multi-label classification methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Tsoumakas, G., Katakis, I.: Multi Label Classification: An Overview. Int. J. Data Warehousing and Mining. 3, 1–13 (2007)
Madjarov, G., Kocev, D., Gjorgjevikj, D., Deroski, S.: An extensive experimental comparison of methods for multi-label learning. Pattern Recogn. 45, 3084–3104 (2012)
Zhang, Y., Schneider, J.G.: Multi-label output codes using canonical correlation analysis. In: International Conference on Artificial Intelligence and Statistics, USA, pp. 873–882 (2011)
Sun, L., Ji, S., Ye, J.: A Least Squares Formulation for Canonical Correlation Analysis. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1024–1031. ACM, New York (2008)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: Theory and applications. Neurocomputing 70, 489–501 (2006)
Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern 42, 513–529 (2012)
Kongsorot, Y., Horata, P.: Multi-label classification with extreme learning machine. In: 6th International Conference on Knowledge and Smart Technology (KST), Thailand, pp. 81–86 (2014)
Dietterich, T.G.: Ensemble methods in machine learning. In: Multiple classifier systems, pp. 1–15. Springer (2000)
Sun, L., Ji, S., Ye, J.: Canonical Correlation Analysis for Multilabel Classification: A Least-Squares Formulation, Extensions, and Analysis. IEEE Trans. Pattern Anal. Mach. Intell. 33, 194–200 (2011)
Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier Chains for Multi-label Classification. In: Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II, pp. 254–269. Berlin (2009)
Read, J., Pfahringer, B., Holmes, G.: Multi-label Classification Using Ensembles of Pruned Sets. In: 8th IEEE International Conference on Data Mining (ICDM 2008), Pisa, pp. 995–1000 (2008)
Tsoumakas, G., Vlahavas, I.P.: Random k-Labelsets: An Ensemble Method for Multilabel Classification. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 406–417. Springer, Heidelberg (2007)
Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective and Efficient Multilabel Classification in Domains with Large Number of Labels. In: Proc. ECML/PKDD 2008 Workshop on Mining Multidimensional Data (MMD 2008), Belgium (2008)
Fürnkranz, J.: Round Robin Classification. J. Mach. Learn. Res. 2, 721–747 (2002)
Wu, T.F., Lin, C.J., Weng, R.C.: Probability Estimates for Multi-class Classification by Pairwise Coupling. J. Mach. Learn. Res. 5, 975–1005 (2004)
Freund, Y., Schapire, R.E.: A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. J. Comput. System Sci. 55, 119–139 (1997)
Zhang, M.L., Zhou, Z.H.: Ml-knn: A lazy learning approach to multi-label learning. Pattern Recogn. 40, 2038–2048 (2007)
Clare, A.J., King, R.D.: Knowledge Discovery in Multi-label Phenotype Data. In: Siebes, A., De Raedt, L. (eds.) PKDD 2001. LNCS (LNAI), vol. 2168, pp. 42–53. Springer, Heidelberg (2001)
Zhang, M.L., Zhou, Z.H.: Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization. IEEE Trans. Knowl. Data Eng. 18, 1338–1351 (2006)
Zhang, M.L.: Ml-rbf: RBF Neural Networks for Multi-Label Learning. Neural Process. Lett. 29, 61–74 (2009)
Elisseeff, A., Weston, J.: A Kernel Method for Multi-Labelled Classification. In: Advances in Neural Information Processing Systems, vol. 14, pp. 681–687. MIT Press (2001)
Hotelling, H.: Relations between two sets of variates. Biometrika 28, 321–377 (1936)
Tibshirani, R.: Regression Shrinkage and Selection Via the Lasso. J. Roy. Statist. Soc. Ser. B. 58, 267–288 (1994)
Penrose, R.: A generalized inverse for matrices. In: Mathematical Proceedings of the Cambridge Philosophical Society, pp. 406–413. Cambridge Univ Press (1955)
Rong, H.J., Huang, G.B., Ong, Y.S.: Extreme learning machine for multi-categories classification applications. In: IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 1709–1713. IEEE (2008)
Huang, G.B., Wang, D.H., Lan, Y.: Extreme learning machines: a survey. Int. J Mach. Learn. and Cyber. 2, 107–122 (2011)
Lan, Y., Soh, Y.C., Huang, G.B.: Ensemble of online sequential extreme learning machine. Neurocomputing 72, 3391–3395 (2009)
Liu, N., Wang, H.: Ensemble Based Extreme Learning Machine. IEEE Signal Process. Lett. 17, 754–757 (2010)
Zhu, Q.Y., Qin, A.K., Suganthan, P.N., Huang, G.B.: Evolutionary extreme learning machine. Pattern Recogn. 38, 1759–1763 (2005)
Freund, Y.: Boosting a weak learning algorithm by majority. Inform. Comput. 121, 256–285 (1995)
Fawcett, T.: ROC graphs: Notes and practical considerations for researchers. Mach. Learn. 31, 1–38 (2004)
Brown, C.D., Davis, H.T.: Receiver operating characteristics curves and related decision measures: A tutorial. Chemometr. Intell. Lab. 80, 24–38 (2006)
Zou, K.H., OMalley, A.J., Mauri, L.: Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation 115, 654–657 (2007)
Zhang, Y., Schneider, J.G.: Multi-label output codes using canonical correlation analysis. In: International Conference on Artificial Intelligence and Statistics, USA, pp. 873–882 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kongsorot, Y., Horata, P., Sunat, K. (2015). Applying Regularization Least Squares Canonical Correlation Analysis in Extreme Learning Machine for Multi-label Classification Problems. In: Cao, J., Mao, K., Cambria, E., Man, Z., Toh, KA. (eds) Proceedings of ELM-2014 Volume 1. Proceedings in Adaptation, Learning and Optimization, vol 3. Springer, Cham. https://doi.org/10.1007/978-3-319-14063-6_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-14063-6_32
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14062-9
Online ISBN: 978-3-319-14063-6
eBook Packages: EngineeringEngineering (R0)