Towards More Efficient Multi-label Classification Using Dependent and Independent Dual Space Reduction
While multi-label classification can be widely applied for problems where multiple classes can be assigned to an object, its effectiveness may be sacrificed due to curse of dimensionality in the feature space and sparseness of dimensionality in the label space. Moreover, it suffers with high computational cost when there exist a high number of dimensions, as well as with lower accuracy when there are a number of noisy examples. As a solution, this paper presents two alternative methods, namely Dependent Dual Space Reduction and Independent Dual Space Reduction, to reduce dimensions in the dual spaces, i.e., the feature and label spaces, using Singular Value Decomposition (SVD). The first approach constructs the covariance matrix to represent dependency between the features and labels, project both of them into a single reduced space, and then perform prediction on the reduced space. On the other hand, the second approach handles the feature space and the label space separately by constructing a covariance matrix for each space to represent feature dependency and label dependency before performing SVD on dependency profile of each space to reduce dimension and for noise elimination and then predicting using their reduced dimensions. A number of experiments evidence that prediction on the reduced spaces for both dependent and independent reduction approaches can obtain better classification performance as well as faster computation, compared to the prediction using the original spaces. The dependent approach helps saving computational time while the independent approach tends to obtain better classification performance.
Keywordsmulti-label classification Singular Value Decomposition SVD dimensionality reduction Problem Transformation
Unable to display preview. Download preview PDF.
- 1.Bi, W., Kwok, J.: Multi-label classification on tree- and dag-structured hierarchies. In: Getoor, L., Scheffer, T. (eds.) Proceedings of the 28th International Conference on Machine Learning (ICML 2011), pp. 17–24. ACM, New York (2011)Google Scholar
- 3.Cherman, E.A., Metz, J., Monard, M.C.: Incorporating label dependency into the binary relevance framework for multi-label classification. Expert Systems with Applications (2011)Google Scholar
- 4.Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Proceedings of the Advances in Neural Information Processing Systems, vol. 14, pp. 681–687. MIT Press (2001)Google Scholar
- 6.Han, Y., Wu, F., Jia, J., Zhuang, Y., Yu, B.: Multi-task sparse discriminant analysis (mtsda) with overlapping categories. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, pp. 469–474 (2010)Google Scholar
- 7.Hsu, D., Kakade, S., Langford, J., Zhang, T.: Multi-label prediction via compressed sensing. In: Proceedings of the Advances in Neural Information Processing Systems, vol. 22, pp. 772–780 (2009)Google Scholar
- 8.Katakis, I., Tsoumakas, G., Vlahavas, I.: Multilabel text classification for automated tag suggestion. In: Proceedings of the the ECML/PKDD 2008 Discovery Challenge (2008)Google Scholar
- 9.Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Machine Learning, 1–27 (2011)Google Scholar
- 11.Tai, F., Lin, H.T.: Multi-label classification with principle label space transformation. In: Proceedings of the 2nd International Workshop on Learning from Multi-Label Data (MLD 2010), pp. 45–52 (2010)Google Scholar
- 12.Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.: Multi-label classification of music into emotions. In: Proceedings of the International Symposium/Conference on Music Information Retrieval, pp. 325–330 (2008)Google Scholar
- 13.Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining Multi-label Data. In: Data Mining and Knowledge Discovery Handbook, 2nd edn. Springer (2010)Google Scholar
- 16.Yu, K., Yu, S., Tresp, V.: Multi-label informed latent semantic indexing. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 258–265 (2005)Google Scholar
- 17.Zhang, M.L., Pea, J.M., Robles, V.: Feature selection for multi-label naive bayes classification. Information Science, 3218–3229 (2009)Google Scholar