Abstract
In this article we describe the approach we applied for the JRS 2012 Data Mining Competition. The task of the competition was the multi-labelled classification of biomedical documents. Our method is motivated by recent work in the machine learning and computer vision communities that highlights the usefulness of feature learning for classification tasks. Our approach uses orthogonal matching persuit to learn a dictionary from PCA-transformed features. Binary relevance with logistic regression is applied to the encoded representations, leading to a fifth place performance in the competition. In order to show the suitability of our approach outside the competition task we also report a state-of-the-art classification performance on the multi-label ASRS dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. Advances in Neural Information Processing Systems 19, 153 (2007)
Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1–127 (2009)
Boureau, Y.-L., Bach, F., LeCun, Y., Ponce, J.: Learning mid-level features for recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2559–2556 (2010)
Coates, A., Ng, A.Y.: The importance of encoding versus training with sparse coding and vector quantization. In: Getoor, L., Scheffer, T. (eds.) International Conference on Machine Learning, pp. 921–928. Omnipress (2011)
Gersho, A., Gray, R.M.: Vector quantization and signal compression. Kluwer Academic Publishers, Norwell (1991)
Goutte, C.: A probabilistic model for fast and confident categorization of textual documents. In: Berry, M.W., Castellanos, M. (eds.) Survey of Text Mining II, vol. 4, pp. 187–202. Springer (2008)
Hinton, G., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)
Liu, D.C., Nocedal, J.: On the limited memory method for large scale optimization. Mathematical Programming 45(3), 503–528 (1989)
Manning, C.D., Raghavan, P., Schütze, H.: An Introduction to Information Retrieval. Cambridge University Press (2008)
NASA: SIAM 2007 – Aviation Safety Reporting System (ASRS) Challenge Dataset (2007), http://web.eecs.utk.edu/events/tmw07/
Pati, Y., Rezaiifar, R., Krishnaprasad, P.: Orthogonal matching pursuit: recursive function approximation with application to wavelet decomposition. In: Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, pp. 40–44 (1993)
Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y.: Self-taught learning, pp. 759–766. ACM Press (2007)
Tsoumakas, G., Katakis, I.: Multi-label classification: An overview. International Journal of Data Warehousing & Mining 3(3), 1–13 (2007)
Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1794–1801 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kiros, R., Soto, A.J., Milios, E., Keselj, V. (2012). Representation Learning for Sparse, High Dimensional Multi-label Classification. In: Yao, J., et al. Rough Sets and Current Trends in Computing. RSCTC 2012. Lecture Notes in Computer Science(), vol 7413. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32115-3_55
Download citation
DOI: https://doi.org/10.1007/978-3-642-32115-3_55
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32114-6
Online ISBN: 978-3-642-32115-3
eBook Packages: Computer ScienceComputer Science (R0)