Fast Sparse Multinomial Regression Applied to Hyperspectral Data
Methods for learning sparse classification are among the state-of-the-art in supervised learning. Sparsity, essential to achieve good generalization capabilities, can be enforced by using heavy tailed priors/ regularizers on the weights of the linear combination of functions. These priors/regularizers favour a few large weights and many to exactly zero. The Sparse Multinomial Logistic Regression algorithm  is one of such methods, that adopts a Laplacian prior to enforce sparseness. Its applicability to large datasets is still a delicate task from the computational point of view, sometimes even impossible to perform. This work implements an iterative procedure to calculate the weights of the decision function that is O(m 2) faster than the original method introduced in  (m is the number of classes). The benchmark dataset Indian Pines is used to test this modification. Results over subsets of this dataset are presented and compared with others computed with support vector machines.
KeywordsTraining Sample Linear Discriminant Analysis Hyperspectral Image Linear Kernel Hyperspectral Data
Unable to display preview. Download preview PDF.
- 8.Lawrence, N.D., Seeger, M., Herbrich, R.: Fast sparse Gaussian process methods: The informative vector machine. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems 15, pp. 609–616. MIT Press, Cambridge (2003)Google Scholar
- 9.Krishnapuram, B., Carin, L., Hartemink, A.J.: Joint classifier and feature optimization for cancer diagnosis using gene expression data. In: Proceedings of the International Conference in Research in Computational Molecular Biology (RECOMB 2003), Berlin, Germany (2003)Google Scholar
- 11.Quarteroni, A., Sacco, R., Saleri, F.: Numerical Mathematics. TAM Series, vol. 37. Springer, New York (2000)Google Scholar
- 12.Bioucas Dias, J.M.: Fast Sparse Multinomial Logistic Regression - Technical Report. Instituto Superior Técnico (2006), Available at: http://www.lx.it.pt/~bioucas/
- 15.Landgrebe, D.A.: NW Indiana’s Indian Pine (1992), Available at: http://dynamo.ecn.purdue.edu/~biehl/MultiSpec/
- 16.The MathWorks: MATLAB The Language of Technical Computing - Using MATLAB: version 6. The Math Works, Inc. (2000)Google Scholar