A Novel Sparsity Based Classification Framework to Exploit Clusters in Data

Abstract

A huge recent advance in machine learning has been the usage of sparsity as a guiding principle to perform classification. Traditionally sparsity has been used to exploit a property of high dimensional vectors–which is that, vectors of the same class lie on or near the same low dimensional subspace within an ambient high dimensional space–this is seen in algorithms like Basis Pursuit, and Sparse Representation classifier. In this paper we use sparsity to exploit a different property of data, which is that data points belonging to the same class constitute a cluster. Here classification is done by determining which cluster’s vectors can best convexly approximate the given test vector. So if the vectors of cluster ‘A’ best approximate or realise the given test vector, then label ‘A’ is assigned as its class. The problem of finding the best approximate is framed as a $\ell _{1}$ norm minimization problem with convex constraints. The optimization framework of the proposed algorithm is convex in nature making the classification algorithm tractable. The proposed algorithm is evaluated by comparing its accuracy with the accuracy of other popular machine learning algorithms on a diverse collection of real datasets. The proposed algorithm on an average provides a 10 % improvement in accuracy over certain standard machine learning algorithms.