A local and global classification machine with collaborative mechanism

Abstract

As an advanced local and global learning machine, the existing maxi–min margin machine (M⁴) still has its heavy time-consuming weakness. Inspired from the fact that covariance matrix of a dataset can characterize its data orientation and compactness globally, a novel large margin classifier called the local and global classification machine with collaborative mechanism (C²M) is constructed to circumvent this weakness in this paper. This classifier divides the whole global data into two independent models, and the final decision boundary is obtained by collaboratively combining two hyperplanes learned from two independent models. The proposed classifier C²M can be individually solved as a quadratic programming problem. The total training time complexity is $O(2N^3)$ which is faster than $O(N^4)$ of M⁴. C²M can be well defined with the clear geometrical interpretation and can also be justified from a theoretical perspective. As an additional contribution, it is shown that C²M can robustly leverage the global information from those datasets with overlapping class margins, while M⁴ does not use such global information. We also use the kernel trick and exploit C²M’s kernelized version. Experiments on toy and real-world datasets demonstrate that compared with M⁴, C²M is a more time-saving local and global learning machine.