Twin maximum entropy discriminations for classification
- 21 Downloads
Maximum entropy discrimination (MED) is an excellent classification method based on the maximum entropy and maximum margin principles, and can produce hard-margin support vector machines (SVMs) under certain condition. In this paper, we propose a novel maximum entropy discrimination classifier called twin maximum entropy discriminations (TMED) which construct two discrimination functions for two classes such that each discrimination function is closer to one of the two classes and is at least γt distance from the other. Therefore, it is more flexible and has better generalization ability than typical MED. Furthermore, it solves a pair of convex optimization problems and has the same advantages as those of non-parallel SVM (NPSVM) which is only the special case of our TMED when the priors and parameters are chosen appropriately. It also owns the inherent sparseness as MED. Experimental results confirm the effectiveness of our proposed method.
KeywordsMaximum entropy discrimination Non-parallel support vector machines Maximum margin principle Convex optimization problems
This work is supported by Ningbo University talent project 421703670 as well as programs sponsored by K.C. Wong Magna Fund in Ningbo University. It is also supported by the Research Foundation of Education Department of Zhejiang Province under Project Y201635608, and the Zhejiang Provincial Natural Science Foundation of China under Project LQ18F020001, the Natural Science Foundation of Ningbo city of Zhejiang Province of China under Project 2018A610155, the Open Project Program of the State Key Lab of CAD&CG in Zhejiang University, under Project A1815, the National Natural Science Foundation of China under Projects 61572266 and 61472194.
- 1.Jaakkola T, Meila M, Jebara T (1999) Maximum entropy discrimination. In: Proceedings of the 13th Annual Conference on Neural Information Processing Systems, pp 470–476Google Scholar
- 3.Jebara T, Jaakkola T (2000) Feature selection and dualities in maximum entropy discrimination. In: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pp 291–300Google Scholar
- 4.Jebara T (2004) Multi-task feature and kernel selection for SVMs. In: Proceedings of the 21st International Conference on Machine Learning, pp 55–62Google Scholar
- 6.Long P, Wu X (2004) Mistake bounds for maximum entropy discrimination. In: Proceedings of the 18th Annual Conference on Neural Information Processing Systems, pp 833–840Google Scholar
- 8.Zhu J, Xing E, Zhang B (2008) Laplace maximum margin Markov networks. In: Proceedings of the 25th International Conference on Machine Learning, pp 1256–1263Google Scholar
- 9.Zhu J, Xing E, Zhang B (2008) Partially observed maximum entropy discrimination Markov networks. In: Proceedings of the 22th Annual Conference on Neural Information Processing Systems, pp 1977–1984Google Scholar
- 11.Sun S, Chao G (2013) Multi-view maximum entropy discrimination. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pp 1706–1712Google Scholar
- 14.Mao L, Sun S (2016) Soft margin consistency based scalable multi-view maximum entropy discrimination. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, pp 1839–1845Google Scholar
- 15.Domingos (2000) Bayesian averaging of classifiers and the overfitting problem. In: Proceedings of the 17th International Conference on Machine Learning, pp 223–230Google Scholar
- 19.Chen S, Wu X, Xu J (2018) Locality preserving projection least squares twin support vector machine for?pattern classifcation. Pattern Analysis and Applications. https://doi.org/10.1007/s10044-018-0728-x