Online Extreme Entropy Machines for Streams Classification and Active Learning
When dealing with large evolving datasets one needs machine learning models able to adapt to the growing number of information. In particular, stream classification is a research topic where classifiers need an ability to rapidly change their solutions and behave stably after many changes in training set structure. In this paper we show how recently proposed Extreme Entropy Machine can be trained in an online fashion supporting not only adding/removing points to/from the model but even changing the size of the internal representation on demand. In particular we show how one can build a well-conditioned covariance estimator in an online scenario. All these operations are guaranteed to converge to the optimal solutions given by their offline counterparts.
KeywordsOnline learning Extreme entropy machine Stream classification Covariance estimation
The work of the first author was partially founded by National Science Centre Poland grant no. 2013/09/N/ST6/03015, while the work of the second one by National Science Centre Poland grant no. 2014/13/B/ST6/01792.
- 1.Bache, K., Lichman, M.: UCI machine learning repository. http://archive.ics.uci.edu/ml (2013)
- 5.Czarnecki, W.M., Tabor, J.: Extreme entropy machines: robust information theoretic classification. Pattern Anal. Appl. (2015). doi: 10.1007/s10044-015-0497-8
- 10.Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings. 2004 IEEE International Joint Conference on Neural Networks, vol. 2, pp. 985–990. IEEE (2004)Google Scholar
- 13.Lakshminarayanan, B., Roy, D.M., Teh, Y.W.: Mondrian forests: efficient online random forests. In: Advances in Neural Information Processing Systems, pp. 3140–3148 (2014)Google Scholar