Derivative-based acceleration of general vector machine
- 204 Downloads
General vector machine (GVM) is one of supervised learning machine, which is based on three-layer neural network. It is capable of constructing a learning model with limited amount of data. Generally, it employs Monte Carlo algorithm (MC) to adjust weights of the underlying network. However, GVM is time-consuming at training and is not efficient when compared with other learning algorithm based on gradient descent learning. In this paper, we present a derivative-based Monte Carlo algorithm (DMC) to accelerate the training of GVM. Our experimental results indicate that DMC algorithm is faster than the original MC method. Specifically, the training time of our DMC algorithm in GVM for function fitting is also less than some gradient descent-based methods, in which we compare DMC with back-propagation neural network. Experimental results indicate that our algorithm is promising for training GVM.
KeywordsGeneral vector machine Neural network Gradient descent Derivative Back-propagation
This work was supported by Dongguan’s Recruitment of Innovation and entrepreneurship talent program, National Natural Science Foundation of China under Grant Nos. 61402210 and 60973137, Program for New Century Excellent Talents in University under Grant No. NCET-12-0250, Strategic Priority Research Program of the Chinese Academy of Sciences with Grant No. XDA03030100, Gansu Sci. and Tech. Program under Grant Nos. 1104GKCA049, 1204GKCA061 and 1304GKCA018, Google Research Awards and Google Faculty Award, China. This research has also been conducted with the support of the Australian Government Research Training Program Scholarship.
Compliance with ethical standards
Conflict of interest
All authors declare that they have no conflicts of interest regarding the publication of this manuscript.
- Chen H, Zhao H, Shen J, Zhou R, Zhou Q (2015) Supervised machine learning model for high dimensional gene data in colon cancer detection. In: IEEE BigData congress, pp 134–141Google Scholar
- Hagan MT, Demuth HB, Beale MH (1995) Neural network design. PWS Publishing Company, Boston, MAGoogle Scholar
- İçen D, Cattaneo MEGV (2016) Different distance measures for fuzzy linear regression with Monte Carlo methods. Soft Comput. doi: 10.1007/s00500-016-2218-7
- Ji S, Xu W, Yang M, Yu K (2013) 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231. doi: 10.1109/TPAMI.2012.59
- Kreinovich V, Sirisaengtaksin O (1993) 3-layer neural networks are universal approximators for functionals and for control strategies. Neural Parallel Sci Comput 1(3):325–346Google Scholar
- Krizhevsky A, Sutskever I, Hinton GE (2012a) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates, Inc., Red Hook, pp 1097–1105Google Scholar
- Krizhevsky A, Sutskever I, Hinton GE (2012b) Imagenet classification with deep convolutional neural networks. In: International conference on neural information processing systems, pp 1097–1105Google Scholar
- Wang H, Raj B (2017) On the origin of deep learning. arXiv preprintGoogle Scholar
- Yong B, Xu Z, Shen J, Chen H, Tian Y, Zhou Q (2017) Neural network model with monte carlo algorithm for electricity demand forecasting in queensland. In: Australasian computer science week multiconference, p 47Google Scholar
- Zhao H (2016) General vector machine. arXiv preprintGoogle Scholar
- Zhou Q, Chen H, Zhao H, Zhang G, Yong J, Shen J (2016) A local field correlated and monte carlo based shallow neural network model for non-linear time series prediction. EAI Endorsed Trans Scalable Inf Syst 3(8):e5-1–e5-7Google Scholar