Advertisement

Soft Computing

, Volume 23, Issue 3, pp 987–995 | Cite as

Derivative-based acceleration of general vector machine

  • Binbin Yong
  • Fucun Li
  • Qingquan Lv
  • Jun Shen
  • Qingguo ZhouEmail author
Methodologies and Application
  • 204 Downloads

Abstract

General vector machine (GVM) is one of supervised learning machine, which is based on three-layer neural network. It is capable of constructing a learning model with limited amount of data. Generally, it employs Monte Carlo algorithm (MC) to adjust weights of the underlying network. However, GVM is time-consuming at training and is not efficient when compared with other learning algorithm based on gradient descent learning. In this paper, we present a derivative-based Monte Carlo algorithm (DMC) to accelerate the training of GVM. Our experimental results indicate that DMC algorithm is faster than the original MC method. Specifically, the training time of our DMC algorithm in GVM for function fitting is also less than some gradient descent-based methods, in which we compare DMC with back-propagation neural network. Experimental results indicate that our algorithm is promising for training GVM.

Keywords

General vector machine Neural network Gradient descent Derivative Back-propagation 

Notes

Acknowledgements

This work was supported by Dongguan’s Recruitment of Innovation and entrepreneurship talent program, National Natural Science Foundation of China under Grant Nos. 61402210 and 60973137, Program for New Century Excellent Talents in University under Grant No. NCET-12-0250, Strategic Priority Research Program of the Chinese Academy of Sciences with Grant No. XDA03030100, Gansu Sci. and Tech. Program under Grant Nos. 1104GKCA049, 1204GKCA061 and 1304GKCA018, Google Research Awards and Google Faculty Award, China. This research has also been conducted with the support of the Australian Government Research Training Program Scholarship.

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflicts of interest regarding the publication of this manuscript.

References

  1. Abdalla A, Buckley JJ (2007) Monte carlo methods in fuzzy linear regression. Soft Comput 11(10):991–996CrossRefzbMATHGoogle Scholar
  2. Abdalla A, Buckley JJ (2008) Monte carlo methods in fuzzy linear regression ii. Soft Comput 12(5):463–468. doi: 10.1007/s00500-007-0179-6 CrossRefzbMATHGoogle Scholar
  3. Chen H, Zhao H, Shen J, Zhou R, Zhou Q (2015) Supervised machine learning model for high dimensional gene data in colon cancer detection. In: IEEE BigData congress, pp 134–141Google Scholar
  4. Freitas JFGD, Niranjan MA, Gee AH, Doucet A (2000) Sequential monte carlo methods to train neural network models. Neural Comput 12(4):955CrossRefGoogle Scholar
  5. Hagan MT, Demuth HB, Beale MH (1995) Neural network design. PWS Publishing Company, Boston, MAGoogle Scholar
  6. İçen D, Cattaneo MEGV (2016) Different distance measures for fuzzy linear regression with Monte Carlo methods. Soft Comput. doi: 10.1007/s00500-016-2218-7
  7. Ji S, Xu W, Yang M, Yu K (2013) 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231. doi: 10.1109/TPAMI.2012.59
  8. Kreinovich V, Sirisaengtaksin O (1993) 3-layer neural networks are universal approximators for functionals and for control strategies. Neural Parallel Sci Comput 1(3):325–346Google Scholar
  9. Krizhevsky A, Sutskever I, Hinton GE (2012a) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates, Inc., Red Hook, pp 1097–1105Google Scholar
  10. Krizhevsky A, Sutskever I, Hinton GE (2012b) Imagenet classification with deep convolutional neural networks. In: International conference on neural information processing systems, pp 1097–1105Google Scholar
  11. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  12. Liang F (2007) Annealing stochastic approximation monte carlo algorithm for neural network training. Mach Learn 68(3):201–233CrossRefGoogle Scholar
  13. Wang H, Raj B (2017) On the origin of deep learning. arXiv preprintGoogle Scholar
  14. Wang L, Shen J, Zhou Q, Shang Z, Chen H, Zhao H (2016) An evaluation of the dynamics of diluted neural network. Int J Comput Intell Syst 9(6):1191–1199CrossRefGoogle Scholar
  15. Yong B, Xu Z, Shen J, Chen H, Tian Y, Zhou Q (2017) Neural network model with monte carlo algorithm for electricity demand forecasting in queensland. In: Australasian computer science week multiconference, p 47Google Scholar
  16. Zhao H (2016) General vector machine. arXiv preprintGoogle Scholar
  17. Zhou Q, Chen H, Zhao H, Zhang G, Yong J, Shen J (2016) A local field correlated and monte carlo based shallow neural network model for non-linear time series prediction. EAI Endorsed Trans Scalable Inf Syst 3(8):e5-1–e5-7Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  • Binbin Yong
    • 1
  • Fucun Li
    • 2
  • Qingquan Lv
    • 3
  • Jun Shen
    • 2
  • Qingguo Zhou
    • 1
    Email author
  1. 1.School of Information Science and EngineeringLanzhou UniversityLanzhouChina
  2. 2.School of Computing and Information TechnologyUniversity of WollongongWollongongAustralia
  3. 3.Wind Power Technology Center of Gansu Electirc Power CompanyLanzhouChina

Personalised recommendations