Advertisement

Spike Encoding Modules Using Neuron Model in Neural Networks

  • Yeeun KimEmail author
  • Seunghee Lee
  • Wonho Song
  • Hyun Myung
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1015)

Abstract

There has been a great increase in performance of deep neural networks. However, for mobile devices which are not equipped with GPU (Graphics Processing Unit) or powerful CPU (Central Processing Unit), it is still impossible to deal with such a large amount of data in real time. In this paper, preliminary results in spike neural encoding methods reducing the amount of the input and computational load by mimicking the neuronal firing are presented. For this, two neuron models, leaky integrate-and-fire (LIF) model and simplified IF model, are exploited for transforming the input image to the spike image. For the evaluation, MNIST datasets are encoded and tested in deep neural networks for checking the loss of information. The proposed spike encoding modules using neuron models will be able to greatly help reduce required computation by using spike input data in low powered mobile devices.

Keywords

Low-powered Neural encoding Spike input Spike conversion Spike encoding using neuron model 

Notes

Acknowledgement

This work was supported by the ICT R&D program of MSIP/IITP (2016-0-00563, Research on Adaptive Machine Learning Technology Development for Intelligent Autonomous Digital Companion). The students are supported by Korea Minister of Ministry of Land, Infrastructure and Transport (MOLIT) as U-City Master and Doctor Course Grant Program.

References

  1. 1.
    Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Proceedings of Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)Google Scholar
  2. 2.
    Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition. IEEE Sig. Process. Mag. 29(6), 82–97 (2012)CrossRefGoogle Scholar
  3. 3.
    Mikolov, T., Karafiat, M., Burget, L., Cernocky, J., Khudanpur, S.: Recurrent neural network based language model. In: Interspeech, vol. 2, p. 3 (2010)Google Scholar
  4. 4.
    Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. In: Proceedings of Neural Information Processing Systems (NIPS) (2015)Google Scholar
  5. 5.
    Tavanaei, A., Maida, A.: Bio-inspired spiking convolutional neural network using layer-wise sparse coding and STDP learning. arXiv preprint arXiv:1611.03000 (2016)
  6. 6.
    Trabelsi, C., et al.: Deep complex networks. arXiv preprint arXiv:1705.09792 (2017)
  7. 7.
    Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or −1. arXiv preprint arXiv:1602.02830 (2016)
  8. 8.
    Long, L.: Scalable biologically inspired neural networks with spike time based learning. In: Proceedings Learning and Adaptive Behaviors for Robotic Systems, LAB-RS 2008, pp. 29–34. ECSIS (2008)Google Scholar
  9. 9.
    Lee, S., Kim, K., Kim, J., Kim, Y., Myung, H.: Spike-inspired deep neural network design using binary weight. In: Proceedings of International Conference on Control, Automation and Systems (ICCAS) (2018)Google Scholar
  10. 10.
    Kim, K., Kim, J., Lee, S., Kim, Y., Myung, H.: Development of the image to spike conversion algorithm for the deep neural networks. In: Proceedings of International Conference on Control, Automation and Systems (ICCAS) (2018)Google Scholar
  11. 11.
    Abbott, L.F.: Lapicque’s introduction of the integrate-and-fire model neuron (1907). Brain Res. Bull. 50(5–6), 303–304 (1999)CrossRefGoogle Scholar
  12. 12.
    Xie, X., Qu, H., Yi, Z., Kurths, J.: Efficient training of supervised spiking neural network via accurate synaptic-efficiency adjustment method. IEEE Trans. Neural Netw. Learn. Syst. 28(6), 1411–14247 (2017)CrossRefGoogle Scholar
  13. 13.
    Izhikevich, E.M.: Simple model of spiking neurons. IEEE Trans. Neural Netw. 14(6), 1569–1572 (2013)CrossRefGoogle Scholar
  14. 14.
    Hodgkin, A., Huxley, A.: A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117(4), 500–544 (1952)CrossRefGoogle Scholar
  15. 15.
    Hodgkin, A., Huxley, A., Katz, B.: Measurement of current-voltage relations in the membrane of the giant axon of Loligo. J. Physiol. 116(4), 424–448 (1952)CrossRefGoogle Scholar
  16. 16.
    Hodgkin, A., Huxley, A.: Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo. J. Physiol. 116(4), 449–472 (1952)CrossRefGoogle Scholar
  17. 17.
    Hodgkin, A., Huxley, A.: The components of membrane conductance in the giant axon of Loligo. J. Physiol. 116(4), 473–496 (1952)CrossRefGoogle Scholar
  18. 18.
    Runge, C.: Über die numerische Auflösung von Differentialgleichungen. Mathematische Annalen 46(2), 167–178 (1895). SpringerMathSciNetCrossRefGoogle Scholar
  19. 19.
    LeCun, Y., Cortes, C., Burges, C.: MNIST handwritten digit database. AT&T Labs, (2018). http://yann.lecun.com/exdb/mnist
  20. 20.
    Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)Google Scholar
  21. 21.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)Google Scholar
  22. 22.
    NVIDIA: DIGITS Deep Learning Framework (2018). https://github.com/NVIDIA/DIGITS
  23. 23.
    Nemoto, I., Saito, K.: A complex-valued version of Nagumo-Sato model of a single neuron and its behavior. Neural Netw. 15(7), 833–853 (2002)CrossRefGoogle Scholar
  24. 24.
    Iakymchuk, T., Rosado-Muñoz, A., Guerrero-Martínez, J., Bataller-Mompeán, M., Francés-Víllora, J.: Simplified spiking neural network architecture and STDP learning algorithm applied to image classification. EURASIP J. Image Video Process. (2015) Google Scholar
  25. 25.
    Bologna, L., et al.: A closed-loop neurobotic system for fine touch sensing. J. Neural Eng. 10(4), 046019 (2016)CrossRefGoogle Scholar
  26. 26.
    Thorpe, S., Delorme, A., Van Rullen, R.: Spike-based strategies for rapid processing. Neural Netw. 14(6-7), 715–725 (2001)CrossRefGoogle Scholar
  27. 27.
    Liu, Q., et al.: Benchmarking spike-based visual recognition: a dataset and evaluation. Front. Neurosci. 10, 496 (2016)Google Scholar
  28. 28.
    Heeger, D.: Poisson model of spike generation. Handout Univ. Standford 5, 1–13 (2000)Google Scholar
  29. 29.
    Fatahi, M., et al.: evt_MNIST: a spike based version of traditional MNIST. arXiv preprint arXiv:1604.06751 (2016)
  30. 30.
    Sen, B., Steve, F.: Evaluating rank-order code performance using a biologically-derived retinal model. In: International Joint Conference on IEEE (IJCNN), pp. 2867–2874 (2009)Google Scholar
  31. 31.
    Serrano-Gotarredona, T., Linares-Barranco, B.: A 128 × 128 1.5% contrast sensitivity 0.9% FPN 3 µs latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. J. Solid-State Circ. 48(3), 827–838 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Yeeun Kim
    • 1
    Email author
  • Seunghee Lee
    • 2
  • Wonho Song
    • 2
  • Hyun Myung
    • 1
    • 2
  1. 1.Robotics ProgramKorean Advanced Institute for Science and Technology (KAIST)DaejeonKorea
  2. 2.Department of Civil and Environmental EngineeringKorean Advanced Institute for Science and Technology (KAIST)DaejeonKorea

Personalised recommendations