A Spiking Neural Network Architecture for Object Tracking

  • Yihao Luo
  • Quanzheng Yi
  • Tianjiang WangEmail author
  • Ling Lin
  • Yan Xu
  • Jing Zhou
  • Caihong Yuan
  • Jingjuan Guo
  • Ping Feng
  • Qi Feng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11901)


Spiking neural network (SNN) has the advantages of high computational efficiency, low energy consumption, low memory resource consumption, and easy hardware implementation. But its training algorithm is immature and inefficiency which limits the applications of SNN. In this paper, we propose a SNN architecture named SiamSNN for object tracking to avoid the training problems. Specifically, we propose a more comprehensive parameter conversion scheme with the processes of standardization, retraining, parameter transfer, and weight normalization, in order to convert a trained CNN to a similar SNN. Then we propose an encoder named Attention with Average Rate Over Time (AAR) in order to encoding images to spiking sequences. By using IF model, the accuracy decreases by only 0.007 on MNIST compared to the original method. Our approach applies SNN to object tracking and achieves certain effects, which is a reference for SNN applications in other computer vision areas in the future.


Spiking neural network Object tracking Conversion Encoder 


  1. 1.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 1097–1105. MIT Press, Lake Tahoe (2012)Google Scholar
  2. 2.
    Zhuang, B., Wang, L., Lu, H.: Visual tracking via shallow and deep collaborative model. Neurocomputing 218(61), 71 (2016)CrossRefGoogle Scholar
  3. 3.
    Tavanaei, A., Ghodrati, M., Kheradpisheh, S.R., et al.: Deep learning in spiking neural networks. Neural Netw. 111(47), 63 (2019)Google Scholar
  4. 4.
    Maass, W.: Lower bounds for the computational power of networks of spiking neurons. Neural Comput. 8(1), 1–40 (1996)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Neil, D., Pfeiffer, M., Liu, S.: Learning to be efficient: algorithms for training low-latency, low-compute deep spiking neural networks. In: ACM, pp. 293–298 (2016)Google Scholar
  6. 6.
    Xu, Q., Qi, Y., Yu, H., et al.: CSNN: an augmented spiking based framework with perceptron-inception. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 1646–1652. Morgan Kaufmann, Stockholm (2018)Google Scholar
  7. 7.
    Cao, Y., Chen, Y., Khosla, D.: Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vis. (IJCV) 113(1), 54–66 (2015)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Diehl, P.U., Neil, D., Binas, J., et al.: Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE, Killarney (2015)Google Scholar
  9. 9.
    Rueckauer, B., Lungu, I.A., Hu, Y., et al.: Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Front. Neurosci. 11, 682 (2017)CrossRefGoogle Scholar
  10. 10.
    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Technical report, University of Toronto (2009)Google Scholar
  11. 11.
    Deng, J., Dong, W., Socher, R., et al.: Imagenet: a large-scale hierarchical image database. In: Computer Vision and Pattern Recognition (CVPR), pp. 248–255. IEEE, Miami (2009)Google Scholar
  12. 12.
    Hu, Y., Tang, H., Wang, Y., et al.: Spiking deep residual network. arXiv preprint arXiv:1805.01352 (2018)
  13. 13.
    Sengupta, A., Ye, Y., Wang, R., et al.: Going deeper in spiking neural networks: VGG and residual architectures. Front. Neurosci. 13, 95 (2019)CrossRefGoogle Scholar
  14. 14.
    He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE, Las Vegas (2016)Google Scholar
  15. 15.
    Danelljan, M., Hager, G., Shahbaz, K.F., et al.: Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV), pp. 58–66. IEEE, Santiago (2015)Google Scholar
  16. 16.
    Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional siamese networks for object tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 850–865. Springer, Cham (2016). Scholar
  17. 17.
    Li, B., Yan, J., Wu, W., et al.: High performance visual tracking with siamese region proposal network. In: Computer Vision and Pattern Recognition (CVPR), pp. 8971–8980. IEEE, Salt Lake City (2018)Google Scholar
  18. 18.
    Ma, J., Shao, W., Ye, H., et al.: Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans. Multimed. 20(11), 3111–3122 (2018)CrossRefGoogle Scholar
  19. 19.
    Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., Hu, W.: Distractor-aware siamese networks for visual object tracking. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11213, pp. 103–119. Springer, Cham (2018). Scholar
  20. 20.
    Farabet, C.: Towards real-time image understanding with convolutional networks. Master Degree thesis. University Paris-Est, Paris (2013)Google Scholar
  21. 21.
    Krause, T.U., Wrtz, P.D.D.R.: Rate coding and temporal coding in a neural network. Master Degree thesis. University of Bochum, Germany (2014)Google Scholar
  22. 22.
    LeCun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yihao Luo
    • 1
  • Quanzheng Yi
    • 1
  • Tianjiang Wang
    • 1
    Email author
  • Ling Lin
    • 1
  • Yan Xu
    • 1
  • Jing Zhou
    • 1
    • 2
  • Caihong Yuan
    • 1
    • 3
  • Jingjuan Guo
    • 1
    • 4
  • Ping Feng
    • 1
    • 5
  • Qi Feng
    • 1
  1. 1.School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina
  2. 2.School of Mathematics and Computer ScienceJianghan UniversityWuhanChina
  3. 3.School of Computer and Information EngineeringHenan UniversityKaifengChina
  4. 4.School of Information Science and TechnologyJiujiang UniversityJiujiangChina
  5. 5.School of InformationGuizhou University of Finance and EconomicsGuiyangChina

Personalised recommendations