Advertisement

A Neural Spiking Approach Compared to Deep Feedforward Networks on Stepwise Pixel Erasement

Conference paper
First Online:
  • 5.5k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11139)

Abstract

In real world scenarios, objects are often partially occluded. This requires a robustness for object recognition against these perturbations. Convolutional networks have shown good performances in classification tasks. The learned convolutional filters seem similar to receptive fields of simple cells found in the primary visual cortex. Alternatively, spiking neural networks are more biological plausible. We developed a two layer spiking network, trained on natural scenes with a biologically plausible learning rule. It is compared to two deep convolutional neural networks using a classification task of stepwise pixel erasement on MNIST. In comparison to these networks the spiking approach achieves good accuracy and robustness.

Keywords

STDP Unsupervised learning Deep convolutional networks 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgement

This work was supported by the European Social Fund (ESF) and the Freistaat Sachsen.

References

  1. 1.
    Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154 (1962)CrossRefGoogle Scholar
  2. 2.
    Jones, J.P., Palmer, L.A.: The two-dimensional spatial structure of simple receptive fields in cat striate cortex. J. Neurophysiol. 85, 187–211 (1987)Google Scholar
  3. 3.
    Beaulieu, C., Kisvarday, Z., Somogyi, P., Cynaer, M., Cowey, A.: Quantitative distribution of GABA-immunopositive and - immunonegative neurons and synapses in the monkey striate cortex (Area 17). Cereb. Cortex 2, 295–309 (1992)CrossRefGoogle Scholar
  4. 4.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)CrossRefGoogle Scholar
  5. 5.
    LeCun, Y., Bottou, L., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  6. 6.
    Priebe, N.J., Ferster, D.: Inhibition, Spike Threshold, and Stimulus Selectivity in Primary Visual Cortex. Neuron 4, 482–497 (2008)CrossRefGoogle Scholar
  7. 7.
    Clopath, C., Büsing, L., Vasilaki, E., Gerstner, W.: Connectivity reflects coding: a model of voltage-based STDP with homeostasis. Nat. Neurosci. 13, 344–352 (2010)CrossRefGoogle Scholar
  8. 8.
    Katzner, S., Busse, L., Carandini, M.: GABAA inhibition controls response gain in visual cortex. J. Neurosci. 31, 5931–5941 (2011)CrossRefGoogle Scholar
  9. 9.
    Vogels, T.P., Sprekeler, H., Zenke, F., Clopath, C., Gerstner, W.: Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science 334, 1569–1573 (2011)CrossRefGoogle Scholar
  10. 10.
    Cireşan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. arXiv:1202.2745 (2012)
  11. 11.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)Google Scholar
  12. 12.
    Zeiler, M.D.: ADADELTA: an adaptive learning rate method arXiv:1212.5701v1 (2012)
  13. 13.
    Potjans, T.C., Diesmann, M.: The cell-type specific cortical microcircuit: relating structure and activity in a full-scale spiking network model. Cereb. Cortex 24, 785–806 (2014)CrossRefGoogle Scholar
  14. 14.
    Bengio, Y., Lee, D.H., Bornschein, J., Lin, Z.: Towards biologically plausible deep learning. arXiv:1703.08245 (2015)
  15. 15.
    Chollet, F., et al.: Keras (2015). https://keras.io. Accessed 23 Apr 2018
  16. 16.
    Diehl, P.U., Cook, M.: Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front. Comput. Neurosci. 9, 99 (2015)CrossRefGoogle Scholar
  17. 17.
    Kermani Kolankeh, A., Teichmann, M., Hamker, F.H.: Competition improves robustness against loss of information. Front. Comput. Neurosci. 9, 35 (2015)CrossRefGoogle Scholar
  18. 18.
    Russakovsky, O., Denk, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2015)Google Scholar
  20. 20.
    Vitay, J., Dinkelbach, H.Ü., Hamker, F.H.: ANNarchy: a code generation approach to neural simulations on parallel hardware. Front. Neuroinformatics 9, 19 (2015).  https://doi.org/10.3389/fninf.2015.00019CrossRefGoogle Scholar
  21. 21.
    Cichy, R.M., Khosla, A., Pantazis, D., Torralba, A., Oliva, A.: Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Sci. Rep. 6, 27755 (2016)CrossRefGoogle Scholar
  22. 22.
    Kheradpisheh, S.R., Ganjtabesh, M., Thorpe, S.J., Masquelier, T.: STDP-based spiking deep convolutional neural networks for object recognition. arXiv:1611.01421 (2017)
  23. 23.
    Tavanaei, A., Maida, A.S.: Multi-layer unsupervised learning in a spiking convolutional neural network. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2023–2030 (2017)Google Scholar
  24. 24.
    Wen, H., Shi, J., Zhang, Y., Lu, K., Cao, J., Liu, Z.: Neural encoding and decoding with deep learning for dynamic natural vision. Cereb. Cortex, 1–25 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceChemnitz University of TechnologyChemnitzGermany

Personalised recommendations

Cite paper

Buy options