Handwritten Digits Recognition by Bio-inspired Hierarchical Networks

  • Antonio G. Zippo
  • Giuliana Gelsomino
  • Sara Nencini
  • Gabriele E. M. Biella
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 19)

Abstract

The human brain processes information showing learning and prediction abilities but the underlying neuronal mechanisms still remain unknown. Recently, many studies prove that neuronal networks are able of both generalizations and associations of sensory inputs.

In this paper, following a set of neurophysiological evidences, we propose a learning framework with a strong biological plausibility that mimics prominent functions of cortical circuitries. We developed the Inductive Conceptual Network (ICN), that is a hierarchical bio-inspired network, able to learn invariant patterns by Variable-order Markov Models implemented in its nodes. The outputs of the top-most node of ICN hierarchy, representing the highest input generalization, allow for automatic classification of inputs. We found that the ICN clusterized MNIST images with an error of 5.73% and USPS images with an error of 12.56%.

Keywords

pattern recognition handwritten digits abstraction process hierarchical network 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bassett, D.S., Greenfield, D.L., Meyer-Lindenberg, A., Weinberger, D.R., Moore, S.W., Bullmore, E.T.: Efficient physical embedding of topologically complex information processing networks in brains and computer circuits. PLoS Computational Biology 6(4), e1000748 (2010); Friston, K.J. (ed.), Public Library of Science, doi:10.1371/journal.pcbi.1000748Google Scholar
  2. 2.
    DiCarlo, J., Zoccolan, D., Rust, N.: How Does the Brain Solve Visual Object Recognition? Neuron 73, 415–434 (2012)CrossRefGoogle Scholar
  3. 3.
    Takahashi, N., Kitamura, K., Matsuo, N., Mayford, M., Kano, M., Matsuki, N., Ikegaya, Y.: Locally Synchronized Synaptic Inputs. Science 335, 353–356 (2012)CrossRefGoogle Scholar
  4. 4.
    Kleindienst, T., Winnubst, J., Roth-Alpermann, C., Bonhoeffer, T., Lohmann, C.: Activity-Dependent Clustering of Functional Synaptic Inputs on Developing Hippocampal Dendrites. Neuron 72(6), 1012–1024 (2011)CrossRefGoogle Scholar
  5. 5.
    Makino, H., Malinow, R.: Compartmentalized versus Global Synaptic Plasticity on Dendrites Controlled by Experience. Neuron 72(6), 1001–1011 (2011)CrossRefGoogle Scholar
  6. 6.
    Meyers, E.M., Qi, X.L., Constantinidis, C.: Incorporation of new information into prefrontal cortical activity after learning working memory tasks. Proc. Natl. Acad. Sci. USA (2012), doi:10.1073/pnas.1201022109Google Scholar
  7. 7.
    Branco, T., Clark, B.A., Hausser, M.: Dendritic Discrimination of Temporal Input Sequences in Cortical Neurons. Science 329(5999), 1671–1675 (2010)CrossRefGoogle Scholar
  8. 8.
    Bühlmann, P., Wyner, A.J.: Variable Length Markov Chains. The Annals of Statistics 27(2), 480–513 (1999)MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Begleiter, R., El-Yaniv, R., Yona, G.: On Prediction Using Variable Order Markov Models. Journal of Artificial Intelligence Research 22, 385–421 (2004)MathSciNetMATHGoogle Scholar
  10. 10.
    Hawkins, J., Blackeslee, S.: On Intellingence. Times Books, New York (2004)Google Scholar
  11. 11.
    Dileep, G., Hawkins, J.: Towards a Mathematical Theory of Cortical Micro-circuits. PLoS Comput. Biol. 5(10), e1000532 (2009)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Baum, L.E., Petrie, T.: Statistical Inference for Probabilistic Functions of Finite State Markov Chains. The Annals of Mathematical Statistics 37(6), 1554–1563 (1966)MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains. The Annals of Mathematical Statistics 41, 164 (1970)MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Rissanen, J.: Universal coding, information, prediction, and estimation. IEEE Transactions on Information Theory 30(4), 629–636 (1984)Google Scholar
  15. 15.
    Cleary, J., Witten, I.: Data Compression Using Adaptive Coding and Partial String Matching. IEEE Transactions on Communications 32(4), 396–402 (1984)CrossRefGoogle Scholar
  16. 16.
    Teahan, W.: Probability estimation for PPM. In: Proceedings of the New Zealand Computer Science Research Students’ Conference. University of Waikato, Hamilton (1995)Google Scholar
  17. 17.
  18. 18.
    Hamming, R.W.: Error detecting and error correcting codes. Bell System Technical Journal 29(2), 147–160 (1950)MathSciNetGoogle Scholar
  19. 19.
    Hebb, D.O.: The organization of behavior; a neuropsychological theory. Wiley, New York (1949)Google Scholar
  20. 20.
    Hubel, D., Wiesel, T.: Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology 195, 215–243 (1968)Google Scholar
  21. 21.
    LeCun, Y., Boser, B., Denker, J.S., et al.: Handwritten Digit Recognition with a Back-Propagation Network. In: Touretzky, D. (ed.) Advances in Neural Information Processing Systems 2 (NIPS 1989). Morgan Kaufman, Denver (1990)Google Scholar
  22. 22.
    LeCun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks. MIT Press (1995)Google Scholar
  23. 23.
    Powell, W.B.: Approximate Dynamic Programming: Solving the Curse of Dimensionality. Wiley, New York (2007)CrossRefGoogle Scholar
  24. 24.
    Kotsiantis, S., Kanellopoulos, D., Pintelas, P.: Data Preprocessing for Supervised Leaning. International Journal of Computer Science 1(2), 111–117 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Antonio G. Zippo
    • 1
  • Giuliana Gelsomino
    • 1
  • Sara Nencini
    • 1
  • Gabriele E. M. Biella
    • 1
  1. 1.Institute of Molecular Bioimaging and Physiology, Consiglio Nazionale delle RicercheMilanItaly

Personalised recommendations