Application of Support Vector Machines, Convolutional Neural Networks and Deep Belief Networks to Recognition of Partially Occluded Objects

  • Joseph Lin Chu
  • Adam Krzyżak
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8467)


Artificial neural networks have been widely used for machine learning tasks such as object recognition. Recent developments have made use of biologically inspired architectures, such as the Convolutional Neural Network, and the Deep Belief Network. We test the hypothesis that generative models such as the Deep Belief Network should perform better on occluded object recognition tasks than purely discriminative models such as Convolutional Neural Networks. We find that the data does not support this hypothesis when the generative models are run in a partially discriminative manner. We also find that the use of Gaussian visible units in a Deep Belief Network trained on occluded image data allows it to also learn to classify non-occluded images.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for boltzmann machines. Cognitive Science 9, 147–169 (1985)CrossRefGoogle Scholar
  2. 2.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1–27:27 (2011), software available at
  3. 3.
    Collobert, R., Bengio, S.: Links between perceptrons, mlps and svms. In: Proceedings of the 21st International Conference on Machine Learning (ICML), p. 23 (2004)Google Scholar
  4. 4.
    Cortes, C., Vapnik, V.N.: Support-vector networks. Machine Learning 20, 273–297 (1995)zbMATHGoogle Scholar
  5. 5.
    Fukushima, K.: Neocognitron for handwritten digit recognition. Neurocomputing 51, 161–180 (2003)CrossRefGoogle Scholar
  6. 6.
    Fukushima, K., Miyake, S.: Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognition 15(6), 455–469 (1982)CrossRefGoogle Scholar
  7. 7.
    Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1771–1800 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Hinton, G.E.: A practical guide to training restricted boltzmann machines. Momentum 9(1), 599–619 (2010)Google Scholar
  9. 9.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18, 1527–1554 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the USA 79(8), 2554–2558 (1982)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Huang, F.J., LeCun, Y.: Large-scale learning with svm and convolutional nets for generic object categorization. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 284–291 (2006)Google Scholar
  13. 13.
    Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in a cat’s visual cortex. Journal of Physiology 160, 106–154 (1962)Google Scholar
  14. 14.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  15. 15.
    LeCun, Y., Huang, F., Bottou, L.: Learning methods for generic object recognition with invariance to pose and lighting. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 97–104 (2004)Google Scholar
  16. 16.
    Martinez, A.M.: Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(6), 748–763 (2002)CrossRefGoogle Scholar
  17. 17.
    Nair, V., Hinton, G.E.: 3d object recognition with deep belief nets. In: Advances in Neural Information Processing Systems (NIPS), pp. 1339–1347 (2009)Google Scholar
  18. 18.
    Ranzato, M., Susskind, J., Mnih, V., Hinton, G.: On deep generative models with applications to recognition. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2857–2864 (2011)Google Scholar
  19. 19.
    Ranzato, M.A., Huang, F.J., Boureau, Y.L., LeCun, Y.: Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)Google Scholar
  20. 20.
    Salakhutdinov, R., Hinton, G.E.: Deep boltzmann machines. In: International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 448–455 (2009)Google Scholar
  21. 21.
    Smolensky, P.: Information processing in dynamical systems: Foundations of harmony theory. In: Rumelhart, D.E., McLelland, J.L. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, ch. 6, pp. 194–281. MIT Press (1986)Google Scholar
  22. 22.
    Tsang, P.W.M., Yuen, P.C.: Recognition of partially occluded objects. IEEE Transactions on Systems, Man and Cybernetics 23(1), 228–236 (1993)CrossRefzbMATHGoogle Scholar
  23. 23.
    Winn, J., Shotton, J.: The layout consistent random field for recognizing and segmenting partially occluded objects. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 37–44 (2006)Google Scholar
  24. 24.
    Wiskott, L., Malsburg, C.V.D.: A neural system for the recognition of partially occluded objects in cluttered scenes: A pilot study. International Journal of Pattern Recognition and Artificial Intelligence 7(4), 935–948 (1993)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Joseph Lin Chu
    • 1
  • Adam Krzyżak
    • 1
  1. 1.Department of Computer Science and Software EngineeringConcordia UniversityMontrealCanada

Personalised recommendations