V1-Inspired Features Induce a Weighted Margin in SVMs

  • Hilton Bristow
  • Simon Lucey
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)


Image representations derived from simplified models of the primary visual cortex (V1), such as HOG and SIFT, elicit good performance in a myriad of visual classification tasks including object recognition/detection, pedestrian detection and facial expression classification. A central question in the vision, learning and neuroscience communities regards why these architectures perform so well. In this paper, we offer a unique perspective to this question by subsuming the role of V1-inspired features directly within a linear support vector machine (SVM). We demonstrate that a specific class of such features in conjunction with a linear SVM can be reinterpreted as inducing a weighted margin on the Kronecker basis expansion of an image. This new viewpoint on the role of V1-inspired features allows us to answer fundamental questions on the uniqueness and redundancies of these features, and offer substantial improvements in terms of computational and storage efficiency.


Neural Information Processing System Linear Support Vector Machine Rank Reduction Kernel Support Vector Machine Convolutional Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Ashraf, A.B., Lucey, S., Chen, T.: Reinterpreting the application of Gabor filters as a manipulation of the margin in linear support vector machines. Pattern Analysis and Machine Learning 32(7), 1335–1341 (2010)CrossRefGoogle Scholar
  2. 2.
    Bergstra, J., Desjardins, G., Lamblin, P., Bengio, Y.: Quadratic polynomials learn better image features. Technical report, Universite de Montreal (2009)Google Scholar
  3. 3.
    Bo, L., Ren, X., Fox, D.: Kernel Descriptors for Visual Recognition. In: Advances in Neural Information Processing Systems, pp. 1–9 (2010)Google Scholar
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, pp. 886–893 (2005)Google Scholar
  5. 5.
    Fan, R., Chang, K., Hsieh, C., Wang, X., Lin, C.: LIBLINEAR: A library for large linear classification. The Journal of Machine Learning Research 9, 1871–1874 (2008)zbMATHGoogle Scholar
  6. 6.
    Fodor, I.: A survey of dimension reduction techniques. Technical report, Lawrence Livermore National Laboratory (2002)Google Scholar
  7. 7.
    Hubel, D., Wiesel, T.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of Physiology 160(1), 106 (1962)CrossRefGoogle Scholar
  8. 8.
    Jarrett, K., Kavukcuoglu, K., Ranzato, M.A., LeCun, Y.: What is the best multi-stage architecture for object recognition? In: International Conference on Computer Vision, pp. 2146–2153 (September 2009)Google Scholar
  9. 9.
    Lades, M., Vorbruggen, J., Buhmann, J., Lange, J., von der Malsburg, C., Wurtz, R., Konen, W.: Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers 42(3), 300–311 (1993)CrossRefGoogle Scholar
  10. 10.
    Lee, H., Battle, A., Raina, R., Ng, A.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems, vol. 19, p. 801 (2007)Google Scholar
  11. 11.
    Liu, C., Wechsler, H.: Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Transactions on Image Processing 11(4), 467–476 (2002)CrossRefGoogle Scholar
  12. 12.
    Lowe, D.: Object recognition from local scale-invariant features. In: International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999)Google Scholar
  13. 13.
    Lucey, P., Lucey, S., Cohn, J.: Registration invariant representations for expression detection. In: International Conference on Digital Image Computing: Techniques and Applications, vol. (i), pp. 255–261 (2010)Google Scholar
  14. 14.
    Pinto, N., Cox, D.D., DiCarlo, J.J.: Why is real-world visual object recognition hard? PLoS computational biology 4(1), e27 (2008)Google Scholar
  15. 15.
    Saxe, A., Koh, P., Chen, Z., Bhand, M., Suresh, B., Ng, A.: On random weights and unsupervised feature learning. In: Advances in Neural Information Processing Systems, pp. 1–9 (2010)Google Scholar
  16. 16.
    Shivaswamy, P., Jebara, T.: Relative margin machines. In: Advances in Neural Information Processing Systems, vol. 21, pp. 1–8 (2008)Google Scholar
  17. 17.
    Vedaldi, A., Zisserman, A.: Efficient additive kernels via explicit feature maps. Computer Vision and Pattern Recognition, vol. (iii), pp. 3539–3546 (2010)Google Scholar
  18. 18.
    Yang, M., Zhang, L.: Gabor Feature Based Sparse Representation for Face Recognition with Gabor Occlusion Dictionary. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 448–461. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Hilton Bristow
    • 1
    • 2
  • Simon Lucey
    • 2
  1. 1.Queensland University of TechnologyAustralia
  2. 2.Commonwealth Scientific and Industrial Research OrganisationAustralia

Personalised recommendations