Fourier Kernel Learning

  • Eduard Gabriel Băzăvan
  • Fuxin Li
  • Cristian Sminchisescu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)


Approximations based on random Fourier embeddings have recently emerged as an efficient and formally consistent methodology to design large-scale kernel machines [23]. By expressing the kernel as a Fourier expansion, features are generated based on a finite set of random basis projections, sampled from the Fourier transform of the kernel, with inner products that are Monte Carlo approximations of the original non-linear model. Based on the observation that different kernel-induced Fourier sampling distributions correspond to different kernel parameters, we show that a scalable optimization process in the Fourier domain can be used to identify the different frequency bands that are useful for prediction on training data. This approach allows us to design a family of linear prediction models where we can learn the hyper-parameters of the kernel together with the weights of the feature vectors jointly. Under this methodology, we recover efficient and scalable linear reformulations for both single and multiple kernel learning. Experiments show that our linear models produce fast and accurate predictors for complex datasets such as the Visual Object Challenge 2011 and ImageNet ILSVRC 2011.


Fourier Domain Kernel Parameter Feature Channel Multiple Kernel Multiple Kernel Learning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bach, F.: Consistency of the group Lasso and multiple kernel learning. JMLR 9, 1179–1225 (2008)zbMATHMathSciNetGoogle Scholar
  2. 2.
    Bach, F.: Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning. In: NIPS (2009)Google Scholar
  3. 3.
    Bach, F., Lanckriet, G.R.G., Jordan, M.I.: Multiple kernel learning, conic duality, and the SMO algorithm. In: ICML (2004)Google Scholar
  4. 4.
    Carreira, J., Sminchisescu, C.: Constrained Parametric Min-Cuts for Automatic Object Segmentation. In: CVPR (2010)Google Scholar
  5. 5.
  6. 6.
    Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S.: Choosing Multiple Parameters for Support Vector Machines. Machine Learning 46, 131–159 (2002)zbMATHCrossRefGoogle Scholar
  7. 7.
    Cortes, C., Mohri, M., Rostamizadeh, A.: Learning Non-linear Combinations of Kernels. In: NIPS (2009)Google Scholar
  8. 8.
    Cortes, C., Mohri, M., Rostamizadeh, A.: Two-Stage Learning Kernel Algorithms. In: ICML (2010)Google Scholar
  9. 9.
    Cristianini, N., Kandola, J., Elisseeff, A., Shawe-Taylor, J.: On Kernel-Target Alignment. In: NIPS (2002)Google Scholar
  10. 10.
    Dekel, O., Shalev-Shwartz, S., Singer, Y.: Smooth epsilon-Insensitive Regression by Loss Symmetrization (2003)Google Scholar
  11. 11.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale Hierarchical Image Database. In: CVPR (2009)Google Scholar
  12. 12.
    Li, F., Fu, Y., Dai, Y.H., Sminchisescu, C., Wang, J.: Kernel learning by unconstrained optimization. In: AISTATS (2009)Google Scholar
  13. 13.
    Maire, M., Arbelaez, P., Fowlkes, C., Malik, J.: Using Contours to Detect and Localize Junctions in Natural Images. In: CVPR (2008)Google Scholar
  14. 14.
    Gehler, P., Nowozin, S.: On Feature Combination for Multiclass Object Classification. In: ICCV (2009)Google Scholar
  15. 15.
    Keerthi, S., Sindhwani, V., Chapelle, O.: An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models. In: NIPS (2007)Google Scholar
  16. 16.
    Kloft. M., Brefeld, U., Sonnenburg, S., Laskov, P., Müller, K. R., Zien, A.: Efficient and Accurate Lp-Norm Multiple Kernel Learning. In: NIPS (2009)Google Scholar
  17. 17.
    Lanckriet, G.R.G., Cristianini, N., Bartlett, P., El Ghaoui, L., Jordan, M.I.: Learning the Kernel Matrix with Semidefinite Programming. JMLR 5, 27–72 (2004)zbMATHGoogle Scholar
  18. 18.
    Li, F., Carreira, J., Sminchisescu, C.: Object Recognition as Ranking Holistic Figure-Ground Hypotheses. In: CVPR (2010)Google Scholar
  19. 19.
    Li, F., Ionescu, C., Sminchisescu, C.: Random Fourier Approximations for Skewed Multiplicative Histogram Kernels. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) DAGM 2010. LNCS, vol. 6376, pp. 262–271. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  20. 20.
    Li, F., Sminchisescu, C.: The Feature Selection Path in Kernel Methods. In: AISTATS (2010)Google Scholar
  21. 21.
    Schmidt, M., van den Berg, E., Friedlander, M.P., Murphy, K.: Optimizing Costly Functions with Simple Constraints: A Limited-Memory Projected Quasi-Newton Algorithm. In: AISTATS (2009)Google Scholar
  22. 22.
    Maji, S., Berg, A.C., Malik, J.: Classification using Intersection Kernel Support Vector Machines is Efficient. In: CVPR (2008)Google Scholar
  23. 23.
    Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: NIPS (2007)Google Scholar
  24. 24.
    Jason, D.M.R.: Maximum-Margin Logistic Regression (2004),
  25. 25.
    Li, F., Lebanon, G., Sminchisescu, C.: Chebyshev Approximations to the Histogram χ2 Kernel. In: ICCV (2012)Google Scholar
  26. 26.
    Rudin, W.: Fourier Analysis on Groups. Wiley Interscience (1990)Google Scholar
  27. 27.
    Schölkopf, B., Smola, A.: Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press (2002)Google Scholar
  28. 28.
    Sreekanth, V., Vedaldi, A., Jawahar, C.V., Zisserman, A.: Generalized RBF feature maps for efficient detection. In: BMVC (2010)Google Scholar
  29. 29.
    van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating Color Descriptors for Object and Scene Recognition. PAMI 9, 1582–1596 (2010)CrossRefGoogle Scholar
  30. 30.
    Varma, M., Babu, B.R.: More Generality in Efficient Multiple Kernel Learning. In: ICML (2009)Google Scholar
  31. 31.
    Vedaldi, A., Fulkerson, B.: VLFeat – An open and portable library of computer vision algorithms. In: ACM International Conference on Multimedia (2010)Google Scholar
  32. 32.
    Vedaldi, A., Gulshan, V., Varma, M., Zisserman, A.: Multiple Kernels for Object Detection. In: ICCV (2009)Google Scholar
  33. 33.
    Vedaldi, A., Zisserman, A.: Efficient Additive Kernels via Explicit Feature Maps. In: CVPR (2010)Google Scholar
  34. 34.
    Visnwanathan, S.V.N., Sun, Z., Theera-Ampornpunt, N., Varma, M.: Multiple Kernel Learning and the SMO Algorithm. In: NIPS (2010)Google Scholar
  35. 35.
    Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge, VOC 2011 (2011)Google Scholar
  36. 36.
    Bazavan, E., Li, F., Sminchisescu, C.: Learning Kernels in Fourier Space. Technical Report, Romanian Academy of Sciences and University of Bonn (July 2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Eduard Gabriel Băzăvan
    • 1
  • Fuxin Li
    • 2
  • Cristian Sminchisescu
    • 3
    • 1
  1. 1.Institute of Mathematics of the Romanian AcademyRomania
  2. 2.College of ComputingGeorgia Institute of TechnologyUSA
  3. 3.Faculty of Mathematics and Natural ScienceUniversity of BonnGermany

Personalised recommendations