Skip to main content

A Family of Maximum Margin Criterion for Adaptive Learning

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 11303)

Abstract

In recent years, pattern analysis plays an important role in data mining and recognition, and many variants have been proposed to handle complicated scenarios. In the literature, it has been quite familiar with high dimensionality of data samples, but either such characteristics or large data sets have become usual sense in real-world applications. In this work, an improved maximum margin criterion (MMC) method is introduced firstly. With the new definition of MMC, several variants of MMC, including random MMC, layered MMC, 2D\( ^2 \) MMC, are designed to make adaptive learning applicable. Particularly, the MMC network is developed to learn deep features of images in light of simple deep networks. Experimental results on a diversity of data sets demonstrate the discriminant ability of proposed MMC methods are component to be adopted in complicated application scenarios.

Keywords

  • Maximum margin criterion (MMC)
  • Adaptive learning
  • Variants of MMC
  • MMC network

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-04182-3_33
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-04182-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.

Notes

  1. 1.

    The implementations are available at: http://mch.one/resources.

  2. 2.

    http://vision.princeton.edu/projects/2010/SUN.

  3. 3.

    http://yann.lecun.com/exdb/mnist.

  4. 4.

    https://keras.io.

  5. 5.

    http://aloi.science.uva.nl.

References

  1. Turk, M.A., Pentland, A.P.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 195–197 (1981)

    Google Scholar 

  2. Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans. Patt. Anal. Machine Intell. 19(7), 711–720 (1997)

    CrossRef  Google Scholar 

  3. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2011)

    MATH  Google Scholar 

  4. Yu, H., Yang, J.: A direct LDA algorithm for high-dimensional data - with application to face recognition. Patt. Recog. 19(7), 2067–2070 (2001)

    CrossRef  Google Scholar 

  5. Li, H., Jiang, T., Zhang, K.: Efficient and robust feature extraction by maximum margin criterion. IEEE Trans. Neural Netw. 17(1), 157–165 (2006)

    CrossRef  Google Scholar 

  6. Liu, J., Chen, S., Tan, X., et al.: Comments on ‘Efficient and robust feature extraction by maximum margin criterion’. IEEE Trans. Neural Netw. 18(6), 1862–1864 (2007)

    CrossRef  Google Scholar 

  7. Cheng, M., Tang, Y.Y., Pun, C.-M.: Nonparametric feature extraction via direct maximum margin alignment. In: Proceedings of IEEE International Conference on Machine Learning and Applications, Hawaii, USA, pp. 585–591 (2010)

    Google Scholar 

  8. Cheng, M., Fang, B., Tang, Y.Y., et al.: Incremental embedding and learning in the local discriminant subspace with application to face recognition. IEEE Trans. Syst. Man. Cybern. Part C Appl. Rev. 40(5), 580–591 (2010)

    CrossRef  Google Scholar 

  9. Yan, S., Xu, D., Zhang, B., et al.: Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans. Patt. Anal. Mach. Intell. 29(1), 40–51 (2007)

    CrossRef  Google Scholar 

  10. Cheng, M., Tsoi, A.C.: CRH: a simple benchmark approach to continuous hashing. In: Proceedings of IEEE Global Conference on Signal and Information Processing, Orlando, USA, pp. 1076–1080 (2015)

    Google Scholar 

  11. Huang, G.-B., Zhou, H., Ding, X., et al.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B Cybern. 42(2), 513–529 (2012)

    CrossRef  Google Scholar 

  12. Yang, J., Zhang, D., Frangi, A.F., et al.: Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans. Patt. Anal. Mach. Intell. 26(1), 131–137 (2004)

    CrossRef  Google Scholar 

  13. Kong, H., Li, X., Wang, L., et al.: Generalized 2D principal component analysis. In: Proceedings of IEEE International Joint Conference on Neural Networks, Montreal, Canada (2005)

    Google Scholar 

  14. Li, M., Yuan, B.: 2D-LDA: a statistical linear discriminant analysis for image matrix. Patt. Recog. Lett. 26(5), 527–532 (2005)

    CrossRef  Google Scholar 

  15. Xiong, H., Swamy, M.N.S., Ahmad, M.O.: Two-dimensional FLD for face recognition. Patt. Recog. 38(7), 1121–1124 (2005)

    CrossRef  Google Scholar 

  16. Ye, J., Janardan, R., Li, Q.: Two-dimensional linear discriminant analysis. In: NIPS (2004)

    Google Scholar 

  17. Chan, T.-H., Jia, K., Gao, S., et al.: PCANet: a simple deep learning baseline for image classification? IEEE Trans. Patt. Anal. Mach. Intell. 24(12), 5017–5032 (2015)

    MathSciNet  Google Scholar 

  18. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Patt. Anal. Mach. Intell. 24(7), 971–987 (2002)

    CrossRef  Google Scholar 

  19. Xiao, J., Ehinger, K.A., Hays, J., et al.: SUN database: exploring a large collection of scene categories. IJCV 119(1), 3–22 (2016)

    MathSciNet  CrossRef  Google Scholar 

  20. Lecun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 569–571 (1998)

    CrossRef  Google Scholar 

  21. Labusch, K., Barth, E., Martinetz, T.: Simple method for high-performance digit recognition based on sparse coding. IEEE Trans. Neural Netw. 19(11), 1985–1989 (2008)

    CrossRef  Google Scholar 

  22. Coates, A., Lee, H., Ng, A.Y.: An analysis of single-layer networks in unsupervised feature learning. In: Proceedings of International Conference on Artificial Intelligence and Statistics, Ft. Lauderdale, USA, pp. 215–223 (2011)

    Google Scholar 

  23. Yang, S., Luo, P., Loly, C.C., et al.: Deep representation learning with target coding. In: AAAI, Austin Texas, USA, pp. 3848–3854 (2015)

    Google Scholar 

  24. Geusebroek, J.M., Burghouts, G.J., Smeulders, W.M.: The amsterdam library of object images. IJCV 61(1), 103–112 (2005)

    CrossRef  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Universität zu Lübeck for sparse coding data set of MNIST, and the Chinese University of Hong Kong for target coding data set of STL-10. The corresponding author of this work is Dr. Miao Cheng.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Miao Cheng .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Cheng, M., Liu, Z., Zou, H., Tsoi, A.C. (2018). A Family of Maximum Margin Criterion for Adaptive Learning. In: Cheng, L., Leung, A., Ozawa, S. (eds) Neural Information Processing. ICONIP 2018. Lecture Notes in Computer Science(), vol 11303. Springer, Cham. https://doi.org/10.1007/978-3-030-04182-3_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-04182-3_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-04181-6

  • Online ISBN: 978-3-030-04182-3

  • eBook Packages: Computer ScienceComputer Science (R0)