Advertisement

Multiview Learning of Weighted Majority Vote by Bregman Divergence Minimization

  • Anil GoyalEmail author
  • Emilie Morvant
  • Massih-Reza Amini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11191)

Abstract

We tackle the issue of classifier combinations when observations have multiple views. Our method jointly learns view-specific weighted majority vote classifiers (i.e. for each view) over a set of base voters, and a second weighted majority vote classifier over the set of these view-specific weighted majority vote classifiers. We show that the empirical risk minimization of the final majority vote given a multiview training set can be cast as the minimization of Bregman divergences. This allows us to derive a parallel-update optimization algorithm for learning our multiview model. We empirically study our algorithm with a particular focus on the impact of the training set size on the multiview learning results. The experiments show that our approach is able to overcome the lack of labeled information.

Keywords

Multiview learning Bregman divergence Majority vote 

Notes

Acknowledgment

This work is partially funded by the French ANR project LIVES ANR-15-CE23-0026-03 and the “Région Rhône-Alpes”.

References

  1. 1.
    Amini, M.R., Usunier, N., Goutte, C.: Learning from multiple partially observed views - an application to multilingual text categorization. In: NIPS (2009)Google Scholar
  2. 2.
    Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)Google Scholar
  3. 3.
    Blum, A., Mitchell, T.M.: Combining labeled and unlabeled data with co-training. In: COLT, pp. 92–100 (1998)Google Scholar
  4. 4.
    Bregman, L.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7(3), 200–217 (1967)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Chen, M., Denoyer, L.: Multi-view generative adversarial networks. In: ECML-PKDD, pp. 175–188 (2017)CrossRefGoogle Scholar
  6. 6.
    Collins, M., Schapire, R.E., Singer, Y.: Logistic regression, adaboost and bregman distances. Mach. Learn. 48(1–3), 253–285 (2002)CrossRefGoogle Scholar
  7. 7.
    Darroch, J.N., Ratcliff, D.: Generalized iterative scaling for log-linear models. Ann. Math. Stat. 43, 1470–1480 (1972)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Della Pietra, S., Della Pietra, V., Lafferty, J.: Inducing features of random fields. IEEE TPAMI 19(4), 380–393 (1997)CrossRefGoogle Scholar
  9. 9.
    Farquhar, J., Hardoon, D., Meng, H., Shawe-taylor, J.S., Szedmák, S.: Two view learning: Svm-2k, theory and practice. In: NIPS, pp. 355–362 (2006)Google Scholar
  10. 10.
    Gönen, M., Alpayd, E.: Multiple kernel learning algorithms. JMLR 12, 2211–2268 (2011)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Huusari, R., Kadri, H., Capponi, C.: Multi-view metric learning in vector-valued kernel spaces. In: AISTATS (2018)Google Scholar
  12. 12.
    Janodet, J.C., Sebban, M., Suchier, H.M.: Boosting classifiers built from different subsets of features. Fundam. Inf. 94(2009), 1–21 (2009)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Koço, S., Capponi, C.: A boosting approach to multiview classification with cooperation. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6912, pp. 209–228. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-23783-6_14CrossRefGoogle Scholar
  14. 14.
    Lafferty, J.: Additive models, boosting, and inference for generalized divergences. In: COLT, pp. 125–133 (1999)Google Scholar
  15. 15.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, pp. 2278–2324 (1998)Google Scholar
  16. 16.
    Pedregosa, F., et al.: Scikit-learn: machine learning in python. JMLR 12, 2825–2830 (2011)Google Scholar
  17. 17.
    Peng, J., Aved, A.J., Seetharaman, G., Palaniappan, K.: Multiview boosting with information propagation for classification. IEEE Trans. Neural Netw. Learn. Syst. 99, 1–13 (2017)Google Scholar
  18. 18.
    Peng, J., Barbu, C., Seetharaman, G., Fan, W., Wu, X., Palaniappan, K.: Shareboost: boosting for multi-view learning with performance guarantees. In: ECML-PKDD, pp. 597–612 (2011)CrossRefGoogle Scholar
  19. 19.
    Powers, D.M.: Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. J. Mach. Learn. Technol. 1(2), 37–63 (2011)Google Scholar
  20. 20.
    Sindhwani, V., Rosenberg, D.S.: An RKHS for multi-view learning and manifold co-regularization. In: ICML, pp. 976–983 (2008)Google Scholar
  21. 21.
    Snoek, C., Worring, M., Smeulders, A.W.M.: Early versus late fusion in semantic video analysis. In: ACM Multimedia, pp. 399–402 (2005)Google Scholar
  22. 22.
    Sun, S.: A survey of multi-view machine learning. Neural Comput. Appl. 23(7–8), 2031–2038 (2013)CrossRefGoogle Scholar
  23. 23.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Berlin (1999)Google Scholar
  24. 24.
    Xiao, M., Guo, Y.: Multi-view Adaboost for multilingual subjectivity analysis. In: COLING, pp. 2851–2866 (2012)Google Scholar
  25. 25.
    Xu, C., Tao, D., Xu, C.: Large-margin multi-viewinformation bottleneck. IEEE TPAMI 36(8), 1559–1572 (2014)CrossRefGoogle Scholar
  26. 26.
    Xu, C., Tao, D., Xu, C.: Multi-view learning with incomplete views. IEEE Trans. Image Process. 24(12), 5812–5825 (2015)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Xu, X., Li, W., Xu, D., Tsang, I.W.: Co-labeling for multi-view weakly labeled learning. IEEE TPAMI 38(6), 1113–1125 (2016)CrossRefGoogle Scholar
  28. 28.
    Xu, Z., Sun, S.: An algorithm on multi-view Adaboost. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds.) ICONIP 2010. LNCS, vol. 6443, pp. 355–362. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-17537-4_44CrossRefGoogle Scholar
  29. 29.
    Zhang, J., Zhang, D.: A novel ensemble construction method for multi-view data using random cross-view correlation between within-class examples. Pattern. Recogn. 44(6), 1162–1171 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Anil Goyal
    • 1
    • 2
    Email author
  • Emilie Morvant
    • 1
  • Massih-Reza Amini
    • 2
  1. 1.Laboratoire Hubert Curien UMR 5516Université de Lyon, UJM-St-Etienne, CNRS, Institut d’Optique Graduate SchoolSt-EtienneFrance
  2. 2.Laboratoire d’Informatique de Grenoble, AMAUniversité Grenoble AlpsGrenobleFrance

Personalised recommendations