Advertisement

ShareBoost: Boosting for Multi-view Learning with Performance Guarantees

  • Jing Peng
  • Costin Barbu
  • Guna Seetharaman
  • Wei Fan
  • Xian Wu
  • Kannappan Palaniappan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6912)

Abstract

Algorithms combining multi-view information are known to exponentially quicken classification, and have been applied to many fields. However, they lack the ability to mine most discriminant information sources (or data types) for making predictions. In this paper, we propose an algorithm based on boosting to address these problems. The proposed algorithm builds base classifiers independently from each data type (view) that provides a partial view about an object of interest. Different from AdaBoost, where each view has its own re-sampling weight, our algorithm uses a single re-sampling distribution for all views at each boosting round. This distribution is determined by the view whose training error is minimal. This shared sampling mechanism restricts noise to individual views, thereby reducing sensitivity to noise. Furthermore, in order to establish performance guarantees, we introduce a randomized version of the algorithm, where a winning view is chosen probabilistically. As a result, it can be cast within a multi-armed bandit framework, which allows us to show that with high probability the algorithm seeks out most discriminant views of data for making predictions. We provide experimental results that show its performance against noise and competing techniques.

Keywords

Data fusion boosting convergence multi-view learning 

References

  1. 1.
    Audibert, J.-Y., Munos, R., Szepesvari, C.: Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theor. Comput. Sci. 410(19), 1876–1902 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine Learning 47, 235–256 (2002)CrossRefzbMATHGoogle Scholar
  3. 3.
    Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.: The non-stochastic multi-armed bandit problem. SIAM Journal on Computing 32(1), 48–77 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: In Proceedings of the Eleventh Annual Conference in Computational Learning TheoryGoogle Scholar
  5. 5.
    Busa-Fekete, R., Kegl, B.: Accelerating adaboost using ucb. In: KDDCup (JMLR W&CP), pp. 111–122 (2009)Google Scholar
  6. 6.
    Busa-Fekete, R., Kegl, B.: Fast boosting using adversarial bandits. In: Proceedings of International Conference on Machine Learning (2010)Google Scholar
  7. 7.
    Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)CrossRefzbMATHGoogle Scholar
  8. 8.
    Culp, M., Michailidis, G., Johnson, K.: Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on Knowledge and Data Engineering 17, 1529–1541 (2005)CrossRefGoogle Scholar
  9. 9.
    Fawcett, T., Niculescu-mizil, A.: Technical note: Pav and the roc convex hull. Machine Learning 68, 97–106 (2007)CrossRefGoogle Scholar
  10. 10.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and Systems Science 55, 119–139 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Kittler, J.: Combining classifiers: A theoretical framework. Pattern Analysis and Applications 1, 18–27 (1998)CrossRefGoogle Scholar
  12. 12.
    Kuncheva, L.I., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion: An experimental comparison. Pattern Recognition 34, 299–314 (2001)CrossRefzbMATHGoogle Scholar
  13. 13.
    Lanckriet, G.R.G., Deng, M.H., Cristianini, N., Jordan, M.I., Noble, W.S.: Kernel-based data fusion and its application to protein function prediction in yeast. In: Proceedings of the Pacific Symposium on Biocomputing, vol. 9, pp. 300–311 (2004)Google Scholar
  14. 14.
    Maturana, J., Fialho, A., Saubion, F., Schoenauer, M., Sebag, M.: Extreme compass and dynamic multi-armed bandits for adaptive operator selection. In: Proceedings of IEEE ICEC, pp. 365–372 (2009)Google Scholar
  15. 15.
    Mesmay, F.D., Rimmel, A., Voronenko, Y., Puschel, M.: Bandit-based optimization on graphs with application to library performance tuning. In: Proceedings of International Conference on Machine Learning, pp. 729–736 (2009)Google Scholar
  16. 16.
    Mewes, H.W., Frishman, D., Gruber, C., Geier, B., Haase, D., Kaps, A., Lemcke, K., Mannhaupt, G., Pfeiffer, F., Schüller, C., Stocker, S., Weil, B.: Mips: a database for genomes and protein sequences. Nucleic Acids Research 28, 37–40 (2000)CrossRefGoogle Scholar
  17. 17.
    Robbins, H.: Some aspects of the sequential design of experiments. Bulletin American Mathematical Society 55, 527–535 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Ross, A., Jain, A.K.: Multimodal biometrics: an overview. In: Proceedings of 12th European Signal Processing Conference, pp. 1221–1224 (2004)Google Scholar
  19. 19.
    Rudin, C., Schapire, R., Daubechies, I.: Precise statements of convergence for adaboost and arc-gv. Contemporary Mathematics 443 (2007)Google Scholar
  20. 20.
    Schapire, R.E., Singer, Y.: Improved boosting algorithms using confidence rated predictions. Machine Learning 3(37), 297–336 (1999)CrossRefzbMATHGoogle Scholar
  21. 21.
    Viola, P., Jones, M.: Fast and robust classification using asymmetric adaboost and a detector cascade. In: Advances in Neural Information Processing Systems, vol. 14 (2002)Google Scholar
  22. 22.
    Wang, W., Hua Zhou, Z.: On multi-view active learning and the combination with semi-supervised learning. In: Proceedings of the 25th International Conference on Machine Learning (2008)Google Scholar
  23. 23.
    Wang, W., Hua Zhou, Z.: A new analysis of co-training. In: Proceedings of the 25th International Conference on Machine Learning (2010)Google Scholar
  24. 24.
    Wolpert, D.H.: Stacked generalization. Neural Networks 5, 241–259 (1992)CrossRefGoogle Scholar
  25. 25.
    Zhang, D., Wang, F., Zhang, C., Li, T.: Multi-view local learning. In: Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI), pp. 752–757 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jing Peng
    • 1
  • Costin Barbu
    • 1
  • Guna Seetharaman
    • 1
  • Wei Fan
    • 1
  • Xian Wu
    • 1
  • Kannappan Palaniappan
    • 1
  1. 1.Montclair State University; MIT Lincoln Lab; AFRL/RITB; IBM T.J. Watson Research; IBM China ResearchUniversity of MissouriColumbiaUSA

Personalised recommendations