Advertisement

Data Mining and Knowledge Discovery

, Volume 28, Issue 4, pp 1078–1106 | Cite as

Multiple instance learning via Gaussian processes

  • Minyoung KimEmail author
  • Fernando De la Torre
Article

Abstract

Multiple instance learning (MIL) is a binary classification problem with loosely supervised data where a class label is assigned only to a bag of instances indicating presence/absence of positive instances. In this paper we introduce a novel MIL algorithm using Gaussian processes (GP). The bag labeling protocol of the MIL can be effectively modeled by the sigmoid likelihood through the max function over GP latent variables. As the non-continuous max function makes exact GP inference and learning infeasible, we propose two approximations: the soft-max approximation and the introduction of witness indicator variables. Compared to the state-of-the-art MIL approaches, especially those based on the Support Vector Machine, our model enjoys two most crucial benefits: (i) the kernel parameters can be learned in a principled manner, thus avoiding grid search and being able to exploit a variety of kernel families with complex forms, and (ii) the efficient gradient search for kernel parameter learning effectively leads to feature selection to extract most relevant features while discarding noise. We demonstrate that our approaches attain superior or comparable performance to existing methods on several real-world MIL datasets including large-scale content-based image retrieval problems.

Keywords

Multiple instance learning Gaussian processes Kernel machines Probabilistic models 

References

  1. Andrews S, Tsochantaridis I, Hofmann T (2003) Support vector machines for multiple-instance learning. Adv Neural Inf Process Syst 15:561–568Google Scholar
  2. Antić B, Ommer B (2012) Robust multiple-instance learning with superbags. In: Proceedings of the 11th Asian conference on computer visionGoogle Scholar
  3. Babenko B, Verma N, Dollár P, Belongie S (2011) Multiple instance learning with manifold bags. In: International conference on machine learningGoogle Scholar
  4. Chen Y, Bi J, Wang JZ (2006) MILES: multiple-instance learning via embedded instance selection. IEEE Trans Pattern Anal Mach Intell 28(12):1931–1947CrossRefGoogle Scholar
  5. Dietterich TG, Lathrop RH, Lozano-Perez T (1997) Solving the multiple-instance problem with axis-parallel rectangles. Artif Intell 89:31–71CrossRefzbMATHGoogle Scholar
  6. Dooly DR, Zhang Q, Goldman SA, Amar RA (2002) Multiple-instance learning of real-valued data. J Mach Learn Res 3:651–678Google Scholar
  7. Friedman J (1999) Greedy function approximation: a gradient boosting machine. Technical Report, Department of Statistics, Stanford UniversityGoogle Scholar
  8. Fu Z, Robles-Kelly A, Zhou J (2011) MILIS: multiple instance learning with instance selection. IEEE Trans Pattern Anal Mach Intell 33(5):958–977CrossRefGoogle Scholar
  9. Gärtner T, Flach PA, Kowalczyk A, Smola AJ (2002) Multi-instance kernels. In: International conference on machine learningGoogle Scholar
  10. Gehler PV, Chapelle O (2007) Deterministic annealing for multiple-instance learning. In: AI and Statistics (AISTATS)Google Scholar
  11. Kim M, De la Torre F (2010) Gaussian process multiple instance learning. In: International conference on machine learningGoogle Scholar
  12. Lázaro-Gredilla M, Quinonero-Candela J, Rasmussen CE, Figueiras-Vidal AR (2010) Sparse spectrum Gaussian process regression. J Mach Learn Res 11:1865–1881zbMATHMathSciNetGoogle Scholar
  13. Li YF, Kwok JT, Tsang IW, Zhou ZH (2009) A convex method for locating regions of interest with multi-instance learning. In: Proceedings of the European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD’09), Bled, SloveniaGoogle Scholar
  14. Liu G, Wu J, Zhou ZH (2012) Key instance detection in multi-instance learning. In: Proceedings of the 4th Asian conference on machine learning (ACML’12), SingaporeGoogle Scholar
  15. Mangasarian OL, Wild EW (2008) Multiple instance classification via successive linear programming. J Optim Theory Appl 137(3):555–568CrossRefzbMATHMathSciNetGoogle Scholar
  16. Maron O, Lozano-Perez T (1998) A framework for multiple-instance learning. In: Jordan MI, Kearns MJ, Solla SA (eds) Advances in neural information processing systems. MIT Press, CambridgeGoogle Scholar
  17. Rahmani R, Goldman SA (2006) MISSL: multiple-instance semi-supervised learning. In: International conference on machine learningGoogle Scholar
  18. Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. MIT Press, CambridgezbMATHGoogle Scholar
  19. Ray S, Page D (2001) Multiple instance regression. In: International conference on machine learningGoogle Scholar
  20. Raykar VC, Krishnapuram B, Bi J, Dundar M, Rao RB (2008) Bayesian multiple instance learning: automatic feature selection and inductive transfer. In: International conference on machine learningGoogle Scholar
  21. Scott S, Zhang J, Brown J (2003) On generalized multiple instance learning. Technical Report UNL-CSE-2003-5, Department of Computer Science and Engineering, University of Nebraska, LincolnGoogle Scholar
  22. Snelson E, Ghahramani Z (2006) Sparse Gaussian processes using pseudo-inputs. In: Weiss Y, Scholkopf B, Platt J (eds) Advances in neural information processing systems. MIT Press, CambridgeGoogle Scholar
  23. Tao Q, Scott S, Vinodchandran NV, Osugi TT (2004) SVM-based generalized multiple-instance learning via approximate box counting. In: International conference on machine learningGoogle Scholar
  24. Viola PA, Platt JC, Zhang C (2005) Multiple instance boosting for object detection. In: Viola P, Platt JC, Zhang C (eds) Advances in neural information processing systems. MIT Press, CambridgeGoogle Scholar
  25. Wang J, Zucker JD (2000) Solving the multiple-instance problem: a lazy learning approach. In: International conference on machine learningGoogle Scholar
  26. Wang H, Huang H, Kamangar F, Nie F, Ding C (2011) A maximum margin multi-instance learning framework for image categorization. In: Advances in neural information processing systemsGoogle Scholar
  27. Weidmann N, Frank E, Pfahringer B (2003) A two-level learning method for generalized multi-instance problem. Lecture notes in artificial intelligence, vol 2837. Springer, BerlinGoogle Scholar
  28. Zhang H, Fritts J (2005) Improved hierarchical segmentation. Technical Report, Washington University in St LouisGoogle Scholar
  29. Zhang T, Oles FJ (2000) Text categorization based on regularized linear classification methods. Inf Retr 4:5–31CrossRefGoogle Scholar
  30. Zhang ML, Zhou ZH (2009) Multi-instance clustering with applications to multi-instance prediction. Appl Intell 31(1):47–68CrossRefGoogle Scholar
  31. Zhang Q, Goldman SA, Yu W, Fritts JE (2002) Content-based image retrieval using multiple-instance learning. In: International conference on machine learningGoogle Scholar
  32. Zhang D, Wang F, Luo S, Li T (2011) Maximum margin multiple instance clustering with its applications to image and text clustering. IEEE Trans Neural Netw 22(5):739–751CrossRefGoogle Scholar

Copyright information

© The Author(s) 2013

Authors and Affiliations

  1. 1.Department of Electronics and IT Media EngineeringSeoul National University of Science & TechnologySeoulSouth Korea
  2. 2.The Robotics InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations