Multiple-Instance Learning with Structured Bag Models

  • Jonathan Warrell
  • Philip H. S. Torr
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6819)


Traditional approaches to Multiple-Instance Learning (MIL) operate under the assumption that the instances of a bag are generated independently, and therefore typically learn an instance-level classifier which does not take into account possible dependencies between instances. This assumption is particularly inappropriate in visual data, where spatial dependencies are the norm. We introduce here techniques for incorporating MIL constraints into Conditional Random Field models, thus providing a set of tools for constructing structured bag models, in which spatial (or other) dependencies are represented. Further, we show how Deterministic Annealing, which has proved a successful method for training non-structured MIL models, can also form the basis of training models with structured bags. Results are given on various segmentation tasks.


Soft Constraint Multiple Instance Learning Instance Label Dual Decomposition Patch Level 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: From Contours to Regions: An Empirical Evaluation. In: CVPR (2009)Google Scholar
  2. 2.
    Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: iCoseg: Interactive Cosegmentation with Intelligent Scribble Guidance. In: CVPR (2010)Google Scholar
  3. 3.
    Babenko, B., Yang, M.H., Belongie, S.: Visual tracking with online multiple instance learning. In: CVPR (2009)Google Scholar
  4. 4.
    Bertsekas, D.: Nonlinear Programming. Athena Scientific (1999)Google Scholar
  5. 5.
    Besag, J.: Statistical analysis of non-lattice data. The Statistician 24, 179–195 (1975)CrossRefGoogle Scholar
  6. 6.
    Gehler, P.V., Chapelle, O.: Deterministic Annealing for Multiple-Instance Learning. In: AISTATS (2007)Google Scholar
  7. 7.
    Gould, S., Gao, T., Koller, D.: Region-based Segmentation and Object Detection. In: NIPS (2009)Google Scholar
  8. 8.
    Kohli, P., Ladicky, L., Torr, P.H.S.: Robust Higher Order Potentials for Enforcing Label Consistency. In: IJCV (2009)Google Scholar
  9. 9.
    Komodakis, N., Paragios, N., Tziritas, G.: MRF Optimization via Dual Decomposition: Message- passing Revisited. In: ICCV (2005)Google Scholar
  10. 10.
    Ladicky, L., Russell, C., Kohli, P., Torr, P.H.S.: Associative Hierarchical CRFs for Object Class Image Segmentation. In: ICCV (2009)Google Scholar
  11. 11.
    Ladicky, L., Russell, C., Kohli, P., Torr, P.H.S.: Graph Cut Based Inference with Co-occurrence Statistics. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 239–253. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Leistner, C., Saffari, A., Bischof, H.: MIForests: Multiple-Instance Learning with Randomized Trees. In: ECCV (2009)Google Scholar
  13. 13.
    Leistner, C., Saffari, A., Santner, J., Bischof, H.: Semi-supervised Random Forests. In: ICCV (2009)Google Scholar
  14. 14.
    Rose, K.: Deterministic annealing, constrained clustering, and optimization. In: IJCNN (1998)Google Scholar
  15. 15.
    Rother, C., Kolmogorov, V., Minka, T., Blake, A.: Cosegmentation of Image Pairs by Histogram Matching - Incorporating a Global Constraint into MRFs. In: CVPR (2006)Google Scholar
  16. 16.
    Saffari, A., Leistner, C., Godec, M., Santner, J., Bischof, H.: On-line Random Forests. In: OLCV (2009)Google Scholar
  17. 17.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation. In: ECCV (2006)Google Scholar
  18. 18.
    Vezhnevets, A., Buhmann, J.: Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning. In: CVPR (2010)Google Scholar
  19. 19.
    Vicente, S., Kolmogorov, V., Rother, C.: Joint optimization of segmentation and appearance models. In: ICCV (2009)Google Scholar
  20. 20.
    Vicente, S., Kolmogorov, V., Rother, C.: Cosegmentation revisited: Models and optimization. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 465–479. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  21. 21.
    Viola, P., Platt, J.C., Zang, C.: Multiple instance boosting for object detection. In: NIPS (2005)Google Scholar
  22. 22.
    Warrell, J., Prince, S., Torr, P.H.S.: StyP-Boost: A Bilinear Boosting Algorithm for Learning Style-Parameterized Classifiers. In: BMVC (2010)Google Scholar
  23. 23.
    Woodford, O.J., Rother, C., Kolmogorov, V.: A Global Perspective on MAP inference for Low-Level Vision. In: ICCV (2009)Google Scholar
  24. 24.
    Zha, Z., Hua, X., Mei, T., Wang, J., Qi, G., Wang, Z.: Joint multi-label multi-instance learning for image classification. In: CVPR (2008)Google Scholar
  25. 25.
    Zhou, Z., Zang, M.: Multiple-Instance Multi-Label Learning with application to Scene Classification. In: NIPS (2006)Google Scholar
  26. 26.
    Zhou, Z., Sun, Y., Li, Y.: Multiple-Instance Learning by Treating Instances as Non-I.I.D. Samples. In: ICML (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jonathan Warrell
    • 1
  • Philip H. S. Torr
    • 1
  1. 1.Oxford Brookes UniversityOxfordUK

Personalised recommendations