Skip to main content
SpringerLink
Log in

Structure-sensitive graph-based multiple-instance semi-supervised learning

  • Published:
Sādhanā Aims and scope Submit manuscript

Abstract

Multiple-instance learning (MIL) is a weakly supervised learning paradigm in which a training dataset contains a set of labeled bags, and each bag contains multiple number of unlabeled instances. Preparation of instance-level labels is resource intensive. Being weakly supervised, MIL is sensitive to several practical issues such as noisy label information and low witness rate. A scenario of high class imbalance and low degree-of-supervision further poses additional challenges. Recent works on graph-based label propagation methods have been shown to be effective in semi-supervised setup to address such issues by propagating the label information over graph-based manifold. Application of such semi-supervised strategies for MIL framework requires the instance-level labeling. Whenever the problem setup contains the three characteristics of high class imbalance, low degree-of-supervision and weak supervision, the state-of-the-art methods of either MIL or graph-based label propagation are inadequate when applied alone. This article proposes a non-convex formulation for instance-level MIL to find the instance-level labels by combining the benefits of both MIL and graph-based label propagation methods. The proposed approach improves the performance of the classifier using density-difference- and distance-based structural smoothness assumptions in the graph structure. This article presents the comparison of the performance of the proposed method to those of several state-of-the-art base-lines in MIL. The experimental results are shown on multiple datasets from four different applications. The proposed method is compared in a total of 616 cases (14 datasets \(\times \) 11 base-line models \(\times \) 4 low degree-of-supervision values). The minimum f-score improvements are 15.22%, 1.14%, and 4.25% in DAP, CIR, and ACSV datasets, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4

Similar content being viewed by others

Notes

  1. \( \tiny \begin{matrix} \left\{ \begin{array}{ll} {\mathscr {D}}&{}{\text { has high class imbalance}}\\ \& {\mathscr {D}}&{}{\text { has low witness rate}} \end{array}\right\} &{}\Rightarrow \left\{ \begin{array}{ll} &{}{\text {noisy labels are created}}\\ &{}{\text {in graph-based multiple-instance SSL}} \end{array}\right\}&\Rightarrow {\text {compromise in performance.}} \end{matrix}\)

  2. CVXPY tutorial link: https://www.cvxpy.org/tutorial/index.html#.

  3. https://drive.google.com/file/d/14hMUagLTLAPQZWj0Sfz4xHlG1L19pALQ/view?usp=sharing

  4. http://www.ics.uci.edu/ mlearn/MLRepository.html

  5. http://www.cs.columbia.edu/ andrews/mil/datasets.html

  6. https://sites.google.com/site/dctresearch/Home/content-based-image-retrieval

References

  1. Thomas G Dietterich, Richard H Lathrop, and Tomás Lozano-Pérez 1997 Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence 89(1–2): 31–71

    Article  MATH  Google Scholar 

  2. Waqas Sultani, Chen Chen, and Mubarak Shah 2018 Real-world anomaly detection in surveillance videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6479–6488

  3. Qi Zhang, Sally A Goldman, Wei Yu, and Jason E Fritts 2002 Content-based image retrieval using multiple-instance learning. In: Proceedings of the ICML. Citeseer, vol. 1, p. 2

  4. Razvan C Bunescu and Raymond J Mooney 2007 Multiple instance learning for sparse positive bags. In: Proceedings of the 24th International Conference on Machine Learning, pp. 105–112

  5. Peng Tang, Xinggang Wang, Song Bai, Wei Shen, Xiang Bai, Wenyu Liu, and Alan Yuille 2018 PCL: Proposal cluster learning for weakly supervised object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(1): 176–191

    Article  Google Scholar 

  6. Maximilian Ilse, Jakub M Tomczak, and Max Welling 2018 Attention-based deep multiple instance learning. In: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, International Machine Learning Society (IMLS), pp. 3376–3391

  7. Annabella Astorino, Antonio Fuduli, and Manlio Gaudioso 2019 A lagrangian relaxation approach for binary multiple instance classification. IEEE Transactions on Neural Networks and Learning Systems 30(9): 2662–2671

    Article  MathSciNet  Google Scholar 

  8. Oded Maron and Tomás Lozano-Pérez 1998 A framework for multiple-instance learning. In: Advances in Neural Information Processing Systems, pp. 570–576

  9. Yixin Chen and James Z Wang 2004 Image categorization by learning and reasoning with regions. Journal of Machine Learning Research 5(Aug): 913–939

  10. Qi Zhang and Sally A Goldman 2002 EM-DD: an improved multiple-instance learning technique. In: Advances in Neural Information Processing Systems, pp. 1073–1080

  11. Yixin Chen, Jinbo Bi, and James Ze Wang 2006 MILES: multiple-instance learning via embedded instance selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(12): 1931–1947

    Article  Google Scholar 

  12. Ming Tu, Jing Huang, Xiaodong He, and Bowen Zhou 2019 Multiple instance learning with graph neural networks. arXiv preprint arXiv:1906.04881

  13. Jun Wang and Jean-Daniel Zucker 2000 Solving the multiple-instance problem: a lazy learning approach. In: Proceedings of the Seventeenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc., pp. 1119–1126

  14. Stuart Andrews, Ioannis Tsochantaridis, and Thomas Hofmann 2003 Support vector machines for multiple-instance learning. In: Advances in Neural Information Processing Systems, pp. 577–584

  15. Zhi-Hua Zhou and Jun-Ming Xu 2007 On the relation between multi-instance learning and semi-supervised learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 1167–1174

  16. Gary Doran and Soumya Ray 2014 A theoretical and empirical analysis of support vector machine methods for multiple-instance classification. Machine Learning 97(1–2): 79–102

    MathSciNet  MATH  Google Scholar 

  17. Gitte Vanwinckelen, Daan Fierens, Hendrik Blockeel, et al 2016 Instance-level accuracy versus bag-level accuracy in multi-instance learning. Data Mining and Knowledge Discovery 30(2): 313–341

    Article  MathSciNet  MATH  Google Scholar 

  18. Thomas Gärtner, Peter A Flach, Adam Kowalczyk, and Alexander J Smola 2002 Multi-instance kernels. In: Proceedings of the ICML, vol. 2, p. 7

  19. Yangqing Jia and Changshui Zhang 2008 Instance-level semisupervised multiple instance learning. In: Proceedings of the AAAI Conference, pp. 640–645

  20. Dan Zhang, Yan Liu, Luo Si, Jian Zhang, and Richard D Lawrence 2011 Multiple instance learning on structured data. In: Advances in Neural Information Processing Systems, pp. 145–153

  21. Rakesh Pimplikar, Dinesh Garg, Deepesh Bharani, and Gyana Parija 2014 Learning to propagate rare labels. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 201–210

  22. Rouhollah Rahmani and Sally A Goldman 2006 MISSL: multiple-instance semi-supervised learning. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 705–712

  23. Sudheendra Vijayanarasimhan and Kristen Grauman 2008 Keywords to visual categories: multiple-instance learning forweakly supervised object categorization. In: 2008 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 1–8

  24. Xiangfa Song, L C Jiao, Shuyuan Yang, Xiangrong Zhang, and Fanhua Shang 2013 Sparse coding and classifier ensemble based multi-instance learning for image categorization. Signal Processing 93(1): 1–11

    Article  Google Scholar 

  25. Judy Hoffman, Deepak Pathak, Trevor Darrell, and Kate Saenko 2015 Detector discovery in the wild: joint multiple instance and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2883–2891

  26. Hyun Oh Song, Ross Girshick, Stefanie Jegelka, Julien Mairal, Zaid Harchaoui, and Trevor Darrell 2014 On learning to localize objects with minimal supervision. In: Proceedings of the ICML-31st International Conference on Machine Learning, vol. 32, pp. 1611–1619

  27. Sriparna Saha, Abhay Kumar Alok, and Asif Ekbal 2016 Brain image segmentation using semi-supervised clustering. Expert Systems with Applications 52: 50–63

    Article  Google Scholar 

  28. Boris Babenko, Ming-Hsuan Yang, and Serge Belongie 2010 Robust object tracking with online multiple instance learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(8): 1619–1632

    Article  Google Scholar 

  29. Kaihua Zhang and Huihui Song 2013 Real-time visual tracking via online weighted multiple instance learning. Pattern Recognition 46(1): 397–411

    Article  MATH  Google Scholar 

  30. Huchuan Lu, Qiuhong Zhou, Dong Wang, and Ruan Xiang 2011 A co-training framework for visual tracking with multiple instance learning. In: Proceedings of Face and Gesture 2011. IEEE, pp. 539–544

  31. Yan Xu, Jun-Yan Zhu, I Eric, Chao Chang, Maode Lai, and Zhuowen Tu 2014 Weakly supervised histopathology cancer image segmentation and classification. Medical Image Analysis 18(3): 591–604

    Article  Google Scholar 

  32. Gwénolé Quellec, Mathieu Lamard, Michael D Abràmoff, Etienne Decencière, Bruno Lay, Ali Erginay, Béatrice Cochener, and Guy Cazuguel 2012 A multiple-instance learning framework for diabetic retinopathy screening. Medical Image Analysis 16(6): 1228–1240

    Article  Google Scholar 

  33. Tong Tong, Robin Wolz, Qinquan Gao, Ricardo Guerrero, Joseph V Hajnal, Daniel Rueckert, Alzheimer’s Disease Neuroimaging Initiative, et al 2014 Multiple instance learning for classification of dementia in brain MRI. Medical Image Analysis 18(5): 808–818

    Article  Google Scholar 

  34. Nikolaos Pappas and Andrei Popescu-Belis 2014 Explaining the stars: weighted multiple-instance learning for aspect-based sentiment analysis. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 455–466

  35. Yi Zhang, Arun C Surendran, John C Platt, and Mukund Narasimhan 2008 Learning from multi-topic web documents for contextual advertisement. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1051–1059

  36. Dan Zhang, Jingrui He, and Richard Lawrence 2013 MI2LS: multi-instance learning from multiple information sources. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 149–157

  37. Soumya Ray and Mark Craven 2005 Supervised versus multiple instance learning: an empirical comparison. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 697–704

  38. Abhay Kumar Alok, Pooja Gupta, Sriparna Saha, and Vineet Sharma 2020 Simultaneous feature selection and clustering of micro-array and RNA-sequence gene expression data using multiobjective optimization. International Journal of Machine Learning and Cybernetics 11: 2541–2563

    Article  Google Scholar 

  39. Charles Bergeron, Gregory Moore, Jed Zaretzki, Curt M Breneman, and Kristin P Bennett 2011 Fast bundle algorithm for multiple-instance learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(6): 1068–1079

    Article  Google Scholar 

  40. Forrest Briggs, Xiaoli Z Fern, and Raviv Raich 2012 Rank-loss support instance machines for MIML instance annotation. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 534–542

  41. Marc-André Carbonneau, Eric Granger, Yazid Attabi, and Ghyslain Gagnon 2017 Feature learning from spectrograms for assessment of personality traits. IEEE Transactions on Affective Computing 11(1): 25–31

    Article  Google Scholar 

  42. Maja Stikic, Diane Larlus, Sandra Ebert, and Bernt Schiele 2011 Weakly supervised recognition of daily life activities with wearable sensors. IEEE Transactions on Pattern analysis and Machine Intelligence 33(12): 2521–2537

    Article  Google Scholar 

  43. Veronika Cheplygina and David M J Tax 2015 Characterizing multiple instance datasets. In: Proceedings of the International Workshop on Similarity-Based Pattern Recognition. Springer, pp. 15–27

  44. James T Kwok and Pak-Ming Cheung 2007 Marginalized multi-instance kernels. In: Proceedings of IJCAI, vol. 7, pp. 901–906

    Google Scholar 

  45. Yan Li, David M J Tax, Robert P W Duin, and Marco Loog 2013 Multiple-instance learning as a classifier combining problem. Pattern Recognition 46(3): 865–874

    Article  Google Scholar 

  46. Fuxin Li and Cristian Sminchisescu 2010 Convex multiple-instance learning by estimating likelihood ratio. In: Advances in Neural Information Processing Systems, pp. 1360–1368

  47. Peter V Gehler and Olivier Chapelle 2007 Deterministic annealing for multiple-instance learning. In: Artificial Intelligence and Statistics, pp. 123–130

  48. Xin-Shun Xu, Yuan Jiang, Xiangyang Xue, and Zhi-Hua Zhou 2012 Semi-supervised multi-instance multi-label learning for video annotation task. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 737–740

  49. AS d’Avila Garcez and Gerson Zaverucha 2012 Multi-instance learning using recurrent neural networks. In: Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN). IEEE, pp. 1–6

  50. Jiajun Wu, Yinan Yu, Chang Huang, and Kai Yu 2015 Deep multiple instance learning for image classification and auto-annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3460–3469

  51. Manlio Gaudioso, Giovanni Giallombardo, Giovanna Miglionico, and Eugenio Vocaturo 2020 Classification in the multiple instance learning framework via spherical separation. Soft Computing 24(7): 5071–5077

    Article  Google Scholar 

  52. Avrim Blum and Shuchi Chawla 2001 Learning from labeled and unlabeled data using graph mincuts. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 19–26

  53. Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty 2003 Semi-supervised learning using Gaussian fields and harmonic functions. In: Proceedings of the 20th International conference on Machine learning (ICML-03), pp. 912–919

  54. Jun Wang, Tony Jebara, and Shih-Fu Chang 2008 Graph transduction via alternating minimization. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1144–1151

  55. Dengyong Zhou, Olivier Bousquet, Thomas N Lal, Jason Weston, and Bernhard Schölkopf 2004 Learning with local and global consistency. In: Advances in Neural Information Processing Systems, pp. 321–328

  56. Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani 2006 Manifold regularization: a geometric framework for learning from labeled and unlabeled examples Journal of Machine Learning Research 7(Nov): 2399–2434

  57. Olivier Bousquet, Olivier Chapelle, and Matthias Hein 2004 Measure based regularization. In: Advances in Neural Information Processing Systems, pp. 1221–1228

  58. Xiaojin Zhu, John Lafferty, and Ronald Rosenfeld 2005 Semi-supervised learning with graphs. PhD Thesis, Carnegie Mellon University, Language Technologies Institute, School of Computer Science

  59. O Chapelle, B Scholkopf, and A Zien 2009 Semi-supervised learning. IEEE Transactions on Neural Networks 3(20): 542

    Article  Google Scholar 

  60. Amarnag Subramanya and Jeff Bilmes 2011 Semi-supervised learning with measure propagation. The Journal of Machine Learning Research 12: 3311–3370

    MathSciNet  MATH  Google Scholar 

  61. Richard O Duda, Peter E Hart, et al 2006 Pattern classification. John Wiley & Sons

  62. Jinhui Tang, Xian-Sheng Hua, Guo-Jun Qi, Meng Wang, Tao Mei, and Xiuqing Wu 2007 Structure-sensitive manifold ranking for video concept detection. In: Proceedings of the 15th ACM International Conference on Multimedia, pp. 852–861

  63. Alan L Yuille and Anand Rangarajan 2003 The concave–convex procedure. Neural Computation 15(4): 915–936

    Article  MATH  Google Scholar 

  64. Jorge Nocedal and Stephen Wright 2006 Numerical optimization. Springer Science & Business Media

  65. Olvi L Mangasarian and Edward W Wild 2008 Multiple instance classification via successive linear programming. Journal of Optimization Theory and Applications 137(3): 555–568

    Article  MathSciNet  MATH  Google Scholar 

  66. Vijay Mahadevan, Weixin Li, Viral Bhalodia, and Nuno Vasconcelos 2010 Anomaly detection in crowded scenes. In: Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp. 1975–1981

  67. Hamidreza Rabiee, Javad Haddadnia, Hossein Mousavi, Moin Nabi, Vittorio Murino, and Nicu Sebe 2016 Emotion-based crowd representation for abnormality detection. arXiv preprint arXiv:1607.07646

  68. Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri 2015 Learning spatiotemporal features with 3D convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4489–4497

  69. Trevor Hastie, Robert Tibshirani, and Jerome Friedman 2009 The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media

Download references

Acknowledgements

We sincerely thank Dr Mridula Verma for valuable suggestions.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology – Warangal, Warangal, Telangana, 506004, India

    Satya Krishna Nunna & D V L N Somayajulu

  2. Centre for Payment Systems, Institute for Development and Research in Banking Technology – Hyderabad, Hyderabad, Telangana, 500057, India

    Satya Krishna Nunna & N V Narendra Kumar

  3. Department of Computer Science and Engineering, National Institute of Technology – Tadepalligudem, Tadepalligudem, Andhra Pradesh, 534146, India

    S Nagesh Bhattu

  4. Department of Computer Science and Engineering, Indian Institute of Information Technology, Design and Manufacturing – Jagannathagattu Hill, Kurnool, Andhra Pradesh, 518007, India

    D V L N Somayajulu

Authors
  1. Satya Krishna Nunna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S Nagesh Bhattu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D V L N Somayajulu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. N V Narendra Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S Nagesh Bhattu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nunna, S.K., Bhattu, S.N., Somayajulu, D.V.L.N. et al. Structure-sensitive graph-based multiple-instance semi-supervised learning. Sādhanā 46, 156 (2021). https://doi.org/10.1007/s12046-021-01659-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12046-021-01659-4

Keywords

Search

Navigation