Skip to main content
SpringerLink
Log in

Multi-instance embedding learning with deconfounded instance-level prediction

  • Regular Paper
  • Published:
International Journal of Data Science and Analytics Aims and scope Submit manuscript

Abstract

Confounded information is an objective fact when using multi-instance learning (MIL) to classify bags of instances, which may be inherited by MIL embedding methods and lead to questionable bag label prediction. To respond to this problem, we propose the multi-instance embedding learning with deconfounded instance-level prediction algorithm. Unlike traditional embedding-based strategies, we design a deconfounded optimization goal to maximize the distinction between instances in positive and negative bags. In addition, we present and use bag-level embedding with feature distillation to reduce the MIL classification task to a single-instance learning problem. Under the theoretical analysis, the embedding cohesiveness and feature magnitude metrics are developed to explain the benefits of the proposed deconfounded technique in MIL settings. Extensive experiments on thirty-four data sets demonstrate that our proposed method has the best overall performance over other state-of-the-art MIL methods. This strategy, in particular, has a substantial advantage on web data sets. Source codes are available at https://github.com/InkiInki/MEDI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Angelidis, S., Lapata, M.: Multiple instance learning networks for fine-grained sentiment analysis. Tran. Assoc. Comput. Linguist. 6, 17–31 (2018). https://doi.org/10.1162/tacl_a_00002

    Article  Google Scholar 

  2. Chen, Y.X., Bi, J.B., Wang, J.Z.: MILES: multiple-instance learning via embedded instance selection. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 1931–1947 (2006). https://doi.org/10.1109/TPAMI.2006.248

    Article  Google Scholar 

  3. Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1–2), 31–71 (1997). https://doi.org/10.1016/S0004-3702(96)00034-3

    Article  MATH  Google Scholar 

  4. Fu, Z.Y., Robles-Kelly, A., Zhou, J.: MILIS: multiple instance learning with instance selection. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 958–977 (2011). https://doi.org/10.1109/TPAMI.2010.155

    Article  Google Scholar 

  5. Hong, R.C., Wang, M., Gao, Y., et al.: Image annotation by multiple-instance learning with discriminative feature mapping and selection. IEEE Trans. Cybern. 44(5), 669–680 (2014). https://doi.org/10.1109/TCYB.2013.2265601

    Article  Google Scholar 

  6. Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2127–2136 (2018)

  7. Li, S., Liu, F., Jiao, L.C.: Self-training multi-sequence learning with transformer for weakly supervised video anomaly detection, pp 1–9 (2022)

  8. Lin, T.C., Xu, H.T., Yang, C.Q. et al.: Interventional multi-instance learning with deconfounded instance-level prediction. In: AAAI Conference on Artificial Intelligence, pp. 1–9 (2022) https://doi.org/10.48550/arXiv.2204.09204

  9. Lin, Y., Zhang, H.G.: Regularized instance embedding for deep multi-instance learning. Appl. Sci. 10(1), 64–77 (2020). https://doi.org/10.3390/app10010064

    Article  Google Scholar 

  10. Shi, X.S., Xing, F.Y., Xie, Y.P. et al.: Loss-based attention for deep multiple instance learning. In: AAAI Conference on Artificial Intelligence, pp. 5742–5749 (2020). https://doi.org/10.1609/aaai.v34i04.6030

  11. Sİvrikaya, Ö.E., Yüksekgönül, M., Baydoğan, M.G.: Learning prototypes for multiple instance learning. Turk. J. Electr. Eng. Comput. Sci. 29(7), 2901–2919 (2021)

    Article  Google Scholar 

  12. Tarragó, D.S., Cornelis, C., Bello, R., et al.: A multi-instance learning wrapper based on the Rocchio classifier for web index recommendation. Knowl. Based Syst. 59, 173–181 (2014). https://doi.org/10.1016/j.knosys.2014.01.008

    Article  Google Scholar 

  13. Tian, Y., Pang, G.S., Chen, Y.H., et al.: Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In: IEEE/CVF International Conference on Computer Vision, pp. 4975–4986 (2021)

  14. Wei, X.S., Zhou, Z.H.: An empirical study on image bag generators for multi-instance learning. Mach. Learn. 105(2), 155–198 (2016). https://doi.org/10.1007/s10994-016-5560-1

    Article  MathSciNet  Google Scholar 

  15. Wei, X.S., Wu, J.X., Zhou, Z.H.: Scalable multi-instance learning. In: IEEE International Conference on Data Mining, pp. 1037–1042 (2014) https://doi.org/10.1109/ICDM.2014.16

  16. Wei, X.S., Wu, J.X., Zhou, Z.H.: Scalable algorithms for multi-instance learning. IEEE Trans. Neural Netw. Learn. Syst. 28(4), 975–987 (2017). https://doi.org/10.1109/TNNLS.2016.2519102

    Article  Google Scholar 

  17. Wu, J., Pan, S.R., Zhu, X.Q., et al.: Multi-instance learning with discriminative bag mapping. IEEE Trans. Knowl. Data Eng. 30(6), 1065–1080 (2018). https://doi.org/10.1109/TKDE.2017.2788430

    Article  Google Scholar 

  18. Xu, B.C., Ting, K.M., Zhou, Z.H.: Isolation set-kernel and its application to multi-instance learning. In: ACM SIGKDD International Conference on Knowledge Discovery & Data Mining July, pp. 941–949 (2019). https://doi.org/10.1145/3292500.3330830

  19. Yang, M., Zhang, Y.X., Wang, X.Z. et al.: Multi-instance ensemble learning with discriminative bags. IEEE Trans. Syst. Man Cybern. Syst. 5456–5467 (2021). https://doi.org/10.1109/TSMC.2021.3125040

  20. Yang, M., Tang, W.T., Min, F.: Multi-instance multi-label learning based on parallel attention and local label manifold correlation. In: International Conference on Data Science and Advanced Analytics, pp. 1–10 (2022a)

  21. Yang, M., Zeng, W.X., Min, F.: Multi-instance embedding learning through high-level instance selection. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 122–133 (2022b). https://doi.org/10.1007/978-3-031-05936-0_10

  22. Yang, M., Zhang, Y.X., Ye, M., et al.: Attention-to-embedding framework for multi-instance learning. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 109–121 (2022c). https://doi.org/10.1007/978-3-031-05936-0_9

  23. Yang, M., Zhang, Y.X., Zhou, Z. et al.: Multi-embedding space set-kernel and its application to multi-instance learning. Neurocomputing 512, 1–14 (2022d)

  24. Zhang, H.R., Meng, Y.D., Zhao, Y.T. et al.: DTFD-MIL: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In: Computer Vision and Pattern Recognition, pp. 18,802–18,812 (2022). https://doi.org/10.48550/arXiv.2203.12081

  25. Zhang, M.L., Zhou, Z.H.: Multi-instance clustering with applications to multi-instance prediction. Appl. Intell. 31(1), 47–68 (2009). https://doi.org/10.1007/s10489-007-0111-x

    Article  Google Scholar 

  26. Zhang, T., Jin, H.: Optimal margin distribution machine for multi-instance learning. In: International Conference on International Joint Conferences on Artificial Intelligence, pp. 2383–2389 (2021)

  27. Zhang, W.J., Liu, L., Li, J.Y.: Robust multi-instance learning with stable instances, pp 1682–1689 (2020). https://doi.org/10.3233/FAIA200280. arXiv:1902.05066

  28. Zhou, Z.H., Jiang, K., Li, M.: Multi-instance learning based web mining. Appl. Intell. 22, 135–147 (2005). https://doi.org/10.1007/s10489-005-5602-z

    Article  Google Scholar 

  29. Zhou, Z.H., Sun, Y.Y., Li, Y.F.: Multi-instance learning by treating instances as non-I.I.D. samples. In: International Conference on Machine Learning, pp. 1249–1256 (2009). https://doi.org/10.1145/1553374.1553534

Download references

Acknowledgements

This work was supported in part by the National Key R &D Program of China (2018YFE0203900), National Natural Science Foundation of China (61773093), Sichuan Science and Technology Program (2020YFG0476), Important Science and Technology Innovation Projects in Chengdu (2018-YF08-00039-GX), and Open Project of Zhejiang Key Laboratory of Marine Big Data Mining and Application (OBDMA202102).

Author information

Author notes

  1. Mei Yang, Zhengchun Zhou and Fan Min contributed equally to this work.

Authors and Affiliations

  1. School of Computer Science, Southwest Petroleum University, Chengdu, China

    Yu-Xuan Zhang, Mei Yang & Fan Min

  2. School of Information Science and Technology, Southwest Jiaotong University, Chengdu, China

    Zhengchun Zhou

  3. Institute for Artificial Intelligence, Southwest Petroleum University, Chengdu, China

    Fan Min

Authors
  1. Yu-Xuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengchun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fan Min
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fan Min.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, YX., Yang, M., Zhou, Z. et al. Multi-instance embedding learning with deconfounded instance-level prediction. Int J Data Sci Anal 16, 391–401 (2023). https://doi.org/10.1007/s41060-022-00372-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41060-022-00372-7

Keywords

Search

Navigation