Skip to main content
SpringerLink
Log in

Learning a Bayesian network with multiple latent variables for implicit relation representation

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

Artificial intelligence applications could be more powerful and comprehensive by incorporating the ability of inference, which could be achieved by probabilistic inference over implicit relations. It is significant yet challenging to represent implicit relations among observed variables and latent ones like disease etiologies and user preferences. In this paper, we propose the BN with multiple latent variables (MLBN) as the framework for representing the dependence relations, where multiple latent variables are incorporated to describe multi-dimensional abstract concepts. However, the efficiency of MLBN learning and effectiveness of MLBN based applications are still nontrivial due to the presence of multiple latent variables. To this end, we first propose the constraint induced and Spark based algorithm for MLBN learning, as well as several optimization strategies. Moreover, we present the concept of variation degree and further design a subgraph based algorithm for incremental learning of MLBN. Experimental results suggest that our proposed MLBN model could represent the dependence relations correctly. Our proposed method outperforms some state-of-the-art competitors for personalized recommendation, and facilitates some typical approaches to achieve better performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Algorithm 2
Algorithm 3
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Data availability

The datasets of this work are available at: Chest-clinic network: (https://www.bnlearn.com/bnrepository/discrete-small.html#asia)

MovieLens-1M: (https://grouplens.org/datasets/movielens/) Renttherunway: (https://cseweb.ucsd.edu/~jmcauley/datasets.html).

Notes

  1. https://www.renttherunway.com.

References

  • Amirkhani H, Rahmati M, Lucas P, Hommersom A (2017) Exploiting experts’ knowledge for structure learning of Bayesian networks. IEEE Trans Pattern Anal Mach Intell 39(11):2154–2170

    Article  PubMed  Google Scholar 

  • Anandkumar A, Chaudhuri K, Hsu DJ, Kakade SM, Song L, Zhang T (2011) Spectral methods for learning multivariate latent tree structure. In: Proceedings of the 25th annual conference Neural Information Processing Systems (NIPS), Granada, Spain, pp 2025–2033

  • Anandkumar A, Hsu DJ, Javanmard A, Kakade SM (2013) Learning linear bayesian networks with latent variables. In: Proceedings of the 30th international conference on machine learning (ICML), Atlanta, GA, USA, vol 28, pp 249–257. JMLR.org

  • Apache Software Foundation (2020) http://spark.apache.org. Apache Spark

  • Bartlett M, Cussens J (2017) Integer linear programming for the Bayesian network structure learning problem. Artif Intell 244:258–271

    Article  MathSciNet  Google Scholar 

  • Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022

    Google Scholar 

  • Buntine WL (1991)Theory refinement on Bayesian networks. In: Proceedings of the 7th conference uncertainty artificial intelligence (UAI), Los Angeles, CA, USA, pp 52–60

  • Chandrasekaran V, Parrilo PA, Willsky AS (2012) Latent variable graphical model selection via convex optimization. Ann Stat 40(4):1935–1967

    MathSciNet  Google Scholar 

  • Chen C, Yuan C (2019) Learning diverse Bayesian networks. In: Proceedings of the 33rd conference artificial intelligence (AAAI), Honolulu, Hawaii, USA, pp 7793–7800

  • Contaldi C, Vafaee F, Nelson PC (2019) Bayesian network hybrid learning using an elite-guided genetic algorithm. Artif Intell Rev 52(1):245–272

    Article  Google Scholar 

  • de Campos CP, Scanagatta M, Corani G, Zaffalon M (2018) Entropy-based pruning for learning Bayesian networks using BIC. Artif Intell 260:42–50

    Article  MathSciNet  Google Scholar 

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B Methodol 39(1):1–38

    MathSciNet  Google Scholar 

  • Do CB, Batzoglou S (2008) What is the expectation maximization algorithm? Nat Biotechnol 26(8):897–899

    Article  CAS  PubMed  Google Scholar 

  • Friedman N (1997) Learning belief networks in the presence of missing values and hidden variables. In: Proceedings of the 14th international conference machine learning (ICML), Nashville, TN, USA, pp 125–133

  • Friedman N (1998) The Bayesian structural EM algorithm. In: Proceedings of the 14th conference uncertainty artificial intelligence (UAI), Madison, Wisconsin, USA, pp 129–138

  • Friedman N, Goldszmidt M (1997) Sequential update of Bayesian network structure. In: Proceedings of the 13rd conference uncertainty artificial intelligence (UAI), Providence, RI, USA, pp 165–174

  • Gámez JA, Mateo JL, Puerta JM (2011) Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood. Data Min Knowl Disc 22(1–2):106–148

    Article  MathSciNet  Google Scholar 

  • Gao T, Fadnis KP, Campbell M (2017) Local-to-global Bayesian network structure learning. In: Proceedings of the 34th international conference machine learning (ICML), Sydney, NSW, Australia, pp 1193–1202

  • GroupLens (2020) http://grouplens.org/datasets/ movielens/1m/. MovieLens-1M Dataset

  • He C, Yue K, Wu H, Liu W (2014) Structure learning of Bayesian network with latent variables by weight-induced refinement. In: Proceedings of the 5th international workshop on web-scale knowledge represent. Retrieval and Reason (Web-KR), Shanghai, China. ACM, pp 37–44

  • He X, Liao L, Zhang H, Nie L, Hu X, Chua T (2017) Neural collaborative filtering. In: Proceedings of the 26th international conference on World Wide Web (WWW), Perth, Australia, pp 173–182

  • He T, Bao J, Ruan S, Li R, Li Y, He H, Zheng Y (2020) Interactive bike lane planning using sharing bikes’ trajectories. IEEE Trans Knowl Data Eng 32(8):1529–1542

    Google Scholar 

  • Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th international conference on learning representations (ICLR), Toulon, France

  • Koller D, Friedman N (2009) Probabilistic graphical models-principles and techniques. MIT Press, Cambridge

    Google Scholar 

  • Lam W, Bacchus F (1994) Using new data to refine a Bayesian network. In: Proceedings of the 10th conference on uncertainty artificial intelligence (UAI), Seattle, Washington, USA, pp 383–390

  • Lauritzen SL (1995) The EM algorithm for graphical association models with missing data. Comput Stat Data Anal 19(2):191–201

    Article  MathSciNet  Google Scholar 

  • Lauritzen SL, Spiegelhalter DJ (1988) Local computations with probabilities on graphical structures and their application to expert systems. J R Stat Soc B 50(2):157–194

    MathSciNet  Google Scholar 

  • Li S, Tryfonas T, Russell G, Andriotis P (2016) Risk assessment for mobile systems through a multilayered hierarchical Bayesian network. IEEE Trans Cybern 46(8):1749–1759

    Article  PubMed  Google Scholar 

  • Liao ZA, Sharma C, Cussens J, van Beek P (2019) Finding all Bayesian network structures within a factor of optimal. In: Proceedings of the 33rd conference on artificial intelligence (AAAI), Honolulu, Hawaii, USA, pp 7892–7899

  • Liu W, Yue K, Yue M, Yin Z, Zhang B (2018) A Bayesian network-based approach for incremental learning of uncertain knowledge. Int J Uncertain Fuzziness Knowl Based Syst 26(1):87–108

    Article  MathSciNet  Google Scholar 

  • Liu W, Yue K, Li J, Li J, Li J, Zhang Z (2022) Inferring range of information diffusion based on historical frequent items. Data Min Knowl Discov 36(1):82–107

    Article  MathSciNet  Google Scholar 

  • Misra R, Wan M, McAuley JJ (2018) Decomposing fit semantics for product size recommendation in metric spaces. In: Proceedings of the 12nd ACM conference recommender systems (RecSys), Vancouver, BC, Canada, pp 422–426

  • Park J, Kim J (2020) Incremental class learning for hierarchical classification. IEEE Trans Cybern 50(1):178–189

    Article  PubMed  Google Scholar 

  • Sarki R, Ahmed K, Wang H, Zhang Y (2020) Automated detection of mild and multi-class diabetic eye diseases using deep learning. Health Inf Sci Syst 8(1):32

    Article  PubMed  PubMed Central  Google Scholar 

  • Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464

    Article  MathSciNet  Google Scholar 

  • Scutari M, Vitolo C, Tucker A (2019) Learning Bayesian networks from big data with greedy search: computational complexity and efficient implementation. Stat Comput 29(5):1095–1108

    Article  MathSciNet  Google Scholar 

  • Shafer G, Shenoy PP (1990) Probability propagation. Ann Math Artif Intell 2:327–351

    Article  MathSciNet  Google Scholar 

  • Sidana S, Trofimov M, Horodnytskyi O, Laclau C, Maximov Y, Amini M (2021) User preference and embedding learning with implicit feedback for recommender systems. Data Min Knowl Discov 35(2):568–592

    Article  Google Scholar 

  • Suzuki J, Kawahara J (2017) Branch and bound for regular Bayesian network structure learning. In: Proceedings of the 33rd conference on uncertainty artificial intelligence (UAI), Sydney, Australia

  • Tajbakhsh MS, Bagherzadeh J (2019) Semantic knowledge LDA with topic vector for recommending hashtags: Twitter use case. Intell Data Anal 23(3):609–622

    Article  Google Scholar 

  • van den Berg R, Kipf TN, Welling M (2017) Graph convolutional matrix completion. CoRR, arXiv:1706.02263

  • Wang H, Yeung D (2020) A survey on Bayesian deep learning. ACM Comput Surv 53(5):108:1-108:37

    Google Scholar 

  • Wang H, Wang N, Yeung D (2015) Collaborative deep learning for recommender systems. In: Proceedings of the 21st ACM SIGKDD international conference on knowledge discovery data mining (SIGKDD), Sydney, NSW, Australia. ACM, pp 1235–1244

  • Wang X, He X, Wang M, Feng F, Chua T (2019) Neural graph collaborative filtering. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval (SIGIR), Paris, France, pp 165–174

  • Yin H, Yang S, Song X, Liu W, Li J (2021) Deep fusion of multimodal features for social media retweet time prediction. World Wide Web 24(4):1027–1044

    Article  Google Scholar 

  • Yue K, Fang Q, Wang X, Li J, Liu W (2015) A parallel and incremental approach for data-intensive learning of Bayesian networks. IEEE Trans Cybern 45(12):2890–2904

    Article  PubMed  Google Scholar 

  • Yue K, Wu X, Duan L, Qiao S, Wu H (2020) A parallel and constraint induced approach to modeling user preference from rating data. Knowl Based Syst 204:106206

    Article  Google Scholar 

  • Zhang NL, Poole DL (1996) Exploiting causal independence in Bayesian network inference. J Artif Intell Res 5:301–328

    Article  MathSciNet  Google Scholar 

  • Zhang Y, Liu J, Liu Y (2018) Bayesian network structure learning: the two-step clustering-based algorithm. In: Proceedings of the 32nd conference on artificial intelligence (AAAI), New Orleans, Louisiana, USA, pp 8183–8184

  • Zheng S, Ding CHQ, Nie F (2018) Regularized singular value decomposition and application to recommender system. The Computing Research Repository, arXiv:1804.05090

Download references

Acknowledgements

This paper was supported by the Joint Key Project of National Natural Science Foundation of China (U23A20298) and Key Project of Fundamental Research of Yunnan Province (202301AS070153).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yunnan University, Dongwaihuan South Road, Kunming, 650500, China

    Xinran Wu, Kun Yue & Liang Duan

  2. Yunnan Key Laboratory of Intelligent Systems and Computing, Yunnan University, Dongwaihuan South Road, Kunming, 650500, China

    Xinran Wu, Kun Yue & Liang Duan

  3. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, No. 727 Jingming South Road, Kunming, 650504, China

    Xiaodong Fu

Authors
  1. Xinran Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kun Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liang Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaodong Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: XW; Formal Analysis: XW; Writing—original draft: XW; Software: XW; Funding acquisition: KY; Investigation: KY; Methodology: KY; Writing—review & editing: KY; Project administration: LD; Supervision: KY; Validation: LD; Data curation: XF; Resources: KY; Visualization: XF.

Corresponding author

Correspondence to Kun Yue.

Ethics declarations

Conflict of interest

The authors have no conflicts of financial or proprietary interests in any material discussed in this paper.

Code availability

The codes of this work are available at https://github.com/wxr72412/MLBN.

Additional information

Responsible editor: Sriraam Natarajan.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Yue, K., Duan, L. et al. Learning a Bayesian network with multiple latent variables for implicit relation representation. Data Min Knowl Disc (2024). https://doi.org/10.1007/s10618-024-01012-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10618-024-01012-3

Keywords

Search

Navigation