Forest construction of Gaussian and discrete variables with the application of Watanabe Bayesian Information Criterion

Abstract

This paper introduces a technique to estimate mutual information in data sets that comprise discrete and continuous variables. Utilizing the Chow–Liu algorithm, our approach constructs a forest that captures the underlying probabilistic dependencies among variables. Conventional methods maximize likelihood, particularly in sequences involving discrete, Gaussian, and discrete variables, thus limiting the class of permissible forests. Our novel methodology overcomes these constraints by accommodating discrete and continuous random variables simultaneously. Initially, we used copula techniques to estimate the joint density of mixed-type variables. Subsequently, we apply the Watanabe Bayesian Information Criterion (WBIC) to compute the free energies, enabling a more sophisticated estimation of mutual information between the discrete and continuous variables. This innovation improves the capabilities of existing mutual information estimation frameworks. When integrated with the Chow-Liu algorithm, our estimator produces a forest topology instead of a mere spanning tree without restrictive assumptions. Our method successfully links genomic expression to single nucleotide polymorphism (SNP) data in genome expression analysis.