Robust inference of trees

  • Marco Zaffalon
  • Marcus Hutter

This paper is concerned with the reliable inference of optimal tree-approximations to the dependency structure of an unknown distribution generating data. The traditional approach to the problem measures the dependency strength between random variables by the index called mutual information. In this paper reliability is achieved by Walley's imprecise Dirichlet model, which generalizes Bayesian learning with Dirichlet priors. Adopting the imprecise Dirichlet model results in posterior interval expectation for mutual information, and in a set of plausible trees consistent with the data. Reliable inference about the actual tree is achieved by focusing on the substructure common to all the plausible trees. We develop an exact algorithm that infers the substructure in time O(m 4), m being the number of random variables. The new algorithm is applied to a set of data sampled from a known distribution. The method is shown to reliably infer edges of the actual tree even when the data are very scarce, unlike the traditional approach. Finally, we provide lower and upper credibility limits for mutual information under the imprecise Dirichlet model. These enable the previous developments to be extended to a full inferential method for trees.


robust inference spanning trees intervals dependence graphical models mutual information imprecise probabilities imprecise Dirichlet model 

AMS subject classification

05C05 41A58 62G35 62H20 68T37 90C35 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    M. Abramowitz and I.A. Stegun, eds., Handbook of Mathematical Functions (Dover, 1974).Google Scholar
  2. 2.
    I.D. Aron and P. Van Hentenryck, On the complexity of the robust spanning tree problem with internal data, Operations Research Letters 32 (2004) 36–40.CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    J.-M. Bernard, 2001, Non-parametric inference about an unknown mean using the imprecise Dirichlet model, in: ISIPTA'01, eds. G. de Cooman, T. Fine and T. Seidenfeld (The Netherlands, 2001) pp. 40–50.Google Scholar
  4. 4.
    J.-M. Bernard, An introduction to the imprecise Dirichlet model for multinomial data, International Journal of Approximate 39(2–3) (2005) 123–150.CrossRefMathSciNetzbMATHGoogle Scholar
  5. 5.
    C.K. Chow and C.N. Liu, Approximating discrete probability distributions with dependence tress, IEEE Transactions on Information Theory, IT-14(3) (1968) 462–468.CrossRefMathSciNetGoogle Scholar
  6. 6.
    N. Friedman, D. Geiger and M. Goldszmidt, Bayesian networks classifiers, Machine Learning 29(2/3) (1997) 131–163.CrossRefzbMATHGoogle Scholar
  7. 7.
    A. Gelman, J.B. Carlin, H.S. Stern and D.B. Rubin, Bayesian Data Analysis (Chapman, 1995).Google Scholar
  8. 8.
    J.B.S. Haldane, The precision of observed values of small frequencies, Biometrika 35 (1948) 297–300.MathSciNetGoogle Scholar
  9. 9.
    M. Hutter, Distribution of mutual information, in: Proceedings of NIPS*2001, eds. T.G. Dietterich, S. Vecker and Z. Ghahramani (Cambridge, MA, 2001).Google Scholar
  10. 10.
    M. Hutter, Robust estimators under the imprecise dirichlet model, in: Proc. 3rd International Symposium on Imprecise Probalities and Their Application (ISIPTA-2003), Proceedings in Informatics Vol. 18 (Canada, 2003) pp. 274–289.Google Scholar
  11. 11.
    M. Hutter and M. Zaffalon, Distribution of mutual information from complete and incomplete data, Computational Statics & Data Analysis 48(3) (2005) 633–657.CrossRefMathSciNetzbMATHGoogle Scholar
  12. 12.
    H. Jeffreys, An invariant form for the prior probability in estimation problems, in: Proceedings Royal Society London A, 186 (1946) pp. 453–461.zbMATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    M.G. Kendall and A. Stuart, The Advanced Theory of Statistics, 2nd edition. (Griffin, London, 1967).Google Scholar
  14. 14.
    G.D. Kleiter, The posterior probability of Bayers nets with strong dependences, Soft Computing 3 (1999) 162–173.Google Scholar
  15. 15.
    J.B. Kruskal Jr., On the shortest spanning subtree of a graph and the traveling salesman problem, in: Proceedings of the American Mathematical Society 7 (1956) 48–50.CrossRefMathSciNetGoogle Scholar
  16. 16.
    S. Kullback, Information Theory and Statistics (Dover, 1968).Google Scholar
  17. 17.
    S. Kullback and R.A. Leiber, On information and sufficiency, Annals of Mathematical Statistics 22 (1951) 79–86.CrossRefMathSciNetzbMATHGoogle Scholar
  18. 18.
    C. Manski, Partial Identification of Probability Distributions (Department of Economics, Northwestern University, USA: Draft book, 2002).Google Scholar
  19. 19.
    R. Montemanni, A Benders decomposition approach for the robust spanning tree problem with interval data, European Journal of Operational Research. Forthcoming.Google Scholar
  20. 20.
    H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity (Prentice Hall, New York, 1982).zbMATHGoogle Scholar
  21. 21.
    J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann, San Mateo, 1988).Google Scholar
  22. 22.
    W. Perks, Some observations on inverse probability, Journal of the Institute of Actuaries 73 (1947) 285–312.MathSciNetGoogle Scholar
  23. 23.
    M. Ramoni and P. Sebastiani, Robust learning with missing data, Machine Learning 45(2) (2001) 147–170.CrossRefzbMATHGoogle Scholar
  24. 24.
    T. Verma and J. Pearl, Equivalence and synthesis of causal models, in: UAI'90, eds. P.P. Bonissone, M. Henrion, L.N. Kanal and J.F. Lemmer (New York, 1990) pp. 220–227.Google Scholar
  25. 25.
    P. Walley, Statistical Reasoning with Imprecise Probabilities (Chapman and Hall, New York, 1991).zbMATHGoogle Scholar
  26. 26.
    P. Walley, Inferences from multinomial data: learning about a bag of marbles, Journal of the Royal Statistical Society B 58(1) (1996) 3–57.zbMATHMathSciNetGoogle Scholar
  27. 27.
    D.H. Wolpert and D.R. Wolf, Estimating functions of distributions from a finite set of samples, Physical Review E 52(6) (1995) 6841–6854.CrossRefMathSciNetGoogle Scholar
  28. 28.
    H. Yaman, O.E. Karaşan and M.C. Pinar, The robust spanning tree problem with interval data, Operations Research Letters 29 (2001) 31–40.CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    M. Zaffalon, Exact credal treatment of missing data, Journal of Statistical Planning and Inference 105(1) (2002) 105–122.CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2006

Authors and Affiliations

  1. 1.IDSIAMannoSwitzerland

Personalised recommendations