The VLDB Journal

, Volume 15, Issue 3, pp 250–262 | Cite as

Dependency trees in sub-linear time and bounded memory

Regular Paper


We focus on the problem of efficient learning of dependency trees. Once grown, they can be used as a special case of a Bayesian network, for PDF approximation, and for many other uses. Given the data, a well-known algorithm can fit an optimal tree in time that is quadratic in the number of attributes and linear in the number of records. We show how to modify it to exploit partial knowledge about edge weights. Experimental results show running time that is near-constant in the number of records, without significant loss in accuracy of the generated trees.


Data mining Probably approximately correct learning Fast algorithms Dependency trees 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blake, C., Merz, C.: UCI repository of machine learning databases (1998)
  2. 2.
    Chow, C.K., Liu, C.N.: Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory, 14, 462–467 (1968)CrossRefGoogle Scholar
  3. 3.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L.: Introduction to Algorithms. McGraw-Hill (1989)Google Scholar
  4. 4.
    Davies, S.: Scalable and practical probability density estimators for scientific anomaly detection. Doctoral dissertation, Carnegie-Mellon University (2002)Google Scholar
  5. 5.
    Domingos, P., Hulten, G.: Mining high-speed data streams. In Proceedings of 6th International Conference on Knowledge Discovery and Data Mining, pp. 71–80, N.Y., ACM Press (2000)Google Scholar
  6. 6.
    Domingos, P., Hulten, G.: A general method for scaling up machine learning algorithms and its application to clustering. In Proceedings of the Eighteenth International Conference on Machine Learning. Morgan Kaufmann (2001a)Google Scholar
  7. 7.
    Domingos, P., Hulten, G.: Learning from infinite data in finite time. In Advances in Neural Information Processing Systems 14, Vancouver, British Columbia, Canada (2001b)Google Scholar
  8. 8.
    Friedman, N., Goldszmidt, M., Lee, T.J.: Bayesian Network Classification with Continuous Attributes: Getting the Best of Both Discretization and Parametric Fitting. In Proceedings of the Fifteenth International Conference on Machine Learning. Morgan Kaufmann, San Francisco, CA (1998)Google Scholar
  9. 9.
    Friedman, N., Nachman, I., Peér, D.: Learning bayesian network structure from massive datasets: The “sparse candidate” algorithm. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI-99), pp. 206–215. Stockholm, Sweden 1999)Google Scholar
  10. 10.
    Goldenberg, A., Moore, A.: Tractable learning of large bayes net structures from sparse data. In Proc. 21st International Conf. on Machine Learning (2004)Google Scholar
  11. 11.
    Hettich, S., Bay, S.D.: The UCI KDD archive, (1999)
  12. 12.
    Maron, O., Moore A.W.: Hoeffding races: Acdelerating model selection search for classification and function approximation. Advances in Neural Information Processing Systems, pp 59–66. Denver, Colorado, Morgan Kaufmann (1994)Google Scholar
  13. 13.
    Meila, M.: An accelerated Chow and Liu algorithm: fitting tree distributions to high dimensional sparse data. In Proceedings of the Sixteenth International Conference on Machine Learning (1999a)Google Scholar
  14. 14.
    Meila, M.: Learning with Mixtures of Trees. Doctoral dissertation. Massachusetts Institute of Technology (1999b)Google Scholar
  15. 15.
    Moore, A.W., Lee M.S.: Efficient algorithms fro minimizing cross validation error. In Proceedings of the Eleventh International Conference on Machine Learning, pp. 190–198 New Brunswick, US: Morgan Kaufmann (1994)Google Scholar
  16. 16.
    Pelleg, D.: Scalable and Practical Probability Density Estimators for Scientific Anomaly Detection. Doctoral dissertation, Carnegie-Mellon University (2004)Google Scholar
  17. 17.
    Reza, F.: An Introduction to Information Theory. Dover Publications, pp. 282–283. New York (1994)Google Scholar
  18. 18.
    SDSS. The Sloan Digital Sky Survey. (1998)
  19. 19.
    Tarjan, R.E.: Data Structures and Network Algorithms, Vol. 44 of CBMS-NSF Reg. Conf. Ser. Appl. Math. SIAM (1983)Google Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  1. 1.IBM Haifa LabsHaifaIsrael
  2. 2.Robotics InstituteCarnegie-Mellon UniversityPittsburgh

Personalised recommendations