Probabilistic Graphical Markov Model Learning: An Adaptive Strategy

  • Elva Diaz
  • Eunice Ponce-de-Leon
  • Pedro Larrañaga
  • Concha Bielza
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5845)


In this paper an adaptive strategy to learn graphical Markov models is proposed to construct two algorithms. A statistical model complexity index (SMCI) is defined and used to classify models in complexity classes, sparse, medium and dense. The first step of both algorithms is to fit a tree using the Chow and Liu algorithm. The second step begins calculating SMCI and using it to evaluate an index (EMUBI) to predict the edges to add to the model. The first algorithm adds the predicted edges and stop, and the second, decides to add an edge when the fitting improves. The two algorithms are compared by an experimental design using models of different complexity classes. The samples to test the models are generated by a random sampler (MSRS). For the sparse class both algorithms obtain always the correct model. For the other two classes, efficiency of the algorithms is sensible to complexity.


Graphical Markov Model Chow and Liu Algorithm Model Learning Entropy Complexity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adami, C.N., Cerf, J.: Physical Complexity of Symbolic Sequences. Physica D 137, 62–69 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Akaike, H.: A New Look at the Statistical Model Identification. IEEE Transactions on Automatic Control 19, 716–723 (1974)zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Chickering, M.C., Heckerman, D., Meck, C.: Large-Sample Learning of Bayesian Network is NP-Hard. Journal of Machine Learning Research 5, 1287–1330 (2004)Google Scholar
  4. 4.
    Chow, C.K., Liu, W.: Approximating Discrete Probability Distributions with Dependency Trees. IEEE Trans. Inf. Theory IT-14(3), 462–467 (1968)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Deming, W.E., Stephan, F.F.: On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals are Known. Ann. of Math. Static. 11, 427–444 (1940)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Diaz, E.: Metaheurísticas Híbridas para el Aprendizaje de Modelos Gráficos Markovianos y Aplicaciones. Tesis para optar por el Grado de Doctor en Ciencias de la Computación. Universidad Autónoma de Aguascalientes, Ags., Mexico (2008)Google Scholar
  7. 7.
    Diaz, E., Ponce de Leon, E.: Discrete Markov model selection by a genetic algorithm. In: Sossa-Azuela, J.H., Aguilar-IbaÑÉz, C., Alvarado-Mentado, M., Gelbukh, A. (eds.) Avances en Ciencias de la Computación e Ingeniería de Cómputo, Mexico, vol. 2, pp. 315–324 (2002)Google Scholar
  8. 8.
    Diaz, E., Ponce de Leon, E.: Markov Structure Random Sampler (MSRS) algorithm from unrestricted discrete graphic Markov models. In: Gelbukh, A., Reyes, C.A. (eds.) Proceedings of the Fifth Mexican International Conference on Artificial Intelligence, pp. 199–206. IEEE Computer Society, Mexico (2006)CrossRefGoogle Scholar
  9. 9.
    Haberman, S.J.: The Analysis of Frequency Data. The University of Chicago Press (1974)Google Scholar
  10. 10.
    Koller, D., Freedman, N., Getoor, L., Taskar, B.: Graphical models in a nutshell in Introduction to Statistical Relational Learning. In: Getoor, L., Taskar, B. (eds.) Stanford (2007)Google Scholar
  11. 11.
    Kruskal, J.B.: On the Shortest Spanning Tree of a Graph and the Traveling Salesman Problem. Proc. Amer. Math. Soc. 7, 48–50 (1956)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Kuratowski, K.: Introduction to Calculus. Pergamon Press, Warsaw (1961)zbMATHGoogle Scholar
  13. 13.
    Lauritzen, S.L.: Graphical models. Oxford University Press, USA (1996)Google Scholar
  14. 14.
    Li, M., Vitanyi, P.M.B.: An Introduction to Kolmogorov Complexity and its Applications. Springer, Heidelberg (1993)zbMATHGoogle Scholar
  15. 15.
    MacKay, J.C.: Information Theory Inference and Learning Algorithms. Cambridge Press (2003)Google Scholar
  16. 16.
    Zvonkin, A.K., Levin, L.A.: The Complexity of Finite Objects and the Development of the Concepts of Information and Randomness by Means of the Theory of Algorithms. Russ. Math. Surv. 256, 83–124 (1970)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Elva Diaz
    • 1
  • Eunice Ponce-de-Leon
    • 1
  • Pedro Larrañaga
    • 2
  • Concha Bielza
    • 2
  1. 1.Computer Science DepartmentAutonomous University of AguascalientesAguascalientesMexico
  2. 2.Department of Artificial IntelligencePolytechnic University of MadridMadridSpain

Personalised recommendations