Measures for the Evaluation and Comparison of Graphical Model Structures

  • Gabriel KronbergerEmail author
  • Bogdan Burlacu
  • Michael Kommenda
  • Stephan Winkler
  • Michael Affenzeller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10671)


Structure learning is the identification of the structure of graphical models based solely on observational data and is NP-hard. An important component of many structure learning algorithms are heuristics or bounds to reduce the size of the search space. We argue that variable relevance rankings that can be easily calculated for many standard regression models can be used to improve the efficiency of structure learning algorithms. In this contribution, we describe measures that can be used to evaluate the quality of variable relevance rankings, especially the well-known normalized discounted cumulative gain (NDCG). We evaluate and compare different regression methods using the proposed measures and a set of linear and non-linear benchmark problems.


Graphical models Structure learning Regression 



The authors gratefully acknowledge financial support by the Austrian Research Promotion Agency (FFG) and the Government of Upper Austria within the COMET Project #843532 Heuristic Optimization in Production and Logistics (HOPL).


  1. 1.
    Campos, C.P., Ji, Q.: Efficient structure learning of Bayesian networks using constraints. J. Mach. Learn. Res. 12, 663–689 (2011)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Chickering, D.M.: Learning Bayesian Networks is NP-Complete, pp. 121–130. Springer, Heidelberg (1996). Google Scholar
  3. 3.
    Elidan, G., Nachman, I., Friedman, N.: “Ideal Parent” structure learning for continuous variable Bayesian networks. J. Mach. Learn. Res. 8(8), 1799–1833 (2007)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)CrossRefzbMATHGoogle Scholar
  5. 5.
    Friedman, N., Nachman, I.: Gaussian process networks. In: Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence (UAI), pp. 211–219. Morgan Kaufmann Publishers (2000)Google Scholar
  6. 6.
    Hand, D.J., Till, R.J.: A simple generalisation of the area under the ROC curve for multiple class classification problems. Mach. Learn. 45(2), 171–186 (2001). CrossRefzbMATHGoogle Scholar
  7. 7.
    Hofmann, R., Tresp, V.: Discovering structure in continuous variables using Bayesian networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 500–506 (1996)Google Scholar
  8. 8.
    Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. (TOIS) 20(4), 422–446 (2002)CrossRefGoogle Scholar
  9. 9.
    Koivisto, M., Sood, K.: Exact Bayesian structure discovery in Bayesian networks. J. Mach. Learn. Res. 5(May), 549–573 (2004)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning. MIT Press, Cambridge (2009)Google Scholar
  11. 11.
    Koski, T.J., Noble, J.: A review of Bayesian networks and structure learning. Math. Applicanda 40(1), 51–103 (2012)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Kronberger, G.: Symbolic Regression for Knowledge Discovery - Bloat, Overfitting, and Variable Interaction Networks. Reihe C: Technik und Naturwissenschaften, Trauner Verlag (2011)Google Scholar
  13. 13.
    Kronberger, G., Fink, S., Kommenda, M., Affenzeller, M.: Macro-economic time series modeling and interaction networks. In: Di Chio, C., Brabazon, A., Di Caro, G.A., Drechsler, R., Farooq, M., Grahl, J., Greenfield, G., Prins, C., Romero, J., Squillero, G., Tarantino, E., Tettamanzi, A.G.B., Urquhart, N., Uyar, A.Ş. (eds.) EvoApplications 2011. LNCS, vol. 6625, pp. 101–110. Springer, Heidelberg (2011). CrossRefGoogle Scholar
  14. 14.
    Meinshausen, N., Bühlmann, P.: High-dimensional graphs and variable selection with the lasso. Annal. Stat. 34(3), 1436–1462 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Rao, R., Lakshminarayanan, S.: Variable interaction network based variable selection for multivariate calibration. Anal. Chim. Acta 599(1), 24–35 (2007)CrossRefGoogle Scholar
  16. 16.
    Singh, A.P., Moore, A.W.: Finding optimal Bayesian networks by dynamic programming. Technical report CMU-CALD-05-1062, School of Computer Science, Carnegie Mellon University, June 2005Google Scholar
  17. 17.
    Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. Springer, New York (1993). CrossRefzbMATHGoogle Scholar
  18. 18.
    Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn. 65(1), 31–78 (2006)CrossRefGoogle Scholar
  19. 19.
    Winker, S., Affenzeller, M., Kronberger, G., Kommenda, M., Wagner, S., Jacak, W., Stekel, H.: Variable interaction networks in medical data. In: Proceedings of the 24th European Modeling and Simulation Symposium EMSS 2012, pp. 265–270. Dime Universitá di Genova (2012)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Heuristic and Evolutionary Algorithms LaboratoryUniversity of Applied Sciences Upper AustriaHagenbergAustria
  2. 2.Institute for Formal Models and VerificationJohannes Kepler UniversityLinzAustria

Personalised recommendations