New Generation Computing

, Volume 35, Issue 1, pp 5–27

On Model Selection, Bayesian Networks, and the Fisher Information Integral

Special Feature

DOI: 10.1007/s00354-016-0002-y

Cite this article as:
Zou, Y. & Roos, T. New Gener. Comput. (2017) 35: 5. doi:10.1007/s00354-016-0002-y
  • 79 Downloads

Abstract

We study BIC-like model selection criteria and in particular, their refinements that include a constant term involving the Fisher information matrix. We perform numerical simulations that enable increasingly accurate approximation of this constant in the case of Bayesian networks. We observe that for complex Bayesian network models, the constant term is a negative number with a very large absolute value that dominates the other terms for small and moderate sample sizes. For networks with a fixed number of parameters, d, the leading term in the complexity penalty, which is proportional to d, is the same. However, as we show, the constant term can vary significantly depending on the network structure even if the number of parameters is fixed. Based on our experiments, we conjecture that the distribution of the nodes’ outdegree is a key factor. Furthermore, we demonstrate that the constant term can have a dramatic effect on model selection performance for small sample sizes.

Keywords

Model selection Bayesian networks Fisher information approximation NML BIC 

Copyright information

© Ohmsha, Ltd. and Springer Japan 2016

Authors and Affiliations

  1. 1.Helsinki Institute for Information Technology HIIT, Department of Computer ScienceUniversity of HelsinkiHelsinkiFinland