MC4: A Tempering Algorithm for Large-Sample Network Inference

  • Daniel James Barker
  • Steven M. Hill
  • Sach Mukherjee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6282)


Bayesian networks and their variants are widely used for modelling gene regulatory and protein signalling networks. In many settings, it is the underlying network structure itself that is the object of inference. Within a Bayesian framework inferences regarding network structure are made via a posterior probability distribution over graphs. However, in practical problems, the space of graphs is usually too large to permit exact inference, motivating the use of approximate approaches. An MCMC-based algorithm known as MC3 is widely used for network inference in this setting. We argue that recent trends towards larger sample size datasets, while otherwise advantageous, can, for reasons related to concentration of posterior mass, render inference by MC3 harder. We therefore exploit an approach known as parallel tempering to put forward an algorithm for network inference which we call MC4. We show empirical results on both synthetic and proteomic data which highlight the ability of MC4 to converge faster and thereby yield demonstrably accurate results, even in challenging settings where MC3 fails.


Markov Chain Monte Carlo Bayesian Network Marginal Likelihood Markov Chain Monte Carlo Method Proposal Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Friedman, N.: Inferring cellular networks using probabilistic graphical models. Science 303(5659), 799–805 (2004)CrossRefPubMedGoogle Scholar
  2. 2.
    Gelman, A., Rubin, D.B.: Inference from iterative simulation using multiple sequences. Stat. Sci. 7, 457–472 (1992)CrossRefGoogle Scholar
  3. 3.
    Geyer, C., Thompson, E.: Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference. Journal of the American Statistical Association 90(431), 909–920 (1995)CrossRefGoogle Scholar
  4. 4.
    Husmeier, D.: Reverse engineering of genetic networks with Bayesian networks. Biochemical Society Transactions 31(6), 1516–1518 (2003)CrossRefPubMedGoogle Scholar
  5. 5.
    Jordan, M.: Graphical Models. Stat. Sci. 19, 140–155 (2004)CrossRefGoogle Scholar
  6. 6.
    Lauritzen, S.: Graphical Models. O.U.P., Oxford (1996)Google Scholar
  7. 7.
    Liu, J.: Monte Carlo Strategies in Scientific Computing. Series in Statistics. Springer, New York (2008)Google Scholar
  8. 8.
    Madigan, D., York, J., Allard, D.: Bayesian Graphical Models for Discrete Data. International Statistical Review/Revue Internationale de Statistique 63(2), 215–232 (1995)Google Scholar
  9. 9.
    Marinari, E., Parisi, G.: Simulated Tempering: a New Monte Carlo Scheme. Europhys. Lett. 19(6), 451–458 (1992)CrossRefGoogle Scholar
  10. 10.
    Mukherjee, S., Speed, T.: Network Inference Using Informative Priors. PNAS 105(38), 14313–14318 (2008)CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Murphy, K.: Dynamic Bayesian Networks: Representation, Inference and Learning. Ph.D. thesis, Computer Science Division, Berkeley CA (2002)Google Scholar
  12. 12.
    Newman, M., Barkema, G.: Monte Carlo Methods in Statistical Physics. O.U.P., Oxford (1999)Google Scholar
  13. 13.
    Nott, D.J., Green, P.J.: Bayesian variable selection and the swendsen-wang algorithm. J. Comput. Graph. Stat. 13, 141–157 (2004)CrossRefGoogle Scholar
  14. 14.
    Robert, C., Casella, G.: Monte Carlo Statistical Methods. Springer, New York (2004)CrossRefGoogle Scholar
  15. 15.
    Robinson, R.: Counting Labeled Acyclic Digraphs. In: New Directions in the Theory of Graphs, pp. 239–273. Academic Press, London (1973)Google Scholar
  16. 16.
    Sachs, K., Perez, O., Pe’er, D., Lauffenburger, D.A., Nolan, G.P.: Causal protein-signaling networks derived from multiparameter single-cell data. Science 308(5721), 523–529 (2005)CrossRefPubMedGoogle Scholar
  17. 17.
    Smith, M., Kohn, R.: Nonparametric regression using Bayesian variable selection. J. Econometrics 75, 317–343 (1996)CrossRefGoogle Scholar
  18. 18.
    Xie, X., Geng, Z.: A recursive method for structural learning of directed acyclic graphs. J. Mach. Learn. Res. 9, 459–483 (2008)Google Scholar
  19. 19.
    Yu, J., Smith, A., Wang, P.P., Hartemink, A.J., Jarvis, E.D.: Advances to Bayesian network inference for generating causal networks from observational biological data. Bioinformatics 20(18), 3594–3603 (2004)CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Daniel James Barker
    • 1
    • 2
  • Steven M. Hill
    • 1
    • 3
  • Sach Mukherjee
    • 3
    • 1
  1. 1.Centre for Complexity ScienceUniversity of WarwickCoventryU.K.
  2. 2.Department of PhysicsUniversity of WarwickCoventryU.K.
  3. 3.Department of StatisticsUniversity of WarwickCoventryU.K.

Personalised recommendations