Reconstruction of Markov Random Fields from Samples: Some Observations and Algorithms

  • Guy Bresler
  • Elchanan Mossel
  • Allan Sly
Conference paper

DOI: 10.1007/978-3-540-85363-3_28

Volume 5171 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Bresler G., Mossel E., Sly A. (2008) Reconstruction of Markov Random Fields from Samples: Some Observations and Algorithms. In: Goel A., Jansen K., Rolim J.D.P., Rubinfeld R. (eds) Approximation, Randomization and Combinatorial Optimization. Algorithms and Techniques. Lecture Notes in Computer Science, vol 5171. Springer, Berlin, Heidelberg

Abstract

Markov random fields are used to model high dimensional distributions in a number of applied areas. Much recent interest has been devoted to the reconstruction of the dependency structure from independent samples from the Markov random fields. We analyze a simple algorithm for reconstructing the underlying graph defining a Markov random field on n nodes and maximum degree d given observations. We show that under mild non-degeneracy conditions it reconstructs the generating graph with high probability using Θ(d logn) samples which is optimal up to a multiplicative constant. Our results seem to be the first results for general models that guarantee that the generating model is reconstructed. Furthermore, we provide an explicit O(dnd + 2 logn) running time bound. In cases where the measure on the graph has correlation decay, the running time is O(n2 logn) for all fixed d. In the full-length version we also discuss the effect of observing noisy samples. There we show that as long as the noise level is low, our algorithm is effective. On the other hand, we construct an example where large noise implies non-identifiability even for generic noise and interactions. Finally, we briefly show that in some cases, models with hidden nodes can also be recovered.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Guy Bresler
    • 1
  • Elchanan Mossel
    • 2
  • Allan Sly
    • 3
  1. 1.Dept. of Electrical Engineering and Computer SciencesU.C. Berkeley 
  2. 2.Dept. of Statistics and Dept. of Electrical Engineering and Computer SciencesU.C. Berkeley 
  3. 3.Dept. of StatisticsU.C. Berkeley