Advertisement

Markov Logic: A Language and Algorithms for Link Mining

  • Pedro Domingos
  • Daniel Lowd
  • Stanley Kok
  • Aniruddh Nath
  • Hoifung Poon
  • Matthew Richardson
  • Parag Singla
Chapter

Abstract

Link mining problems are characterized by high complexity (since linked objects are not statistically independent) and uncertainty (since data is noisy and incomplete). Thus they necessitate a modeling language that is both probabilistic and relational. Markov logic provides this by attaching weights to formulas in first-order logic and viewing them as templates for features of Markov networks. Many link mining problems can be elegantly formulated and efficiently solved using Markov logic. Inference algorithms for Markov logic draw on ideas from satisfiability testing, Markov chain Monte Carlo, belief propagation, and resolution. Learning algorithms are based on convex optimization, pseudo-likelihood, and inductive logic programming. Markov logic has been used successfully in a wide variety of link mining applications and is the basis of the open-source Alchemy system.

Keywords

Markov Chain Monte Carlo Link Prediction Inductive Logic Programming Horn Clause Ground Atom 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

This research was partly supported by ARO grant W911NF-08-1-0242, DARPA contracts FA8750-05-2-0283, FA8750-07-D-0185, HR0011-06-C-0025, HR0011-07-C-0060 and NBCH-D030010, NSF grants IIS-0534881 and IIS-0803481, ONR grants N-00014-05-1-0313 and N00014-08-1-0670, an NSF CAREER Award (first author), a Sloan Research Fellowship (first author), an NSF Graduate Fellowship (second author), and a Microsoft Research Graduate Fellowship (second author). The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ARO, DARPA, NSF, ONR, or the US Government.

References

  1. 1.
    J. Besag. Statistical analysis of non-lattice data. The Statistician, 24:179–195, 1975.CrossRefGoogle Scholar
  2. 2.
    M. Collins. Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, pages 1–8, ACL Philadelphia, PA, 2002.Google Scholar
  3. 3.
    M. Craven and S. Slattery. Relational learning with statistical predicate invention: Better models for hypertext. Machine Learning, 43(1/2):97–119, 2001.CrossRefGoogle Scholar
  4. 4.
    P. Damien, J. Wakefield, and S. Walker. Gibbs sampling for Bayesian non-conjugate and hierarchical models by auxiliary variables. Journal of the Royal Statistical Society, Series B, 61:331–344, 1999.CrossRefGoogle Scholar
  5. 5.
    L. De Raedt and L. Dehaspe. Clausal discovery. Machine Learning, 26:99–146, 1997.CrossRefGoogle Scholar
  6. 6.
    S. Della Pietra, V. Della Pietra, and J. Lafferty. Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19:380–392, 1997.CrossRefGoogle Scholar
  7. 7.
    P. Domingos and D. Lowd. Markov Logic: An Interface Layer for AI. Morgan & Claypool, San Rafael, CA, 2009.Google Scholar
  8. 8.
    P. Domingos and M. Richardson. Mining the network value of customers. In Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 57–66, ACM Press, San Francisco, CA, 2001.Google Scholar
  9. 9.
    M. R. Genesereth and N. J. Nilsson. Logical Foundations of Artificial Intelligence. Morgan Kaufmann, San Mateo, CA, 1987.Google Scholar
  10. 10.
    W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, editors. Markov Chain Monte Carlo in Practice. Chapman and Hall, London, 1996.Google Scholar
  11. 11.
    A. Jaimovich, O. Meshi, and N. Friedman. Template based inference in symmetric relational Markov random fields. In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence, pages 191–199, AUAI Press, Vancouver, BC, 2007.Google Scholar
  12. 12.
    H. Kautz, B. Selman, and Y. Jiang. A general stochastic approach to solving problems with hard and soft constraints. In D. Gu, J. Du, and P. Pardalos, editors, The Satisfiability Problem: Theory and Applications, pages 573–586. American Mathematical Society, New York, NY, 1997.Google Scholar
  13. 13.
    S. Kok and P. Domingos. Learning the structure of Markov logic networks. In Proceedings of the 22nd International Conference on Machine Learning, pages 441–448, ACM Press, Bonn, 2005.Google Scholar
  14. 14.
    S. Kok and P. Domingos. Statistical predicate invention. In Proceedings of the 24th International Conference on Machine Learning, pages 433–440, ACM Press, Corvallis, OR, 2007.Google Scholar
  15. 15.
    S. Kok and P. Domingos. Extracting semantic networks from text via relational clustering. In Proceedings of the 19th European Conference on Machine Learning, pages 624–639, Springer, Antwerp, 2008.Google Scholar
  16. 16.
    S. Kok and P. Domingos. Hypergraph lifting for structure learning in Markov logic networks. In Proceedings of the 26th International Conference on Machine Learning, ACM Press, Montreal, QC, 2009.Google Scholar
  17. 17.
    S. Kok, M. Sumner, M. Richardson, P. Singla, H. Poon, D. Lowd, and P. Domingos. The Alchemy system for statistical relational AI. Technical report, Department of Computer Science and Engineering, University of Washington, Seattle, WA, 2007. http://alchemy.cs.washington.edu.
  18. 18.
    N. Lavrač and S. Džeroski. Inductive Logic Programming: Techniques and Applications. Ellis Horwood, Chichester, 1994.Google Scholar
  19. 19.
    D. C. Liu and J. Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(3):503–528, 1989.CrossRefGoogle Scholar
  20. 20.
    J. W. Lloyd. Foundations of Logic Programming. Springer, Berlin, 1987.CrossRefGoogle Scholar
  21. 21.
    D. Lowd and P. Domingos. Efficient weight learning for Markov logic networks. In Proceedings of the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, pages 200–211, Springer, Warsaw, 2007.Google Scholar
  22. 22.
    D. J. Lunn, A. Thomas, N. Best, and D. Spiegelhalter. WinBUGS — a Bayesian modeling framework: concepts, structure, and extensibility. Statistics and Computing, 10:325–337, 2000.CrossRefGoogle Scholar
  23. 23.
    L. Mihalkova and R. Mooney. Bottom-up learning of Markov logic network structure. In Proceedings of the 24th International Conference on Machine Learning, pages 625–632, ACM Press, Corvallis, OR, 2007.Google Scholar
  24. 24.
    M. Møller. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6:525–533, 1993.CrossRefGoogle Scholar
  25. 25.
    A. Nath and P. Domingos. A language for relational decision theory. In Proceedings of the International Workshop on Statistical Relational Learning, Leuven, 2009.Google Scholar
  26. 26.
    J. Nocedal and S. Wright. Numerical Optimization. Springer, New York, NY, 2006.Google Scholar
  27. 27.
    J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, CA, 1988.Google Scholar
  28. 28.
    H. Poon and P. Domingos. Sound and efficient inference with probabilistic and deterministic dependencies. In Proceedings of the 21st National Conference on Artificial Intelligence, pages 458–463, AAAI Press, Boston, MA, 2006.Google Scholar
  29. 29.
    H. Poon, P. Domingos, and M. Sumner. A general method for reducing the complexity of relational inference and its application to MCMC. In Proceedings of the 23rd National Conference on Artificial Intelligence, pages 1075–1080, AAAI Press, Chicago, IL, 2008.Google Scholar
  30. 30.
    J. R. Quinlan. Learning logical definitions from relations. Machine Learning, 5:239–266, 1990.Google Scholar
  31. 31.
    M. Richardson and P. Domingos. Mining knowledge-sharing sites for viral marketing. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 61–70, ACM Press, Edmonton, AB, 2002.Google Scholar
  32. 32.
    M. Richardson and P. Domingos. Markov logic networks. Machine Learning, 62:107–136, 2006.CrossRefGoogle Scholar
  33. 33.
    S. Riedel. Improving the accuracy and efficiency of MAP inference for Markov logic. In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, pages 468–475, AUAI Press, Helsinki, 2008.Google Scholar
  34. 34.
    J. A. Robinson. A machine-oriented logic based on the resolution principle. Journal of the ACM, 12:23–41, 1965.CrossRefGoogle Scholar
  35. 35.
    D. Roth. On the hardness of approximate reasoning. Artificial Intelligence, 82:273–302, 1996.CrossRefGoogle Scholar
  36. 36.
    P. Singla and P. Domingos. Discriminative training of Markov logic networks. In Proceedings of the 20th National Conference on Artificial Intelligence, pages 868–873, AAAI Press, Pittsburgh, PA, 2005.Google Scholar
  37. 37.
    P. Singla and P. Domingos. Entity resolution with Markov logic. In Proceedings of the 6th IEEE International Conference on Data Mining, pages 572–582, IEEE Computer Society Press, Hong Kong, 2006.Google Scholar
  38. 38.
    P. Singla and P. Domingos. Memory-efficient inference in relational domains. In Proceedings of the 21st National Conference on Artificial Intelligence, pages 488–493, AAAI Press, Boston, MA, 2006.Google Scholar
  39. 39.
    P. Singla and P. Domingos. Markov logic in infinite domains. In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence, pages 368–375, AUAI Press, Vancouver, BC, 2007.Google Scholar
  40. 40.
    P. Singla and P. Domingos. Lifted first-order belief propagation. In Proceedings of the 23rd National Conference on Artificial Intelligence, pages 1094–1099, AAAI Press, Chicago, IL, 2008.Google Scholar
  41. 41.
    A. Srinivasan. The Aleph manual. Technical report, Computing Laboratory, Oxford University, 2000.Google Scholar
  42. 42.
    B. Taskar, P. Abbeel, and D. Koller. Discriminative probabilistic models for relational data. In Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, pages 485–492, Morgan Kaufmann, Edmonton, AB, 2002.Google Scholar
  43. 43.
    S. Wasserman and K. Faust. Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge, 1994.CrossRefGoogle Scholar
  44. 44.
    W. Wei, J. Erenrich, and B. Selman. Towards efficient sampling: Exploiting random walk strategies. In Proceedings of the 19th National Conference on Artificial Intelligence, pages 670–676, AAAI Press, San Jose, CA, 2004.Google Scholar
  45. 45.
    M. Wellman, J. S. Breese, and R. P. Goldman. From knowledge bases to decision models. Knowledge Engineering Review, 7:35–53, 1992.CrossRefGoogle Scholar
  46. 46.
    J. S. Yedidia, W. T. Freeman, and Y. Weiss. Generalized belief propagation. In T. Leen, T. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 689–695. MIT Press, Cambridge, MA, 2001.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Pedro Domingos
    • 1
  • Daniel Lowd
    • 2
  • Stanley Kok
    • 1
  • Aniruddh Nath
    • 1
  • Hoifung Poon
    • 1
  • Matthew Richardson
    • 3
  • Parag Singla
    • 4
  1. 1.Department of Computer Science and EngineeringUniversity of WashingtonSeattleUSA
  2. 2.Department of Computer and Information ScienceUniversity of OregonEugeneUSA
  3. 3.Microsoft ResearchRedmondUSA
  4. 4.Department of Computer ScienceThe University of Texas at AustinAustinUSA

Personalised recommendations