Dyna: Extending Datalog for Modern AI

  • Jason Eisner
  • Nathaniel W. Filardo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6702)

Abstract

Modern statistical AI systems are quite large and complex; this interferes with research, development, and education. We point out that most of the computation involves database-like queries and updates on complex views of the data. Specifically, recursive queries look up and aggregate relevant or potentially relevant values. If the results of these queries are memoized for reuse, the memos may need to be updated through change propagation. We propose a declarative language, which generalizes Datalog, to support this work in a generic way. Through examples, we show that a broad spectrum of AI algorithms can be concisely captured by writing down systems of equations in our notation. Many strategies could be used to actually solve those systems. Our examples motivate certain extensions to Datalog, which are connected to functional and object-oriented programming paradigms.

Keywords

Logic Program Logic Programming Parse Tree Full Version Arithmetic Circuit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Acar, U.A., Ley-Wild, R.: Self-adjusting computation with Delta ML. In: Koopman, P.W.M., Plasmeijer, R., Swierstra, S.D. (eds.) AFP 2008. LNCS, vol. 5832, pp. 1–38. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Ahmad, Y., Koch, C.: DBToaster: A SQL compiler for high-performance delta processing in main-memory databases. In: Proc. of VLDB, pp. 1566–1569 (2009)Google Scholar
  3. 3.
    Allauzen, C., Riley, M., Schalkwyk, J., Skut, W., Mohri, M.: OpenFST: A general and efficient weighted finite-state transducer library. In: Holub, J., Žďárek, J. (eds.) CIAA 2007. LNCS, vol. 4783, pp. 11–23. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Apt, K.R., Blair, H.A., Walker, A.: Towards a theory of declarative knowledge. In: Minker, J. (ed.) Foundations of Deductive Databases and Logic Programming, ch. 2. Morgan Kaufmann, San Francisco (1988)Google Scholar
  5. 5.
    Berg-Kirkpatrick, T., Bouchard-Côté, A., DeNero, J., Klein, D.: Painless unsupervised learning with features. In: Proc. of NAACL, pp. 582–590. ACL (2010)Google Scholar
  6. 6.
    Bidoit, N., Hull, R.: Minimalism, justification and non-monotonicity in deductive databases. Journal of Computer and System Sciences 38(2), 290–325 (1989)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Breck, E.: zymake: A computational workflow system for machine learning and natural language processing. In: Software Engineering, Testing, and Quality Assurance for Natural Language Processing, SETQA-NLP 2008, pp. 5–13. ACL (2008)Google Scholar
  8. 8.
    Brodie, M.L.: Future Intelligent Information Systems: AI and Database Technologies Working Together. Morgan Kaufmann, San Francisco (1988)Google Scholar
  9. 9.
    Burstall, R.M., Collins, J.S., Popplestone, R.J.: Programming in POP-2. Edinburgh University Press, Edinburgh (1971)MATHGoogle Scholar
  10. 10.
    Ceri, S., Gottlob, G., Tanca, L.: What you always wanted to know about datalog (and never dared to ask). IEEE Transactions on Knowledge and Data Engineering 1, 146–166 (1989)CrossRefGoogle Scholar
  11. 11.
    Ceri, S., Gottlob, G., Tanca, L.: Logic Programming and Databases. Springer, Heidelberg (1990)CrossRefGoogle Scholar
  12. 12.
    Clark, K.L.: Negation as failure. In: Gallaire, H., Minker, J. (eds.) Logic and Data Bases, pp. 293–322. Plenum, New York (1978)CrossRefGoogle Scholar
  13. 13.
    Cohen, S.B., Simmons, R.J., Smith, N.A.: Products of weighted logic programs. Theory and Practice of Logic Programming (2010)Google Scholar
  14. 14.
    Cohen, S., Nutt, W., Serebrenik, A.: Algorithms for rewriting aggregate queries using views. In: Masunaga, Y., Thalheim, B., Štuller, J., Pokorný, J. (eds.) ADBIS 2000 and DASFAA 2000. LNCS, vol. 1884, pp. 65–78. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  15. 15.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995)MATHGoogle Scholar
  16. 16.
    Courtney, A., Elliott, C.: Genuinely functional user interfaces. In: 2001 Haskell Workshop (2001)Google Scholar
  17. 17.
    The functional logic language Curry, http://www.informatik.uni-kiel.de/~curry/
  18. 18.
    Davis, M., Logemann, G., Loveland, D.: A machine program for theorem-proving. Communications of the ACM 5(7), 394–397 (1962)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Dechter, R.: Constraint Processing. Morgan Kaufmann, San Francisco (2003)MATHGoogle Scholar
  20. 20.
    Eisner, J.: Parameter estimation for probabilistic finite-state transducers. In: Proc. of ACL, pp. 1–8 (2002)Google Scholar
  21. 21.
    Eisner, J., Blatz, J.: Program transformations for optimization of parsing algorithms and other weighted logic programs. In: Wintner, S. (ed.) Proc. of FG 2006: The 11th Conference on Formal Grammar, pp. 45–85. CSLI Publications, Stanford (2007)Google Scholar
  22. 22.
    Eisner, J., Filardo, N.W.: Dyna: Extending Datalog for modern AI (full version). Tech. rep., Johns Hopkins University (2011); Extended version of the present paper, http://dyna.org/Publications
  23. 23.
    Eisner, J., Goldlust, E., Smith, N.A.: Compiling comp ling: Weighted dynamic programming and the Dyna language. In: Proc. of HLT-EMNLP, pp. 281–290. Association for Computational Linguistics (2005)Google Scholar
  24. 24.
    Eisner, J., Kornbluh, M., Woodhull, G., Buse, R., Huang, S., Michael, C., Shafer, G.: Visual navigation through large directed graphs and hypergraphs. In: Proc. of IEEE InfoVis, Poster/Demo Session, pp. 116–117 (2006)Google Scholar
  25. 25.
    Elliott, C., Hudak, P.: Functional reactive animation. In: International Conference on Functional Programming (1997)Google Scholar
  26. 26.
    Felzenszwalb, P.F., McAllester, D.: The generalized A* architecture. J. Artif. Int. Res. 29(1), 153–190 (2007)MathSciNetMATHGoogle Scholar
  27. 27.
    Fidler, S., Boben, M., Leonardis, A.: Learning hierarchical compositional representations of object structure. In: Dickinson, S., Leonardis, A., Schiele, B., Tarr, M.J. (eds.) Object Categorization: Computer and Human Vision Perspectives, Cambridge University Press, Cambridge (2009)Google Scholar
  28. 28.
    Finkel, J.R., Grenager, T., Manning, C.: Incorporating non-local information into information extraction systems by Gibbs sampling. In: Proc. of ACL, pp. 363–370. ACL (2005)Google Scholar
  29. 29.
    Gelfond, M., Lifschitz, V.: The stable model semantics for logic programming. In: Proc. of the 5th International Conference and Symposium Logic Programming, pp. 1070–1080 (1988)Google Scholar
  30. 30.
    Goodman, J.: Semiring parsing. Computational Linguistics 25(4), 573–605 (1999)MathSciNetGoogle Scholar
  31. 31.
    Green, T.J., Karvounarakis, G., Tannen, V.: Provenance semirings. In: Proc. of PODS, pp. 31–40 (2007)Google Scholar
  32. 32.
    Griewank, A., Corliss, G. (eds.): Automatic Differentiation of Algorithms. SIAM, Philadelphia (1991)Google Scholar
  33. 33.
    Guo, H.-F., Gupta, G.: Simplifying dynamic programming via tabling. In: Jayaraman, B. (ed.) PADL 2004. LNCS, vol. 3057, pp. 163–177. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  34. 34.
    Gupta, A., Mumick, I.S.: Maintenance of materialized views: Problems, techniques, and applications. IEEE Data Eng. Bull. 18(2), 3–18 (1995)Google Scholar
  35. 35.
    Hinton, G.: Products of experts. In: Proc. of ICANN, vol. 1, pp. 1–6 (1999)Google Scholar
  36. 36.
    Johnson, M.: Transforming projective bilexical dependency grammars into efficiently-parsable CFGs with unfold-fold. In: Proc. of ACL, pp. 168–175 (2007)Google Scholar
  37. 37.
    Kemp, D.B., Stuckey, P.J.: Semantics of logic programs with aggregates. In: Proc. of the International Logic Programming Symposium, pp. 338–401 (1991)Google Scholar
  38. 38.
    Kifer, M., Subrahmanian, V.S.: Theory of generalized annotated logic programming and its applications. Journal of Logic Programming 12(4), 335–368 (1992)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Klein, D., Manning, C.D.: A* parsing: Fast exact Viterbi parse selection. In: Proc. of HLT-NAACL (2003)Google Scholar
  40. 40.
    Kline, M.: Mathematics in the modern world; readings from Scientific American. With introductions by Morris Kline. W.H. Freeman, San Francisco (1968)Google Scholar
  41. 41.
    LogicBlox: Datalog for enterprise applications: from industrial applications to research (2010), http://www.logicblox.com/research/presentations/arefdatalog20.pdf, presented by Molham Aref at Datalog 2.0 Workshop
  42. 42.
    LogicBlox: Modular and reusable Datalog (2010), http://www.logicblox.com/research/presentations/morebloxdatalog20.pdf, presented by Shan Shan Huang at Datalog 2.0 Workshop
  43. 43.
    Loo, B.T., Condie, T., Garofalakis, M.N., Gay, D.E., Hellerstein, J.M., Maniatis, P., Ramakrishnan, R., Roscoe, T., Stoica, I.: Declarative networking. Commun. ACM 52(11), 87–95 (2009)CrossRefGoogle Scholar
  44. 44.
    Marek, V., Truszczyński, M.: Stable models and an alternative logic programming paradigm. In: Apt, K., Marek, V., Truszczyński, M., Warren, D. (eds.) The Logic Programming Paradigm: A 25-Year Perspective, pp. 375–398. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  45. 45.
    McAllester, D.A.: On the complexity analysis of static analyses. J. ACM 49(4), 512–537 (2002)MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
  47. 47.
    Minnen, G.: Magic for filter optimization in dynamic bottom-up processing. In: ACL, pp. 247–254 (1996)Google Scholar
  48. 48.
    Mohr, R., Henderson, T.: Arc and path consistency revised. Artificial Intelligence 28, 225–233 (1986)CrossRefGoogle Scholar
  49. 49.
    Mumick, I.S., Pirahesh, H., Ramakrishnan, R.: The magic of duplicates and aggregates. In: Proc. of VLDB, pp. 264–277 (1990)Google Scholar
  50. 50.
    Nádas, A.: On Turing’s formula for word probabilities. IEEE Transactions on Acoustics, Speech, and Signal Processing ASSP-33(6), 1414–1416 (1985)CrossRefMATHGoogle Scholar
  51. 51.
    Ngai, G., Florian, R.: Transformation-based learning in the fast lane. In: Proc. of NAACL-HLT (2001)Google Scholar
  52. 52.
    van Noord, G., Gerdemann, D.: Finite state transducers with predicates and identities. Grammars 4(3) (2001)Google Scholar
  53. 53.
    Overton, D.: Precise and Expressive Mode Systems for Typed Logic Programming Languages. Ph.D. thesis, University of Melbourne (2003)Google Scholar
  54. 54.
    Pelov, N.: Semantics of Logic Programs With Aggregates. Ph.D. thesis, Katholieke Universiteit Leuven (2004)Google Scholar
  55. 55.
    Ramakrishnan, R., Srivastava, D., Sudarshan, S., Seshadri, P.: The coral deductive system. The VLDB Journal 3(2), 161–210 (1994); Special Issue on Prototypes of Deductive Database SystemsCrossRefGoogle Scholar
  56. 56.
    Ramamohanarao, K.: Special issue on prototypes of deductive database systems. VLDB 3(2) (1994)Google Scholar
  57. 57.
    Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62(1-2), 107–136 (2006)CrossRefGoogle Scholar
  58. 58.
    Ross, K.A., Sagiv, Y.: Monotonic aggregation in deductive databases. In: Proc. of PODS, pp. 114–126 (1992)Google Scholar
  59. 59.
    Schmid, H., Rooth, M.: Parse forest computation of expected governors. In: Proc. of ACL (2001)Google Scholar
  60. 60.
    Shieber, S.M., Schabes, Y., Pereira, F.: Principles and implementation of deductive parsing. Journal of Logic Programming 24(1-2), 3–36 (1995)MathSciNetCrossRefMATHGoogle Scholar
  61. 61.
    Singla, P., Domingos, P.: Lifted first-order belief propagation. In: Proc. of AAAI, pp. 1094–1099. AAAI Press, Menlo Park (2008)Google Scholar
  62. 62.
    Van Emden, M.H., Kowalski, R.A.: The semantics of predicate logic as a programming language. JACM 23(4), 733–742 (1976)MathSciNetCrossRefMATHGoogle Scholar
  63. 63.
    Van Gelder, A., Ross, K.A., Schlipf, J.S.: The well-founded semantics for general logic programs. Journal of the ACM 38(3), 620–650 (1991)MathSciNetMATHGoogle Scholar
  64. 64.
    Williams, R., Zipser, D.: A learning algorithm for continually running fully recurrent neural networks. Neural Computation 1(2), 270–280 (1989)CrossRefGoogle Scholar
  65. 65.
  66. 66.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Understanding belief propagation and its generalizations. In: Exploring Artificial Intelligence in the New Millennium, ch. 8. Science & Technology Books (2003)Google Scholar
  67. 67.
    Younger, D.H.: Recognition and parsing of context-free languages in time n 3. Information and Control 10(2), 189–208 (1967)CrossRefMATHGoogle Scholar
  68. 68.
    Zhang, M., Zhang, H., Li, H.: Convolution kernel over packed parse forest. In: Proc. of ACL, pp. 875–885 (2010)Google Scholar
  69. 69.
    Zhu, S.C., Mumford, D.: A stochastic grammar of images. Foundations and Trends in Computer Graphics and Vision 2(4), 259–362 (2006)CrossRefMATHGoogle Scholar
  70. 70.
    Zukowski, U., Freitag, B.: The deductive database system LOLA. In: Fuhrbach, U., Dix, J., Nerode, A. (eds.) LPNMR 1997. LNCS (LNAI), vol. 1265, pp. 375–386. Springer, Heidelberg (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jason Eisner
    • 1
  • Nathaniel W. Filardo
    • 1
  1. 1.Computer Science DepartmentJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations