Abstract
We first review a logical-statistical framework called statistical abduction and identify its three computational tasks, one of which is the learning of parameters from observations by ML (maximum likelihood) estimation. Traditionally, in the presence of missing values, the EM algorithm has been used for ML estimation. We report that the graphical EM algorithm, a new EM algorithm developed for statistical abduction, achieved the same time complexity as specialized EM algorithms developed in each discipline such as the Inside-Outside algorithm for PCFGs (probabilistic context free grammars). Furthermore, learning experiments using two corpora revealed that it can outperform the Inside-Outside algorithm by orders of magnitude. We then specifically look into a family of extensions of PCFGs that incorporate context sensitiveness into PCFGs. Experiments show that they are learnable by the graphical EM algorithm using at most twice as much time as plain PCFGs even though these extensions have higher time complexity.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Baker, J. K., Trainable grammars for speech recognition, Proc. of Spring Conference of the Acoustical Society of America, pp.547–550, 1979.
Castillo, E., Gutierrez, J.M., and Hadi, A.S., Expert Systems and Probabilistic Network Models, Springer-Verlag, 1997.
Charniak, E. and Carroll, G., Context-sensitive statistics for improved grammatical language models, Proc. of AAAI’94, pp.728–733, 1994.
EDR Electronic Dictionary Technical Guide (2nd edition), Japan Electronic Dictionary Research Institute, Ltd., Technical report EDR TR-045, http://www.iijnet.or.jp/edr/E Struct.html, 1995.
Fujisaki, T. and Jelinek, F. and Cocke, J. and Black, E. and Nishino, T., A Probabilistic Parsing Method for Sentence Disambiguation, Proc. of International Parsing Workshop’ 89, pp.85–94,1989.
Fung, T.H. and Kowalski, R., The iff procedure for abductive logic programming, Journal of Logic Programming, 33, pp.151–165, 1997.
Inoue, K. and Sakama, C., Computing extended abduction through transaction programs, Annals of Mathematics and Artificial Intelligence, 25(3,4), pp.339–367, 1999.
Kakas, A.C., Kowalski, R.A. and Toni, F., Abductive Logic Programming, J. Logic Computation, Vol.2 No.6, pp.719–770, 1992.
Kakas, A.C., Kowalski, R.A. and Toni, F., The Role of Abduction in Logic Programming, in Handbook of Logic in Artificial Intelligence and Logic Programming 5, D.M. Gabbay, J.J. Hogger and J.A. Robinson eds. Oxford University Press, pp.235–324, 1998.
Kameya, Y. and Sato, T., Efficient EM learning for parameterized logic programs, Proc. of CL2000, LNAI 1861, pp.269–294, 2000.
Kita, K., Morimoto, T., Ohkura, K., Sagayama, S. and Yano, Y., Spoken sentence recognition based on HMM-LR with hybrid language modeling, IEICE Trans. on Info. & Syst., Vol.E77-D, No.2, 1994.
Lloyd, J. W., Foundations of Logic Programming, Springer-Verlag, 1984.
Manning, C. D. and Schütze, H., Foundations of Statistical Natural Language Processing, The MIT Press, 1999.
McLachlan, G. J. and Krishnan, T., The EM Algorithm and Extensions, Wiley Interscience, 1997.
Rabiner, L. R. and Juang, B., Foundations of Speech Recognition, Prentice-Hall, 1993.
Sato, T., A statistical learning method for logic programs with distribution semantics, Proc. of ICLP’95, pp.715–729, 1995.
Sato, T. and Kameya, Y., PRISM:A Language for Symbolic-Statistical Modeling, Proc. of IJCAI’97, pp.1330–1335, 1997.
Sato, T. and Kameya, Y., “A Viterbi-like algorithm and EM learning for statistical abduction”, Proc. of UAI2000 Workshop on Fusion of Domain Knowledge with Data for Decision Support, 2000.
Sato, T. and Kameya, Y., Parameter Learning of Logic Programs for Symbolicstatistical Modeling, submitted for publication, 2000.
Sato, T., Parameterized Logic Programs where Computing Meets Learning Proc. of FLOPS 2001, LNCS 2024, pp.40–60, 2001.
Sato, T., Kameya, Y., Abe, S. and Shirai, K., Fast EM learning of a Family of PCFGs, Titech Technical Report (Dept. of CM) TR01-0006, Tokyo Institute of Technology, 2001.
Stolcke, A., An efficient probabilistic context-free parsing algorithm that computes prefix probabilities, Computational Linguistics, Vol.21 No.2, pp.165–201, 1995.
Tamaki, H. and Sato, T., OLD resolution with tabulation, Proc. of ICLP’86, London, LNCS 225, pp.84–98, 1986.
Tanaka, H. and Takezawa, T. and Etoh, J., Japanese grammar for speech recognition considering the MSLR method (in Japanese), Proc. of the meeting of SIGSLP (Spoken Language Processing), 97-SLP-15-25, Information Processing Society of Japan, pp.145–150, 1997.
Uratani, N. and Takezawa, T. and Matsuo, H. and Morita, C., ATR Integrated Speech and Language Database (in Japanese), TR-IT-0056, ATR Interpreting Telecommunications Research Laboratories, 1994.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Sato, T. (2002). EM Learning for Symbolic-Statistical Models in Statistical Abduction. In: Arikawa, S., Shinohara, A. (eds) Progress in Discovery Science. Lecture Notes in Computer Science(), vol 2281. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45884-0_11
Download citation
DOI: https://doi.org/10.1007/3-540-45884-0_11
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43338-5
Online ISBN: 978-3-540-45884-5
eBook Packages: Springer Book Archive