Hierarchical Hidden Conditional Random Fields for Information Extraction

  • Satoshi Kaneko
  • Akira Hayashi
  • Nobuo Suematsu
  • Kazunori Iwata
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6683)


Hidden Markov Models (HMMs) are very popular generative models for time series data. Recent work, however, has shown that for many tasks Conditional Random Fields (CRFs), a type of discriminative model, perform better than HMMs. Information extraction is the task of automatically extracting instances of specified classes or relations from text. A method for information extraction using Hierarchical Hidden Markov Models (HHMMs) has already been proposed. HHMMs, a generalization of HMMs, are generative models with a hierarchical state structure. In previous research, we developed the Hierarchical Hidden Conditional Random Field (HHCRF), a discriminative model corresponding to HHMMs. In this paper, we propose information extraction using HHCRFs, and then compare the performance of HHMMs and HHCRFs through an experiment.


Hide Markov Model Information Extraction Viterbi Algorithm Discriminative Model Test Sentence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. 18th Int. Conf. Machine Learning (2001)Google Scholar
  2. 2.
    Fine, S., Singer, Y., Tishby, N.: The hierarchical hidden Markov model: Analysis and applications. Machine Learning 32(1) (1998)Google Scholar
  3. 3.
    Murphy, K., Paskin, M.: Linear time inference in hierarchical HMMs. In: Advances in Neural Information Processing Systems, vol. 14 (2001)Google Scholar
  4. 4.
    Sugiura, T., Goto, N., Hayashi, A.: A discriminative model corresponding to hierarchical hMMs. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 375–384. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Tamada, H., Hayashi, A.: Sports video segmentation using a hierarchical hidden CRF. In: Köppen, M., Kasabov, N., Coghill, G. (eds.) ICONIP 2008. LNCS, vol. 5506, pp. 715–722. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Skounakis, M., Craven, M., Ray, S.: Hierarchical hidden Markov models for information extraction. In: Proc. 18th Int. Joint Conf. Artificial Intelligence (2003)Google Scholar
  7. 7.
    Huang, C., Darwiche, A.: Inference in belief networks: A procedural guide. Int. J. of Approximate Reasoning 15(3) (1996)Google Scholar
  8. 8.
    Scheffer, T., Decomain, C., Wrobel, S.: Active hidden markov models for information extraction. In: Hoffmann, F., Adams, N., Fisher, D., Guimarães, G., Hand, D.J. (eds.) IDA 2001. LNCS, vol. 2189, p. 309. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  9. 9.
    Cestnik, B.: Estimating probabilities. In: Proc. 9th European Conf. Artificial Intelligence (1990)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Satoshi Kaneko
    • 1
  • Akira Hayashi
    • 1
  • Nobuo Suematsu
    • 1
  • Kazunori Iwata
    • 1
  1. 1.Graduate School of Information SciencesHiroshima City UniversityAsaminami-kuJapan

Personalised recommendations