Advertisement

Detecting Sentence Boundaries in Japanese Speech Transcriptions Using a Morphological Analyzer

  • Sachie Tajima
  • Hidetsugu Nanba
  • Manabu Okumura
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3248)

Abstract

We present a method to automatically detect sentenceboundaries(SBs) in Japanese speech transcriptions. Our method uses a Japanese morphological analyzer that is based on a cost calculation and selects as the best result the one with the minimum cost. The idea behind using a morphological analyzer to identify candidates for SBs is that the analyzer outputs lower costs for better sequences of morphemes. After the candidate SBs have been identified, the unsuitable candidates are deleted by using lexical information acquired from the training corpus. Our method had a 77.24% precision, 88.00% recall, and 0.8277 F-Measure, for a corpus consisting of lecture speech transcriptions in which the SBs are not given.

Keywords

Automatic Speech Recognition Training Corpus Input String Connection Cost Deletion Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Matsumoto, Y., Kitauchi, A., Tatsuo Yamashita, Y.H., Matsuda, H., Takaoka, K., Asahara, M.: Morphological Analysis System ChaSen version 2.2.9 Manual (2002)Google Scholar
  2. 2.
    The National Institute for Japanese Language: The Corpus of Spontaneous Japanese (The monitor version 2001) Guidance of monitor public presentation (2001), http://www.kokken.go.jp/public/monitor_kokai001.html
  3. 3.
    The National Institute for Japanese Language: The Corpus of Spontaneous Japanese (The monitor version 2002) Guidance of monitor public presentation (2002), http://www.kokken.go.jp/public/monitor_kokai002.html
  4. 4.
    Stevenson, M., Gaizauskas, R.: Experiments on Sentence Boundary Detection. In: Proc. of ANLP-NAACL 2000, pp. 84–89 (2000)Google Scholar
  5. 5.
    Shitaoka, K., Kawahara, T., Okuno, H.G.: Automatic Transformation of Lecture Transcription into Document Style using Statistical Framework. In: IPSJ SIG Notes on Spoken Language Processing, vol. 41-3 (2002) (in Japanese)Google Scholar
  6. 6.
    Nobata, C., Sekine, S., Uchimoto, K., Isahara, H.: Sentence Segmentation and Sentence Extraction. In: Proc. of the Second Spontaneous Speech Science and Technology Workshop, pp. 527–534 (2002) (in Japanese)Google Scholar
  7. 7.
    Seligman, M., Hosaka, J., Singer, H.: “Pause Units” and Analysis of Spontaneous Japanese Dialogues: Preliminary Studies. In: ECAI 1996 workshop on “Dialogue Processing in Spoken Language Systems”, pp. 100–112 (1996)Google Scholar
  8. 8.
    Palmer, D.D., Hearst, M.A.: Adaptive sentence boundary disambiguation. In: Proc. of the fourth Conference on Applied Natural Language Processing, pp. 78–83 (1994)Google Scholar
  9. 9.
    Reynar, J.C., Ratnaparkhi, A.: A maximum entropy approach to identifying sentence boundaries. In: Proc. of the fifth Conference on Applied Natural Language Processing, pp. 16–19 (1997)Google Scholar
  10. 10.
    Beeferman, D., Berger, A., Lafferty, J.: CYBERPUNC: A lightweight punctu IEEE International Conference on Acoustics, Speech and Signaation annotation system for speech. In: Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 689–692 (1998)Google Scholar
  11. 11.
    Shieber, S.M., Tao, X.: Comma restoration using constituency information. In: Proc. of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pp. 221–227 (2003)Google Scholar
  12. 12.
    Tooyama, Y., Nagata, M.: Insertion methods of punctuation marks for speech recognition systems. In Technical Report of IEICE, NLC2000-5 (2000) (in Japanese)Google Scholar
  13. 13.
    Asahara, M., Matsumoto, Y.: Extended Hidden Markov Model for Japanese Morphological Analyzer. In: IPSJ SIG Notes on Spoken Language Processing, vol. 031 (2000) (in Japanese)Google Scholar
  14. 14.
    Asahara, M., Matsumoto, Y.: IPADIC user’s manual version 2.5 (2002)Google Scholar
  15. 15.
    Liu, D., Zong, C.: Utterance Segmentation Using Combined Approach Based on Bi-directional N-gram and Maximum Entropy. In: Proc. of ACL 2003 Workshop: The Second SIGHAN Workshop on ChineeseLanguage Processing, pp. 16–23 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Sachie Tajima
    • 1
  • Hidetsugu Nanba
    • 2
  • Manabu Okumura
    • 3
  1. 1.Interdisciplinary Graduate School of Science and EngineeringTokyo Institute of TechnologyYokohamaJapan
  2. 2.Graduate School of Information Sciences Hiroshima City UniversityHiroshimaJapan
  3. 3.Precision and Intelligence LaboratoryTokyo Institute of TechnologyYokohamaJapan

Personalised recommendations