Advertisement

Extraction of Displayed Objects Corresponding to Demonstrative Words for Use in Remote Transcription

  • Yoshinori Takeuchi
  • Hajime Ohta
  • Noboru Ohnishi
  • Daisuke Wakatsuki
  • Hiroki Minagawa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6180)

Abstract

A previously proposed system for extracting target objects displayed during lectures by using demonstrative words and phrases and pointing gestures has now been evaluated. The system identifies pointing gestures by analyzing the trajectory of the stick pointer and extracts the objects to which the speaker points. The extracted objects are displayed on the transcriber’s monitor at a remote location, thereby helping the transcriber to translate the demonstrative word or phrase into a short description of the object. Testing using video of an actual lecture showed that the system had a recall rate of 85.7% and precision of 84.8%. Testing using two extracted scenes showed that transcribers replaced significantly more demonstrative words with short descriptions of the target objects when the extracted objects were displayed on the transcriber’s screen. A transcriber using this system can thus transcribe speech more easily and produce more meaningful transcriptions for hearing-impaired listeners.

Keywords

Speech Recognition Target Object Short Description Recall Rate Automatic Speech Recognition System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kato, N., Kawano, S., Miyoshi, S., Nishioka, T., Murakami, H., Minagawa, H., Wakatsuki, D., Shirasawa, M., Ishihara, Y., Naito, I.: Subjective Evaluation of Displaying Keywords for Speech to Text Service Operators. The Transactions of Human Interface Society 9(2), 195–203 (2007) (in Japanese)Google Scholar
  2. 2.
    Miyoshi, S., Kawano, S., Nishioka, T., Kato, N., Shirasawa, M., Murakami, H., Minagawa, H., Ishihara, Y., Naito, I., Wakatsuki, D., Kuroki, H., Kobayashi, M.: A Basic Study on Supplementary Visual Information for Real-Time Captionists in the Lecture of Information Science. IEICE Transactions on Information and Systems (Japanese edition) J91-D(9), 2236–2246 (2008)Google Scholar
  3. 3.
    Miyoshi, S., Kuroki, H., Kawano, S., Shirasawa, M., Ishihara, Y., Kobayashi, M.: Support Technique for Real-Time Captionist to Use Speech Recognition Software. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds.) ICCHP 2008. LNCS, vol. 5105, pp. 647–650. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  4. 4.
    Wald, M., Bain, K.: Universal access to communication and learning: role of automatic speech recognition. Universal Access in the Information Society 6(4), 435–447 (2007)CrossRefGoogle Scholar
  5. 5.
    Sakiyama, T., Mukunoki, M., Katsuo, I.: Detection of the Indicated Area with an Indication Stick. In: Int. Conf. on Multimodal Interfaces, pp. 480–487 (2000)Google Scholar
  6. 6.
    Marutani, T., Nishiguchi, S., Kakusho, K., Minoh, M.: Making a lecture content with deictic information about indicated objects in lecture materials. In: AEARU Workshop on Network Education, pp. 70–75 (2005)Google Scholar
  7. 7.
    Takeuchi, Y., Saito, K., Ito, A., Ohnishi, N., Iizuka, S., Nakajima, S.: Extracting Pointing Object with Demonstrative Speech Phrase for Remote Transcription in Lecture. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds.) ICCHP 2008. LNCS, vol. 5105, pp. 624–631. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Ito, A., Saito, K., Takeuchi, Y., Ohnishi, N., Iizuka, S., Nakajima, S.: A Study on Demonstrative Words Extraction in Instructor Utterance on Communication Support for Hearing Impaired Persons. ibid, pp. 632–639 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Yoshinori Takeuchi
    • 1
  • Hajime Ohta
    • 2
  • Noboru Ohnishi
    • 2
  • Daisuke Wakatsuki
    • 3
  • Hiroki Minagawa
    • 3
  1. 1.Information and Communications HeadquartersNagoya UniversityNagoyaJapan
  2. 2.Graduate School of Information ScienceNagoya UniversityNagoyaJapan
  3. 3.Tsukuba University of TechnologyJapan

Personalised recommendations