Abstract
We have been doing research on machine lip-reading. In the process of the study, we proposed a reproduction method of utterance images without voice from Japanese kana. Firstly, sequence of codes that is called “Mouth Shapes Sequence Code” is generated. The Mouth Shapes Sequence Code expresses the order of mouth shapes when a Japanese word is uttered. The utterance images are generated by using the mouth images corresponding to the Mouth Shapes Sequence Code and the deformed mouth shape images generated with morphing. However, the deformation rate of the mouth shapes has been decided from the real utterance images experimentally. Therefore, there were cases in which the utterance images with the sense of incongruity about mouth shape deformation were generated. In this paper, the deformation rate of the mouth shapes is analyzed using real utterance images captured by a high-speed camera, and we propose a generation method of utterance images based on the results. Finally, the mean opinion score of the subjects are shown, and we evaluate the effectiveness of proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Asa, H., Bertil, L.: Visual speech synthesis with concatenative speech. In: Auditory-Visual Speech Processing, pp. 181–184 (1998)
Kiyotsugu, K., Satoshi, N., Kiyohiro, S.: Facial movement synthesis by HMM from audio speech. Trans. Inst. Electr. Inf. Commun. Eng. J83-D-I I(11), 2498–2506 (2000) (in Japanese)
Shinichi, K., Hiroshi, S., Tsuneo, N., Takuya, N., Satoshi, N., Katsunobu, I., Shigeo, M., Tatsuo, Y., Atsuhiko, K., Akinobu, L., Yoichi, Y., Takao, K., Keiichi, T., Keikichi, H., Nobuaki, M., Atsushi, Y., Yasuharu, D., Takehito, U., Shigeki, S.: Design of software toolkit for anthropomorphic spoken dialog agent software with customization-oriented features. Inf. Process. Soc. Jpn. (IPSJ) J 43(7), 2249–2263 (2002) (in Japanese)
Tsuyoshi, M., Toyoshiro, N.: The codification of distinctive mouth shapes and the expression method of data concerning changes in mouth shape when uttering Japanese. IEEJ Trans. Electr. Inf. Syst. 129(12), 2108–2114 (2009) (in Japanese)
Tsuyoshi, M., Toyoshiro, N.: Development of lip-reading training application for smartphones. Multimedia Distrib. Coop Mob. Symp. 2012, 1863–1868 (2012) (in Japanese)
Tsuyoshi, M., Toyoshiro, N., Naohiro, I.: Evaluation for an automatic generation of lips movement images based on mouth shapes sequence code in Japanese pronunciation. In: Proceedings of Japan-Cambodia Joint Symposium on Information Systems and Communication Technology, 2011 (JCAICT 2011), pp. 89–92 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Miyazaki, T., Nakashima, T. (2015). Analysis of Mouth Shape Deformation Rate for Generation of Japanese Utterance Images Automatically. In: Lee, R. (eds) Software Engineering Research, Management and Applications. Studies in Computational Intelligence, vol 578. Springer, Cham. https://doi.org/10.1007/978-3-319-11265-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-11265-7_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11264-0
Online ISBN: 978-3-319-11265-7
eBook Packages: EngineeringEngineering (R0)