Speakers’ Empowerment with Keywords: The Speaking Competency Development for Multilingual to Contribute in a Real-Time Interaction

  • Hiromi HanawaEmail author
  • Xiaoyu SongEmail author
  • Mengyuan Tang
  • Tomoo Inoue
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10391)


Multilingual communication often causes difficulty when speakers have different language proficiency. Real-time keywords potentially demonstrated positive effects when they are provided in cross-cultural communication. Previously it was found that keywords enhanced mutual understanding and knowledge in conversation, and it was beneficial when it was shown with the latency of 3 s of all matched utterance. However participants’ speech have not ever been investigated to reveal how speakers improved utterance in multilingual discussion with the textual support. In this paper speakers’ interaction were examined from the data set collected through the experiment as well as the use of keywords. It was found that dyad contribution to conversation was facilitated in mean length of utterance with the use of keywords, which was consistent with the data that speakers increased episodic utterance to refashion and expand own prior speech. Consequently speakers extended speech length by keywords, which was supported by speakers’ large number of sequence completion with repair sequence. These findings indicated real-time keywords boosted speakers’ utterance and progressed intercultural discussion via keywords.


Intercultural communication Keyword typing Non-native speaker Text-enhanced audio conference 



This research was partially supported by the JSPS Grants-in-Aid for Scientific Research No. 26330218.


  1. 1.
    Gao, G., Yamashita, N., Hautasaari, A., Echenique, A., Fussell, S.R.: Effects of public vs. private automated transcripts on multiparty communication between native and non-native English speakers. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 843–852, April 2014Google Scholar
  2. 2.
    Gao, G., Wang, H., Cosley, D., Fussell, S.R.: Same translation but different experience: the effects of highlighting on machine-translated conversation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 449–458, May 2013Google Scholar
  3. 3.
    Chapanis, A.: Human factors in teleconferencing systems. Final report, vol. 53, p. 30, John Hopkins University, Department of Psychology (1976)Google Scholar
  4. 4.
    Kuramoto, I., Munemori, J., Yuizono, T., Sudo, M.: Effects of text -based communication on a groupware for a new-idea generation support system. J. Inf. Process. Soc. Jpn. 39(10), 2778–2787 (1998)Google Scholar
  5. 5.
    Echenique, A., Yamashita, N., Kuzuoka, H., Hautasaari, A.: Effects of video and text support on grounding in multilingual multiparty audio conferencing. In: Proceedings of the 5th ACM International Conference on Collaboration Across Boundaries: Culture, Distance and Technology, pp. 73–81, August 2014Google Scholar
  6. 6.
    Yamashita, N., Echenique, A., Ishida, T., Hautasaari, A.: Lost in transmittance: how transmission lag enhances and deteriorates multilingual collaboration. In: Proceedings of the Conference on Computer Supported Cooperative Work, pp. 923–934, February 2013Google Scholar
  7. 7.
    Swain, M.: The output hypothesis and beyond: mediating acquisition through collaborative dialogue. In: Sociocultural Theory and Second Language Learning, pp. 97–114 (2000)Google Scholar
  8. 8.
    Kushalnagar, R.S., Lasecki, W.S., Bigham, J.P.: Accessibility evaluation of classroom captions. J. ACM Trans. Accessible Comput. 5(3) (2014). Article 7Google Scholar
  9. 9.
    Imai, T., Oku, T., Kobayashi, A.: Advances in real-time closed-captioning for live broadcast by speech recognition, IPSJ SIG Technical report, vol. 2011, SLP-88, No. 4, pp. 1–6, October 2011Google Scholar
  10. 10.
    Hanawa, H., Song, X., Inoue, T.: Key-typing on teleconference: collaborative effort on cross-cultural discussion. In: Proceedings of the 8th International Conference on Collaboration Technologies and Social Computing. Communications in Computer and Information Science, vol. 647, pp. 74–88, September 2016Google Scholar
  11. 11.
    Inoue, T., Hanawa, H., Song, X.: With a little help from my native friends: a method to boost non-native’s language use in collaborative work. In: Proceedings of the 9th International Workshop on Informatics, pp. 223–226, September 2015Google Scholar
  12. 12.
    Clark, H.H., Wilkes-Gibbs, D.: Referring as a collaborative process. Cognition 22(1), 1–39 (1986)CrossRefGoogle Scholar
  13. 13.
    Jothilakshmi, S.: Spoken keyword detection using autoassociative neural networks. Int. J. Speech Technol. 17(1), 83–89 (2014)CrossRefGoogle Scholar
  14. 14.
    Japanese Language Proficiency Test.
  15. 15.
    Sirintrapun, S.J., Adela, C.: Dynamic nonrobotic telemicroscopy via skype: a cost effective solution to teleconsultation. J. Pathol. Inform. 3(1), 28 (2012)Google Scholar
  16. 16.
    Rosenfelder, I.: A short introduction to transcribing with elan, Technical report, University of Pennsylvania, January 2011Google Scholar
  17. 17.
    Hautasaari, A., Yamashita, N.: Do automated transcripts help non-native speakers catch up on missed conversation in audio conferences? Proceedings of the 5th ACM International Conference on Collaboration Across Boundaries, pp. 65–72 (2014)Google Scholar
  18. 18.
    Schegloff, E.A.: Sequence Organization in Interaction: A Primer in Conversation Analysis, vol. 1, pp. 115–127. Cambridge University Press, Cambridge (2007)Google Scholar
  19. 19.
    Heritage, J.C., Roth, A.L.: Grammar and institution: questions and questioning in the broadcast news interview. Res. Lang. Soc. Interact. 28(1), 1–60 (1995)CrossRefGoogle Scholar
  20. 20.
    Levitan, R., Hirschberg, J.: Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions. In: Interspeech (2011)Google Scholar
  21. 21.
    Hanawa, H., Song, X., Inoue, T.: Keyword generation by native speaker is quick and useful in conversation between native and non-native speaker. In: IEEE 21st International Conference on Computer Supported Cooperative Work in Design, April 2017Google Scholar
  22. 22.
    Coulthard, M.: An Introduction to Discourse Analysis. Longman, London (1985)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Graduate School of Library, Information and Media StudiesUniversity of TsukubaTsukubaJapan
  2. 2.Faculty of Library, Information and Media ScienceUniversity of TsukubaTsukubaJapan

Personalised recommendations