Integration of Multiple Sound Source Localization Results for Speaker Identification in Multiparty Dialogue System

  • Taichi Nakashima
  • Kazunori Komatani
  • Satoshi Sato
Conference paper


Humanoid robots need to head toward human participants when answering to their questions in multiparty dialogues. Some positions of participants are difficult to localize from robots in multiparty situations, especially when the robots can only use their own sensors. We present a method for identifying the speaker more accurately by integrating the multiple sound source localization results obtained from two robots: one talking mainly with participants and the other also joining the conversation when necessary. We place them so that they can compensate for each other’s localization capabilities and then integrate their two results. Our experimental evaluation revealed that using two robots improved speaker identification compared with using only one robot. We furthermore implemented our method into humanoid robots and constructed a demo system.


Sound Source Humanoid Robot Localization Result Confidence Measure Speaker Identification 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bennewitz, M., Faber, F., Joho, D., Schreiber, M., Behnke, S.: Integrating vision and speech for conversations with multiple persons. In: Proceedings of IEEE/RSJ the International Conference on Intelligent Robots and Systems (IROS), pp. 2523–2528 (2005). doi: 10.1109/IROS.2005.1545158Google Scholar
  2. 2.
    Bohus, D., Horvitz, E.: Models for multiparty engagement in open-world dialog. In: Proceedings of the SIGDIAL 2009 Conference, pp. 225–234 (2009)Google Scholar
  3. 3.
    Gruenstein, A., Seneff, S.: Releasing a multimodal dialogue system into the wild: User support mechanisms. In: Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, pp. 111–119 (2007)Google Scholar
  4. 4.
    Haider, F., Moubayed, S.A.: Towards speaker detection using lips movements for human-machine multiparty dialogue. In: FONETIK 2012 (2012)Google Scholar
  5. 5.
    Jovanovic, N., op den Akker, R., Nijholt, A.: Addressee identification in face-to-face meetings. In: Proceedings of the 11th Conference of the EACL (2006)Google Scholar
  6. 6.
    Matsuyama, Y., Taniyama, H., Fujie, S., Kobayashi, T.: Framework of communication activation robot participating in multiparty conversation. In: Proceedings of AAAI Fall Symposium, Dialog with Robots, pp. 68–73 (2010)Google Scholar
  7. 7.
    Moubayed, S.A., Beskow, J., Blomberg, M., Granström, B., Gustafson, J., Mirnig, N., Skantze, G.: Talking with furhat - multi-party interaction with a back-projected robot head. In: FONETIK 2012 (2012)Google Scholar
  8. 8.
    Mutlu, B., Shiwa, T., Kanda, T., Ishiguro, H., Hagita, N.: Footing in human-robot conversations: how robots might shape participant roles using gaze cues. In: Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, pp. 61–68 (2009)Google Scholar
  9. 9.
    Nakadai, K., Takahashi, T., Okuno, H.G., Nakajima, H., Hasegawa, Y., Tsujino, H.: Design and implementation of robot audition system ‘HARK’ - open source software for listening to three simultaneous speakers. Adv. Robotics 5, 739–761 (2010)CrossRefGoogle Scholar
  10. 10.
    Schmidt, R.O.: Multiple emitter location and signal parameter estimation. IEEE Trans. Antenn. Propagat. 34, 276–280 (1986). doi: 10.1109/TAP.1986.1143830CrossRefGoogle Scholar
  11. 11.
    Stiefelhagen, R., Yang, J., Waibel, A.: Modeling focus of attention for meeting indexing based on multiple cues. IEEE Trans. Neural Networ. 13, 928–938 (2002)CrossRefGoogle Scholar
  12. 12.
    Traum, D.: Issues in multi-party dialogues. In: Advances in Agent Communication. Lecture Notes in Artificial Intelligence, vol. 2922, pp. 201–211. Springer, Berlin (2004)Google Scholar
  13. 13.
    Traum, D., Rickel, J.: Embodied agents for multi-party dialogue in immersive virtual worlds. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 766–773 (2002)Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Taichi Nakashima
    • 1
  • Kazunori Komatani
    • 1
  • Satoshi Sato
    • 1
  1. 1.Graduate School of Engineering, Nagoya UniversityNagoyaJapan

Personalised recommendations