Skip to main content
Book cover

International Conference on Practical Applications of Agents and Multi-Agent Systems

PAAMS 2020: Highlights in Practical Applications of Agents, Multi-Agent Systems, and Trust-worthiness. The PAAMS Collection pp 238–249Cite as

Learning to Communicate Proactively in Human-Agent Teaming

Part of the Communications in Computer and Information Science book series (CCIS,volume 1233)

Abstract

Artificially intelligent agents increasingly collaborate with humans in human-agent teams. Timely proactive sharing of relevant information within the team contributes to the overall team performance. This paper presents a machine learning approach to proactive communication in AI-agents using contextual factors. Proactive communication was learned in two consecutive experimental steps: (a) multi-agent team simulations to learn effective communicative behaviors, and (b) human-agent team experiments to refine communication suitable for a human team member. Results consist of proactive communication policies for communicating both beliefs and goals within human-agent teams. Agents learned to use minimal communication to improve team performance in simulation, while they learned more specific socially desirable behaviors in the human-agent team experiment.

Keywords

  • Human-agent teaming
  • Reinforcement Learning
  • BDI-agent
  • Human-agent communication
  • Proactive
  • Context-sensitive

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-51999-5_20
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-51999-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.

References

  1. Bernsen, N.O., Dybkjær, L.: Building usable spoken dialogue systems: some approaches. Sprache und Datenverarbeitung 28(2), 111–131 (2004)

    Google Scholar 

  2. Bradshaw, J.M., et al.: Adjustable autonomy and human-agent teamwork in practice: an interim report on space applications. In: Weiss, G., Hexmoor, H., Castelfranchi, C., Falcone, R. (eds.) Agent Autonomy, vol. 7, pp. 243–280. Kluwer Academic Press, Boston (2003). https://doi.org/10.1007/978-1-4419-9198-0_11. http://link.springer.com/10.1007/978-1-4419-9198-0 11

  3. Butchibabu, A.: Anticipatory Communication Strategies for Human Robot Team Coordination (2016)

    Google Scholar 

  4. Chu-Carroll, J.: MIMIC: an adaptive mixed initiative spoken dialogue system for information queries. In: Proceedings of the Sixth Conference on Applied Natural Language Processing, pp. 97–104. Association for Computational Linguistics, Seattle (2000). https://doi.org/10.3115/974147.974161, http://portal.acm.org/citation.cfm?doid=974147.974161

  5. Costa, A.C., Anderson, N.: Measuring trust in teams: development and validation of a multifaceted measure of formative and reflective indicators of team trust. Eur. J. Work Organ. Psychol. 20(1), 119–154 (2011). https://doi.org/10.1080/13594320903272083, http://www.tandfonline.com/doi/abs/10.1080/13594320903272083

  6. Foerster, J., Assael, I.A., de Freitas, N., Whiteson, S.: Learning to communicate with deep multi-agent reinforcement learning. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29, pp. 2137–2145. Curran Associates, Inc. (2016). http://papers.nips.cc/paper/6042-learning-to-communicate-with-deep-multi-agent-reinforcement-learning.pdf

  7. Goldman, C.V., Zilberstein, S.: Optimizing information exchange in cooperative multi-agent systems. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multi Agent Systems, pp. 137–144. ACM Press, Melbourne (2003)

    Google Scholar 

  8. Jarvenpaa, S.L., Leidner, D.E.: Communication and trust in global virtual teams. Organ. Sci. 10(6), 791–815 (1999). https://doi.org/10.1287/orsc.10.6.791, http://pubsonline.informs.org/doi/abs/10.1287/orsc.10.6.791

  9. Johnson, M., Bradshaw, J.M., Feltovich, P.J., Jonker, C.M., van Riemsdijk, B., Sierhuis, M.: The fundamental principle of coactive design: interdependence must shape autonomy. In: De Vos, M., Fornara, N., Pitt, J.V., Vouros, G. (eds.) COIN -2010. LNCS (LNAI), vol. 6541, pp. 172–191. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21268-0_10. http://link.springer.com/10.1007/978-3-642-21268-0 10

    CrossRef  Google Scholar 

  10. Johnson, M., Jonker, C., van Riemsdijk, B., Feltovich, P.J., Bradshaw, J.M.: Joint activity testbed: blocks world for teams (BW4T). In: Aldewereld, H., Dignum, V., Picard, G. (eds.) ESAW 2009. LNCS (LNAI), vol. 5881, pp. 254–256. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10203-5_26. http://link.springer.com/10.1007/978-3-642-10203-5 26

    CrossRef  Google Scholar 

  11. Klein, G., Woods, D.D., Bradshaw, J.M., Hoffman, R.R., Feltovich, P.J.: Ten challenges for making automation a “team player” in joint human-agent activity. IEEE Intell. Syst. 19(6), 91–95 (2004). https://doi.org/10.1109/MIS.2004.74

    CrossRef  Google Scholar 

  12. Kruijff, G.J.M., Janıcek, M., Keshavdas, S., Larochelle, B., Zender, H.: Experience in system design for human-robot teaming in urban search & rescue. In: 8th International Conference on Field and Service Robotics, Matsushima, Japan, pp. 1–14 (2012)

    Google Scholar 

  13. Lazaridou, A., Peysakhovich, A., Baroni, M.: Multi-agent cooperation and the emergence of (natural) language. arXiv:1612.07182 [cs], December 2016

  14. Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum.-Comput. Interact. 7, 57–78 (1995)

    CrossRef  Google Scholar 

  15. Singh, D., Hindriks, K.V.: Learning to improve agent behaviours in GOAL. In: Dastani, M., Hübner, J.F., Logan, B. (eds.) ProMAS 2012. LNCS (LNAI), vol. 7837, pp. 158–173. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38700-5_10. http://link.springer.com/10.1007/978-3-642-38700-5 10

    CrossRef  Google Scholar 

  16. Sordoni, A., et al.: A neural network approach to context-sensitive generation of conversational responses. arXiv:1506.06714 [cs], June 2015

Download references

Author information

Authors and Affiliations

  1. TNO, Kampweg 55, 3769 DE, Soesterberg, The Netherlands

    Emma M. van Zoelen, Anita Cremers, Jurriaan van Diggelen & Marieke M. Peeters

  2. Utrecht University, Domplein 29, 3512 JE, Utrecht, The Netherlands

    Frank P. M. Dignum

  3. University of Applied Sciences, Heidelberglaan 7, 3584 CS, Utrecht, The Netherlands

    Anita Cremers

  4. Umeå University, 901 87, Umeå, Sweden

    Frank P. M. Dignum

  5. Czech Technical University, Zikova 1903/4, 166 36, Praha 6, Czech Republic

    Frank P. M. Dignum

Authors
  1. Emma M. van Zoelen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anita Cremers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frank P. M. Dignum
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jurriaan van Diggelen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marieke M. Peeters
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emma M. van Zoelen .

Editor information

Editors and Affiliations

  1. University of Salamanca, Salamanca, Spain

    Dr. Fernando De La Prieta

  2. University of Lille, Villeneuve d’Ascq, France

    Philippe Mathieu

  3. Polytechnic University of Valencia, Valencia, Spain

    Jaime Andrés Rincón Arango

  4. German University in Cairo, New Cairo City, Egypt

    Alia El Bolock

  5. Polytechnic University of Valencia, Valencia, Spain

    Dr. Elena Del Val

  6. University of Valencia, Valencia, Spain

    Jaume Jordán Prunera

  7. Instituto Superior de Engenharia do Port, Porto, Portugal

    João Carneiro

  8. Complutense University of Madrid, Madrid, Spain

    Rubén Fuentes

  9. National Laboratory for Energy and Geology, Amadora, Portugal

    Fernando Lopes

  10. Polytechnic University of Valencia, Valencia, Spain

    Vicente Julian

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

van Zoelen, E.M., Cremers, A., Dignum, F.P.M., van Diggelen, J., Peeters, M.M. (2020). Learning to Communicate Proactively in Human-Agent Teaming. In: , et al. Highlights in Practical Applications of Agents, Multi-Agent Systems, and Trust-worthiness. The PAAMS Collection. PAAMS 2020. Communications in Computer and Information Science, vol 1233. Springer, Cham. https://doi.org/10.1007/978-3-030-51999-5_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-51999-5_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-51998-8

  • Online ISBN: 978-3-030-51999-5

  • eBook Packages: Computer ScienceComputer Science (R0)