Skip to main content
SpringerLink
Log in
Book cover

Recent Advances in Agent-based Complex Automated Negotiation pp 175–192Cite as

Using Transfer Learning to Model Unknown Opponents in Automated Negotiations

  • Chapter
  • First Online:

Part of the book series: Studies in Computational Intelligence ((SCI,volume 638))

Abstract

Modeling unknown opponents is known as a key factor for the efficiency of automated negotiations. The learning processes are however challenging because of (1) the indirect way the target function can be observed, and (2) the limited amount of experience available to learn from an unknown opponent at a single session. To address these difficulties we propose to adopt two approaches from transfer learning. Both approaches transfer knowledge from previous tasks to the current negotiation of an agent to aid learn the latent behavior model of an opposing agent. The first approach achieves knowledge transfer by weighting the encounter offers of previous tasks and the ongoing task, while the second one by weighting the models learnt from the previous negotiation tasks and the model learnt from the current negotiation session. Extensive experimental results show the applicability and effectiveness of both approaches. Moreover, the robustness of the proposed approaches is evaluated using empirical game theoretic analysis.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Because both an agent’s utility function and bidding strategy are hidden, we will often use the term behavior model to encompass both as the force governing the agents negotiating behavior.

  2. 2.

    Outcome space of a domain refers to the number of possible agreements that could be agreed upon between participants.

References

  1. Brzostowski, J., Kowalczyk, R.: Predicting partner’s behaviour in agent negotiation. In: Proceedings of AAMAS ’06, pp. 355–361. ACM (2006)

    Google Scholar 

  2. Carbonneau, R., Kersten, G.E., Vahidov, R.: Predicting opponent’s moves in electronic negotiations using neural networks. Expert Syst. Appl. 34, 1266–1273 (2008)

    Article  Google Scholar 

  3. Chen, S., Ammar, H.B., Tuyls, K., Weiss, G.: Optimizing complex automated negotiation using sparse pseudo-input Gaussian processes. In: Proceedings of AAMAS’2013, pp. 707–714. ACM (2013)

    Google Scholar 

  4. Chen, S., Ammar, H.B., Tuyls, K., Weiss, G.: Using conditional restricted boltzmann machine for highly competitive negotiation tasks. In: Proceedings of the 23th International Joint Conference on Artificial Intelligence, pp. 69–75, Beijing, China. AAAI Press (2013)

    Google Scholar 

  5. Chen, S., Hao, J., Weiss, G., Tuyls, K., Leung, H.-F.: Evaluating practical automated negotiation based on spatial evolutionary game theory. In: Lutz, C., Thielscher, M. (eds.) KI 2014: Advances in Artificial Intelligence, Lecture Notes in Computer Science, vol. 8736, pp. 147–158. Springer International Publishing (2014)

    Google Scholar 

  6. Chen, S., Weiss, G.: An efficient and adaptive approach to negotiation in complex environments. In: Proceedings of ECAI’2012, pp. 228–233. IOS Press (2012)

    Google Scholar 

  7. Chen, S., Weiss, G.: An efficient automated negotiation strategy for complex environments. Eng. Appl. Artif. Intell. 26(10), 2613–2623 (2013)

    Article  Google Scholar 

  8. Chen, S., Weiss, G.: An intelligent agent for bilateral negotiation with unknown opponents in continuous-time domains. ACM Trans. Auton. Adapt. Syst. 9(3):16:1–16:24 (2014)

    Google Scholar 

  9. Chen, S., Weiss, G.: An approach to complex agent-based negotiations via effectively modeling unknown opponents. Expert Syst. Appl. 42(5), 2287–2304 (2015)

    Article  Google Scholar 

  10. Coehoorn, R.M., Jennings, N.R.: Learning on opponent’s preferences to make effective multi-issue negotiation trade-offs. In: Proceedings of the 6th International Conference on Electronic Commerce, pp. 59–68, New York, NY, USA. ACM (2004)

    Google Scholar 

  11. Dai, W., Yang, Q., Xue, G., Yu, Y.: Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 193–200. ACM (2007)

    Google Scholar 

  12. Drucker, H.: Improving regressors using boosting techniques. In: Proceedings of ICML ’97, pp. 107–115 (1997)

    Google Scholar 

  13. Faratin, P., Sierra, C., Jennings, N.R.: Negotiation decision functions for autonomous agents. Rob. Autom. Syst. 24(4), 159–182 (1998)

    Article  Google Scholar 

  14. Hao, J., Leung, H.: ABiNeS: an adaptive bilateral negotiating strategy over multiple items. In: Proceedings of the 2012 IEEE Conference on IAT, pp. 95–102 (2012)

    Google Scholar 

  15. Hindriks, K., Jonker, C., Kraus, S., Lin, R., Tykhonov, D.: Genius: negotiation environment for heterogeneous agents. In: Proceedings of AAMAS’2009, pp. 1397–1398 (2009)

    Google Scholar 

  16. Hou, C.: Predicting agents tactics in automated negotiation. In: Proceedings of the 2004 IEEE Conference on IAT, pp. 127–133 (2004)

    Google Scholar 

  17. Jennings, N.R., Faratin, P., Lomuscio, A.R., Parsons, S., Sierra, C., Wooldridge, M.: Automated negotiation: prospects, methods and challenges. Int. J. Group Decis. Negot. 10(2), 199–215 (2001)

    Article  Google Scholar 

  18. Jordan, P.R., Kiekintveld, C., Wellman, M.P.: Empirical game-theoretic analysis of the tac supply chain game. In: Proceedings of AAMAS’2007, pp. 1188–1195 (2007)

    Google Scholar 

  19. Lin, R., Kraus, S., Wilkenfeld, J., Barry, J.: Negotiating with bounded rational agents in environments with incomplete information using an automated agent. Artif. Intell. 172, 823–851 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  20. Pan, S., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)

    Article  Google Scholar 

  21. Pardoe, D., Stone, P.: Boosting for regression transfer. In: Proceedings of the 27th International Conference on Machine Learning, pp. 863–870 (2010)

    Google Scholar 

  22. Rettinger, A., Zinkevich, M., Bowling, M.: Boosting expert ensembles for rapid concept recall. In: Proceedings Of AAAI’2006, vol. 21, pp. 464–469 (2006)

    Google Scholar 

  23. Rubinstein, A.: Perfect equilibrium in a bargaining model. Econometrica 50(1), 97–109 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  24. Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In Advances In Neural Information Processing Systems, pp. 1257–1264. MIT press (2006)

    Google Scholar 

  25. Taylor, M.E., Stone, P.: Transfer learning for reinforcement learning domains: a survey. J. Mach. Learn. Res. 10, 1633–1685 (2009)

    MathSciNet  MATH  Google Scholar 

  26. Williams, C., Robu, V., Gerding, E., Jennings, N.: Using gaussian processes to optimise concession in complex negotiations against unknown opponents. In: Proceedings of IJCAI’2011, pp. 432–438. AAAI Press (2011)

    Google Scholar 

  27. Yao, Y., Doretto, G.: Boosting for transfer learning with multiple sources. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1855–1862. IEEE (2010)

    Google Scholar 

Download references

Acknowledgments

We greatly appreciate the fruitful discussions with our colleagues at the Department of Knowledge Engineering, Maastricht University, especially those suggestions from Kurt Driessens, Haitham Bou Ammar and Evgueni Smirnov.

Author information

Authors and Affiliations

  1. School of Computer and Information Science, Southwest University, Chongqing, 400715, China

    Siqi Chen

  2. Department of Knowledge Engineering, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands

    Shuang Zhou & Gerhard Weiss

  3. University of Liverpool, Ashton Street, Liverpool, L69 3BX, UK

    Karl Tuyls

Authors
  1. Siqi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerhard Weiss
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Karl Tuyls
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Siqi Chen .

Editor information

Editors and Affiliations

  1. Hamamatsu, Shizuoka, Japan

    Naoki Fukuta

  2. Nagoya Institute of Technology, Nagoya, Japan

    Takayuki Ito

  3. School of Computer Science, University of Wollongong, Wollongong, New South Wales, Australia

    Minjie Zhang

  4. Tokyo University of Agriculture and, Tokyo, Japan

    Katsuhide Fujita

  5. Institute of Sensors, Signals, Heriot-Watt University, Riccarton, United Kingdom

    Valentin Robu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Chen, S., Zhou, S., Weiss, G., Tuyls, K. (2016). Using Transfer Learning to Model Unknown Opponents in Automated Negotiations. In: Fukuta, N., Ito, T., Zhang, M., Fujita, K., Robu, V. (eds) Recent Advances in Agent-based Complex Automated Negotiation. Studies in Computational Intelligence, vol 638. Springer, Cham. https://doi.org/10.1007/978-3-319-30307-9_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-30307-9_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-30305-5

  • Online ISBN: 978-3-319-30307-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation