Using Transfer Learning to Model Unknown Opponents in Automated Negotiations

  • Siqi ChenEmail author
  • Shuang Zhou
  • Gerhard Weiss
  • Karl Tuyls
Part of the Studies in Computational Intelligence book series (SCI, volume 638)


Modeling unknown opponents is known as a key factor for the efficiency of automated negotiations. The learning processes are however challenging because of (1) the indirect way the target function can be observed, and (2) the limited amount of experience available to learn from an unknown opponent at a single session. To address these difficulties we propose to adopt two approaches from transfer learning. Both approaches transfer knowledge from previous tasks to the current negotiation of an agent to aid learn the latent behavior model of an opposing agent. The first approach achieves knowledge transfer by weighting the encounter offers of previous tasks and the ongoing task, while the second one by weighting the models learnt from the previous negotiation tasks and the model learnt from the current negotiation session. Extensive experimental results show the applicability and effectiveness of both approaches. Moreover, the robustness of the proposed approaches is evaluated using empirical game theoretic analysis.



We greatly appreciate the fruitful discussions with our colleagues at the Department of Knowledge Engineering, Maastricht University, especially those suggestions from Kurt Driessens, Haitham Bou Ammar and Evgueni Smirnov.


  1. 1.
    Brzostowski, J., Kowalczyk, R.: Predicting partner’s behaviour in agent negotiation. In: Proceedings of AAMAS ’06, pp. 355–361. ACM (2006)Google Scholar
  2. 2.
    Carbonneau, R., Kersten, G.E., Vahidov, R.: Predicting opponent’s moves in electronic negotiations using neural networks. Expert Syst. Appl. 34, 1266–1273 (2008)CrossRefGoogle Scholar
  3. 3.
    Chen, S., Ammar, H.B., Tuyls, K., Weiss, G.: Optimizing complex automated negotiation using sparse pseudo-input Gaussian processes. In: Proceedings of AAMAS’2013, pp. 707–714. ACM (2013)Google Scholar
  4. 4.
    Chen, S., Ammar, H.B., Tuyls, K., Weiss, G.: Using conditional restricted boltzmann machine for highly competitive negotiation tasks. In: Proceedings of the 23th International Joint Conference on Artificial Intelligence, pp. 69–75, Beijing, China. AAAI Press (2013)Google Scholar
  5. 5.
    Chen, S., Hao, J., Weiss, G., Tuyls, K., Leung, H.-F.: Evaluating practical automated negotiation based on spatial evolutionary game theory. In: Lutz, C., Thielscher, M. (eds.) KI 2014: Advances in Artificial Intelligence, Lecture Notes in Computer Science, vol. 8736, pp. 147–158. Springer International Publishing (2014)Google Scholar
  6. 6.
    Chen, S., Weiss, G.: An efficient and adaptive approach to negotiation in complex environments. In: Proceedings of ECAI’2012, pp. 228–233. IOS Press (2012)Google Scholar
  7. 7.
    Chen, S., Weiss, G.: An efficient automated negotiation strategy for complex environments. Eng. Appl. Artif. Intell. 26(10), 2613–2623 (2013)CrossRefGoogle Scholar
  8. 8.
    Chen, S., Weiss, G.: An intelligent agent for bilateral negotiation with unknown opponents in continuous-time domains. ACM Trans. Auton. Adapt. Syst. 9(3):16:1–16:24 (2014)Google Scholar
  9. 9.
    Chen, S., Weiss, G.: An approach to complex agent-based negotiations via effectively modeling unknown opponents. Expert Syst. Appl. 42(5), 2287–2304 (2015)CrossRefGoogle Scholar
  10. 10.
    Coehoorn, R.M., Jennings, N.R.: Learning on opponent’s preferences to make effective multi-issue negotiation trade-offs. In: Proceedings of the 6th International Conference on Electronic Commerce, pp. 59–68, New York, NY, USA. ACM (2004)Google Scholar
  11. 11.
    Dai, W., Yang, Q., Xue, G., Yu, Y.: Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 193–200. ACM (2007)Google Scholar
  12. 12.
    Drucker, H.: Improving regressors using boosting techniques. In: Proceedings of ICML ’97, pp. 107–115 (1997)Google Scholar
  13. 13.
    Faratin, P., Sierra, C., Jennings, N.R.: Negotiation decision functions for autonomous agents. Rob. Autom. Syst. 24(4), 159–182 (1998)CrossRefGoogle Scholar
  14. 14.
    Hao, J., Leung, H.: ABiNeS: an adaptive bilateral negotiating strategy over multiple items. In: Proceedings of the 2012 IEEE Conference on IAT, pp. 95–102 (2012)Google Scholar
  15. 15.
    Hindriks, K., Jonker, C., Kraus, S., Lin, R., Tykhonov, D.: Genius: negotiation environment for heterogeneous agents. In: Proceedings of AAMAS’2009, pp. 1397–1398 (2009)Google Scholar
  16. 16.
    Hou, C.: Predicting agents tactics in automated negotiation. In: Proceedings of the 2004 IEEE Conference on IAT, pp. 127–133 (2004)Google Scholar
  17. 17.
    Jennings, N.R., Faratin, P., Lomuscio, A.R., Parsons, S., Sierra, C., Wooldridge, M.: Automated negotiation: prospects, methods and challenges. Int. J. Group Decis. Negot. 10(2), 199–215 (2001)CrossRefGoogle Scholar
  18. 18.
    Jordan, P.R., Kiekintveld, C., Wellman, M.P.: Empirical game-theoretic analysis of the tac supply chain game. In: Proceedings of AAMAS’2007, pp. 1188–1195 (2007)Google Scholar
  19. 19.
    Lin, R., Kraus, S., Wilkenfeld, J., Barry, J.: Negotiating with bounded rational agents in environments with incomplete information using an automated agent. Artif. Intell. 172, 823–851 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Pan, S., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  21. 21.
    Pardoe, D., Stone, P.: Boosting for regression transfer. In: Proceedings of the 27th International Conference on Machine Learning, pp. 863–870 (2010)Google Scholar
  22. 22.
    Rettinger, A., Zinkevich, M., Bowling, M.: Boosting expert ensembles for rapid concept recall. In: Proceedings Of AAAI’2006, vol. 21, pp. 464–469 (2006)Google Scholar
  23. 23.
    Rubinstein, A.: Perfect equilibrium in a bargaining model. Econometrica 50(1), 97–109 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In Advances In Neural Information Processing Systems, pp. 1257–1264. MIT press (2006)Google Scholar
  25. 25.
    Taylor, M.E., Stone, P.: Transfer learning for reinforcement learning domains: a survey. J. Mach. Learn. Res. 10, 1633–1685 (2009)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Williams, C., Robu, V., Gerding, E., Jennings, N.: Using gaussian processes to optimise concession in complex negotiations against unknown opponents. In: Proceedings of IJCAI’2011, pp. 432–438. AAAI Press (2011)Google Scholar
  27. 27.
    Yao, Y., Doretto, G.: Boosting for transfer learning with multiple sources. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1855–1862. IEEE (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Siqi Chen
    • 1
    Email author
  • Shuang Zhou
    • 2
  • Gerhard Weiss
    • 2
  • Karl Tuyls
    • 3
  1. 1.School of Computer and Information ScienceSouthwest UniversityChongqingChina
  2. 2.Department of Knowledge EngineeringMaastricht UniversityMaastrichtThe Netherlands
  3. 3.University of LiverpoolLiverpoolUK

Personalised recommendations