Skip to main content

CPMetric: Deep Siamese Networks for Metric Learning on Structured Preferences

Part of the Lecture Notes in Computer Science book series (LNAI,volume 12158)

Abstract

Preferences are central to decision making by both machines and humans. Representing, learning, and reasoning with preferences is an important area of study both within computer science and across the social sciences. When we give our preferences to an AI system we expect the system to make decisions or recommendations that are consistent with our preferences but the decisions should also adhere to certain norms, guidelines, and ethical principles. Hence, when working with preferences it is necessary to understand and compute a metric (distance) between preferences – especially if we encode both the user preferences and ethical systems in the same formalism. In this paper we investigate the use of CP-nets as a formalism for representing orderings over actions for AI systems. We leverage a recently proposed metric for CP-nets and propose a neural network architecture to learn an approximation of the metric, CPMetric. Using these two tools we look at how one can build a fast and flexible value alignment system (This is an expanded version of our paper, “Metric Learning for Value Alignment” [38]. In this version we have added the classification and regression results and significantly expanded the description of the CPMetric network.).

Keywords

  • Metric learning
  • Value alignment
  • Artificial intelligence
  • Ethics

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-56150-5_11
  • Chapter length: 18 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-56150-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   69.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

References

  1. Abdollahpouri, H., Burke, R.: Multi-stakeholder recommendation and its connection to multi-sided fairness. arXiv preprint arXiv:1907.13158 (2019)

  2. Alkoby, S., Rath, A., Stone, P.: Teaching social behavior through human reinforcement for ad hoc teamwork-the star framework. In: Proceedings of 18th AAMAS, pp. 1773–1775 (2019)

    Google Scholar 

  3. Allen, C., Smit, I., Wallach, W.: Artificial morality: top-down, bottom-up, and hybrid approaches. Ethics Inf. Technol. 7(3), 149–155 (2005)

    Google Scholar 

  4. Allen, T.E., et al.: Beyond theory and data in preference modeling: bringing humans into the loop. In: Walsh, T. (ed.) ADT 2015. LNCS (LNAI), vol. 9346, pp. 3–18. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23114-3_1

    CrossRef  Google Scholar 

  5. Allen, T.E., Goldsmith, J., Justice, H.E., Mattei, N., Raines, K.: Uniform random generation and dominance testing for CP-nets. JAIR 59, 771–813 (2017)

    MathSciNet  MATH  Google Scholar 

  6. Amor, N.B., Dubois, D., Gouider, H., Prade, H.: Graphical models for preference representation: an overview. In: Schockaert, S., Senellart, P. (eds.) SUM 2016. LNCS (LNAI), vol. 9858, pp. 96–111. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45856-4_7

    CrossRef  Google Scholar 

  7. Bache, K., Lichman, M.: UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences (2013). http://archive.ics.uci.edu/ml

  8. Balakrishnan, A., Bouneffouf, D., Mattei, N., Rossi, F.: Using contextual bandits with behavioral constraints for constrained online movie recommendation. In: Proceedings of the 27th IJCAI (2018)

    Google Scholar 

  9. Balakrishnan, A., Bouneffouf, D., Mattei, N., Rossi, F.: Incorporating behavioral constraints in online AI systems. In: Proceedings of the 33rd AAAI (2019)

    Google Scholar 

  10. Bellet, A., Habrard, A., Sebban, M.: Metric Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, San Rafael (2015)

    MATH  Google Scholar 

  11. Boutilier, C., Brafman, R., Domshlak, C., Hoos, H., Poole, D.: CP-nets: a tool for representing and reasoning with conditional ceteris paribus preference statements. J. Artif. Intell. Res. 21, 135–191 (2004)

    MathSciNet  MATH  Google Scholar 

  12. Brandt, F., Conitzer, V., Endriss, U., Lang, J., Procaccia, A.D. (eds.): Handbook of Computational Social Choice. Cambridge University Press, Cambridge (2016)

    MATH  Google Scholar 

  13. Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. arXiv abs/1312.6203 (2013)

    Google Scholar 

  14. Cohen, P.R.: Empirical Methods for Artificial Intelligence. MIT Press, Cambridge (1995)

    MATH  Google Scholar 

  15. Cornelio, C., Goldsmith, J., Mattei, N., Rossi, F., Venable, K.B.: Updates and uncertainty in CP-Nets. In: Cranefield, S., Nayak, A. (eds.) AI 2013. LNCS (LNAI), vol. 8272, pp. 301–312. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03680-9_32

    CrossRef  Google Scholar 

  16. Cornelio, C., Grandi, U., Goldsmith, J., Mattei, N., Rossi, F., Venable, K.: Reasoning with PCP-nets in a multi-agent context. In: Proceedings of the 14th AAMAS (2015)

    Google Scholar 

  17. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th NeurIPS, pp. 3837–3845 (2016)

    Google Scholar 

  18. Domshlak, C., Hüllermeier, E., Kaci, S., Prade, H.: Preferences in AI: an overview. AI 175(7), 1037–1052 (2011)

    MathSciNet  Google Scholar 

  19. Fagin, R., Kumar, R., Mahdian, M., Sivakumar, D., Vee, E.: Comparing partial rankings. SIAM J. Discret. Math. 20(3), 628–648 (2006). https://doi.org/10.1137/05063088X

    MathSciNet  MATH  CrossRef  Google Scholar 

  20. Fattah, S.M.M., Bouguettaya, A., Mistry, S.: A CP-net based qualitative composition approach for an IaaS provider. In: Hacid, H., Cellary, W., Wang, H., Paik, H.-Y., Zhou, R. (eds.) WISE 2018. LNCS, vol. 11234, pp. 151–166. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02925-8_11

    CrossRef  Google Scholar 

  21. Fürnkranz, J., Hüllermeier, E.: Preference learning. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning. Springer, Boston (2010). https://doi.org/10.1007/978-0-387-30164-8

    MATH  CrossRef  Google Scholar 

  22. Goldsmith, J., Junker, U.: Preference handling for artificial intelligence. AI Mag. 29(4), 9 (2009)

    Google Scholar 

  23. Goldsmith, J., Lang, J., Truszczyński, M., Wilson, N.: The computational complexity of dominance and consistency in CP-nets. J. Artif. Intell. Res. 33(1), 403–432 (2008)

    MathSciNet  MATH  Google Scholar 

  24. Goyal, P., Ferrara, E.: Graph embedding techniques, applications, and performance: a survey. CoRR abs/1705.02801 (2017)

    Google Scholar 

  25. Henaff, M., Bruna, J., LeCun, Y.: Deep convolutional networks on graph-structured data. arXiv abs/1506.05163 (2015)

    Google Scholar 

  26. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    MathSciNet  MATH  Google Scholar 

  27. Kahneman, D.: Thinking, Fast and Slow. Farrar, Straus and Giroux, New York (2011)

    Google Scholar 

  28. Kambhampati, S.: Synthesizing explainable behavior for human-AI collaboration. In: Proceedings of the 18th AAMAS (2019)

    Google Scholar 

  29. Kendall, M.G.: A new measure of rank correlation. Biometrika 30(1/2), 81–93 (1938)

    MATH  Google Scholar 

  30. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)

  31. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv abs/1609.02907 (2016)

    Google Scholar 

  32. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural and Information Processing Systems (NeurIPS), pp. 1097–1105 (2012)

    Google Scholar 

  33. Lecun, Y., Bengio, Y.: Convolutional Networks for Images, Speech and Time Series, pp. 255–258. The MIT Press, Cambridge (1995)

    Google Scholar 

  34. Li, M., Kazimipour, B.: An efficient algorithm to compute distance between lexicographic preference trees. In: Proceedings of 27th IJCAI, pp. 1898–1904 (2018)

    Google Scholar 

  35. Loreggia, A., Mattei, N., Rossi, F., Venable, K.B.: On the distance between CP-nets. In: Proceedings of the 17th AAMAS (2018)

    Google Scholar 

  36. Loreggia, A., Mattei, N., Rossi, F., Venable, K.B.: Preferences and ethical principles in decision making. In: Proceedings of the 1st AAAI/ACM Conference on AI, Ethics, and Society (AIES) (2018)

    Google Scholar 

  37. Loreggia, A., Mattei, N., Rossi, F., Venable, K.B.: Value alignment via tractable preference distance. In: Yampolskiy, R.V. (ed.) Artificial Intelligence Safety and Security. CRC Press, Boca Raton (2018)

    Google Scholar 

  38. Loreggia, A., Mattei, N., Rossi, F., Venable, K.B.: Metric learning for value alignment. In: Proceedings of the Workshop on Artificial Intelligence Safety 2019 held at IJCAI 2019. CEUR Workshop Proceedings, vol. 2419. CEUR-WS.org (2019)

    Google Scholar 

  39. Mattei, N., Walsh, T.: PrefLib: a library for preferences. In: Proceedings of the 3rd ADT (2013). http://www.preflib.org

  40. Mattei, N., Walsh, T.: A PrefLib.Org Retrospective: Lessons Learned and New Directions. In: Endriss, U. (ed.) Trends in Computational Social Choice, pp. 289–309. AI Access Foundation (2017)

    Google Scholar 

  41. Noothigattu, R., et al.: Teaching AI agents ethical values using reinforcement learning and policy orchestration. In: Proceedings of the 28th IJCAI (2019)

    Google Scholar 

  42. Popova, A., Regenwetter, M., Mattei, N.: A behavioral perspective on social choice. AMAI 68(1–3), 135–160 (2013). https://doi.org/10.1007/s10472-012-9326-6

    MathSciNet  CrossRef  Google Scholar 

  43. Pu, P., Faltings, B., Chen, L., Zhang, J., Viappiani, P.: Usability guidelines for product recommenders based on example critiquing research. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 511–545. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-85820-3_16

    CrossRef  Google Scholar 

  44. Rossi, F., Loreggia, A.: Preferences and ethical priorities: thinking fast and slow in AI. In: Proceedings of the 18th AAMAS, pp. 3–4 (2019)

    Google Scholar 

  45. Rossi, F., Venable, K., Walsh, T.: A Short Introduction to Preferences: Between Artificial Intelligence and Social Choice. Morgan and Claypool, San Rafael (2011)

    Google Scholar 

  46. Rossi, F., Mattei, N.: Building ethically bounded AI. In: Proceedings of the 33rd AAAI (2019)

    Google Scholar 

  47. Russell, S., Dewey, D., Tegmark, M.: Research priorities for robust and beneficial artificial intelligence. AI Mag. 36(4), 105–114 (2015)

    Google Scholar 

  48. Sen, A.: Choice, Ordering, and Morality. Blackwell, Oxford (1974)

    Google Scholar 

  49. Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P.: The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013)

    Google Scholar 

  50. Sohn, K.: Improved deep metric learning with multi-class n-pair loss objective. In: Advances in Neural Information Processing Systems (NeruIPS), pp. 1857–1865 (2016)

    Google Scholar 

  51. Wallach, W., Allen, C.: Moral Machines: Teaching Robots Right From Wrong. Oxford University Press, Oxford (2008)

    Google Scholar 

  52. Wang, H., Shao, S., Zhou, X., Wan, C., Bouguettaya, A.: Web service selection with incomplete or inconsistent user preferences. In: Baresi, L., Chi, C.-H., Suzuki, J. (eds.) ICSOC/ServiceWave -2009. LNCS, vol. 5900, pp. 83–98. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10383-4_6

    CrossRef  Google Scholar 

  53. Wong, S.C., Gatt, A., Stamatescu, V., McDonnell, M.D.: Understanding data augmentation for classification: When to warp? In: Proceedings of the 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6 (2016)

    Google Scholar 

  54. Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.J.: Distance metric learning with application to clustering with side-information. In: Proceedings of the 15th NeurIPS, pp. 505–512 (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Andrea Loreggia .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Loreggia, A., Mattei, N., Rossi, F., Venable, K.B. (2020). CPMetric: Deep Siamese Networks for Metric Learning on Structured Preferences. In: El Fallah Seghrouchni, A., Sarne, D. (eds) Artificial Intelligence. IJCAI 2019 International Workshops. IJCAI 2019. Lecture Notes in Computer Science(), vol 12158. Springer, Cham. https://doi.org/10.1007/978-3-030-56150-5_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-56150-5_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-56149-9

  • Online ISBN: 978-3-030-56150-5

  • eBook Packages: Computer ScienceComputer Science (R0)