Abstract
Preferences are central to decision making by both machines and humans. Representing, learning, and reasoning with preferences is an important area of study both within computer science and across the social sciences. When we give our preferences to an AI system we expect the system to make decisions or recommendations that are consistent with our preferences but the decisions should also adhere to certain norms, guidelines, and ethical principles. Hence, when working with preferences it is necessary to understand and compute a metric (distance) between preferences – especially if we encode both the user preferences and ethical systems in the same formalism. In this paper we investigate the use of CP-nets as a formalism for representing orderings over actions for AI systems. We leverage a recently proposed metric for CP-nets and propose a neural network architecture to learn an approximation of the metric, CPMetric. Using these two tools we look at how one can build a fast and flexible value alignment system (This is an expanded version of our paper, “Metric Learning for Value Alignment” [38]. In this version we have added the classification and regression results and significantly expanded the description of the CPMetric network.).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abdollahpouri, H., Burke, R.: Multi-stakeholder recommendation and its connection to multi-sided fairness. arXiv preprint arXiv:1907.13158 (2019)
Alkoby, S., Rath, A., Stone, P.: Teaching social behavior through human reinforcement for ad hoc teamwork-the star framework. In: Proceedings of 18th AAMAS, pp. 1773–1775 (2019)
Allen, C., Smit, I., Wallach, W.: Artificial morality: top-down, bottom-up, and hybrid approaches. Ethics Inf. Technol. 7(3), 149–155 (2005)
Allen, T.E., et al.: Beyond theory and data in preference modeling: bringing humans into the loop. In: Walsh, T. (ed.) ADT 2015. LNCS (LNAI), vol. 9346, pp. 3–18. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23114-3_1
Allen, T.E., Goldsmith, J., Justice, H.E., Mattei, N., Raines, K.: Uniform random generation and dominance testing for CP-nets. JAIR 59, 771–813 (2017)
Amor, N.B., Dubois, D., Gouider, H., Prade, H.: Graphical models for preference representation: an overview. In: Schockaert, S., Senellart, P. (eds.) SUM 2016. LNCS (LNAI), vol. 9858, pp. 96–111. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45856-4_7
Bache, K., Lichman, M.: UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences (2013). http://archive.ics.uci.edu/ml
Balakrishnan, A., Bouneffouf, D., Mattei, N., Rossi, F.: Using contextual bandits with behavioral constraints for constrained online movie recommendation. In: Proceedings of the 27th IJCAI (2018)
Balakrishnan, A., Bouneffouf, D., Mattei, N., Rossi, F.: Incorporating behavioral constraints in online AI systems. In: Proceedings of the 33rd AAAI (2019)
Bellet, A., Habrard, A., Sebban, M.: Metric Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, San Rafael (2015)
Boutilier, C., Brafman, R., Domshlak, C., Hoos, H., Poole, D.: CP-nets: a tool for representing and reasoning with conditional ceteris paribus preference statements. J. Artif. Intell. Res. 21, 135–191 (2004)
Brandt, F., Conitzer, V., Endriss, U., Lang, J., Procaccia, A.D. (eds.): Handbook of Computational Social Choice. Cambridge University Press, Cambridge (2016)
Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. arXiv abs/1312.6203 (2013)
Cohen, P.R.: Empirical Methods for Artificial Intelligence. MIT Press, Cambridge (1995)
Cornelio, C., Goldsmith, J., Mattei, N., Rossi, F., Venable, K.B.: Updates and uncertainty in CP-Nets. In: Cranefield, S., Nayak, A. (eds.) AI 2013. LNCS (LNAI), vol. 8272, pp. 301–312. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03680-9_32
Cornelio, C., Grandi, U., Goldsmith, J., Mattei, N., Rossi, F., Venable, K.: Reasoning with PCP-nets in a multi-agent context. In: Proceedings of the 14th AAMAS (2015)
Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th NeurIPS, pp. 3837–3845 (2016)
Domshlak, C., Hüllermeier, E., Kaci, S., Prade, H.: Preferences in AI: an overview. AI 175(7), 1037–1052 (2011)
Fagin, R., Kumar, R., Mahdian, M., Sivakumar, D., Vee, E.: Comparing partial rankings. SIAM J. Discret. Math. 20(3), 628–648 (2006). https://doi.org/10.1137/05063088X
Fattah, S.M.M., Bouguettaya, A., Mistry, S.: A CP-net based qualitative composition approach for an IaaS provider. In: Hacid, H., Cellary, W., Wang, H., Paik, H.-Y., Zhou, R. (eds.) WISE 2018. LNCS, vol. 11234, pp. 151–166. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02925-8_11
Fürnkranz, J., Hüllermeier, E.: Preference learning. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning. Springer, Boston (2010). https://doi.org/10.1007/978-0-387-30164-8
Goldsmith, J., Junker, U.: Preference handling for artificial intelligence. AI Mag. 29(4), 9 (2009)
Goldsmith, J., Lang, J., Truszczyński, M., Wilson, N.: The computational complexity of dominance and consistency in CP-nets. J. Artif. Intell. Res. 33(1), 403–432 (2008)
Goyal, P., Ferrara, E.: Graph embedding techniques, applications, and performance: a survey. CoRR abs/1705.02801 (2017)
Henaff, M., Bruna, J., LeCun, Y.: Deep convolutional networks on graph-structured data. arXiv abs/1506.05163 (2015)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Kahneman, D.: Thinking, Fast and Slow. Farrar, Straus and Giroux, New York (2011)
Kambhampati, S.: Synthesizing explainable behavior for human-AI collaboration. In: Proceedings of the 18th AAMAS (2019)
Kendall, M.G.: A new measure of rank correlation. Biometrika 30(1/2), 81–93 (1938)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv abs/1609.02907 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural and Information Processing Systems (NeurIPS), pp. 1097–1105 (2012)
Lecun, Y., Bengio, Y.: Convolutional Networks for Images, Speech and Time Series, pp. 255–258. The MIT Press, Cambridge (1995)
Li, M., Kazimipour, B.: An efficient algorithm to compute distance between lexicographic preference trees. In: Proceedings of 27th IJCAI, pp. 1898–1904 (2018)
Loreggia, A., Mattei, N., Rossi, F., Venable, K.B.: On the distance between CP-nets. In: Proceedings of the 17th AAMAS (2018)
Loreggia, A., Mattei, N., Rossi, F., Venable, K.B.: Preferences and ethical principles in decision making. In: Proceedings of the 1st AAAI/ACM Conference on AI, Ethics, and Society (AIES) (2018)
Loreggia, A., Mattei, N., Rossi, F., Venable, K.B.: Value alignment via tractable preference distance. In: Yampolskiy, R.V. (ed.) Artificial Intelligence Safety and Security. CRC Press, Boca Raton (2018)
Loreggia, A., Mattei, N., Rossi, F., Venable, K.B.: Metric learning for value alignment. In: Proceedings of the Workshop on Artificial Intelligence Safety 2019 held at IJCAI 2019. CEUR Workshop Proceedings, vol. 2419. CEUR-WS.org (2019)
Mattei, N., Walsh, T.: PrefLib: a library for preferences. In: Proceedings of the 3rd ADT (2013). http://www.preflib.org
Mattei, N., Walsh, T.: A PrefLib.Org Retrospective: Lessons Learned and New Directions. In: Endriss, U. (ed.) Trends in Computational Social Choice, pp. 289–309. AI Access Foundation (2017)
Noothigattu, R., et al.: Teaching AI agents ethical values using reinforcement learning and policy orchestration. In: Proceedings of the 28th IJCAI (2019)
Popova, A., Regenwetter, M., Mattei, N.: A behavioral perspective on social choice. AMAI 68(1–3), 135–160 (2013). https://doi.org/10.1007/s10472-012-9326-6
Pu, P., Faltings, B., Chen, L., Zhang, J., Viappiani, P.: Usability guidelines for product recommenders based on example critiquing research. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 511–545. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-85820-3_16
Rossi, F., Loreggia, A.: Preferences and ethical priorities: thinking fast and slow in AI. In: Proceedings of the 18th AAMAS, pp. 3–4 (2019)
Rossi, F., Venable, K., Walsh, T.: A Short Introduction to Preferences: Between Artificial Intelligence and Social Choice. Morgan and Claypool, San Rafael (2011)
Rossi, F., Mattei, N.: Building ethically bounded AI. In: Proceedings of the 33rd AAAI (2019)
Russell, S., Dewey, D., Tegmark, M.: Research priorities for robust and beneficial artificial intelligence. AI Mag. 36(4), 105–114 (2015)
Sen, A.: Choice, Ordering, and Morality. Blackwell, Oxford (1974)
Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P.: The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013)
Sohn, K.: Improved deep metric learning with multi-class n-pair loss objective. In: Advances in Neural Information Processing Systems (NeruIPS), pp. 1857–1865 (2016)
Wallach, W., Allen, C.: Moral Machines: Teaching Robots Right From Wrong. Oxford University Press, Oxford (2008)
Wang, H., Shao, S., Zhou, X., Wan, C., Bouguettaya, A.: Web service selection with incomplete or inconsistent user preferences. In: Baresi, L., Chi, C.-H., Suzuki, J. (eds.) ICSOC/ServiceWave -2009. LNCS, vol. 5900, pp. 83–98. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10383-4_6
Wong, S.C., Gatt, A., Stamatescu, V., McDonnell, M.D.: Understanding data augmentation for classification: When to warp? In: Proceedings of the 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6 (2016)
Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.J.: Distance metric learning with application to clustering with side-information. In: Proceedings of the 15th NeurIPS, pp. 505–512 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Loreggia, A., Mattei, N., Rossi, F., Venable, K.B. (2020). CPMetric: Deep Siamese Networks for Metric Learning on Structured Preferences. In: El Fallah Seghrouchni, A., Sarne, D. (eds) Artificial Intelligence. IJCAI 2019 International Workshops. IJCAI 2019. Lecture Notes in Computer Science(), vol 12158. Springer, Cham. https://doi.org/10.1007/978-3-030-56150-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-56150-5_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-56149-9
Online ISBN: 978-3-030-56150-5
eBook Packages: Computer ScienceComputer Science (R0)