Abstract
Kidney Exchange Programs allow an incompatible patient-donor pair, whose donor cannot provide a kidney to the respective patient, to have a transplant exchange with another pair in a similar situation. The associated combinatorial problem of finding such exchanges can be represented by a graph: nodes represent incompatible pairs and arcs represent compatibility between donor in one pair and patient in the other. This problem has some uncertainty, which in the literature has been commonly addressed in the following ways: expected utility, fall-back mechanisms and robust optimization. We propose an alternative interactive tool to support decision makers (DMs) on choosing a solution, taking into account that some pairs may become unavailable. For a given solution the predicted performance is evaluated under multiple scenarios generated by Monte Carlo Tree Search (MCTS). The root node of the tree corresponds to no failures. From there, a tree of failure possibilities is generated, each of them corresponding to a different scenario. A solution is determined for every particular scenario. At the end, each solution is evaluated under each scenario. Scenarios are grouped based on the cardinality of the set of failing vertices, and average results for each cardinality are considered. Finally, Pareto dominated solutions are filtered out and the non-dominated average solutions are displayed and compared with the worst case scenario. The tool visually drives DMs in the process of choosing the best solution for their particular preferences.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available in: http://www.dcc.fc.up.pt/~jpp/code/KEP/small/.
References
Abraham, D, Blum, A., Sandholm, T.: Clearing algorithms for Barter exchange markets: enabling nationwide kidney exchanges. In: Proceedings of the 8th ACM conference on Electronic commerce, June 13–16, pp. 295–304 (2007)
Bray, M., Wang, W., Song, P.X.-K., Kalbfleisch, J.D.: Valuing sets of potential transplants in a kidney paired donation network. Stat. Biosci. (2018)
Bray, M., Wang, W., Song, P.X.K., Leichtman, A.B., Rees, M., Ashby, V.B., Eikstadt, R., Goulding, A., Kalbfleisch, J.: Planning for uncertainty and fallbacks can increase the number of transplants in a kidney-paired donation program. Am. J. Transplant. 15, 08 (2015)
Browne, C., Powley, E., Whitehouse, D., Lucas, S., Cowling, P., Rohlfshagen, P., Tavener, S., Perez Liebana, D., Samothrakis, S., Colton, S.: A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)
Chen, Y., Li, Y., Kalbfleisch, J.D., Zhou, Y., Leichtman, A., Song, P.X.-K.: Graph-based optimization algorithm and software on kidney exchanges. IEEE Trans. Biomed. Eng. 59(7), 1985–1991 (2012)
Constantino, M., Klimentova, X., Viana, A., Rais, A.: New insights on integer-programming models for the kidney exchange problem. Eur. J. Oper. Res. 231(1), 57–68 (2013)
Dickerson, J., Procaccia, A., Sandholm, T.: Failure-aware kidney exchange. In: EC-13: Proceedings of 14th ACM Conference on Electronic Commerce, June, 2013
Edmonds, J.: Paths, trees, and flowers. Can. J. Math. 17, 449–467 (1965)
Glorie, K., Carvalho, M., Constantino, M., Viana, A., Klimentova, X.: Robust models for the kidney exchange problem. Technical report DS4DM-2018-007, Data science for real-time decision-making at Polytechnique Montréal (2018)
Klimentova, X., Pedroso, J.P., Viana, A.: Maximising expectation of the number of transplants in kidney exchange programmes. Comput. Oper. Res. 73, 1–11 (2016)
Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) Machine Learning: ECML 2006, pp. 282–293. Berlin, Heidelberg (2006)
Li, Y., Song, P.X.-K., Zhou, Y., Leichtman, A.B., Rees, M., Kalbfleisch, J.: Optimal decisions for organ exchanges in a kidney paired donation program. Stat. Biosci. 6, 05 (2014)
Manlove, D., O’Malley, G: Paired and altruistic kidney donation in the UK: algorithms and experimentation. In: Klasing, R. (ed.) Experimental Algorithms. SEA 2012. Lecture Notes in Computer Science, vol. 7276, pp. 271–282 (2012)
Pedroso, J.P.: Maximizing expectation on vertex-disjoint cycle packing. In: Murgante, B. et al. (eds.) ICCSA 2014. Lecture Notes in Computer Science, vol. 8580, pp. 32–46 (2014)
Pedroso, J.P., Rei, R.: Tree search and simulation. In: Mujica Mota, M.E.A. (ed.) Applied Simulation and Optimization: In Logistics, Industrial and Aeronautical Practice, pp. 109–131. Springer International Publishing (2015)
Rapaport, F.: The case for a living emotionally related international kidney donor exchange registry. Transpl. Proc. 18, 5–9 (1986)
Rei, R.: Monte Carlo tree search for combinatorial optimization. PhD thesis, Faculdade de Ciências, Universidade do Porto (2018)
Roth, A., Sönmez, T., Ünver, M.: Efficient kidney exchange: coincidence of wants in markets with compatibility-based preferences. Am. Econ. Rev. 97(3), 828–851 (2007)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Acknowledgements
This work is financed by the ERDF European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation - COMPETE 2020 Programme, and by National Funds through the Portuguese funding agency, FCT - Fundação para a Ciência e a Tecnologia, within project “mKEP - Models and optimisation algorithms for multicountry kidney exchange programs” (POCI-01-0145-FEDER-016677), and by COST Action CA15210, ENCKEP, supported by COST (European Cooperation in Science and Technology) – http://www.cost.eu/.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Monteiro, T., Pedroso, J.P., Viana, A., Klimentova, X. (2021). Reinforcement Learning for Robust Optimization: An Application in Kidney Exchange Programs. In: Relvas, S., Almeida, J.P., Oliveira, J.F., Pinto, A.A. (eds) Operational Research . APDIO 2019. Springer Proceedings in Mathematics & Statistics, vol 374. Springer, Cham. https://doi.org/10.1007/978-3-030-85476-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-85476-8_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85475-1
Online ISBN: 978-3-030-85476-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)