On-the-Fly Exact Computation of Bisimilarity Distances
This paper proposes an algorithm for exact computation of bisimilarity distances between discrete-time Markov chains introduced by Desharnais et. al. Our work is inspired by the theoretical results presented by Chen et. al. at FoSSaCS’12, proving that these distances can be computed in polynomial time using the ellipsoid method. Despite its theoretical importance, the ellipsoid method is known to be inefficient in practice. To overcome this problem, we propose an efficient on-the-fly algorithm which, unlike other existing solutions, computes exactly the distances between given states and avoids the exhaustive state space exploration. It is parametric in a given set of states for which we want to compute the distances. Our technique successively refines over-approximations of the target distances using a greedy strategy which ensures that the state space is further explored only when the current approximations are improved. Tests performed on a consistent set of (pseudo)randomly generated Markov chains shows that our algorithm improves, on average, the efficiency of the corresponding iterative algorithms with orders of magnitude.
KeywordsMarkov Chain Discount Factor Transportation Problem Exact Computation Greedy Strategy
Unable to display preview. Download preview PDF.
- 1.Baier, C., Katoen, J.P.: Principles of Model Checking. MIT Press (2008)Google Scholar
- 3.Chatterjee, K., de Alfaro, L., Majumdar, R., Raman, V.: Algorithms for Game Metrics. Logical Methods in Computer Science 6(3) (2010)Google Scholar
- 5.Comanici, G., Panangaden, P., Precup, D.: On-the-Fly Algorithms for Bisimulation Metrics. In: Proceedings of the 9th International Conference on Quantitative Analysis of Systems, QEST, September 17-20, pp. 94–103 (2012)Google Scholar
- 6.Comanici, G., Precup, D.: Basis function discovery using spectral clustering and bisimulation metrics. In: AAMAS 2011, vol. 3, pp. 1079–1080. International Foundation for Autonomous Agents and Multiagent Systems, Richland (2011)Google Scholar
- 7.Dantzig, G.B.: Application of the Simplex method to a transportation problem. In: Koopmans, T. (ed.) Activity Analysis of Production and Allocation, pp. 359–373. J. Wiley, New York (1951)Google Scholar
- 9.Ferns, N., Panangaden, P., Precup, D.: Metrics for finite Markov Decision Processes. In: Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, UAI, pp. 162–169. AUAI Press (2004)Google Scholar
- 13.Mitzenmacher, M., Upfal, E.: Probability and Computing - randomized algorithms and probabilistic analysis. Cambridge University Press (2005)Google Scholar
- 16.van Breugel, F., Sharma, B., Worrell, J.: Approximating a Behavioural Pseudometric without Discount for Probabilistic Systems. Logical Methods in Computer Science 4(2), 1–23 (2008)Google Scholar