Abstract
In this paper we consider the problem of representing and reasoning about systems, especially probabilistic systems, with hidden state. We consider transition systems where the state is not completely visible to an outside observer. Instead, there are observables that partly identify the state. We show that one can interchange the notions of state and observation and obtain what we call a dual system. In the case of deterministic systems, the double dual gives a minimal representation of the behaviour of the original system. We extend these ideas to probabilistic transition systems and to partially observable Markov decision processes (POMDPs).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arbib, M.A., Manes, E.G.: Machines in a category: An expository introduction. SIAM Review 16, 163–192 (1974)
Arbib, M.A., Manes, E.G.: Foundations of system theory: decomposable systems. Automatica 10, 285–302 (1974)
Arbib, M.A., Manes, E.G.: Adjoint machines, state behavior machines and duality. J. Pure Appl. Algebra 6, 313–343 (1975)
Bainbridge, E.S.: The fundamental duality of system theory. In: Hartnett, W.E. (ed.) Systems: Approaches, Theories, Applications, pp. 45–61. Reidel (1977)
Barr, M.: The chu construction. Theory Appl. Categories 2, 17–35 (1996)
Barr, M.: The separated extensional Chu category. Theory Appl. Categories 4(6), 137–147 (1998)
Bidoit, M., Hennicker, R., Kurz, A.: Observational logic, constructor-based logic, and their duality. Theor. Comput. Sci. 3(298), 471–510 (2003)
Bonchi, F., Bonsangue, M.M., Rutten, J.J.M.M., Silva, A.: Brzozowski’s Algorithm (Co)Algebraically. In: Constable, R.L., Silva, A. (eds.) Logic and Program Semantics, Kozen Festschrift. LNCS, vol. 7230, pp. 12–23. Springer, Heidelberg (2012)
Bonsangue, M.M., Kurz, A.: Duality for Logics of Transition Systems. In: Sassone, V. (ed.) FOSSACS 2005. LNCS, vol. 3441, pp. 455–469. Springer, Heidelberg (2005)
Brzozowski, J.A.: Canonical regular expressions and minimal state graphs for definite events. In: Fox, J. (ed.) Proceedings of the Symposium on Mathematical Theory of Automata. MRI Symposia Series, vol. 12, pp. 529–561. Polytechnic Press of the Polytechnic Institute of Brooklyn (April 1962), book appeared in 1963
Chrisman, L.: Reinforcement learning with perceptual aliasing: The perceptual distinctions approach. In: Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 183–188 (1992)
Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999)
Pavlovic, D., Mislove, M., Worrell, J.B.: Testing Semantics: Connecting Processes and Process Logics. In: Johnson, M., Vene, V. (eds.) AMAST 2006. LNCS, vol. 4019, pp. 308–322. Springer, Heidelberg (2006)
Desharnais, J., Edalat, A., Panangaden, P.: Bisimulation for labeled Markov processes. Information and Computation 179(2), 163–193 (2002)
Desharnais, J., Gupta, V., Jagadeesan, R., Panangaden, P.: A metric for labelled Markov processes. Theoretical Computer Science 318(3), 323–354 (2004)
Hopcroft, J.E., Motwani, R., Ullman, J.D.: Introduction to automata theory, languages, and computation, 2nd edn. Addison-Wesley Series in Computer Science. Addison-Wesley-Longman (2001)
Hundt, C., Panangaden, P., Pineau, J., Precup, D.: Representing systems with hidden state. In: The Twenty-First National Conference on Artificial Intelligence, AAAI (2006)
James, M., Singh, S.: Learning and discovery of predictive state representations in dynamical systems with reset. In: International Conference on Machine Learning, vol. 21, pp. 417–424 (2004)
Johnstone, P.: Stone Spaces, Cambridge Studies in Advanced Mathematics, vol. 3. Cambridge University Press (1982)
Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artificial Intelligence 101 (1998)
Kalman, R.E., Falb, P.L., Arbib, M.A.: Topics in Mathematical Systems Theory. McGraw Hill (1969)
Kozen, D.: A probabilistic PDL. Journal of Computer and Systems Sciences 30(2), 162–178 (1985)
Kozen, D.: Automata and computability. Undergraduate texts in computer science. Springer (1997)
Littman, M., Sutton, R., Singh, S.: Predictive representations of state. In: Advances in Neural Information Processing Systems 14 (NIPS), vol. 14, pp. 1555–1561 (2002)
McCallum, A.: Reinforcement Learning with Selective Perception and Hidden State. Ph.D. thesis, University of Rochester (1995)
Milner, R.: A Calculus of Communication Systems. LNCS, vol. 92. Springer, Heidelberg (1980)
Milner, R.: Communication and Concurrency. Prentice-Hall (1989)
Mislove, M., Ouaknine, J., Pavlovic, D., Worrell, J.: Duality for Labelled Markov Processes. In: Walukiewicz, I. (ed.) FOSSACS 2004. LNCS, vol. 2987, pp. 393–407. Springer, Heidelberg (2004)
Park, D.: Concurrency and Automata on Infinite Sequences. In: Deussen, P. (ed.) GI-TCS 1981. LNCS, vol. 104, pp. 167–183. Springer, Heidelberg (1981)
Park, D.: Title unknown (1981), slides for Bad Honnef Workshop on Semantics of Concurrency
Pratt, V.R.: The Duality of Time and Information. In: Cleaveland, W.R. (ed.) CONCUR 1992. LNCS, vol. 630, pp. 237–253. Springer, Heidelberg (1992)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley (1994)
Rivest, R.L., Schapire, R.E.: Diversity-based inference of finite automata. Journal of the ACM 41(3), 555–589 (1994)
Singh, S., James, M.R., Rudary, M.R.: Predictive state representations: A new theory for modeling dynamical systems. In: Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence (UAI), pp. 512–519 (2004)
Sipser, M.: Introduction to the theory of computation. PWS Publishing Company (1997)
Smyth, M.: Powerdomains and Predicate Transformers. In: Díaz, J. (ed.) ICALP 1983. LNCS, vol. 154, pp. 662–676. Springer, Heidelberg (1983)
Sutton, R.S., Tanner, B.: Temporal-difference networks. In: Advances in Neural Information Processing Systems, vol. 17 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dinculescu, M., Hundt, C., Panangaden, P., Pineau, J., Precup, D. (2013). The Duality of State and Observation in Probabilistic Transition Systems. In: Bezhanishvili, G., Löbner, S., Marra, V., Richter, F. (eds) Logic, Language, and Computation. TbiLLC 2011. Lecture Notes in Computer Science, vol 7758. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36976-6_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-36976-6_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-36975-9
Online ISBN: 978-3-642-36976-6
eBook Packages: Computer ScienceComputer Science (R0)