Abstract
Directed information is an information theoretic measure which accounts for the direction of information flow over causal systems with feedback, such as network communication and communication for control problems. In this chapter, we discuss several functional and topological properties of directed information for general Polish spaces (complete separable metric spaces) using the topology of weak convergence of probability measures. These include, convexity/concavity of directed information, weak compactness of families of causally conditioned convolutional distributions, lower semicontinuity of directed information, continuity of directed information, and extremum problems of directed information, including variational equalities [utilized in Blahut–Arimoto algorithm (BAA)], which are important in nonanticipative or real-time joint source-channel coding (JSCC). These basic functional and topological properties of directed information are analogous to those of mutual information. Throughout the chapter, the importance of the properties of directed information is discussed in the context of extremum problems of directed information, such as point to point and network applications.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Marko H (1973) The bidirectional communication theory—a generalization of information theory. IEEE Trans Commun 21(12):1345–1351
Massey JL (1990) Causality, feedback and directed information. In: International symposium on information theory and its applications (ISITA), 27–30 Nov, pp 303–305
Kramer G (1998) Directed information for channels with feedback. PhD thesis, Swiss Federal Institute of Technology, Zurich, Switzerland
Charalambous CD, Stavrou PA, Ahmed NU (2014) Nonanticipative rate distortion function and relations to filtering theory. IEEE Trans Autom Control 59(4):937–952
Charalambous CD, Stavrou PA (2014) Optimization of directed information and relations to filtering theory. In: European control conference (ECC) (to appear). Strasbourg, France, pp 24–27
Csiszár I, Körner J (1981) Information theory: coding theorems for discrete memoryless systems. Academic Press, New York
Charalambous CD, Stavrou PA (2013) Directed information on abstract spaces: properties and variational equalities. IEEE Trans Inf Theo (Online). Available: http://arxiv.org/abs/1302.3971
Stavrou PA, Kourtellaris CK, Charalambous CD (2014) Applications of information nonanticipative rate distortion function. In: IEEE international symposium on information theory (ISIT) (to appear), Honolulu, HI, USA, 29 June–5 July 2014 (Online). Available: http://arxiv.org/abs/1401.5828v4
Charalambous CD, Stavrou PA (2012) Directed information on abstract spaces: properties and extremum problems. In: IEEE international symposium on information theory (ISIT), Cambridge, MA, USA, 1-â 6 July 2012, pp 518–522
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Charalambous, C.D., Stavrou, P.A., Kourtellaris, C.K. (2015). Directed Information on Abstract Spaces: Properties and Extremum Problems. In: van Schuppen, J., Villa, T. (eds) Coordination Control of Distributed Systems. Lecture Notes in Control and Information Sciences, vol 456. Springer, Cham. https://doi.org/10.1007/978-3-319-10407-2_36
Download citation
DOI: https://doi.org/10.1007/978-3-319-10407-2_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10406-5
Online ISBN: 978-3-319-10407-2
eBook Packages: EngineeringEngineering (R0)