Relations Between Information and Estimation in the Presence of Feedback
We discuss some of the recent literature on relations between information- and estimation-theoretic quantities. We begin by exploring the connections between mutual information and causal/non-causal, matched/mismatched estimation for the setting of a continuous-time source corrupted by white Gaussian noise. Relations involving causal estimation, in both matched and mismatched cases, and mutual information persist in the presence of feedback. We present a new unified framework, based on Girsanov theory and Itô’s Calculus, to derive these relations. We conclude by deriving some new results using this framework.
KeywordsMutual Information Relative Entropy Estimation Loss Standard Brownian Motion Gaussian Channel
This research was supported by LCCC—Linnaeus Grant VR 2007-8646, Swedish Research Council.
- 15.Polyanskiy, Y., Poor, H.V., Verdú, S.: New channel coding achievability bounds. In: IEEE Int. Symposium on Information Theory 2008, Toronto, Ontario, Canada, 6–11 July 2008 Google Scholar
- 17.Steele, J.M.: Stochastic Calculus and Financial Applications. Springer, Berlin (2010) Google Scholar