Abstract
Fisher information generally decreases by summarizing observed data into encoded messages. The present paper studies the amount of Fisher information included in independently summarized messages from correlated information sources; that is, the amount of Fisher information when sequences x N and y N of N independent observations of random variables x and y are encoded (summarized) independently of each other into meassages m X and m Y . The problem is to obtain the maximal amount of Fisher information when the size of the summarized data or Shannon message information is limited. The problem is solved in the case of completely compressed symmetric data summarization. An achievable bound is given in the general case. Information geometry, which is a powerful new differential geometrical method applicable to statistics and systems theory, is applied to this problem, proving its usefulness in information theory as well.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahlswede, R. and Burnashev, M. V. (1989). On minimax estimation in the presence of side information about remote data, Ann. Probab. (to appear).
Ahlswede, R. and Csiszár, I. (1986). Hypothesis testing with communication, IEEE Trans. Inform. Theory. 32, 533–542.
Amari, S. (1982a). Differential geometry of curved exponential families—curvatures and information loss. Ann. Statist., 10, 357–385.
Amari, S. (1982b). Geometrical theory of asymptotic ancillarity and conditional inference, Biometrika. 69–1, 1–17.
Amari, S. (1985). Differential geometrical methods in statistics, Leeture Notes in Statisties, 28, Springer, Berlin-Heidelberg.
Amari, S. (1986). Data compression and statistical inference, J. Inst. Electr. & Comm. Eng. Japan, J 69-A, 757–763 (in Japanese).
Amari, S. (1987a). Differential geometrical theory of statistics, Differential Geometry in Statistical Inference, IMS Monograph Series, 10, 19–94, IMS, California.
Amari, S. (1987b). Differential geometry of a parametric family of invertible linear systems—Riemannian metric, dual affine connections, and divergence, Math. Systems Theory, 20, 53–82.
Amari, S. and Han, T. S. (1989). Statistical inference under multi-terminal rate restrictions—a differential geometrical approach, IEEE Trans. Inform. Theory, 35, 217–227.
Amari, S. and Kumon, M. (1983). Differential geometry of Edgeworth expansions in curved exponential family, Ann. Inst. Statist. Math., 35, 1–24.
Campbell, L. L. (1985). The relation between information theory and the differential geometry approach to statistics, Inform. Sci., 35, 199–210.
Han, T. S. (1987). Hypothesis testing with multi-terminal data compression, IEEE Trans. Inform. Theory, 33, 759–772.
Kumon, M. and Amari, S. (1983). Geometrical theory of higher-order asymptotics of test, interval estimator and conditional inference, Proc. Roy. Soc. London Ser. A, 387, 429–458.
Nagaoka, H. and Amari, S. (1982). Differential geometry of smooth families of probability distributions, Tech. Report, METR 82-7, University of Tokyo.
Zhang, Z. and Berger, T. (1988). Estimation via compressed information, IEEE Trans. Inform. Theory, 34, 198–211.
Author information
Authors and Affiliations
Additional information
The present work is supported in part by Grant-in-Aid for Scientific Research #61030014, Ministry of Education, Science and Culture of Japan.
About this article
Cite this article
Amari, Si. Fisher information under restriction of Shannon information in multi-terminal situations. Ann Inst Stat Math 41, 623–648 (1989). https://doi.org/10.1007/BF00057730
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF00057730