Abstract
This work examines then computational cost of processing the information required by Bayesian updating of beliefs. The standard statistical approach adopted by economists, restricted to the exponential family, ignores thesecomputational aspects. To fill this lacuna, two models of probabilisticreasoning are put forward: a model of associative memory and a well established tool of Artificial Intelligence called `Bayesian Networks'. These models are used to evaluate the time complexity and hence the computational cost. The associativememory model shows processing cost to be proportional to the entropy of the signal. This result is applied to classes of informationally equivalentsignals to characterise the least expensive signals within the class. TheBayesian Network Model comprises a graphical representation of the causaland/or probabilistic relations among the random variables that generate thesignal. According to this model, the computational cost depends on the sizeand connectivity of the graphical structure. The belief that the cost ofinference is monotonically increasing in its precision is shown incorrect.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arrow, K. (1992). Informational equivalence of signals. In Dasgupta et al. (eds.), Economic Analysis of Markets and Games. MIT Press.
Cooper, G.F. (1987). Probabilistic inference using Bayesian Networks is NP-hard. Research Report KSL27. Medical Computer Science Group. Stanford University.
Cover, T.M. and Thomas, J.A. (1991). Elements of Information Theory. Wiley and Sons.
Cowell, R., David, A.P., Lauritzen, S. and Spiegelhalter, D. (1999). Probabilistic Networks and Expert Systems. Springer.
Dawid, P. (1992). Applications of a general propagation algorithm for probabilistic expert systems. Statistics and Computing, 2, 25-36.
Elias, P. (1975). An informational-theoretic approach to computational complexity. In Csiszár and Elias (eds.). Topics in Information Theory. North-Holland.
Friedman, N. and Yakhini, Z. (1996). On the sample complexity of learning Bayesian Networks. Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence. Morgan-Kaufmann, San Francisco.
Heckerman, Dj. (1996). A tutorial on learning with Bayesian Networks. Technical Report n. MSRTR-95. Microsoft Research, Advanced Technology Division, Microsoft Corporation.
Huang, C. and Derwiche, A. (1994). Inference in Bayesian Networks: A procedural guide. International Journal of Applied Reasoning, 11, 1-158.
Jensen, F., Lauritzen, S. and Olesen, K. (1990). Bayesian updating in recursive graphical models by local computation. Computational Statistical Quarterly, 4, 269-282.
Kohonen, T. (1977). Associative Memory: A System-Theoretical Approach. Springer-Verlag Series in Communication and Cybernetics.
Lauritzen, S.L. and Spiegelhalter, D.J. (1988). Local computations with probabilities on graphical structures and their applications to expert systems. Journal of the Royal Statistic Society, B 50, 2, 154-227.
Marschack, J. and Miyasawa, K. (1968). Economic comparability of information systems. International Economic Review, 9 2, 137-174, June.
McGuire, C.B. (1972). Comparison of information structures. In McGuire and Radner (eds.). Decision and Organization. North-Holland, Amsterdam.
Morris, S.B., Cork, D. and Neapolitan, R.E. (1997). The cognitive processing of causal knowledge. Proceedings of the Thirteen Conference on Uncertainty in Artificial Intelligence. Morgan-Kaufmann, San Francisco.
Nermuth, M. (1982). Information Structures in Economics. Berlin, Springer-Verlag.
Pearl, J. (1986). Fusion, propagation and structuring in belief networks. Artificial Intelligence, 29 (3), 241-288.
Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan-Kaufmann, San Francisco.
Seiferas, J. (1990). Machine-independent complexity theory. In J. van Leeuwen (ed.). Algorithms and Complexity, Handbook of Theoretical Computer Science, Volume A. Elsevier, MIT Press.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Eboli, M. Two Models of Information Costs Based on Computational Complexity. Computational Economics 21, 87–105 (2003). https://doi.org/10.1023/A:1022291016063
Issue Date:
DOI: https://doi.org/10.1023/A:1022291016063