Though we have access to a wealth of information, the main issue is always how to process the available information. How to make sense of all we observe and know. Just like the English alphabet: we know there are 26 letters but unless we put these letters together in a meaningful way, they convey no information. There are infinitely many ways of putting these letters together. Only a small number of those make sense. Only some of those convey exactly what we wish to convey though the message may be interpreted differently by different individuals. That same issue comes up with information: how can we process the information we have? How can we infer and reason under conditions of incomplete observed information? In his seminal book on the philosophy of information, Floridi (2011a) raises a number of open questions. I discuss here one of these questions. That question is how to process information. To do so, I take the more realistic view that information is always limited, incomplete and possibly noisy. I define types of information, relate it to Floridi’s definitions and discuss a basic formulation for processing information under a unified framework. I relate it to some of the basic concepts discussed in the book.
KeywordsEfficiency Entropy Information Generalized Maximum Entropy Generalized Cross Entropy Maximum Entropy Noise Truth
I thank Luciano Floridi and Ariel Caticha for many enchanting conversations on the topics discussed here, and for providing me with comments on earlier versions of this paper. I also benefited from comments during recent seminars on the topic. Finally, I thank Patrick Allo for his thoughtful comments, and to the Editor, Tony Beavers.
- Adriaans, P. (2011). Some open problems in the study of information and computation. http://staff.science.uva.nl/~pietera/open_problems.html.
- Adriaans, P. (forthcoming). Philosophy of Information. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy (Fall 2013 Edition).Google Scholar
- Caticha, A. (2012). Entropic inference and the foundations of physics (monograph commissioned by the 11th Brazilian Meeting on Bayesian Statistics). EBEB-2012, USP Press, São Paulo, Brazil. Online at http://www.albany.edu/physics/ACaticha-EIFP-book.pdf.
- Floridi, L. (2011b). Semantic conceptions of information. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy (Spring 2013 Edition). Google Scholar
- Golan, A. (1994). A multivariable stochastic theory of size distribution of firms with empirical evidence. Advances in Econometrics, 10, 1–46.Google Scholar
- Golan, A. (2008). Information and entropy econometrics—a review and synthesis. Foundations and Trends® in Econometric, 2(1–2), 1–145.Google Scholar
- Golan, A. (2012). On the Foundations and Philosophy of Info-Metrics. In Lecture Notes in Computer Science (LNCS), 2012, Vol. 7318.Google Scholar
- Golan A., & Dose, V. (2001). A generalized information theoretical approach to tomographic reconstruction. Journal of Physics A: Mathematical and General, 34, 1271–1283. Google Scholar
- Hartley, R. V. L. (1928). Transmission of information. Bell System Technical Journal, 7(3), 535–563.Google Scholar
- Proceedings. (2011). Info-Metrics Institute Workshop on the Philosophy of Information. American University, Washington, DC. http://www.american.edu/cas/economics/info-metrics/workshop/proceedings-2011-october.cfm
- van Benthem, J., & Adriaans, P. (2008). Philosophy of information. Amsterdam: North Holland.Google Scholar
- Zellner, A. (1988). Optimal information processing and bayes theorem. American Statistician, 42, 278–284.Google Scholar
- Zellner, A. (2002). Information processing and Bayesian analysis. Journal of Econometrics, 107, 41–50.Google Scholar