The Transition from A Priori to A Posteriori Information: Bayesian Procedures in Distributed Large-Scale Data Processing Systems
- 21 Downloads
The procedure of transition from a priori to a posteriori information for a linear experiment in the context of Big Data systems is considered. At first glance, this process is fundamentally sequential, namely: as a result of observation, a priori information is transformed into a posteriori information, which is later interpreted as a priori for the next observation, etc. It is shown that such a procedure can be parallelized and unified due to the transformation of both the measurement results and the original a priori information into some special type. The properties of various forms of information representation are studied and compared. This approach makes it possible to effectively scale the Bayesian estimation procedure and, thus, adapt it to the problems of processing large amounts of distributed data.
KeywordsBig Data a priori and a posteriori information linear estimation canonical information distributed data collection and processing systems information algebra information space
Unable to display preview. Download preview PDF.
- 10.Oravecz, Z., Huentelman, M., and Vandekerckhove, J., Sequential Bayesian updating for Big Data, in Big Data in Cognitive Science, Jones, M.N., Ed., New York: Taylor & Francis, 2016, ch. 2, pp. 13–33.Google Scholar
- 11.Bekkerman, R., Bilenko, M., and Langford, J., Scaling up Machine Learning: Parallel and Distributed Approaches, New York: Cambridge University Press, 2012.Google Scholar
- 17.Robert, C.P., On the relevance of the Bayesian approach to statistics, Rev. Econ. Anal., 2010, vol. 2, no. 2, pp. 139–152.Google Scholar
- 21.White, T., Hadoop: The Definitive Guide, Sebastopol, CA: O’Reilly, 2015.Google Scholar
- 24.Ekanayake, J., Pallickara, S., and Fox, G., MapReduce for data intensive scientific analyses, Fourth IEEE International Conference on eScience, Indianapolis: IN, 2008, pp. 277–284.Google Scholar
- 25.Ryza, S., Laserson, U., Owen, S., and Wills, J., Advanced Analytics with Spark: Patterns for Learning from Data at Scale, Sebastopol, CA: O’Reilly, 2015.Google Scholar