Abstract
The expressive power of Gaussian process (GP) models comes at a cost of poor scalability in the size of the data. To improve their scalability, this paper presents an overview of our recent progress in scaling up GP models for large spatiotemporally correlated data through parallelization on clusters of machines, online learning, and nonmyopic active sensing/learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Active sensing/learning in machine learning is also known as adaptive sampling in oceanography and control [17].
- 2.
GP regression in machine learning is equivalent to the data assimilation scheme called objective analysis or optimal interpolation or 3DVAR in oceanography and meteorology [2, 17] when the domain is reduced to a finite set of grid points and all observations are at the grid points. It is also equivalent to kriging in geostatistics [8].
- 3.
One of the N machines can be assigned to be the master.
References
Cao, N., Low, K.H., Dolan, J.M.: Multi-robot informative path planning for active sensing of environmental phenomena: a tale of two algorithms. In: Proceedings of AAMAS, pp. 7–14 (2013)
Chao, Y., Li, Z., Farrara, J.D., Hung, P.: Blending sea surface temperatures from multiple satellites and in situ observations for coastal oceans. J. Atmos. Ocean. Technol. 26(7), 1415–1426 (2009)
Chen, J., Cao, N., Low, K.H., Ouyang, R., Tan, C.K.Y., Jaillet, P.: Parallel Gaussian process regression with low-rank covariance matrix approximations. In: Proceedings of UAI (2013)
Chen, J., Low, K.H., Tan, C.K.Y.: Gaussian process-based decentralized data fusion and active sensing for mobility-on-demand system. In: Proceedings of RSS (2013)
Chen, J., Low, K.H., Tan, C.K.Y., Oran, A., Jaillet, P., Dolan, J.M., Sukhatme, G.S.: Decentralized data fusion and active sensing with mobile sensors for modeling and predicting spatiotemporal traffic phenomena. In: Proceedings of UAI, pp. 163–173 (2012)
Choudhury, A., Nair, P.B., Keane, A.J.: A data parallel approach for large-scale Gaussian process modeling. In: Proceedings of SDM, pp. 95–111 (2002)
Cressie, N., Johannesson, G.: Fixed rank kriging for very large spatial data sets. J. R. Stat. Soc. B 70(1), 209–226 (2008)
Cressie, N., Wikle, C.K.: Statistics for Spatio-Temporal Data. Wiley, Hoboken (2011)
Csató, L., Opper, M.: Sparse online Gaussian processes. Neural Comput. 14, 641–669 (2002)
Dolan, J.M., Podnar, G., Stancliff, S., Low, K.H., Elfes, A., Higinbotham, J., Hosler, J.C., Moisan, T.A., Moisan, J.: Cooperative aquatic sensing using the telesupervised adaptive ocean sensor fleet. In: Proceedings of SPIE Conference on Remote Sensing of the Ocean, Sea Ice, and Large Water Regions, vol. 7473 (2009)
Furrer, R., Genton, M.G., Nychka, D.: Covariance tapering for interpolation of large spatial datasets. JCGS 15(3), 502–523 (2006)
Hoang, T.N., Low, K.H., Jaillet, P., Kankanhalli, M.: Nonmyopic \(\epsilon \)-Bayes-optimal active learning of Gaussian processes. In: Proceedings of ICML, pp. 739–747 (2014)
Hoang, T.N., Low, K.H., Jaillet, P., Kankanhalli, M.: Active learning is planning: nonmyopic \(\epsilon \)-Bayes-Optimal active learning of Gaussian processes. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014, Part III. LNCS, vol. 8726, pp. 494–498. Springer, Heidelberg (2014)
Ingram, B., Cornford, D.: Parallel geostatistics for sparse and dense datasets. In: Atkinson, P.M., Lloyd, C.D. (eds.) geoENV VII. Quantitative Geology and Geostatistics, vol. 16, pp. 371–381. Springer, Netherlands (2010)
Krause, A., Guestrin, C.: Nonmyopic active learning of Gaussian processes: an exploration-exploitation approach. In: Proceedings of ICML, pp. 449–456 (2007)
Krause, A., Singh, A., Guestrin, C.: Near-optimal sensor placements in Gaussian processes: theory, efficient algorithms and empirical studies. JMLR 9, 235–284 (2008)
Leonard, N.E., Palley, D.A., Lekien, F., Sepulchre, R., Fratantoni, D.M., Davis, R.E.: Collective motion, sensor networks, and ocean sampling. Proc. IEEE 95(1), 48–74 (2007)
Low, K.H., Dolan, J.M., Khosla, P.: Adaptive multi-robot wide-area exploration and mapping. In: Proceedings of AAMAS, pp. 23–30 (2008)
Low, K.H., Dolan, J.M., Khosla, P.: Information-theoretic approach to efficient adaptive path planning for mobile robotic environmental sensing. In: Proceedings of ICAPS, pp. 233–240 (2009)
Low, K.H., Dolan, J.M., Khosla, P.: Active Markov information-theoretic path planning for robotic environmental sensing. In: Proceedings of AAMAS, pp. 753–760 (2011)
Low, K.H., Podnar, G., Stancliff, S., Dolan, J.M., Elfes, A.: Robot boats as a mobile aquatic sensor network. In: Proceedings of IPSN-09 Workshop on Sensor Networks for Earth and Space Science Applications (2009)
Low, K.H., Xu, N., Chen, J., Lim, K.K., Özgül, E.B.: Generalized online sparse gaussian processes with application to persistent mobile robot localization. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014, Part III. LNCS, vol. 8726, pp. 499–503. Springer, Heidelberg (2014)
Martin, R.J.: Comparing and contrasting some environmental and experimental design problems. Environmetrics 12(3), 303–317 (2001)
Ouyang, R., Low, K.H., Chen, J., Jaillet, P.: Multi-robot active sensing of non-stationary Gaussian process-based environmental phenomena. In: Proceedings of AAMAS, pp. 573–580 (2014)
Park, C., Huang, J.Z., Ding, Y.: Domain decomposition approach for fast Gaussian process regression of large spatial data sets. JMLR 12, 1697–1728 (2011)
Podnar, G., Dolan, J.M., Low, K.H., Elfes, A.: Telesupervised remote surface water quality sensing. In: Proceedings of IEEE Aerospace Conference (2010)
Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. JMLR 6, 1939–1959 (2005)
Sang, H., Huang, J.Z.: A full scale approximation of covariance functions for large spatial data sets. J. R. Stat. Soc. B 74(1), 111–132 (2012)
Schwaighofer, A., Tresp, V.: Transductive and inductive methods for approximate Gaussian process regression. In: Proceedings of NIPS, pp. 953–960 (2002)
Snelson, E.: Local and global sparse Gaussian process approximations. In: Proceedings of AISTATS (2007)
Wikle, C.K.: Low-rank representations for spatial processes. In: Gelfand, A.E., Diggle, P., Guttorp, P., Fuentes, M. (eds.) Handbook of Spatial Statistics, pp. 107–118. Chapman and Hall, Boca Raton (2010)
Xu, N., Low, K.H., Chen, J., Lim, K.K., Özgül, E.B.: GP-Localize: persistent mobile robot localization using online sparse Gaussian process observation model. In: Proceedings of AAAI, pp. 2585–2592 (2014)
Acknowledgments
This work was supported by the Singapore-MIT Alliance for Research & Technology Subaward Agreements No. 41 and No. 52.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Low, K.H., Chen, J., Hoang, T.N., Xu, N., Jaillet, P. (2015). Recent Advances in Scaling Up Gaussian Process Predictive Models for Large Spatiotemporal Data. In: Ravela, S., Sandu, A. (eds) Dynamic Data-Driven Environmental Systems Science. DyDESS 2014. Lecture Notes in Computer Science(), vol 8964. Springer, Cham. https://doi.org/10.1007/978-3-319-25138-7_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-25138-7_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25137-0
Online ISBN: 978-3-319-25138-7
eBook Packages: Computer ScienceComputer Science (R0)