Variational Hilbert Regression with Applications to Terrain Modeling

  • Vitor Campanholo GuiziliniEmail author
  • Fabio Tozeto Ramos
Conference paper
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 10)


The ability to generate accurate terrain models is of key importance in a wide variety of robotics tasks, ranging from path planning and trajectory optimization to environment exploration and mining applications. This paper introduces a novel regression methodology for terrain modeling that takes place in a Reproducing Kernel Hilbert Space, and can approximate arbitrarily complex functions using Variational Bayesian inference. A sparse kernel is used to efficiently project input points into a high-dimensional feature vector, based on cluster information generated automatically from training data. Each kernel maintains its own regression model, and the entire set is simultaneously optimized in an iterative fashion as more data is collected, to maximize a global variational bound. Additionally, we show how kernel parameters can be jointly learned alongside the regression model parameters, to achieve a better approximation of the underlying function. Experimental results show that the proposed methodology consistently outperforms current state-of-the-art techniques, while maintaining a fully probabilistic treatment of uncertainties and high scalability to large-scale datasets.



This research was supported by funding from the Faculty of Engineering and Information Technologies, The University of Sydney, under the Faculty Research Cluster Program.


  1. 1.
    Arthur, D., Vassilvitskii, S.: K-means++: the advantages of careful seeding. In: Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 1027–1035 (2007)Google Scholar
  2. 2.
    Bishop, C.: Pattern Recognition and Machine Learning (Information and Statistics). Springer, New York (2006)zbMATHGoogle Scholar
  3. 3.
    Broderick, T., Boyd, N., Wibisono, A., Wilson, A., Jordan, M.: Streaming variational Bayes. In: Advances in Neural Information Processing Systems (NIPS), pp. 1727–1735 (2013)Google Scholar
  4. 4.
    Carlin, B., Louis, T.: Bayesian Methods for Data Analysis, 3rd edn. Chapman and Hall, London (2008)Google Scholar
  5. 5.
    Doherty, K., Wang, J., Englot, B.: Probabilistic map fusion for fast, incremental occupancy mapping with 3d Hilbert maps. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2016)Google Scholar
  6. 6.
    Drugowitsch, J.: Variational Bayesian inference for linear and logistic regression. Technical report (2013). arXiv:1310.5438
  7. 7.
    Frankhauser, P., Bloesch, M., Gehring, C., Hutter, M., Siegwart, R.: Robot-centric elevation mapping with uncertainty estimates. In: Proceedings of the International Conference on Climbing and Walking Robots (CLAWAR) (2014)Google Scholar
  8. 8.
    Gilks, W., Richardson, S., Spiegelhalter, D.: Markov Chain Monte Carlo in Practice, 1st edn. Chapman and Hall, London (1995)Google Scholar
  9. 9.
    Gu, J., Cao, Q., Huang, Y.: Rapid traversability assessment in 2.5d grid-based map on rough terrain. Int. J. Adv. Robot. Syst. 5(4) (2008)Google Scholar
  10. 10.
    Guizilini, V., Ramos, F.: Large-scale 3d scene reconstruction with Hilbert maps. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS) (2016)Google Scholar
  11. 11.
    Guizilini, V., Ramos, F.: Unsupervised feature learning for 3d scene reconstruction with occupancy maps. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)Google Scholar
  12. 12.
    Hadsell, R., Bagnell, J., Huber, D., Hebert, M.: Space-carving kernels for accurate rough terrain estimation. Int. J. Robot. Res. (IJRR) 29(8), 981–996 (2010)CrossRefGoogle Scholar
  13. 13.
    Honkela, A., Valpola, H.: On-line variational bayesian learning. In: International Symposium on Independent Component Analysis and Blind Signal Separation (ICA) (2003)Google Scholar
  14. 14.
    Kidner, D., Dorey, M., Smith, D.: What’s the point? interpolation and extrapolation with a regular grid dem. In: Proceedings of the International Conference on Geocomputation (1999)Google Scholar
  15. 15.
    Krotkov, E., Hoffman, R.: Terrain mapping for a walking planetary rover. IEEE Trans. Robot. Autom. 10(6), 278–292 (1994)CrossRefGoogle Scholar
  16. 16.
    Lacroix, S., Mallet, A., Bonnafous, D., Bauzil, G., Fleury, S., Herrb, M., Chatila, R.: Autonomous rover navigation on unknown terrains: functions and integration. Int. J. Robot. Res. (IJRR) 21(10), 917–942 (2002)CrossRefGoogle Scholar
  17. 17.
    Lloyd, S.: Least-squares quantization in PCM. IEEE Trans. Inf. Theory 28, 129–136 (1982)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Matheron, G.: Principles of geostatistics. Econ. Geol. 58, 1246–1266 (1963)CrossRefGoogle Scholar
  19. 19.
    Melkumyan, A., Ramos, F.: A sparse covariance function for exact Gaussian process inference in large datasets. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 1936–1942 (2009)Google Scholar
  20. 20.
    Muja, M., Lowe, D.: Fast approximate nearest neighbours with automatic algorithm configuration. In: Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), vol. 4, pp. 331–340 (2009)Google Scholar
  21. 21.
    Murphy, K.: Machine Learning: A Probabilistic Perspective, 1st edn. The MIT Press, Cambridge (2012)Google Scholar
  22. 22.
    Pébay, P.: Formulas for robust, one-pass parallel computation of covariances and arbitrary-order statistical moments. Technical report, Sandia National Laboratories (2008)CrossRefGoogle Scholar
  23. 23.
    Plagemann, C., Kersting, K., Burgard, W.: Nonstationary Gaussian process regression using point estimates of local smoothness. In: Proceedings of the European Conference on Machine Learning (ECML) (2008)Google Scholar
  24. 24.
    Plagemann, C., Mischke, S., Prentice, S., Kersting, K., Roy, N., Burgard, W.: Learning predictive terrain models for legged robot locomotion. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS) (2008)Google Scholar
  25. 25.
    Quinonero-Candelas, J., Rasmussen, C.: An unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. (JMLR) 6, 1939–1959 (2005)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Ramos, F., Ott, L.: Hilbert maps: scalable continuous occupancy mapping with stochastic gradient descent. In: Proceedings of Robotics: Science and Systems (RSS) (2015)Google Scholar
  27. 27.
    Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2005)Google Scholar
  28. 28.
    Rekleitis, I., Bedwani, J., Gingras, D., Dupuis, E.: Experimental results for over-the-horizon planetary exploration using a LIDAR sensor (2008)Google Scholar
  29. 29.
    Sato, M.: Online model selection based on the variational Bayes. Neural Comput. 13(7), 1649–1681 (2001)CrossRefGoogle Scholar
  30. 30.
    Schölkopf, B., Muandet, K., Fukumizu, K., Harmeling, S., Peters, J.: Computing functions of random variables via reproducing kernel Hilbert space representations. Stat. Comput. 25(4), 755–766 (2015)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge (2001)Google Scholar
  32. 32.
    Sculley, D.: Web-scale k-means clustering. In: Proceedings of the International Conference on World Wide Web (WWW), vol. 19, pp. 1177–1178 (2010)Google Scholar
  33. 33.
    Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: Advances in Neural Information Processing Systems (NIPS), pp. 1257–1264 (2006)Google Scholar
  34. 34.
    Triebel, R., Pfaff, P., Burgard, W.: Multi-level surface maps for outdoor terrain mapping and loop closing. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS) (2006)Google Scholar
  35. 35.
    Vasudevan, S., Ramos, F., Nettleton, E., Durrant-Whyte, H.: Gaussian process modeling of large-scale terrain. J. Field Robot. (JFR) 26(10), 812–840 (2010)CrossRefGoogle Scholar
  36. 36.
    Yuan, M., Cai, T.: A reproducing kernel Hilbert space approach to functional linear regression. Ann. Stat. 38(6), 3412–3444 (2010)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Vitor Campanholo Guizilini
    • 1
    Email author
  • Fabio Tozeto Ramos
    • 1
  1. 1.School of Information TechnologiesThe University of SydneySydneyAustralia

Personalised recommendations