Abstract
We present a new Gaussian Process inference algorithm, called Online Sparse Matrix Gaussian Processes (OSMGP), and demonstrate its merits with a few vision applications. The OSMGP is based on the observation that for kernels with local support, the Gram matrix is typically sparse. Maintaining and updating the sparse Cholesky factor of the Gram matrix can be done efficiently using Givens rotations. This leads to an exact, online algorithm whose update time scales linearly with the size of the Gram matrix. Further, if approximate updates are permissible, the Cholesky factor can be maintained at a constant size using hyperbolic rotations to remove certain rows and columns corresponding to discarded training examples. We demonstrate that, using these matrix downdates, online hyperparameter estimation can be included without affecting the linear runtime complexity of the algorithm. The OSMGP algorithm is applied to head-pose estimation and visual tracking problems. Experimental results demonstrate that the proposed method is accurate, efficient and generalizes well using online learning.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Rasmussen, C.E., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: Advances in Neural Information Processing Systems, pp. 1259–1266 (2006)
Csato, L., Opper, M.: Sparse online gaussian processes. Neural Computation 14(2), 641–669 (2002)
Quinonero-Candela, J., Rasmussen, C., Williams, C.: Approximation methods for gaussian process regression. In: Large-Scale Kernel Machines, pp. 203–224. MIT Press, Cambridge (2007)
Hamers, B., Suykens, J., Moor, B.D.: Compactly Supported RBF Kernels for Sparsifying the Gram Matrix in LS-SVM Regression Models. In: Proceedings of the International Conference on Artificial Neural Networks, pp. 720–726 (2002)
Golub, G., Loan, C.V.: Matrix Computations. Johns Hopkins University Press (1996)
Kaess, M., Ranganathan, A., Dellaert, F.: Fast incremental square root information smoothing. In: Proceedings of International Joint Conference on Artificial Intelligence, pp. 2129–2134 (2007)
Davis, T., Gilbert, J., Larimore, S., Ng, E.: A column approximate minimum degree ordering algorithm. ACM Transactions on Mathematical Software 30(3), 353–376 (2004)
Kernighan, B., Lin, S.: An efficient heuristic procedure for partitioning graphs. The Bell System Technical Journal 49(2), 291–307 (1970)
Zhao, K., Fuyun, L., Lev-Ari, H., Proakis, J.: Sliding window order-recursive least-squares algorithms. IEEE Transactions on Acoustics, Speech, and Signal Processing 42(8), 1961–1972 (1994)
Bjorck, A., Park, H., Elden, L.: Accurate downdating of least-squares solutions. SIAM Journal on Matrix Analysis and Applications 15(2), 549–568 (1994)
Kruger, N., Potzsch, M., von der Malsburg, C.: Determination of face position and pose with a learned representation based on labeled graphs. Image and Vision Computing 15(8), 665–673 (1997)
Yang, R., Zhang, Z.: Model-based head pose tracking with stereo vision. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 242–247 (2002)
Yao, P., Evans, G., Calway, A.: Using affine correspondence to estimate 3D facial pose. In: Proceedings of IEEE International Conference on Image Processing, pp. 919–922 (2001)
Rae, R., Ritter, H.: Recognition of human head orientation based on artificial neural networks. IEEE Transactions on Neural Networks 9(2), 257–265 (1998)
Li, Y., Gong, S., Liddell, H.: Support vector regression and classification based multi-view face detection and recognition. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 300–305 (2000)
Williams, O., Blake, A., Cipolla, R.: Sparse and semi-supervised visual mapping with the S3GP. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 230–237 (2006)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 511–518 (2001)
Ross, D., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. International Journal of Computer Vision 1–3, 125–141 (2008)
Thayananthan, A., Navaratnam, R., Stenger, B., Torr, P., Cipolla, R.: Multivariate relevance vector machines for tracking. In: Proceedings of European Conference on Computer Vision, vol. 3, pp. 124–138 (2006)
Lawrence, N.: Gaussian process latent variable models for visualization of high dimensional data. In: Advances in Neural Information Processing Systems, pp. 329–336 (2004)
Williams, O., Blake, A., Cipolla, R.: Sparse bayesian regression for efficient visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(8), 1292–1304 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ranganathan, A., Yang, MH. (2008). Online Sparse Matrix Gaussian Process Regression and Vision Applications. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5302. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88682-2_36
Download citation
DOI: https://doi.org/10.1007/978-3-540-88682-2_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88681-5
Online ISBN: 978-3-540-88682-2
eBook Packages: Computer ScienceComputer Science (R0)