Abstract
In this chapter, we explore the surprising result that gradient-based continuous optimization methods perform well for the alignment of image/object models when using densely sampled sparse features (HOG, dense SIFT, etc.). Gradient-based approaches for image/object alignment have many desirable properties—inference is typically fast and exact, and diverse constraints can be imposed on the motion of points. However, the presumption that gradients predicted on sparse features would be poor estimators of the true descent direction has meant that gradient-based optimization is often overlooked in favor of graph-based optimization. We show that this intuition is only partly true: sparse features are indeed poor predictors of the error surface, but this has no impact on the actual alignment performance. In fact, for general object categories that exhibit large geometric and appearance variation, sparse features are integral to achieving any convergence whatsoever. How the descent directions are predicted becomes an important consideration for these descriptors. We explore a number of strategies for estimating gradients, and show that estimating gradients via regression in a manner that explicitly handles outliers improves alignment performance substantially. To illustrate the general applicability of gradient-based methods to the alignment of challenging object categories, we perform unsupervised ensemble alignment on a series of nonrigid animal classes from ImageNet.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In our experiments we actually estimate our linearization function from the template image \(\mathcal{T} (\mathbf{0}) \rightarrow \nabla \mathcal{T} (\mathbf{0})\) using a technique commonly known within LK literature as the inverse compositional approach. This was done due to the substantial computational benefit enjoyed by the inverse compositional approach, since one can estimate \(\mathcal{T} (\mathbf{0}) \rightarrow \nabla \mathcal{T} (\mathbf{0})\) once, as opposed to the classical approach of estimating \(\mathcal{R}(\mathbf{p}) \rightarrow \nabla \mathcal{R}(\mathbf{p})\) at each iteration. See [2, 3] for more details.
- 2.
We removed those elephants whose out-of-plane rotation from the mean image could not be reasonably captured by an affine warp. The requirement of a single basis is a known limitation of the congealing algorithm.
References
Avidan, S.: Support vector tracking. Pattern Anal. Mach. Intell. (PAMI) 26(8), 1064–72 (2004)
Baker, S., Matthews, I.: Lucas-Kanade 20 years on: a unifying framework: part 1. Int. J. Comput. Vis. 56(3), 221–255 (2004)
Baker, S., Matthews, I.: Lucas-Kanade 20 years on: a unifying framework: part 2. Int. J. Comput. Vis. (IJCV) 56(3), 221–255 (2004)
Black, M.J., Jepson, A.: Eigentracking: Robust matching and tracking of articulated objects using a view-based representation. Int. J. Comput. Vis. (IJCV) 26(1), 63–84 (1998)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 6, 681–685 (2001)
Cox, M., Sridharan, S., Lucey, S.: Least-squares congealing for large numbers of images. In: International Conference on Computer Vision (ICCV) (2009)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference of Computer Vision and Pattern Recognition (CVPR), pp. 886–893 (2005)
Dowson, N., Bowden, R.: Mutual information for Lucas-Kanade Tracking (MILK): an inverse compositional formulation. IEEE Trans. Pattern Anal. Mach. Intell. 30(1), 180–5 (2008)
Drucker, H., Burges, C., Kaufman, L., Smola, A., Vapnik, V.: Support vector regression machines. In: Advances in Neural Information Processing Systems (NIPS), pp. 155–161 (1997)
Felzenszwalb, P.F.: Representation and detection of deformable shapes. Pattern Anal. Mach. Intell. (PAMI) 27(2), 208–20 (2005)
Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. Pattern Anal. Mach. Intell. (PAMI) 32(9), 1627–45 (2010)
Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)
Liu, C., Yuen, J., Torralba, A.: SIFT flow: dense correspondence across scenes and its applications. Pattern Anal. Mach. Intell. (PAMI) 33(5), 978–94 (2011)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. (IJCV) 60(2), 91–110 (2004)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence (IJCAI) (1981)
Rakêt, L., Roholm, L., Nielsen, M., Lauze, F.: TV-L 1 optical flow for vector valued images. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 1–14 (2011)
Ramakrishna, V., Munoz, D., Hebert, M., Bagnell, J.A., Sheikh, Y.: Pose machines: articulated pose estimation via inference machines. In: European Conference on Computer Vision (ECCV), pp. 1–15 (2014)
Saragih, J., Lucey, S., Cohn, J.: Face alignment through subspace constrained mean-shifts. In: International Conference on Computer Vision (ICCV) (2009)
Saragih, J.M., Lucey, S., Cohn, J.F.: Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vis. (IJCV) 91(2), 200–215 (2011)
Simoncelli, E.P., Olshausen, B.A.: Natural image statistics and neural representation. Annu. Rev. Neurosci. 24(1), 1193–1216 (2001)
Tsagkatakis, G., Tsakalides, P., Woiselle, A.: Compressed sensing reconstruction of convolved sparse signals. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014)
Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 4, 51–52 (2001)
Weber, J., Malik, J.: Robust computation of optical flow in a multi-scale differential framework. Int. J. Comput. Vis. (IJCV) 81, 67–81 (1995)
Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: DeepFlow: large displacement optical flow with deep matching. In: International Conference on Computer Vision (ICCV), pp. 1385–1392 (2013)
Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: International Conference of Computer Vision and Pattern Recognition (CVPR), pp. 532–539 (2013)
Yuille, A.: Deformable templates for face recognition. J. Cogn. Neurosci. 3(1), 59–70 (1991)
Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: International Conference of Computer Vision and Pattern Recognition (CVPR), pp. 2879–2886 (2012)
Zimmermann, K., Matas, J., Svoboda, T.: Tracking by an optimal sequence of linear predictors. Pattern Anal. Mach. Intell. (PAMI) 31(4), 677–92 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Bristow, H., Lucey, S. (2016). In Defense of Gradient-Based Alignment on Densely Sampled Sparse Features. In: Hassner, T., Liu, C. (eds) Dense Image Correspondences for Computer Vision. Springer, Cham. https://doi.org/10.1007/978-3-319-23048-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-23048-1_7
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23047-4
Online ISBN: 978-3-319-23048-1
eBook Packages: EngineeringEngineering (R0)