Advertisement

International Journal of Computer Vision

, Volume 89, Issue 1, pp 84–105 | Cite as

Unified Direct Visual Tracking of Rigid and Deformable Surfaces Under Generic Illumination Changes in Grayscale and Color Images

  • Geraldo SilveiraEmail author
  • Ezio Malis
Article

Abstract

The fundamental task of visual tracking is considered in this work as an incremental direct image registration problem. Direct methods refer to those that exploit the pixel intensities without resorting to image features. We propose new transformation models and optimization methods for directly and robustly registering images (including color ones) of rigid and deformable objects, all in a unified manner. We also show that widely adopted models are in fact particular cases of the proposed ones. Indeed, the proposed general models combine various classes of image warps and ensure robustness to generic lighting changes. Finally, the proposed optimization method together with the exploitation of all possible image information allow the algorithm to achieve high levels of accuracy. Extensive experiments are reported to demonstrate that visual tracking can indeed be highly accurate and robust despite deforming objects and severe illumination changes.

Keywords

Visual tracking Image registration Nonlinear optimization Direct methods Nonrigid objects Lighting changes Robust techniques Robotics 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

Below is the link to the electronic supplementary material. (3577KB)

Below is the link to the electronic supplementary material. (24170KB)

Below is the link to the electronic supplementary material. (16757KB)

Below is the link to the electronic supplementary material. (20219KB)

Below is the link to the electronic supplementary material. (1139KB)

Below is the link to the electronic supplementary material. (6023KB)

Below is the link to the electronic supplementary material. (19308KB)

Below is the link to the electronic supplementary material. (3831KM)

Below is the link to the electronic supplementary material. (1357KB)

References

  1. Baker, S., & Matthews, I. (2001). Equivalence and efficiency of image alignment algorithms. In IEEE computer vision and pattern recognition (pp. 1090–1097). Google Scholar
  2. Baker, S., & Matthews, I. (2004). Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56(3), 221–255. CrossRefGoogle Scholar
  3. Bartoli, A. (2008). Groupwise geometric and photometric direct image registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(12), 2098–2108. CrossRefGoogle Scholar
  4. Bartoli, A., & Zisserman, A. (2004). Direct estimation of non-rigid registration. In Proc. of the British machine vision conference (pp. 899–908). Google Scholar
  5. Benhimane, S., & Malis, E. (2007). Homography-based 2D visual tracking and servoing. International Journal of Robotics Research, 26(7), 661–676. Special Issue on Vision and Robotics joint with the International Journal of Computer Vision. CrossRefGoogle Scholar
  6. Black, M. J., Fleet, D. J., & Yacoob, Y. (2000). Robustly estimating changes in image appearance. Computer Vision and Image Understanding, 78, 8–31. CrossRefGoogle Scholar
  7. Blinn, J. F. (1977). Models of light reflection for computer synthesized pictures. In SIGGRAPH (pp. 192–198). Google Scholar
  8. Brown, L. G. (1992). A survey of image registration techniques. ACM Computing Surveys, 24, 325–376. CrossRefGoogle Scholar
  9. Carr, J., Fright, W., & Beatson, R. (1997). Surface interpolation with Radial Basis Functions for medical imaging. IEEE Transactions on Medical Imaging, 16(1). Google Scholar
  10. Comaniciu, D., Ramesh, V., & Meer, P. (2000). Real-time tracking of non-rigid objects using mean-shift. In IEEE computer vision and pattern recognition. Google Scholar
  11. Cook, R., & Torrance, K. (1982). A reflectance model for computer graphics. ACM Transactions on Graphics, 1, 7–24. CrossRefGoogle Scholar
  12. Faugeras, O., Luong, Q.-T., & Papadopoulo, T. (2001). The geometry of multiple images. Cambridge: MIT. zbMATHGoogle Scholar
  13. Finlayson, G., Drew, M., & Funt, B. (1994). Color constancy: Generalized diagonal transforms suffice. Journal of the Optical Society of America A, 11(11), 3011–3020. CrossRefGoogle Scholar
  14. Gouiffès, M., Collewet, C., Fernandez-Maloigne, C., & Trémeau, A. (2006). Feature points tracking using photometric model and colorimetric invariants. In Proc. Eur. conf. on colour in graph., imag., and vis. (pp. 18–23). Google Scholar
  15. Hager, G., & Belhumeur, P. (1998). Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10), 1025–1039. CrossRefGoogle Scholar
  16. Hartley, R., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge Univ. Press. zbMATHGoogle Scholar
  17. Haussecker, H. W., & Fleet, D. J. (2001). Computing optical flow with physical models of brightness variation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6). Google Scholar
  18. Horst, R., & Pardalos, P. M. (Eds.) (1995). Handbook of global optimization. Dordrecht: Kluwer. zbMATHGoogle Scholar
  19. Huber, P. J. (1981). Robust statistics. New York: Wiley. zbMATHCrossRefGoogle Scholar
  20. Irani, M., & Anandan, P. (1999). All about direct methods. In Proc. workshop on vision alg.: theory and practice. Google Scholar
  21. Jin, H., Favaro, P., & Soatto, S. (2001). Real-time feature tracking and outlier rejection with changes in illumination. In Proc. of the IEEE international conference on computer vision (pp. 684–689). Google Scholar
  22. Jin, H., Favaro, P., & Soatto, S. (2003). A semi-direct approach to structure from motion. The Visual Computer, 6, 377–394. CrossRefGoogle Scholar
  23. Jurie, F., & Dhome, M. (2002). Real time robust template matching. In Proc. of the British machine vision conference (pp. 123–131). Google Scholar
  24. Klinker, G. J., Shafer, S. A., & Kanade, T. (1990). The measurement of highlights in color images. International Journal of Computer Vision, 2, 7–32. CrossRefGoogle Scholar
  25. La Cascia, M., Sclaroff, S., & Athitsos, V. (2000). Fast, reliable head tracking under varying illumination: An approach based on robust registration of texture-mapped 3d models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 322–336. CrossRefGoogle Scholar
  26. Lai, S.-H., & Fang, M. (1999). Robust and efficient image alignment with spatially varying illumination models. In IEEE computer vision and pattern recognition. Google Scholar
  27. Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proc. of the int. joint conf. on art. intell. (pp. 674–679). Google Scholar
  28. Maintz, J. B., & Viergever, M. A. (1998). A survey of medical image registration. Medical Image Analysis, 2(1), 1–36. CrossRefGoogle Scholar
  29. Malis, E. (2004). Improving vision-based control using efficient second-order minimization techniques. In Proc. of the IEEE int. conf. on robotics and automation, USA. Google Scholar
  30. Malis, E. (2007). An efficient unified approach to direct visual tracking of rigid and deformable surfaces. In Proc. of the IEEE/RSJ int. conf. on intell. robots and syst., USA. Google Scholar
  31. Montesinos, P., Gouet, V., Deriche, R., & Pele, D. (1999). Matching color uncalibrated images using differential invariants. Image and Vision Computing, 18(9), 659–671. CrossRefGoogle Scholar
  32. Mégret, R., Authesserre, J.-B., & Berthoumieu, Y. (2008). The bi-directional framework for unifying parametric image alignment approaches. In Proc. of the European conference on computer vision. Google Scholar
  33. Nastar, C., Moghaddam, B., & Pentland, A. (1996). Generalized image matching: Statistical learning of physically-based deformations. In Proc. Eur. conf. on comp. vision. Google Scholar
  34. Negahdaripour, S. (1998). Revised definition of optical flow: Integration of radiometric and geometric cues for dynamic scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(9), 961–979. CrossRefGoogle Scholar
  35. Shum, H. Y., & Szeliski, R. (2000). Construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 16(1), 63–84. Google Scholar
  36. Silveira, G., & Malis, E. (2007a). Direct visual servoing with respect to rigid objects. In Proc. of the IEEE/RSJ int. conf. on intelligent robots and systems, USA. Google Scholar
  37. Silveira, G., & Malis, E. (2007b). Real-time visual tracking under arbitrary illumination changes. In IEEE computer vision and pattern recognition, USA. Google Scholar
  38. Silveira, G., Malis, E., & Rives, P. (2008). An efficient direct approach to visual SLAM. IEEE Transactions on Robotics, 24, 969–979. CrossRefGoogle Scholar
  39. Szeliski, R. (2005). Image alignment and stitching. In Paragios, N., Chen, Y., & Faugeras, O. (Eds.) Handbook of math. models in comp. vision (pp. 273–292). Berlin: Springer. Google Scholar
  40. Tan, R., & Ikeuchi, K. (2005). Separating reflection components of textured surfaces using a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2), 178–193. CrossRefGoogle Scholar
  41. Varadarajan, V. (1974). Lie groups, Lie algebras, and their representations. New York: Prentice-Hall. zbMATHGoogle Scholar
  42. Warner, F. W. (1987). Foundations of differential manifolds and Lie groups. Berlin: Springer. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.CTI Renato ArcherCampinasBrazil
  2. 2.INRIA Sophia-AntipolisSophia-Antipolis CedexFrance

Personalised recommendations