Skip to main content
SpringerLink
Log in

System-Theoretic Aspects of Dynamic Vision

  • Conference paper
Trends in Control

Abstract

Our aim in this minicourse is twofold. On one hand, we aim to show that the introduction of computer vision in control systems, i.e. “Vision in the Loop”, raises exciting and yet unexplored problems in system theory. On the other hand, we explain how tools from control and estimation theory are, nowadays applied [3, 6, 7, 26, 27, 38, 54, 57, 63, 66] to “Dynamic Vision” problems with rather encouraging results in traditionally difficult applications, such as autonomous vehicle navigation [17, 18, 19], vision-based tracking and servo [9, 35, 36, 50], vision-based manipulation [5, 35, 36], docking [18, 42], vision-based planning [12], active sensing [67]. The course is therefore divided in two parts. In the first part, we shall pose two fundamental dynamic vision problems, namely 3-D motion and scene structure recovery, in a system theoretical framework. In the second part, we shall illustrate some relevant applications of dynamic vision to control systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. E. Arbogast and R. Mohr. An egomotion algorithm based on the tracking of arbitrary curves. Proc. of the 2nd Europ. Conf. on Computer Vision, 1992.

    Google Scholar 

  2. H. Asada and J.J. Slotine. Robot Analysis and Control. John Wiley, 1987.

    Google Scholar 

  3. A. Azarbayejani, B. Horowitz, and A. Pentland. Recursive estimation of structure and motion using relative orientation constraints. Proc. CVPR, New York, 1993.

    Google Scholar 

  4. G. Bastin and M. Gevers. Stable adaptive observers for nonlinear time-varying systems. IEEE trans. Aut. Control, 1988.

    Google Scholar 

  5. A. Blake, M. Taylor, and A. Cox. Grasping visual simmetry. Proc. of the ICCV, 1993.

    Google Scholar 

  6. T. Broida and R. Chellappa. Estimating the kinematics and structure of a rigid object from a sequence of monocular frames. IEEE Trans. Pattern Anal. Mach. Intell., 1991.

    Google Scholar 

  7. T. Broida and R. Chellappa. Estimation of object motion parameters from noisy images. IEEE Trans. Pattern Anal. Mach. Intell., Jan. 1986.

    Google Scholar 

  8. Bryant, Chern, Goldberg, and Goldsmith. Exterior Differential Systems. Mathematical Research Institute. Springer Verlag, 1992.

    Google Scholar 

  9. F. Chaumette and A. Santos. Tracking a moving object by visual servoing. Proc. of the 12th IFAC World Congr. Vol. 9 pp. 409–414, 1993.

    Google Scholar 

  10. R. Cipolla and A. Blake. Surface orientation and time to crash from image divergence and deformation. Proc. of the European Conf. on Comp. Vision, 1992.

    Google Scholar 

  11. P. Crouch and F. Silva Leite. The dynamic interpolation problem on riemannian manifolds, lie groups and symmetric spaces. Technical report,

    Google Scholar 

  12. R. Curwen, A. Blake, and A. Zisserman. Real-time visual tracking for surveillance and path planning. Proc. of the ECCV, 1992.

    Google Scholar 

  13. Darmon. A recursive method to apply the hough transform to a set of moving objects. Proc. IEEE, CE 1746 7/82, 1982.

    Google Scholar 

  14. W. Dayawansa, B. Ghosh, C. Martin, and X. Wang. A necessary and sufficient condition for the perspective observability problem. Systems and Control Letters, in press, 1994.

    Google Scholar 

  15. C. Canudas de Wit et. al. Theory of Robot control. CNRS, France, 1992. Lect. notes of the Ecole d’ete’ d’automatique de Grenoble, sect. 6, Sept. 7–11 1992, CNRS, Grenoble France.

    Google Scholar 

  16. E. Di-Bernardo, L. Toniutti, R. Frezza, and G. Picci. Stima del moto dell’osservatore e della struttura della scena mediante visione monoculare. Tesi di Laurea-Universita di Padova, 1993.

    Google Scholar 

  17. E. D. Dickmanns and Th. Christians. Relative 3d-state estimation for autonomous visual guidance of road vehicles. In Intelligent autonomous system 2 (IAS-2), Amsterdam, 11–14 December 1989.

    Google Scholar 

  18. E. D. Dickmanns and V. Graefe. Applications of dynamic monocular machine vision. Machine Vision and Applications, 1:241–261, 1988.

    Article  Google Scholar 

  19. E. D. Dickmanns and V. Graefe. Dynamic monocular machine vision. Machine Vision and Applications, 1:223–240, 1988.

    Article  Google Scholar 

  20. B. Espiau, F. Chaumette, and P. Rives. A new approach to visual servoing in robotics. IEEE Trans, on Robotics and Autom., 8:313–326, 1992.

    Article  Google Scholar 

  21. O. Faugeras. What can be seen in three dimensions with an uncalibrated stereo rig. Proc. of the 2 ECCV, 1992.

    Google Scholar 

  22. O. Faugeras. Three dimensional vision, a geometric viewpoint. MIT Press, 1993.

    Google Scholar 

  23. O. D. Faugeras and S. Maybank. Motion from point mathces: multiplicity of solutions. Int. J. of Computer Vision, 1990.

    Google Scholar 

  24. O.D. Faugeras, Q.T. Luong, and S.J. Maybank. Camera self-calibration: theory and experiments. Proc. of the ECCV92, Vol. 588 of LNCS, Springer Verlag, 1992.

    Google Scholar 

  25. R. Frezza and G. Picci. On line path following by recursive spline updating. In submitted to 34th Conf. on Decision and Control, New Orleans, December 1995.

    Google Scholar 

  26. D.B. Gennery. Tracking known 3-dimensional object. In Proc. AAAI 2nd Natl. Conf. Artif. Intell, pages 13–17, Pittsburg, PA, 1982.

    Google Scholar 

  27. D.B. Gennery. Visual tracking of known 3-dimensional object. Int. J. of Computer Vision, 7(3):243–270, 1992.

    Article  Google Scholar 

  28. B. Ghosh, M. Jankovic, and Y. Wu. Perspective problems in systems theory and its application in machine vision. Journal of Math. Systems, Est. and Control, 1994.

    Google Scholar 

  29. B. K. Ghosh, E. P. Loucks, and M. Jankovic. An introduction to perspective observability and recursive identification problems in machine vision. In Proc. of the 33rd Conf. on Decision and Control, volume 4, pages 3229–3234, 1994.

    Google Scholar 

  30. B. K. Ghosh and J. Rosenthal. A generalized Popov Belevitch Hautus test of observability. IEEE Trans. Autom. Control, volume 40, pages 176–180, 1995.

    Article  MathSciNet  MATH  Google Scholar 

  31. G. Golub and C. Van Loan. Matrix computations. Johns Hopkins University Press, 2 edition, 1989.

    MATH  Google Scholar 

  32. R. M. Haralik and L. Shapiro. Computer and Robot Vision, volume 2. Addison-Wesley Publishing Company, Reading, MA, 1993.

    Google Scholar 

  33. R. Hartley. Estimation of relative camera positions for uncalibrated cameras. In Proc. 2 nd Europ. Conf. Comput. Vision, G. Sandini (Ed.), LNCS-Series Vol. 588, Springer-Verlag, 1992.

    Google Scholar 

  34. K. Hashimoto and H. Kimura. Dynamic visual servoing with nonlinear model based control. In Proc. of the 13th IFAC World Congress, volume 9, pages 405–408, 1993.

    Google Scholar 

  35. K. Hasimoto, T. Kimoto, T. Ebine, and H. Kimura. Image-based dynamic visual servo for a hand-eye manipulator. In Kodama Kimura, editor, Recent advances in mathematical theory of systems, control, networks, and signal processing II, pages 609–614. Proceedings of the international symposium of MTNS, Mita Press, 1991.

    Google Scholar 

  36. K. Hasimoto, T. Kimoto, T. Ebine, and H. Kimura. Manipulator control with image-based visual servo. In IEEE Intl’ Conference on Robotics and Automation, pages 2267–2272, 1991.

    Google Scholar 

  37. D. Heeger and A. Jepson. Subspace methods for recovering rigid motion i: algorithm and implementation. RBCV TR-90–35, University of Toronto — CS dept., November 1990. Revised July 1991.

    Google Scholar 

  38. J. Heel. Direct estimation of structure and motion from multiple frames. AI Memo 1190, MIT AI Lab, March 1990.

    Google Scholar 

  39. J. Heel. Temporal integration of 3-d surface reconstruction. To appear on IEEE trans. PAMI, special issue on the interpretation of 3-D scenes., March 1991.

    Google Scholar 

  40. L. Hinsken. A singukarity-free algorithm for spatial orientation of bundles. International Archives of Photogrammetry and Remote Sensing, 27, 1988.

    Google Scholar 

  41. C.C. Ho and N.H. McClamroch. Autonomous spacecraft docking using a computer vision system. Proc. of the 31st CDC Tucson AZ, 1992.

    Google Scholar 

  42. C.C. Ho and N.H. McClamrock. Autonomous spacecraft docking using a computer vision systm. Proc. of the 31st CDC — Tucson, AZ, 1992.

    Google Scholar 

  43. B. Horn. Robot Vision. MIT press, 1986.

    Google Scholar 

  44. B.K.P. Horn. Closed-form solution of absolute orientation using unit quaternions. Journ. of Optical Society of America A, 4(4):629–642, 1987.

    Article  MathSciNet  Google Scholar 

  45. B.K.P. Horn. Relative orientation. Int. J. of Computer Vision, 4:59–78, 1990.

    Article  Google Scholar 

  46. M. Jankovic and B. K. Ghosh. Visually guided ranging from observations of points, lines and curves via an identifier based nonlinear observer. -, June 30, 1993.

    Google Scholar 

  47. A.H. Jazwinski. Stochastic Processes and Filtering Theory. Academic Press, 1970.

    MATH  Google Scholar 

  48. O. Khatib. A unified approach for motion and force control of robot manipulators: the operational space formulation. IEEE Journ. of Robotics and Autom., RA-3:43–53, 1987.

    Article  Google Scholar 

  49. G. Heinzinger L. Noakes and B. Paden. Cubic splines on curved spaces. IMA J. Math. Control and Information, 1989.

    Google Scholar 

  50. M. Lei and B. K. Ghosh. A new nonlinear feedback controller for visually-guided robotic motion tracking. Proc. of the ECC, 1993.

    Google Scholar 

  51. H. C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133–135, 1981.

    Article  Google Scholar 

  52. H.C. Longuet-Higgins. Configurations that defeat the eight-point algorithm. Mental processes: studies in cognitive science, MIT press, 1987.

    Google Scholar 

  53. R. Marino. Adaptive observers for single output nonlinear systems. IEEE Trans. Aut. Control (35) 9, 1990.

    Google Scholar 

  54. L. Matthies, R. Szelisky, and T. Kanade. Kaiman filter-based algorithms for estimating depth from image sequences. Int. J. of computer vision, 1989.

    Google Scholar 

  55. S. Maybank. Theory of reconstruction from image motion, volume 28 of Information Sciences. Springer-Verlag, 1992.

    Google Scholar 

  56. R. Mohr. Projective reconstruction, in Geometric Invariance in Computer Vision, 1992.

    Google Scholar 

  57. J. Oliensis and J. Inigo-Thomas. Recursive multi-frame structure from motion incorporating motion error. Proc. DARPA Image Understanding Workshop, 1992.

    Google Scholar 

  58. C. Samson, M. Le Borgne, and B. Espiau. Robot Control, The Task Function Approach. Oxford Engineering science series, vol. 22, Clarendon Press, Oxford, 1991.

    Google Scholar 

  59. A. Shashua. Projective structure reconstruction. AI Memo MIT AI Lab, March 1993.

    Google Scholar 

  60. S. Soatto. Observability of rigid motion under perspective projection with application to visual motion estimation. Technical Report CIT-CDS 94–001, California Institute of Technology, 1994.

    Google Scholar 

  61. S. Soatto, R. Prezza, and P. Perona. Motion estimation via dynamic vision. Submitted to the IEEE Trans, on Automatic Control, 1994. Extended version in Technical Report CIT-CDS-94–004, California Institute of Technology.

    Google Scholar 

  62. S. Soatto, R. Prezza, and P. Perona. Recursive estimation of camera motion from uncalibrated image sequences. Technical Report CIT-CDS 94–003, California Institute of Technology. In the proc. of the first IEEE conf. on Image Processing, 1994.

    Google Scholar 

  63. S. Soatto, R. Prezza, and P. Perona. Recursive motion estimation on the essential manifold. In Proc. 3 rd Europ. Conf. Comput. Vision, J.-O. Eklundh (Ed.), LNCS-Series Vol. 800–801, Spring er-Verlag, pages 61–72, Stockholm, May 1994.

    Google Scholar 

  64. S. Soatto, R. Prezza, and P. Perona. Structure from visual motion as a nonlinear observation problem. To appear in the Proc. of the NOLCOS Conference, 1995.

    Google Scholar 

  65. S. Soatto and P. Perona. Structure-independent visual motion control on the essential manifold. In Proc. of the IFAC Symposium on Robot Control (SYROCO), Capri, Italy, Sept. 1994.

    Google Scholar 

  66. S. Soatto, P. Perona, R. Prezza, and G. Picci. Recursive motion and structure estimation with complete error characterization. In Proc. IEEE Comput. Soc. Conf. Comput. Vision and Pattern Recogn., pages 428–433, New York, June 1993.

    Google Scholar 

  67. M. Swain and M. Stricker (editors). Promising directions in active vision. Technical Report T.R. CS 91–27, University of Chicago, November 1991. Written by the attendees of the NSF Active Vision Workshop — August 5–7 1991.

    Google Scholar 

  68. R. Szeliski. Recovering 3d shape and motion from image streams using nonlinear least squares. J. visual communication and image representation, 1994.

    Google Scholar 

  69. G. Taubin. Estimation of planar curves, surfaces and nonplanar space curves defined by implicit equations. IEEE Trans. Pattern Anal. Mach. Intell., 1991.

    Google Scholar 

  70. C. J. Taylor and D. J. Kriegman. Structure and motion from fine segments in multiple views. Technical Report 9402, Yale University, 1994.

    Google Scholar 

  71. C. Tomasi and T. Kanade. Shape and motion from image streams: a factorization method — 3. detection and tracking of point features. CMU-CS 91–132, School of CS — CMU, April 1991.

    Google Scholar 

  72. C. Tomasi and T. Kanade. Shape and motion from image streams: a factorization method — 2. point features in 3d motion. CMU-CS 91–105, School of CS — CMU, January 1991.

    Google Scholar 

  73. C. Tomasi and T. Kanade. Shape and motion from image streams: a factorization method — 1. planar motion. CMU-CS 90–166, School of CS — CMU, September 1990.

    Google Scholar 

  74. J. Weng, T. Huang, and N. Ahuja. Motion and structure from two perspective views: algorithms, error analysis and error estimation. IEEE Trans. Pattern Anal. Mach. Intell., 11(5):451–476, 1989.

    Article  Google Scholar 

  75. J. Weng, T.S. Huang, and N. Ahuja. Motion and structure from line correspondences: closed-form solution, uniqueness and optimization. IEEE Trans. Pattern Anal. Mach. Intell, 14(3):318–336, 1992.

    Article  Google Scholar 

  76. Z. Zhang and O. Faugeras. Estimation of displacement from two 3d frames obtained form stereo. TR 1440 — INRIA, 1991.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Computer Science, University of Padua, via Gradenigo 6/A, 35131, Padua, Italy

    Ruggero Frezza

  2. Department of Electronics and Computer Science, University of Padua, via Gradenigo 6/A, 35131, Padua, Italy

    Giorgio Picci

  3. LADSEB-CNR, Padua, Italy

    Giorgio Picci

  4. Department of Electrical Engineering, California Institute of Technology, Pasadena, Ca., USA

    Pietro Perona & Stefano Soatto

Authors
  1. Ruggero Frezza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pietro Perona
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giorgio Picci
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefano Soatto
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Informatica e Sistemistica, Università degli Studi di Roma “La Sapienza”, Via Eudossiana 18, 00184, Roma, Italy

    Alberto Isidori

Rights and permissions

Reprints and permissions

Copyright information

© 1995 Springer-Verlag London Limited

About this paper

Cite this paper

Frezza, R., Perona, P., Picci, G., Soatto, S. (1995). System-Theoretic Aspects of Dynamic Vision. In: Isidori, A. (eds) Trends in Control. Springer, London. https://doi.org/10.1007/978-1-4471-3061-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-3061-1_11

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-3063-5

  • Online ISBN: 978-1-4471-3061-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation