Skip to main content
SpringerLink
Log in

An Optical Flow-Based Solution to the Problem of Range Identification in Perspective Vision Systems

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

A classical problem in machine vision is the range identification of an object moving in three-dimensional space from the two-dimensional image sequence obtained with a monocular camera. This study presents a novel reduced-order optical flow-based nonlinear observer that renders the proposed scheme suitable for depth estimation applications in both well-structured and unstructured environments. In this study, a globally exponentially stable observer is synthesized, where optical flow estimates are derived from tracking feature trajectory on the image plane over successive camera frames, to yield asymptotic estimates of feature depth at a desired convergence rate. Furthermore, the observer is shown to be finite-gain \(\mathcal {L}_{p}\) stable ∀p∈[1,∞] in the presence of exogenous disturbance influencing camera motion, and is applicable to a wider class of perspective systems than those considered by alternative designs. The observer requires minor apriori system information for convergence, and the convergence condition arises in a natural manner with an apparently intuitive interpretation. Numerical and experimental studies are used to validate and demonstrate robust observer performance in the presence of significant measurement noise.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Soatto, S., Perona, P.: Reducing structure from motion”: A general framework for dynamic vision part 1: modeling. IEEE Trans. Pattern Anal. Mach. Intell. 20(9), 933–942 (1998)

    Article  Google Scholar 

  2. Chen, X., Kano, H.: A new state observer for perspective systems. IEEE Trans. Autom. Control 47(4), 658–663 (2002)

    Article  MathSciNet  Google Scholar 

  3. Karagiannis, D., Astol, A.: A new solution to the problem of range identification in perspective vision systems. IEEE Trans. Autom. Control 50(12), 2074–2077 (2005)

  4. Jankovic, M., Ghosh, B.K.: Visually guided ranging from observations of points, lines and curves via an identier based nonlinear observer. Syst. Control Lett. 25, 63–73 (1995)

    Article  Google Scholar 

  5. Dixon, W.E., Fang, Y., Dawson, D.M., Flynn, T.J.: Range identication for perspective vision systems. IEEE Trans. Autom. Control 48(12), 2232–2238 (2003)

    Article  Google Scholar 

  6. Luca, A.D., Oriolo, G., Giordano, P.R.: On-Line Estimation of Feature Depth for Image-Based Visual Servoing Schemes. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 2823–2828, Roma (2007)

  7. Dani, A.P., Fischer, N.R., Kan, Z., Dixon, W.E.: Globally exponentially stable observer for vision-based range estimation. Mechatronics 22(4), 381–389 (2012)

    Article  Google Scholar 

  8. Ma, L., Chen, Y., Moore, K.L.: Range Identi?cation for Perspective Dynamic System with Single Homogeneous Observation. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 5207 −5211, New Orleans (2004)

  9. Morbidi, F., Prattichizzo, D.: Range Estimation from a Moving Camera: an Immersion and Invariance Approach. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 2810–2815, Kobe (2009)

  10. Grave, I., Tang, Y.: A new observer for perspective vision systems under noisy measurements. IEEE Trans. Autom. Control 60(2), 503–508 (2015)

    Article  MathSciNet  Google Scholar 

  11. Karagiannis, D., Carnevale, D., Astolfi, A.: Invariant manifold based reduced-order observer design for nonlinear systems. IEEE Trans. Autom. Control 53(11), 2602–2614 (2008)

    Article  MathSciNet  Google Scholar 

  12. Dahl, O., Wang, Y., Lynch, A.F., Heyden, A.: Observer forms for perspective systems. Automatica 46(11), 1829–1834 (2010)

    Article  MathSciNet  Google Scholar 

  13. Nath, N., Braganza, D., Dawson, D.M., Burg, T.: Range identification for perspective vision systems: a position based approach. Int. J. Robot. Autom. 206. doi:10.2316/Journal.206.2011.2.206-3409

  14. Abdursal, R., Inaba, H., Ghosh, B.: Nonlinear observers for perspective time-invariant linear systems. Automatica 40(3), 481–490 (2004)

    Article  MathSciNet  Google Scholar 

  15. Soatto, S., Frezza, R., Perona, P.: Motion Estimation via Dynamic Vision. IEEE Trans. Autom. Control 41(41), 393–412 (1996)

    Article  MathSciNet  Google Scholar 

  16. Tsai, R. Y., Huang, T. S.: Estimating three-dimensional motion parameters of a rigid planar patch. IEEE Trans. Acoust. Speech Signal Process. 29(6), 1147–1152 (1981)

    Article  Google Scholar 

  17. Dahl, O., Nyberg, F., Heyden, A.: Nonlinear and Adaptive Observers for Perspective Dynamic Systems. In: Proceedings of American Control Conference (ACC), pp. 1966–1971, New York (2007)

  18. Chen, X., Kano, H.: State observer for a class of nonlinear systems and its application to machine vision. IEEE Trans. Autom. Control 49(11), 2085–2091 (2004)

    Article  MathSciNet  Google Scholar 

  19. Chitrakaran, V.K., Dawson, D.M., Dixon, W.E., Chen, J.: Identification of a moving objects velocity with a fixed camera. Automatica 41(3), 553–562 (2005)

    Article  MathSciNet  Google Scholar 

  20. Dani, A., Fischer, N., Dixon, W.E.: Single camera structure motion. IEEE Trans. Autom. Control 57(1), 238–243 (2012)

    Article  MathSciNet  Google Scholar 

  21. Matthies, L., Kanade, T., Szeliski, R.: Kalman filter based algorithms for estimating depth from image sequences. Int. J. Comput. Vis. 3(3), 209–236 (1989)

    Article  Google Scholar 

  22. Sridhar, B., Soursa, R., Hussein, B.: Passive range estimation for rotorcraft low-altitude flight. Mach. Vis. Appl. 6(1), 10–24 (1993)

    Article  Google Scholar 

  23. Chaumette, F., Boukir, S., Bouthemy, P., Juvin, D.: Structure from controlled motion. IEEE Trans. Pattern Anal. Mach. Intell. 18(5), 492–504 (1996)

    Article  Google Scholar 

  24. Chiuso, A., Favaro, P., Jin, H., Soatto, S.: Structure from motion causally integrated over time. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 523–535 (2002)

    Article  Google Scholar 

  25. Durrant-Whyte, H., Bailey, T.: Simultaneous localization and mapping (slam): Part II. IEEE Robot. Autom. Mag. 13(3), 108–117 (2006)

    Article  Google Scholar 

  26. Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: Monoslam: Real-time single camera slam. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Article  Google Scholar 

  27. Ayache, N., Faugeras, O.: Maintaining representations of the environment of a mobile robot. IEEE Trans. Robot. Autom. 6(5), 804–819 (1989)

    Article  Google Scholar 

  28. Broida, T.J., Chellappa, R.: Estimation of object motion parameters from noisy images. IEEE Trans. Pattern Anal. Mach. Intell. 8(1), 90–99 (1986)

    Article  Google Scholar 

  29. Dickmanns, E.D., Graefe, V.: Dynamic monocular machine vision. Mach. Vis. Appl. 1, 223–240 (1988)

    Article  Google Scholar 

  30. Faugeras, O.D., Ayache, N., Faverjon, B.: Building Visual Maps by Combining Noisy Stereo Measurements. In: Proceedings of IEEE Conference on Robotics and Automation (ICRA), San Francisco (1986)

  31. Young, G.S., Chellappa, R.: 3D motion estimation using a sequence of noisy stereo images: models, estimation and uniqueness. IEEE Trans. Pattern Anal. Mach. Intell. 12(8), 735–759 (1990)

    Article  Google Scholar 

  32. Reif, K., Sonnemann, F., Unbehauen, R.: An EKF-based nonlinear observer with a prescribed degree of stability. Automatica 34(9), 1119–1123 (1998)

    Article  Google Scholar 

  33. Gurfil, P., Rotstein, H.: Partial aircraft state estimation from visual motion using the subspace constraints approach. J. Guid. Control. Dyn. 25(5), 1016–1028 (2001)

    Article  Google Scholar 

  34. Keshavan, J., Escobar-Alvarez, H., Dimble, K. D., Humbert, J. S., Goerzen, C. L., Whalley, M. S.: Application of a nonlinear recursive visual-depth observer using UH60 flight data. J. Guid. Control. Dyn. 39(7), 1501–1512 (2016)

    Article  Google Scholar 

  35. Ghosh, B. K., Inaba, H., Takahashi, S.: Identification of Riccati dynamics under perspective and orthographic observations. IEEE Trans. Autom. Control 45(7), 1267–1278 (2000)

    Article  MathSciNet  Google Scholar 

  36. Khalil, H. K.: Nonlinear Systems. Prentice Hall. Upper Saddle River, New Jersey (2002)

    Google Scholar 

  37. Conroy, J., Humbert, S. J.: Structure from Motion in Computationally Constrained Systems. In: Proceedings of SPIE 8725 (2013), doi:10.1117/12.2015296

  38. Levant, A.: Robust exact differentiation via sliding mode technique. Automatica 34(3), 379–384 (1998)

    Article  MathSciNet  Google Scholar 

  39. Chawda, V., Celik, O., O’Malley, M. K.: Application of Levant’s Differentiator for Velocity Estimation and Increased Z-Width in Haptic Interfaces. In: IEEE World Haptics Conference, pp. 403–408, Istanbul (2011)

  40. Lucas, B. D., Kanade, T.: An Iterative Image Registration Technique with an Application to Stereo Vision. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)

  41. Birchfield, S.: KLT: An implementation of the Kanade-Lucas-Tomasi feature tracker http://www.ces.clemson.edu/stb/klt/ (2007)

  42. Shi, J., Tomasi, C.: Good Features to Track. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 593–600, Seattle (1994)

  43. Bouguet, J.: Camera calibration toolbox for MATLAB http://www.vision.caltech.edu/bouguetj (2007)

Download references

Author information

Authors and Affiliations

  1. Autonomous Vehicles Laboratory, University of Maryland, College Park, 20742, USA

    Jishnu Keshavan & J. Sean Humbert

Authors
  1. Jishnu Keshavan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Sean Humbert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jishnu Keshavan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Keshavan, J., Humbert, J.S. An Optical Flow-Based Solution to the Problem of Range Identification in Perspective Vision Systems. J Intell Robot Syst 85, 651–662 (2017). https://doi.org/10.1007/s10846-016-0404-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-016-0404-6

Keywords

Search

Navigation