Automation and Remote Control

, Volume 78, Issue 12, pp 2211–2221 | Cite as

UAV navigation based on videosequences captured by the onboard video camera

  • B. M. Miller
  • K. V. Stepanyan
  • A. K. Popov
  • A. B. Miller
Control in Technical Systems


We propose an approach to navigation for an unmanned aerial vehicle based on finding elements of motion (EM) (linear and angular velocities) by processing the field of local velocities for the motion of an image taken by an onboard video camera. The field of velocities for the image motion, the so-called optical flow (OF), is a linear function of the EM, which allows to use it to find the latter and thus provides an additional way of navigation, which can be rather efficiently used for certain specific problems solved by an UAV in autonomous flight. In this work, we use an algorithm for computing the OF for a given motion of the vehicle (direct problem) and show how to reconstruct the motion of the vehicle by OF observations with methods of statistical estimation (inverse problem).


navigation UAV optical flow Kalman filter 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chernous’ko, F.L. and Kolmanovskii, V.B., Optimal’noe upravlenie pri sluchainykh vozmushcheniyakh (Optimal Control under Random Disturbances), Moscow: Nauka, 1978.Google Scholar
  2. 2.
    Grigor’ev, F.N., Kuznetsov, N.A., and Serebrovskii, A.P., Upravlenie nablyudeniyami v avtomaticheskikh sistemakh (Control over Observations in Automatic Systems), Moscow: Nauka, 1986.Google Scholar
  3. 3.
    Rubinovich, E.Ya., Trajectory Control of Observations in Discrete Stochastic Optimization Problems, Autom. Remote Control, 1980, vol. 41, no. 3, pp. 365–372.MathSciNetzbMATHGoogle Scholar
  4. 4.
    Miller, B.M. and Rubinovich, E.Ya., Optimizatsiya dinamicheskikh sistem s impul’snymi upravleniyami (Dynamical Systems Optimization with Impulse Controls), Moscow: Nauka, 2005.Google Scholar
  5. 5.
    Miller, B.M. and Rubinovich, E.Ya., Complexing Problems for Optical Electronic Surveillance Systems with UAV Navigation Systems, Proc. XII All-Russian Seminar on Control Problems (VSPU-2014), June 16–19, Moscow, 2014, pp. 3657–3670.Google Scholar
  6. 6.
    Amelin, K.S. and Miller, A.B., An Algorithm for Refinement of the Position of a Light UAV on the Basis of Kalman Filtering of Bearing Measurements, J. Commun. Technol. Electron., 2014, vol. 59, no. 6, pp. 622–631. Scholar
  7. 7.
    Miller, A.B., Development of the Motion Control on the Basis of Kalman Filtering of Bearing-only Measurements, Autom. Remote Control, 2015, vol. 76, no. 6, pp. 1018–1035. S0005117915060065MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Lowe, D.G., Object Recognition from Local Scale-invariant Features, Proc. Int. Conf. on Computer Vision, September 20–27, Kerkyra, Greece, 1999, vol. 2, pp. 1150–1157.Google Scholar
  9. 9.
    Konovalenko, I., Miller, A., Miller, B., and Nikolaev, D., UAV Navigation on the Basis of the Feature Points Detection on Underlying Surface, Proc. 29 Eur. Conf. on Modelling and Simulation, May 26–29, Albena (Varna), Bulgaria, 2015, pp. 499–505. Scholar
  10. 10.
    Karpenko, S., Konovalenko, I., Miller, A., Miller, B., and Nikolaev, D., Stochastic Control of UAV on the Basis of Robust Filtering of 3D Natural Landmarks Observations, Proc. 39 IITP RAS Interdisciplinary Conf. and School, September 7–11, Olympic Village, Sochi, Russia, 2015, pp. 442–455.Google Scholar
  11. 11.
    Karpenko, S., Konovalenko, I., Miller, A., Miller, B., and Nikolaev, D., UAV Control on the Basis of 3D Landmarks Bearing-Only Observations, Sensors, 2015, vol. 15, no. 12, pp. 29802–29820. http:// Scholar
  12. 12.
    Chao, H., Gu Yu, Gross, J., Guo, G., Fravolini, M., and Napolitano, M., A Comparative Study of Optical Flow and Traditional Sensors in UAV Navigation, 2013 Am. Control Conf., Washington, USA, 2013, pp. 3858–3863.CrossRefGoogle Scholar
  13. 13.
    Serra, P., Le Bras, F., Hamel, T., et al., Nonlinear IBVS Controller for the Flare Maneuver of Fixed-Wing Aircraft Using Optical Flow, 49 IEEE Conf. on Decision and Control (CDC), Atlanta, USA, 2010, pp. 1656–1661.CrossRefGoogle Scholar
  14. 14.
    McCarthy, C. and Barnes, N., A Unified Strategy for Landing and Docking Using Spherical Flow Divergence, IEEE Trans. Patt. Anal. Machine Intellig., 2012, vol. 34, no. 5, pp. 1024–1031.CrossRefGoogle Scholar
  15. 15.
    Serra, P., Cunha, R., Silvestre, C., and Hamel, T., Visual Servo Aircraft Control for Tracking Parallel Curves, 2012 IEEE 51 IEEE Conf. on Decision and Control (CDC), Maui, USA, 2012, pp. 1148–1153.CrossRefGoogle Scholar
  16. 16.
    Liau, Y.S., Zhang, Q., Li, Y., and Ge, S.S., Non-Metric Navigation for Mobile Robot Using Optical Flow, 2012 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Vilamoura, 2012, pp. 4953–4958.CrossRefGoogle Scholar
  17. 17.
    Miller, B.M., Fedchenko, G.I., and Morskova, M.N., Computing Image Shift in Panoramic Photography, Izv. Vyssh. Uchebn. Zaved., Geodeziya Aerofotos”emka, 1984, no. 4, pp. 81–89.Google Scholar
  18. 18.
    Miller, B.M. and Fedchenko, G.I., The Influence of Orientation Errors on Image Shift in Photography from a Moving Object, Izv. Vyssh. Uchebn. Zaved., Geodeziya Aerofotos”emka, 1984, no. 5, pp. 75–80.Google Scholar
  19. 19.
    Kistlerov, V.L., Kitsul, P.I., and Miller, B.M., Computer-Aided Design of the Optical Devices Control Systems Based on the Language of Algebraic Computation FLAC, Math. Comp. Simulat., 1991, vol. 33, pp. 303–307. Scholar
  20. 20.
    Miller, B.M., and Rubinovich, E.Ya., Image Motion Compensation at Charge-coupled Device Photographing in Delay-IntegrationMode, Autom. Remote Control, 2007, vol. 68, no. 3, pp. 564–571. http:// Scholar
  21. 21.
    Lucas, B. and Kanade, T., An Iterative Image Registration Technique with an Application to Stereo Vision, IJCAI’81, Proc. 7 Int. Joint Conf. on Artificial Intelligence, Vancouver, Canada, 1981, vol. 2, pp. 674–679.Google Scholar
  22. 22.
    Popov, A.K., Stepanyan, K.V., Miller, B.M., and Miller, A.B., The IMODEL Software Suite for the Study of the Properties of UAV Control and Navigation Algorithms Based on Observations of the Underlying Surface, Proc. XX Int. Conf. Comp. Mechanics and Modern Applied Software (VMSPPS’2017), May 24–31, 2017, Alushta, 2017, pp. 607–608.Google Scholar
  23. 23.
    Popov, A., Miller, B., Miller, A., and Stepanyan, K., Optical Flow as a Navigation Means for UAVs with Opto-Electronic Cameras, Proc. 56 Israel Annual Conf. on Aerospace Sciences, March 9–10, Tel-Aviv and Haifa, Israel, 2016, code ThL2T5.2.–s2.0–84983156866&partnerID=MN8TOARSGoogle Scholar
  24. 24.
    Popov, A., Miller, A., Miller, B., and Stepanyan, K., Application of the Optical Flow as a Navigation Sensor for UAV, Proc. 39 IITP RAS Interdisciplin. Conf. and School, September 7–11, Olympic Village, Sochi, Russia, 2015, pp. 390–398. Scholar
  25. 25.
    Popov, A., Miller, A., Miller, B., and Stepanyan, K., Optical Flow and Inertial Navigation System Fusion in UAV Navigation, Conf. Unmanned/Unattended Sensors and Sensor Networks XII, September 26, Edinburgh, United Kingdom, 2016, Proc. SPIE, 2016, vol. 9986, pp. 998606–(1–16). 10.1117/12.2241204CrossRefGoogle Scholar
  26. 26.
    Sinitsyn, I.N., Fil’try Kalmana i Pugacheva (Kalman and Pugachev Filters), Moscow: Universitetskaya Kniga, Logos, 2006.Google Scholar
  27. 27.
    Popov, A., Miller, A., Miller, B., and Stepanyan, K., Estimation of Velocities via Optical Flow, Proc. 2016 Int. Conf. on Robotics and Machine Vision, September 14, Moscow, Russia, 2016, Proc. SPIE, 2017, vol. 10253, pp. 1025303–(1–5). Scholar
  28. 28.
    Forsythe, D.A. and Ponce, J., Computer Vision: A Modern Approach, Upper Saddle River: Prentice Hall, 2002. Translated under the title Komp’yuternoe zrenie. Sovremennyi podkhod, Moscow: Vil’yams, 2004.Google Scholar
  29. 29.
    Pratt, W.K., Digital Image Processing, New York: Wiley, 1978. Translated under the title Tsifrovaya obrabotka izobrazhenii, Moscow: Nauka, 1982.zbMATHGoogle Scholar
  30. 30.
    Farnebäck, G., Orientation Estimation Based onWeighted Projection onto Quadratic Polynomials, Proc. Conf. on Vision, Modeling and Visualization, 2000, pp. 89–96. diva2:273875/FULLTEXT01.pdfGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2017

Authors and Affiliations

  • B. M. Miller
    • 1
  • K. V. Stepanyan
    • 1
  • A. K. Popov
    • 1
  • A. B. Miller
    • 1
  1. 1.Institute for Information Transmission Problems (Kharkevich Institute)Russian Academy of SciencesMoscowRussia

Personalised recommendations