Skip to main content
SpringerLink
Log in

Vision-based formation control of aerial robots in the presence of sensor failure

  • Published:
Journal of Mechanical Science and Technology Aims and scope Submit manuscript

Abstract

This article presents vision-based formation flight control for aerial robots with a special focus on failure conditions in visual communication. Then, by proposing and combining two strategies, a new solution is presented for formation control. In vision-based formation flight, the state variables of the leader are estimated using image processing and unscented Kalman filter. The follower adjusts its position with respect to the leader based on the results of the estimation. In the case of visual communication failure an error will occur in the estimation of variables, which would increase with the decreased image quality. In the first proposed strategy, during the failure emergence, the position of the follower aerial robot is obtained by combining the unscented Kalman filter's estimated velocity vector and the velocity vector before failure. The weighting coefficient of each velocity vector is obtained by fuzzy logic and based on image quality. In the second strategy, to reduce the possibility of collision between the members, the geometry of the formation pattern is expanded as a function of image quality and the distance between the members. The expansion coefficient is also extracted by a fuzzy inference method, and the desired distance between the members is increased as a function of expansion coefficient. These two strategies are combined to be used during failure periods. Finally, simulation studies are presented which are conducted based on the system nonlinear equations, a model with 6 degrees of freedom for each member, and the proposed visual noise model. Obtained results reveal the proper capability of the proposed hybrid strategy in terms of controlling the formation flight during failure conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. C. Han, L. Cho and J. Cho, Wake shapes behind wings in close formation flight near the ground, J. of Mechanical Science and Technology, 19 (2005) 674–681.

    Article  Google Scholar 

  2. H. Cho, S. Lee and C. Han, Experimental study on the aerodynamic characteristics of a fighter-type aircraft model in close formation flight, J. of Mechanical Science and Technology, 28 (2014) 3059–3065.

    Article  Google Scholar 

  3. K. Do and J. Pan, Nonlinear formation control of unicycletype mobile robots, Robotics and Autonomous Systems, 55 (2007) 191–204.

    Article  Google Scholar 

  4. A. Kahn, J. Marzat and H. Piet-Lahanier, Formation flying control via elliptical virtual structure, Networking, Sensing and Control (ICNSC), 2013 10th IEEE International Conference on (2013) 158–163.

    Google Scholar 

  5. A. Abbaspour, S. Ali, A. Moosavian and K. Alipour, A virtual structure-based approach to formation control of cooperative wheeled mobile robots, RSI/ISM International Conference on Robotics and Mechatronics (ICRoM 2013), Sharif University of Technology, Tehran, Iran (2013).

    Google Scholar 

  6. S. Kang, H. Choi and Y. Kim, Formation flight and collision avoidance for multiple UAVs using concept of elastic weighting factor, International J. of Aeronautical and Space Sciences, 14 (2013) 75–84.

    Article  Google Scholar 

  7. D. Cai, J. Sun and S. Wu, UAVs formation flight control based on behavior and virtual structure, AsiaSim 2012, Ed: Springer (2012) 429–438.

    Google Scholar 

  8. R. K. Chetty, M. Singaperumal and T. Nagarajan, Behavior based multi robot formations with active obstacle avoidance based on switching control strategy, Advanced Materials Research, 433 (2012) 6630–6635.

    Article  Google Scholar 

  9. K. Seungkeun and K. Youdan, Three dimensional optimum controller for multiple UAV formation flight using behaviorbased decentralized approach, Control, Automation and Systems, 2007. ICCAS '07. International Conference on (2007) 1387–1392.

    Chapter  Google Scholar 

  10. S.-H. Oh and J. Suk, Evolutionary controller design for area search using multiple UAVs with minimum altitude maneuver, J. of Mechanical Science and Technology, 27 (2013) 541–548.

    Article  Google Scholar 

  11. S.-M. Oh and E. N. Johnson, Relative motion estimation for vision-based formation flight using unscented Kalman filter, AIAA Guidance, Navigation and Control Conference and Exhibit (2007) 20–23.

    Google Scholar 

  12. G. F. Ivey and E. N. Johnson, Investigation of methods for target state estimation using vision sensors, AIAA Guidance, Navigation, and Control Conference and Exhibit, San Francisco, CA, Aug. (2005) 15–18.

    Google Scholar 

  13. Z. Mahboubi, Z. Kolter, T. Wang, G. Bower and A. Y. Ng, Camera based localization for autonomous UAV formation flight, Proceedings of the AIAA@ Infotech Conference (2011).

    Google Scholar 

  14. W. Hou and X. Gao, Saliency-guided deep framework for image quality assessment, IEEE MultiMedia, 22 (2015) 46–55.

    Article  Google Scholar 

  15. B. Veeramallu, C. L. Susanna and S. Sahitya, Survey on an image quality assessment metric based on early vision features, IEEE Trans. Image Process, 2 (2013).

  16. B. Ghanem, E. Resendiz and N. Ahuja, Segmentationbased perceptual image quality assessment (SPIQA), Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on (2008) 393–396.

    Google Scholar 

  17. M. Latroch, O. Abdelhafid, H. Koivo and A. la Cour-Harbo, A Radio-vision localization for small UAVs flying in swam, IEEE Transactions on Aerospace and Electronic Systems (2012).

    Google Scholar 

  18. I. Prodan, S. Olaru, R. Bencatel, J. B. De Sousa, C. Stoica and S.-I. Niculescu, Receding horizon flight control for trajectory tracking of autonomous aerial vehicles, Control Engineering Practice, 21 (2013) 1334–1349.

    Article  Google Scholar 

  19. H. A. Izadi, B. W. Gordon and Y. Zhang, Decentralized model predictive control for cooperative multiple vehicles subject to communication loss, International J. of Aerospace Engineering, Article 198308 (2011).

  20. E. Franco, L. Magni, T. Parisini, M. M. Polycarpou and D. M. Raimondo, Cooperative constrained control of distributed agents with nonlinear dynamics and delayed information exchange: A stabilizing receding-horizon approach, Automatic Control, IEEE Transactions on, 53 (2008) 324–338.

    MathSciNet  Google Scholar 

  21. X. Zhang, R. Xu, C. Kwan, L. Haynes, Y. Yang and M. Polycarpou, Fault tolerant formation flight control of UAVs, International J. of Vehicle Autonomous Systems, 2 (2004) 217–235.

    Article  Google Scholar 

  22. L. DeVries and D. A. Paley, Wake estimation and optimal control for autonomous aircraft in formation flight, Proc. of AIAA Guidance, Navigation, and Control Conf., Boston, MA (2013).

    Google Scholar 

  23. M. S. Hemati, J. D. Eldredge and J. L. Speyer, Wake sensing for aircraft formation flight, J. of Guidance, Control, and Dynamics, 37 (2014) 513–524.

    Article  Google Scholar 

  24. L. DeVries and D. A. Paley, Wake sensing and estimation for control of autonomous aircraft in formation flight, J. of Guidance, Control, and Dynamics (2015) 1–10.

    Google Scholar 

  25. T. Keviczky, F. Borrelli, K. Fregene, D. Godbole and G. J. Balas, Decentralized receding horizon control and coordination of autonomous vehicle formations, Control Systems Technology, IEEE Transactions on, 16 (2008) 19–33.

    Google Scholar 

  26. B. S. Kim, A. J. Calise and R. Sattigeri, Adaptive, integrated guidance and control design for line-of-sight-based formation flight, J. of Guidance, Control, and Dynamics, 30 (2007) 1386–1399.

    Article  Google Scholar 

  27. M. Rhudy and Y. Gu, Understanding nonlinear Kalman filters, Part II: An implementation guide, Interactive Robotics Letters (2013).

    Google Scholar 

  28. J. N. Gross, Y. Gu, M. B. Rhudy, S. Gururajan and M. R. Napolitano, Flight-test evaluation of sensor fusion algorithms for attitude estimation, Aerospace and Electronic Systems, IEEE Transactions on, 48 (2012) 2128–2139.

    Google Scholar 

  29. M. Rhudy, Y. Gu, J. Gross and M. R. Napolitano, Evaluation of matrix square root operations for UKF within a UAV GPS/INS sensor fusion application, International J. of Navigation and Observation, 2011 (2012).

  30. S. Gururajan, M. L. Fravolini, H. Chao, M. Rhudy and M. R. Napolitano, Performance evaluation of neural network based approaches for airspeed sensor failure accommodation on a small UAV, Control & Automation (MED), 2013 21st Mediterranean Conference on (2013) 603–608.

    Google Scholar 

  31. Y. Abbasi, S. A. A. Moosavian and A. B. Novinzadeh, Formation control of aerial robots using virtual structure and new fuzzy-based self-tuning synchronization, Transactions of the Institute of Measurement and Control, 014233121 6649021 (2016).

  32. B. L. Stevens and F. L. Lewis, Aircraft control and simulation, Wiley (2003).

    Google Scholar 

  33. J. Roskam, Airplane flight dynamics and automatic flight controls: Part I. DARcorporation, Lawrence, Kansas (2001).

    Google Scholar 

  34. R. Kandepu, B. Foss and L. Imsland, Applying the unscented Kalman filter for nonlinear state estimation, J. of Process Control, 18 (2008) 753–768.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Aerospace Engineering, Malekashtar Univ. of Technology, Tehran, Iran

    Y. Abbasi

  2. Dept. of Mechanical Engineering, K. N. Toosi Univ. of Technology, Tehran, Iran

    S. Ali A. Moosavian

  3. Dept. of Aerospace Engineering, K. N. Toosi Univ. of Technology, Tehran, Iran

    Alireza B. Novinzadeh

Authors
  1. Y. Abbasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Ali A. Moosavian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alireza B. Novinzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Y. Abbasi.

Additional information

Recommended by Associate Editor Deok Jin Lee

Yosef Abbasi received his M.Sc. degree from Tarbiat Modaress University, in 1998, and his Ph.D. in 2016 in K. N. Toosi University of Technology (KNTU), both in Aerospace Engineering, Tehran, Iran. He is an Assistant Professor with the Aerospace Engineering Department at Malekashtar University of Technology (MUT) in Tehran since 2016. His research interests include flight dynamic, control, path planning, formation flight and simulation with applications to aerial robots.

S. Ali A. Moosavian received his B.S. in 1986 from Sharif University of Technology and the M.S. in 1990 from Tarbiat Modaress University (both in Tehran), and his Ph.D. in 1996 from McGill University (Montreal, Canada), all in Mechanical Engineering. He is a Professor of Mechanical Engineering at K. N. Toosi University of Technology (KNTU) in Tehran since 1997. His research interests are in the areas of dynamics modeling and motion/impedance control of terrestrial, legged and space robotic systems.

A. B. Novinzadeh received his Ph.D. in Aerospace Engineering from the Sharif University of Technology, Tehran, Iran in 2005. He has been a Faculty Member at K.N. Toosi University of Technology, Iran since 2005. During his professional career, Dr. Novinzadeh has experienced several academic Head of Flight Dynamics & Space Engineering group and Head of Guidance, Control and Systems Dynamics Laboratory. Dr. Novinzadeh's areas of research are focused on Non-Linear and Closed Loop Optimal Control, Space Systems Design, Dynamics Systems Modeling using Bond graph method, Interplanetary Mission Design and Trajectory, Applied Mathematics, Model Free Control Design. He has developed and taught many Aerospace- related courses and has published more than 60 scientific paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abbasi, Y., Moosavian, S.A.A. & Novinzadeh, A.B. Vision-based formation control of aerial robots in the presence of sensor failure. J Mech Sci Technol 31, 1413–1426 (2017). https://doi.org/10.1007/s12206-017-0242-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12206-017-0242-x

Keywords

Search

Navigation