Skip to main content
SpringerLink
Log in

DNN Based Ranging in Support of Autonomous Aerial Refueling

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Uncrewed aircraft deployed on aircraft carriers will be expected to have ever-increasing levels of autonomous functionality in the near future. This work presents early progress toward the certification of an uncrewed system to exhibit autonomous behavior while acting as the receiver during probe and drogue aerial refueling. The paper focuses on using a computer vision-based approach for identifying a drogue deployed by a tanker aircraft and determining its relative position to the refueling probe tip of the receiver. This will be critical if a vision-based solution will eventually feed into the control loop for the uncrewed system. This research used a fleet representative refueling drogue and probe. A computer vision system installed at the base of the refueling probe was used to feed imagery to a deep neural network trained to identify the refueling drogue and determine the drogue’s position relative to the probe tip. Ground truth experiments validated via a motion capture system demonstrated that the error in relative position measurements was within tolerances to allow the deep neural network to be used in feedback control for a notional uncrewed system. Finally, the laboratory-trained deep neural network demonstrated its ability to identify the drogue outside of the laboratory with mission-representative backgrounds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

N/A

References

  1. Tucker, P.: Drones could one day make up 40 (2021). https://www.defenseone.com/technology/2021/03/drones-could-one-day-make-40-carrier-air-wing-navy-says/172799/. Accessed 10 Nov 2022

  2. Lisa, M., Staub, J.: EA-18G Sea trials, flying qualities, performance and precision approach and landing systems. In: 2008 Society of Experimental Test Pilots Annual Symposium (2008)

  3. Ross, B.L., Mauldin, C.A., Parry, J.K., Costello, D.H.: First steps toward certifying an UAS to receive fuel airborne. In: 2022 International Conference on Unmanned Aircraft Systems (ICUAS), Dubrovnik, Croatia (2022). https://doi.org/10.1109/ICUAS54217.2022.9836174

  4. Airworthiness Policy, NAVAIRINST 13034.1G. NAVAIR (2022)

  5. Naval Air Training and Operations Procedures Standardization (NATOPS) General Flight and Operating Instructions. Commander Naval Air Forces (2016)

  6. Costello, D.H., Xu, H.: Generating certification evidence for autonomous aerial vehicles decision-making. J. Aerospace Inf. Syst. 18(1), 3–13 (2021). https://doi.org/10.2514/1.I010848

    Article  Google Scholar 

  7. Costello, D.H., Jewell, J., Xu, H.: Autonomous flight-test data in support of safety of flight certification. J. Air Transp. 29(2), 93–106 (2021). https://doi.org/10.2514/1.D0220

    Article  Google Scholar 

  8. Costello, D.H., Xu, H.: Relating sensor degradation to vehicle situational awareness for autonomous air vehicle certification. J. Aerospace Inf. Syst. 18(4), 193–202 (2021). https://doi.org/10.2514/1.I010905

    Article  Google Scholar 

  9. Costello, D.H.: Certifying an autonomous system to complete tasks currently reserved for qualified pilots (doctoral dissertation). Clark School Of Engineering. University of Maryland, College Park, MD (2020)

    Google Scholar 

  10. Concepts of design assurance for neural networks (CoDANN) II. European Union Aviation Safety Agency (2021)

  11. Systems Engineering Fundamentals. Defense Acquisition University Press (2001). https://acqnotes.com/wp-content/uploads/2017/07/DAU-Systems-Engineering-Fundamentals.pdf

  12. Shriver, D., Elbaum, S., Dwyer, M.B.: Dnnv: A framework for deep neural network verification. (2021). arXiv:2105.12841

  13. Torens, C., Durak, U., Dauer, J.C.: Guidelines and regulatory framework for machine learning in aviation, pp. 1–10 (2022). https://doi.org/10.2514/6.2022-1132, https://arc.aiaa.org/doi/abs/10.2514/6.2022-1132

  14. NASA Formal Methods 13th International Symposium, NFM 2021, Virtual Event, May 24-28, 2021, Proceedings, 1st ed. 2021. edn. Programming and Software Engineering; 12673. Springer, Cham (2021)

  15. Purton, L., Kourousis, K.: Military airworthiness management frameworks: A critical review. In: Procedia Engineering, vol. 80, pp. 545–564. Elsevier Ltd, (2014)

  16. Wing, T., Livermore, R.A., Leonard, A.W.: Technical report (2020)

  17. Kose, O., Oktay, T.: Hexarotor longitudinal flight control with deep neural network, pid algorithm and morphing. Eur. J. Sci. Technol. 27, 115–124 (2021). https://doi.org/10.31590/ejosat.946884

    Article  Google Scholar 

  18. Kose, O., Oktay, T.: Hexarotor lateral flight control using deep neural network and morphing. In: Baryrak, S., Aydin, N., Boztoprak, Y. (eds.) Research & Reviews in Engineering - II, pp. 33–48. Gece Publishing, (2021)

  19. Kose, O., Oktay, T.: Simultaneous design of morphing hexarotor and autopilot system by using deep neural network and spsa. Aircraft Eng. Aerospace Technol. 95(6), 939–949 (2023)

  20. Padhy, R.P., Verma, S., Ahmad, S., Choudhury, S.K., Sa, P.K.: Deep neural network for autonomous uav navigation in indoor corridor environments. Procedia Comput. Sci. 133, 643–650 (2018)

    Article  Google Scholar 

  21. Costello, D..H., Adams, R.: A framework for airworthiness certification of autonomous systems within united states naval aviation. J. Aviat. 1(7), 7–16 (2023). https://doi.org/10.30518/jav.1161725

    Article  Google Scholar 

  22. Parry, J., Costello, D., Rupert, J., Taylor, G.: The national airworthiness council artificial intelligence working group (nacaiwg) summit proceedings 2022. Systems Engineering, 1–6. https://doi.org/10.1002/sys.21703https://arxiv.org/abs/https://incose.onlinelibrary.wiley.com/doi/pdf/10.1002/sys.21703

  23. Schweikhard, K.: Results of NASA/DARPA automatic probe and drogue refueling flight test. In: SAE Guidance and Control Subcommittee Meeting, Williamsburg, VA (2006)

  24. X-47B Unmanned aircraft demonstrates the first autonomous aerial refueling (2015). https://news.northropgrumman.com/news/releases/photo-release-x-47b-unmanned-aircraft-demonstrates-the-first-autonomous-aerial-refueling. Accessed 10 Nov 2022

  25. Costello, D., Xu, H.: Run time assurance approach to certifying autonomy withing naval aviation. In: Xponential 2022 (2022)

  26. Costello, D., Xu, H.: Test conditions for using run time assurance approach for certificaion of autonomy. Submitted to the International Test and Evaluation Jouranl (2022)

  27. Sharma, T., Debaque, B., Duclos, N., Chehri, A., Kinder, B., Fortier, P.: Deep learning-based object detection and scene perception under bad weather conditions. Electronics. 11(4), 563 (2022)

    Article  Google Scholar 

  28. Kasdin, N.J., Paley, D.A.: Engineering dynamics: a comprehensive introduction. Princeton University Press, Princeton, NJ (2011)

    Book  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Office of Naval Research for its support of this project. Additionally, the Technical Support Division of the Weapons, Robotics, and Control, Engineering Department at the United States Naval Academy provided invaluable assistance in building the test stands and helping to develop the lab experiments. We also are grateful to Mr. Jonathon Parry, a Ph.D. student at Purdue University, for his help in researching autonomy certification throughout the DoD.

Funding

The authors completed this research while performing their duties. No extra funding was required.

Author information

Author notes

  1. All authors are contributed equally to this work.

Authors and Affiliations

  1. Weapons, Robotics and Control Engineering, United States Naval Academy, 597 McNair Road, Annapolis, 21402, MD, USA

    Donald Costello III, Levi DeVries, Caleb Mauldin & Benjamin Ross

Authors
  1. Donald Costello III
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Levi DeVries
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Caleb Mauldin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

This work summarizes the work performed by Ensigns Ross and Mauldin as part of their undergraduate degree at USNA. The project was their Capstone project. Dr DeVries and CDR Costello served as advisers to the Capstone Project. All equally contributed to the work.

Corresponding author

Correspondence to Donald Costello III.

Ethics declarations

Ethics approval

N/A

Consent to participate

N/A

Consent for publication

N/A

Competing interest

The authors have no financial or proprietary interests in any material discussed in this article. \(\bullet \) The authors did not receive support from any organization for the submitted work. \(\bullet \) The authors have no financial or proprietary interests in any material discussed in this article.

Conflicts of interest

The work details the use of a DNN to perform a task currently peformed by pilots. There is not a conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Costello, D., DeVries, L., Mauldin, C. et al. DNN Based Ranging in Support of Autonomous Aerial Refueling. J Intell Robot Syst 109, 49 (2023). https://doi.org/10.1007/s10846-023-01969-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-023-01969-1

Keywords

Search

Navigation