Skip to main content
SpringerLink
Log in

Real-time machine-vision-based position sensing system for UAV aerial refueling

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper describes the design of a Real Time Machine-Vision (MV) Position Sensing System for the problem of Semi-Autonomous Docking within Aerial Refueling (AR) for Unmanned Aerial Vehicles (UAVs). In this effort, techniques and algorithms have been developed and extensively tested in the MATLAB/Simulink® Soft Real-Time environment as well as in Linux/RTAI Hard Real-Time environment. The overall MV software performs several tasks, such as the image acquisition from a real camera, the Feature Extraction (FE) from the acquired image, the Detection and Labeling (DAL) of the features, and the tanker-UAV Pose Estimation (PE). A Cyclic Asynchronous Buffer (CAB) mechanism was implemented for inter-process communication among Real Time and Non Real Time processes. The entire sensing system was tested using an 800 MHz PC-104 computer. The results confirmed the feasibility of executing image processing algorithms in real-time using off-the-shelf commercial hardware to obtain reliable relative position and orientation estimations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Abbreviations

3DW:

3 Dimensional window

AR:

Aerial refueling

CAB:

Cyclic asynchronous buffer

CG:

Center of gravity

DAL:

Detection and labeling

FE:

Feature extraction

GPS:

Global positioning system

LXRT:

LinuX real time

MV:

Machine vision

NRT:

Non real time

OTS:

Off the shelf

PE:

Pose estimation

RGB:

Red green blue

RPOS:

Relative position and orientation sensor

RT:

Real-time

RTAI:

Real-time application interface

RTOS:

Real-time operating system

UAV:

Unmanned aerial vehicle

USB:

Universal serial bus

VRML:

Virtual reality modeling language

VW:

Virtual World

WVU:

West Virginia University

References

  1. Korbly, R., Sensong, L.: Relative attitudes for automatic docking. AIAA J. Guidance Control Dyn. 6, 213–215 (1983)

    Article  Google Scholar 

  2. Valasek, J., Gunnam, K., Kimmett, J., Tandale, M.D., Junkins, J.L., Hughes, D.: Vision-based sensor and navigation system for autonomous air refueling. J. Guidance Control Dyn. 28(5) (2005)

  3. Fravolini, M.L., Ficola, A., Campa, G., Napolitano, M.R., Seanor, B.: Modeling and control issues for autonomous aerial refueling for UAVs using a probe-drogue refueling system. J. Aerosp. Sci. Technol. 8(7), 611–618 (2004)

    Article  Google Scholar 

  4. Vendra, S., Campa, G., Napolitano, M.R., Mammarella, M., Fravolini, M.L., Perhinschi, M.: Addressing corner detection issues for machine vision based UAV aerial refueling. Mach. Vis. Appl. (in press) doi:10.1007/s00138-006-0056-9 (2007)

  5. Fravolini, M.L., Brunori V., Ficola, A., La Cava M., Campa, G.: Feature matching algorithms for machine vision based autonomous aerial refueling. In: Mediterranean Control Conference 2006, June 28–30, Ancona, Italy (2006)

  6. Campa, G., Napolitano, M.R., Fravolini, M.L: A simulation environment for machine vision based aerial refueling for UAVs. IEEE Trans. Aerosp. Electron. Syst. (accepted with minor revisions) (2007)

  7. Mantegazza, P.: DIAPM RTAI for Linux: WHYs, WHATs and HOWs. In: Real Time Linux Workshop Conference, Vienna University of Technology (1999)

  8. Giorgio C. Buttazzo,: HARTIK: a real-time kernel for robotics applications. In: Proceedings of the 14th IEEE Real_Time Systems Symposium (RTSS 1993), pp. 201–205 (1993)

  9. Napolitano, M.R.,: Development of formation flight control algorithms using 3 YF-22 flying models. Final Report, Air Force Office of Scientific Research, AFOSR Grant Number F49620-01-1-0373 (2005)

  10. Daga, L.: http://www.digilander.libero.it/LeoDaga/Simulink/RTBlockset.htm, and http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId= 3175 (2003)

  11. Saillard, L.: http://www.saillard.org/linux/pwc/ (2006)

  12. Video4Linux, http://www.linuxtv.org/v4lwiki/index.php/Main_Page (2006)

  13. Mei, C.: Omnidirectional calibration toolbox extension. INRIA Sophia-Antipolis, Caltech University, Pasadena California, http://www.vision.caltech.edu/bouguetj/calib_doc, 10th August 2005

  14. AA. VV.: Image acquisition toolbox: user’s guide ver. 1. http://www.mathworks.com/access/helpdesk/help/pdf_doc/imaq/imaq_print. pdf, Mathworks, March 2006

  15. Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the 4th Alvey vision conference, Manchester, pp. 147–151 (1988)

  16. Noble, A.: Finding corners. Image Vis. Comput. J. 6(2), 121–128 (1988)

    Article  Google Scholar 

  17. Smith, S.M., Brady, J.M.: SUSAN: a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997)

    Article  Google Scholar 

  18. Hutchinson, S., Hager, G., Corke, P.: A tutorial on visual servo control. IEEE Trans. Robot. Automat. 12(5), 651–670 (1996)

    Article  Google Scholar 

  19. Pla, F., Marchant, J.A.: Matching feature points in image sequences through a region-based method. Comput. Vis. Image Understand. 66(3), 271–285 (1997)

    Article  Google Scholar 

  20. Umeyama, S.: Parameterized point pattern matching and its application to recognition of object families. IEEE Trans. Pattern Anal. Mach. Intell. 15(2), 136–144 (1993)

    Article  Google Scholar 

  21. Fravolini, M.L., Campa, G., Napolitano, M.R., Ficola, A.: Evaluation of machine vision algorithms for autonomous aerial refueling for unmanned aerial vehicles. AIAA J. Aerosp. Comput. Inform. Commun. (accepted) (2006)

  22. Haralick, R.M, et al.: Pose estimation from corresponding point data. IEEE Trans. Syst. Man Cybern. 19(6), 1426–1446 (1989)

    Article  Google Scholar 

  23. Campa, G., Mammarella, M., Napolitano, M.R., Fravolini, M.L., Pollini, L., Stolarik, B.: A comparison of Pose Estimation algorithms for Machine Vision based Aerial Refueling for UAV, Mediterranean Control Conference 2006, June 28–30, Ancona, Italy, (2006)

  24. Mantegazza, P.: DIAPM RTAI—beginner’s guide. RTAI Documentation Article, http://www.rtai.org/ 24th January 2006

  25. Bianchi, E., Dozio, L., Mantegazza, P.: DIAPM-RTAI: a hard real time support for LINUX’. Dipartimento di Ingegneria Aerospaziale, Milano, http://www.aero.polimi.it/∼rtai/documentation/reference/rtai_man.pdf, 2000

  26. Lineo Inc, Mantegazza P.: DIAPM RTAI Programming Guide 1.0. http://www.aero.polimi.it/∼rtai/documentation/reference/rtai_prog_guide. pdf, 2000

  27. Sarolathi, P.: Real-time application interface. In: Research seminar on Real-Time and Java. University of Helsinki, Department of Computer Science, 26th Feb 2001

  28. Buttazzo, G.: Materials for the course on real-time systems. http://www.robot.unipv.it/toolleeo/thesis/dea.pdf, Department of Computer Science University of Pavia, School of Ph. D. 2001

  29. Campa, G.: Saving data to a file at the end of the simulation. http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId= 9986, MATLAB Central File Exchange, 14th February 2006

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical and Aerospace Engineering, West Virginia University, P.O. Box 6106, 535 Eng. Science Bldg, Evansdale Drive, Morgantown, WV, 26506/6106, USA

    Rocco V. Dell’Aquila, Giampiero Campa, Marcello R. Napolitano & Marco Mammarella

Authors
  1. Rocco V. Dell’Aquila
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giampiero Campa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcello R. Napolitano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marco Mammarella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giampiero Campa.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dell’Aquila, R.V., Campa, G., Napolitano, M.R. et al. Real-time machine-vision-based position sensing system for UAV aerial refueling. J Real-Time Image Proc 1, 213–224 (2007). https://doi.org/10.1007/s11554-007-0023-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-007-0023-3

Keywords

Search

Navigation