Skip to main content

SIGS: Synthetic Imagery Generating Software for the Development and Evaluation of Vision-based Sense-And-Avoid Systems

Journal of Intelligent & Robotic Systems volume 84, pages 559–574 (2016)Cite this article

Abstract

Unmanned Aerial Systems (UASs) have recently become a versatile platform for many civilian applications including inspection, surveillance and mapping. Sense-and-Avoid systems are essential for the autonomous safe operation of these systems in non-segregated airspaces. Vision-based Sense-and-Avoid systems are preferred to other alternatives as their price, physical dimensions and weight are more suitable for small and medium-sized UASs, but obtaining real flight imagery of potential collision scenarios is hard and dangerous, which complicates the development of Vision-based detection and tracking algorithms. For this purpose, user-friendly software for synthetic imagery generation has been developed, allowing to blend user-defined flight imagery of a simulated aircraft with real flight scenario images to produce realistic images with ground truth annotations. These are extremely useful for the development and benchmarking of Vision-based detection and tracking algorithms at a much lower cost and risk. An image processing algorithm has also been developed for automatic detection of the occlusions caused by certain parts of the UAV which carries the camera. The detected occlusions can later be used by our software to simulate the occlusions due to the UAV that would appear in a real flight with the same camera setup. Additionally this algorithm could be used to mask out pixels which do not contain relevant information of the scene for the visual detection, making the image search process more efficient. Finally an application example of the imagery obtained with our software for the benchmarking of a state-of-art visual tracker is presented.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Nagy, C.J., Skoog, M.A., Somers, I.A., Cox, T.H., Warner, R.: Civil uav capability assessment. Technical report, NASA Dryden Flight Research Center (2004)

  2. DeGarmo, M.T.: Issues concerning integration of unmanned aerial vehicles in civil airspace. Technical report, MITRE Center for Advanced Aviation System Development (2004)

  3. Dempsey, M.: U.s army unmanned aircraft systems roadmap 2010-2035, Technical report, U.S. Army UAS Center of Excellence (2010)

  4. Williamson, T., Spencer, N.A.: Development and operation of the traffic alert and collision avoidance system (tcas). In: Proceedings of the IEEE, vol. 77, pp 1735–1744 (1989)

  5. Finn, A., Franklin, S.: Acoustic sense & avoid for uav’s. In: 2011 Seventh International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), pp 586–589 (2011)

  6. Barott, W.C., Coyle, E., Dabrowski, T., Hockley, C., Stansbury, R.S.: Passive multispectral sensor architecture for radar-eoir sensor fusion for low swap uas sense and avoid. In: Position, Location and Navigation Symposium - PLANS 2014, 2014 IEEE/ION, pp 1188–1196 (2014)

  7. Zarchan, P.: Tactical and Strategic Missile Guidance, American Institute of Aeronautics and Astronautics, Reston, VA, 4 edition (2002)

  8. Shneydor, N.: Missile Guidance and Pursuit: Kinematics, Dinamics and Control. Horwood Publishing, UK (1998)

    Book  Google Scholar 

  9. Limitations of the see-and-avoid principle, Technical report, Australian Transport Safety Bureau (1991)

  10. Mian, A.S.: Realtime visual tracking of aircrafts. In: Digital Image Computing: Techniques and Applications (DICTA), p 351 (2008)

  11. Mejias, L., McNamara, S., Lai, J., Ford, J.: Vision-based detection and tracking of aerial targets for uav collision avoidance. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 87–92 (2010)

  12. Lai, J., Ford, J.J., Mejias, L., O’Shea, P., Walker, R.: Detection versus false alarm characterisation of a vision-based airborne dim-target collision detection system. In: 2011 International Conference on Digital Image Computing Techniques and Applications (DICTA), pp 448–455 (2011)

  13. Lai, J., Mejias, L., Ford, J.J.: Airborne vision-based collision-detection system. J. Field Rob. 28(2), 137–157 (2011)

    Article  MATH  Google Scholar 

  14. Dey, D., Geyer, C.M., Singh, S., DiGioia, M.E.: Passive, long-range detection of aircraft: Towards a field deployable sense and avoid system. In: Proceedings Field & Service Robotics (FSR ’09) (2009)

  15. Forlenza, L., Fasano, G., Accardo, D., Moccia, A., Rispoli, A.: A hardware in the loop facility for testing multisensor sense and avoid systems. In: Digital Avionics Systems Conference, 2009. DASC ’09. IEEE/AIAA 28th, pp pages 5.C.4-1–5.C.4-10 (2009)

  16. Fasano, G., Accardo, D., Forlenza, L., Renga, A., Rufino, G., Tancredi, U., Moccia, A.: Real-time hardware-in-the-loop laboratory testing for multisensor sense and avoid systems. Int. J. Aerosp. Eng. 2013(748751) (2013)

  17. Zsedrovits, T., Zarandy, A., Vanek, B., Peni, T., Bokor, J., Roska, T.: Collision avoidance for uav using visual detection. In: 2011 IEEE International Symposium on Circuits and Systems (ISCAS), pp 2173–2176 (2011)

  18. Zsedrovits, T., Zarandy, A., Vanek, B., Peni, T., Bokor, J., Roska, T.: Estimation of relative direction angle of distant, approaching airplane in sense-and-avoid. J. Intell. Robot. Syst. 69(1–4), 407–415 (2013)

    Article  Google Scholar 

  19. Delahaye, D., Puechmorel, S., Tsiotras, P., Feron, E.: Mathematical models for aircraft trajectory design: A survey. In: Air Traffic Management and Systems, Lecture Notes in Electrical Engineering, vol. 290, pp 205–247. Springer, Japan (2014)

  20. Brown, D.C.: Close-range camera calibration. Photogramm. Eng. 37(8), 855–866 (1971)

    Google Scholar 

  21. Fryer, J.G., Brown, D.C.: Lens distortion for close-range photogrammetry. Photogramm. Eng. Remote. Sens. 52, 51–58 (1986)

    Google Scholar 

  22. Carrio, A., Changhong, F., Pestana, J., Campoy, P.: A ground-truth video dataset for the development and evaluation of vision-based sense-and-avoid systems. In: 2014 International Conference on Unmanned Aircraft Systems (ICUAS), pp 441–446 (2014)

  23. Changhong, F., Carrio, A., Olivares-Mendez, M.A., Suarez-Fernandez, R., Campoy, P.: Robust real-time vision-based aircraft tracking from unmanned aerial vehicles. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp 5441–5446 (2014)

Download references

Author information

Authors and Affiliations

  1. Computer Vision Group(CVG), Centro de Automática y Robótica(CAR), UPM-CSIC, Universidad Politécnica de Madrid, C/ José Gutiérrez Abascal 2, 28006, Madrid, Spain

    Adrian Carrio, Changhong Fu, Jean-François Collumeau & Pascual Campoy

  2. Robotics Institute, Delft University of Technology (TU Delft), Mekelweg 2, 2628, CD, Delft, Netherlands

    Pascual Campoy

Authors
  1. Adrian Carrio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changhong Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-François Collumeau
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pascual Campoy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrian Carrio.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Carrio, A., Fu, C., Collumeau, JF. et al. SIGS: Synthetic Imagery Generating Software for the Development and Evaluation of Vision-based Sense-And-Avoid Systems. J Intell Robot Syst 84, 559–574 (2016). https://doi.org/10.1007/s10846-015-0286-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-015-0286-z

Keywords

  • Sense-And-Avoid
  • Collision avoidance
  • UAV