Advertisement

Augmenting Flight Imagery from Aerial Refueling

  • James D. Anderson
  • Scott NyklEmail author
  • Thomas Wischgoll
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11845)

Abstract

When collecting real-world imagery, objects in the scene may be occluded by other objects from the perspective of the camera. However, in some circumstances an occluding object is absent from the scene either for practical reasons or the situation renders it infeasible. Utilizing augmented reality techniques, those images can be altered to examine the affect of the object’s occlusion. This project details a novel method for augmenting real images with virtual objects in a virtual environment. Specifically, images from automated aerial refueling (AAR) test flights are augmented with a virtual refueling boom arm, which occludes the receiving aircraft. The occlusion effects of the boom are quantified in order to determine which pixels are not viable for stereo image processing to reduce noise and increase efficiency of estimating aircraft pose from stereo images.

Keywords

Augmented reality Virtual reality simulation Vision occlusion 

References

  1. 1.
    Bobick, A.F., Intille, S.S.: Large occlusion stereo. Int. J. Comput. Vision 33, 181–200 (1999).  https://doi.org/10.1023/A:1008150329890CrossRefGoogle Scholar
  2. 2.
    Campa, G., Mammarella, M., Napolitano, M.R., Fravolini, M.L., Pollini, L., Stolarik, B.: A comparison of pose estimation algorithms for machine vision based aerial refueling for UAVs. In: 14th Mediterranean Conference on Control and Automation, MED 2006 (2006).  https://doi.org/10.1109/MED.2006.328769
  3. 3.
    Campa, G., Napolitano, M.R., Fravolini, M.L.: Simulation environment for machine vision based aerial refueling for UAVs. IEEE Trans. Aerosp. Electron. Syst. 45, 138–151 (2009).  https://doi.org/10.1109/TAES.2009.4805269CrossRefGoogle Scholar
  4. 4.
    Crow, F.C.: Shadow algorithms for computer graphics. ACM SIGGRAPH Comput. Graphics 11, 242–248 (2005).  https://doi.org/10.1145/965141.563901CrossRefGoogle Scholar
  5. 5.
    Dallmann, W.E.: Infrared and electro-optical stereo vision for automated aerial refueling. Master’s thesis, Air Force Institute of Technology (2019)Google Scholar
  6. 6.
    Duan, H., Zhang, Q.: Visual measurement in simulation environment for vision-based UAV autonomous aerial refueling. IEEE Trans. Instrum. Meas. 64, 2468–2480 (2015).  https://doi.org/10.1109/TIM.2014.2343392CrossRefGoogle Scholar
  7. 7.
    Fravolini, M.L., Brunori, V., Ficola, A., La Cava, M., Campa, G.: Feature matching algorithms for machine vision based autonomous aerial refueling. In: 14th Mediterranean Conference on Control and Automation, MED 2006 (2006).  https://doi.org/10.1109/MED.2006.328792
  8. 8.
    Fravolini, M.L., Campa, G., Napolitano, M.R.: Evaluation of machine vision algorithms for autonomous aerial refueling for unmanned aerial vehicles. J. Aerosp. Comput. Inf. Commun. 4, 968–985 (2008).  https://doi.org/10.2514/1.17269CrossRefGoogle Scholar
  9. 9.
    Huang, X., Walker, I., Birchfield, S.: Occlusion-aware reconstruction and manipulation of 3D articulated objects. In: Proceedings - IEEE International Conference on Robotics and Automation (2012).  https://doi.org/10.1109/ICRA.2012.6224911
  10. 10.
    Johnson, D.T., Nykl, S.L., Raquet, J.F.: Combining stereo vision and inertial navigation for automated aerial refueling. J. Guidance Control Dyn. 40, 2250–2259 (2017).  https://doi.org/10.2514/1.g002648CrossRefGoogle Scholar
  11. 11.
    Kimmett, J., Valasek, J., Junkins, J.: Autonomous aerial refueling utilizing a vision based navigation system. In: AIAA Guidance, Navigation, and Control Conference and Exhibit. American Institute of Aeronautics and Astronautics (2002).  https://doi.org/10.2514/6.2002-4469
  12. 12.
    Lepetit, V.: On computer vision for augmented reality. In: Proceedings - International Symposium on Ubiquitous Virtual Reality, ISUVR 2008 (2008).  https://doi.org/10.1109/ISUVR.2008.10
  13. 13.
    Mammarella, M., Campa, G., Napolitano, M.R., Fravolini, M.L.: Comparison of point matching algorithms for the UAV aerial refueling problem. Mach. Vis. Appl. 21, 241–251 (2010).  https://doi.org/10.1007/s00138-008-0149-8CrossRefGoogle Scholar
  14. 14.
    Nykl, S., Mourning, C., Leitch, M., Chelberg, D., Franklin, T., Liu, C.: An overview of the STEAMiE educational game engine. In: Proceedings - Frontiers in Education Conference, FIE (2008).  https://doi.org/10.1109/FIE.2008.4720454
  15. 15.
    Parsons, C.A.: Improving automated aerial refueling stereo vision pose estimation using a shelled reference model. Master’s thesis, Air Force Institute of Technology (2017)Google Scholar
  16. 16.
    Parsons, C., Nykl, S.: Real-time automated aerial refueling using stereo vision. In: Bebis, G., et al. (eds.) ISVC 2016. LNCS, vol. 10073, pp. 605–615. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-50832-0_59CrossRefGoogle Scholar
  17. 17.
    Parsons, C., Paulson, Z., Nykl, S., Dallman, W., Woolley, B.G., Pecarina, J.: Analysis of simulated imagery for real-time vision-based automated aerial refueling. J. Aerosp. Inf. Syst. 16(3), 77–93 (2019).  https://doi.org/10.2514/1.I010658CrossRefGoogle Scholar
  18. 18.
    Paulson, Z., Nykl, S., Pecarina, J., Woolley, B.: Mitigating the effects of boom occlusion on automated aerial refueling through shadow volumes. J. Defense Model. Simul. 16, 175–189 (2019).  https://doi.org/10.1177/1548512918808408CrossRefGoogle Scholar
  19. 19.
    Pollini, L., Campa, G., Giulietti, F., Innocenti, M.: Virtual simulation set-up for UAVs aerial refuelling. In: AIAA Modeling and Simulation Technologies Conference and Exhibit. American Institute of Aeronautics and Astronautics (2012).  https://doi.org/10.2514/6.2003-5682
  20. 20.
    Shah, M.M., Arshad, H., Sulaiman, R.: Occlusion in augmented reality. In: 8th International Conference on Information Science and Digital Content Technology (ICIDT 2012) (2012)Google Scholar
  21. 21.
    Zanfir, A., Sminchisescu, C.: Large displacement 3D scene flow with occlusion reasoning. In: Proceedings of the IEEE International Conference on Computer Vision (2015).  https://doi.org/10.1109/ICCV.2015.502

Copyright information

© This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2019

Authors and Affiliations

  1. 1.Wright State UniversityDaytonUSA
  2. 2.Air Force Institute of TechnologyDaytonUSA

Personalised recommendations