Skip to main content
SpringerLink
Log in

Landmark-Based Virtual Path Estimation for Assisted UAV FPV Tele-Operation with Augmented Reality

  • Conference paper
  • First Online:
Intelligent Robotics and Applications (ICIRA 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11745))

Included in the following conference series:

Abstract

In this paper we proposed an Assisted UAV Tele-Operation System, specifically for FPV navigation based on Artificial Landmarks in obstacle free environments. The system estimates the optimal path through landmarks and traces an artificial route to be followed. Path recognition uses color space and morphological transformation such as eroding and dilating to reduce noise due to different lighting environments. Once path is recognized ORB detector is used for getting a set of the most representative pixels coordinates, this is done for each ROI (Region of Interest) in the camera image. Later, the median of each pixel coordinate in the specific ROI is considered for interpolation needed to trace the route. Parrot’s drone Bebop 2 was used for the purpose of this study as it has a fisheye lens camera that allows us to face downwards to detect the landmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Orbea, D., Moposita, J., Aguilar, W.G., Paredes, M., Reyes, R.P., Montoya, L.: Vertical take off and landing with fixed rotor. In: Chilean Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), Pucón, Chile (2017)

    Google Scholar 

  2. Orbea, D., Moposita, J., Aguilar, W.G., Paredes, M., León, G., Jara-Olmedo, A.: Math model of UAV multi rotor prototype with fixed wing aerodynamic structure for a flight simulator. In: De Paolis, L.T., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10324, pp. 199–211. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60922-5_15

    Chapter  Google Scholar 

  3. Aguilar, W.G., Rodríguez, G.A., Álvarez, L., Sandoval, S., Quisaguano, F., Limaico, A.: Visual SLAM with a RGB-D camera on a quadrotor UAV using on-board processing. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2017. LNCS, vol. 10306, pp. 596–606. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59147-6_51

    Chapter  Google Scholar 

  4. Aguilar, W.G., Rodríguez, G.A., Álvarez, L., Sandoval, S., Quisaguano, F., Limaico, A.: Real-time 3D modeling with a RGB-D camera and on-board processing. In: De Paolis, L.T., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10325, pp. 410–419. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60928-7_35

    Chapter  Google Scholar 

  5. Basantes, J.: Capture and processing of geospatial data with laser scanner system for 3D modeling and virtual reality of Amazonian Caves. In: IEEE Ecuador Technical Chapters Meeting (ETCM), Samborondón, Ecuador (2018)

    Google Scholar 

  6. Pardo, J.A., Aguilar, W.G., Toulkeridis, T.: Wireless communication system for the transmission of thermal images from a UAV. In: Chilean Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), Pucón, Chile (2017)

    Google Scholar 

  7. Aguilar, W.G., Angulo, C.: Real-time model-based video stabilization for microaerial vehicles. Neural Proc. Lett. 43(2), 459–477 (2016)

    Article  Google Scholar 

  8. Aguilar, W.G., Angulo, C.: Real-time video stabilization without phantom movements for micro aerial vehicles. EURASIP J. Image Video Process. 1, 1–13 (2014)

    Google Scholar 

  9. Aguilar, W.G., et al.: Real-time detection and simulation of abnormal crowd behavior. In: De Paolis, L.T., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10325, pp. 420–428. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60928-7_36

    Chapter  Google Scholar 

  10. Aguilar, W.G., et al.: Statistical abnormal crowd behavior detection and simulation for real-time applications. In: Huang, Y., Wu, H., Liu, H., Yin, Z. (eds.) ICIRA 2017. LNCS (LNAI), vol. 10463, pp. 671–682. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65292-4_58

    Chapter  Google Scholar 

  11. Jara-Olmedo, A., Medina-Pazmiño, W., Tozer, T., Aguilar, W.G., Pardo, J.A.: E-services from emergency communication network: aerial platform evaluation. In: International Conference on eDemocracy & eGovernment (ICEDEG) (2018)

    Google Scholar 

  12. Jara-Olmedo, A., Medina-Pazmiño, W., Mesías, R., Araujo-Villaroel, B., Aguilar, W.G., Pardo, J.A.: Interface of optimal electro-optical/infrared for unmanned aerial vehicles. In: Rocha, Á., Guarda, T. (eds.) MICRADS 2018. SIST, vol. 94, pp. 372–380. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78605-6_32

    Chapter  Google Scholar 

  13. Aguilar, W., Morales, S.: 3D environment mapping using the kinect V2 and path planning based on RRT algorithms. Electronics 5, 70 (2016)

    Article  Google Scholar 

  14. Zhang, J., Liu, W., Wu, Y.: Novel technique for vision-based UAV navigation. IEEE Trans. Aerosp. Electron. Syst. 47(4), 2731–2741 (2011)

    Article  Google Scholar 

  15. Zhang, C., Kovacs, J.M.: The application of small unmanned aerial systems for precision agriculture: a review. Precis. Agriculture 13(6), 693–712 (2012)

    Article  Google Scholar 

  16. Nex, F., Fabio, R.: UAV for 3D mapping applications: a review. Appl. Geomat. 6(1), 1–15 (2013)

    Article  Google Scholar 

  17. Aguilar, W.G., Angulo, C., Costa-Castello, R.: Autonomous navigation control for quadrotors in trajectories tracking. In: Huang, Y., Wu, H., Liu, H., Yin, Z. (eds.) ICIRA 2017. LNCS (LNAI), vol. 10464, pp. 287–297. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65298-6_27

    Chapter  Google Scholar 

  18. Aguilar, W.G., Salcedo, V.S., Sandoval, D.S., Cobeña, B.: Developing of a video-based model for UAV autonomous navigation. In: Barone, D.A.C., Teles, E.O., Brackmann, C.P. (eds.) LAWCN 2017. CCIS, vol. 720, pp. 94–105. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71011-2_8

    Chapter  Google Scholar 

  19. Salcedo, V.S., Aguilar, W.G., Cobeña, B., Pardo, J.A., Proaño, Z.: On-board target virtualization using image features for UAV autonomous tracking. In: Boudriga, N., Alouini, M.-S., Rekhis, S., Sabir, E., Pollin, S. (eds.) UNet 2018. LNCS, vol. 11277, pp. 384–391. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02849-7_34

    Chapter  Google Scholar 

  20. Fong, T., Thorpe, C.: Vehicle teleoperation interfaces. Auton. Robots 11(1), 9–18 (2001)

    Article  Google Scholar 

  21. Aguilar, W.G., Luna, M., Moya, J., Abad, V., Parra, H., Ruiz, H.: Pedestrian detection for UAVs using cascade classifiers with meanshift. In: IEEE 11th International Conference on Semantic Computing (ICSC), San Diego (2017)

    Google Scholar 

  22. Aguilar, W., Rodriguez, G., Álvarez, L.: On-board visual SLAM on a UGV using a RGB-D camera. Intell. Robot. Appl. 10464, 298–308 (2017)

    Article  Google Scholar 

  23. Amaguaña, F., Collaguazo, B., Tituaña, J.: Simulation system based on augmented reality for optimization of training tactics on military operations. Augment. Reality Virtual Reality Comput. Graph. 10850, 298–308 (2017)

    Google Scholar 

  24. Aguilar, W.G., Manosalvas, J.F., Guillén, J.A., Collaguazo, B.: Robust motion estimation based on multiple monocular camera for indoor autonomous navigation of micro aerial vehicle. In: De Paolis, L.T., Bourdot, P. (eds.) AVR 2018. LNCS, vol. 10851, pp. 547–561. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95282-6_39

    Chapter  Google Scholar 

  25. Aguilar, W.G., Abad, V., Ruiz, H., Aguilar, J., Aguilar-Castillo, F.: RRT-based path planning for virtual bronchoscopy simulator. In: De Paolis, L.T., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10325, pp. 155–165. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60928-7_13

    Chapter  Google Scholar 

  26. Aguilar, W.G., Cobeña, B., Rodriguez, G., Salcedo, V.S., Collaguazo, B.: SVM and RGB-D sensor based gesture recognition for UAV control. In: De Paolis, L.T., Bourdot, P. (eds.) AVR 2018. LNCS, vol. 10851, pp. 713–719. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95282-6_50

    Chapter  Google Scholar 

  27. RifaïM, H., Hua, D., Hamel, T., Morin, P.: Haptic-based bilateral teleoperation of underactuated unmanned aerial vehicles. IFAC Proc. Volumes (IFAC-PapersOnline) 44, 13782–13788 (2011)

    Article  Google Scholar 

  28. Lee, D., Franchi, A., Son, H.I., Ha, C., Heinrich, H., Giordano, P.R.: Semi-autonomous haptic teleoperation control architecture of multiple unmanned aerial vehicles. IEEE/ASME Trans. Mechatron. 18, 1334–1345 (2013)

    Article  Google Scholar 

  29. Lam, T., D’Amelio, V., Mulder, M., Van Paassen, M.M: UAV tele-operation using haptics with a degraded visual interface. In: Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, vol. 3, pp. 2440–2445 (2007)

    Google Scholar 

  30. Chen, J.: UAV-guided navigation for ground robot tele-operation in a military reconnaissance environment. Ergonomics 53(8), 940–950 (2010)

    Article  Google Scholar 

  31. Galarza, J., Pérez, E., Serrano, E., Tapia, A., Aguilar, Wilbert G.: Pose estimation based on monocular visual odometry and lane detection for intelligent vehicles. In: De Paolis, L.T., Bourdot, P. (eds.) AVR 2018. LNCS, vol. 10851, pp. 562–566. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95282-6_40

    Chapter  Google Scholar 

  32. Omari, S., Hua, M., Dueard, G., Hamel, T.: Bilateral haptic teleoperation of VTOL UAVs. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2393–2399 (2013)

    Google Scholar 

  33. Smisek, J., Sunil, E., Van Paassen, M.M., Abbink, D.A., Mulder, M.: Neuromuscular-system-based tuning of a haptic shared control interface for UAV teleoperation. IEEE Trans. Hum.-Mach. Syst. 47(4), 449–461 (2017)

    Article  Google Scholar 

  34. Kanso, A., Elhajj, I.H., Shammas, E., Asmar, D.: Enhanced teleoperation of UAVs with haptic feedback. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM, pp. 305–310 (2015)

    Google Scholar 

  35. Lee, D., Franchi, A., Giordano, P., Son, H., Bülthoff, H.H.: Haptic teleoperation of multiple unmanned aerial vehicles over the internet. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 1341–1347 (2011)

    Google Scholar 

  36. Guoqin, G.: Navigating path recognition for greenhouse mobile robot based on K-means algorithm. Trans. Chin. Soc. Agric. Eng. 30, 25–33 (2014)

    Google Scholar 

  37. Al-kaff, A., Meng, Q., Mart, D.: Monocular vision-based obstacle detection/avoidance for unamnned aerial vehicles. In: IEEE Intelligent Vehicles Symposium (2016)

    Google Scholar 

  38. De Cristóforis, P., Nitsche, M.A., Krajník, T., Mejail, M.: Real-time monocular image-based path detection: a GPU-based embedded solution for on-board execution on mobile robots. J. Real-Time Image Process. 11(2), 335–348 (2016)

    Article  Google Scholar 

  39. Aguilar, W., Casaliglla, V., Pólit, J.: Obstacle avoidance based-visual navigation for micro aerial vehicles. Electronics 6, 10 (2017)

    Article  Google Scholar 

  40. Aguilar, W.G., et al.: Pedestrian detection for UAVs using cascade classifiers and saliency maps. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2017. LNCS, vol. 10306, pp. 563–574. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59147-6_48

    Chapter  Google Scholar 

  41. Odelga, M., Stegagno, P., Bulthoff, H.H.: Obstacle detection, tracking and avoidance for a teleoperated UAV. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 2984–2990 (2016)

    Google Scholar 

  42. Aleotti, J., et al.: Detection of nuclear sources by UAV teleoperation using a visuo-haptic augmented reality interface. Sensors 17(10), 1–22 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CICTE, Universidad de las Fuerzas Armadas ESPE, Sangolquí, Ecuador

    Santiago Grijalva & Wilbert G. Aguilar

  2. FIS, Escuela Politécnica Nacional, Quito, Ecuador

    Wilbert G. Aguilar

  3. GREC, Universitat Politècnica de Catalunya, Barcelona, Spain

    Wilbert G. Aguilar

Authors
  1. Santiago Grijalva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wilbert G. Aguilar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wilbert G. Aguilar .

Editor information

Editors and Affiliations

  1. Shenyang Institute of Automation, Shenyang, China

    Haibin Yu

  2. Shenyang Institute of Automation, Shenyang, China

    Jinguo Liu

  3. Shenyang Institute of Automation, Shenyang, China

    Lianqing Liu

  4. University of Portsmouth, Portsmouth, UK

    Zhaojie Ju

  5. Shenyang Institute of Automation, Shenyang, China

    Yuwang Liu

  6. University of Portsmouth, Portsmouth, UK

    Dalin Zhou

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Grijalva, S., Aguilar, W.G. (2019). Landmark-Based Virtual Path Estimation for Assisted UAV FPV Tele-Operation with Augmented Reality. In: Yu, H., Liu, J., Liu, L., Ju, Z., Liu, Y., Zhou, D. (eds) Intelligent Robotics and Applications. ICIRA 2019. Lecture Notes in Computer Science(), vol 11745. Springer, Cham. https://doi.org/10.1007/978-3-030-27529-7_58

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27529-7_58

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27528-0

  • Online ISBN: 978-3-030-27529-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation