Abstract
Combining the agility of Micro Aerial Vehicles (MAV) with the dexterity of robotic arms leads to a new era of Aerial Robotic Workers (ARW) targeting infrastructure inspection and maintenance tasks. Towards this vision, this work focuses on the autonomous guidance of the aerial end-effector to either reach or keep desired distance from areas/objects of interest. The proposed system: 1) is structured around a real-time object tracker, 2) employs stereo depth perception to extract the target location within the surrounding scene, and finally 3) generates feasible poses for both the arm and the MAV relative to the target. The performance of the proposed scheme is experimentally demonstrated in multiple scenarios of increasing complexity.
Article PDF
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
References
Mansouri, S.S., Kanellakis, C., Fresk, E., Kominiak, D., Nikolakopoulos, G.: Cooperative uavs as a tool for aerial inspection of the aging infrastructure. In: Field and Service Robotics, pp 177–189. Springer (2018)
Lee, A.C., Dahan, M., Weinert, A.J., Amin, S.: Leveraging suas for infrastructure network exploration and failure isolation. J. Intell. Robot. Syst., 1–29 (2018)
Yuan, C., Liu, Z., Zhang, Y.: Learning-based smoke detection for unmanned aerial vehicles applied to forest fire surveillance. Journal of Intelligent & Robotic Systems. https://doi.org/10.1007/s10846-018-0803-y (2018)
Sampedro, C., Rodriguez-Ramos, A., Bavle, H., Carrio, A., de la Puente, P., Campoy, P.: A fully-autonomous aerial robot for search and rescue applications in indoor environments using learning-based techniques. Journal of Intelligent & Robotic Systems. https://doi.org/10.1007/s10846-018-0898-1 (2018)
Kondak, K., Ollero, A., Maza, I., Krieger, K., Albu-Schaeffer, A., Schwarzbach, M., Laiacker, M.: Unmanned aerial systems physically interacting with the environment: Load transportation, deployment, and aerial manipulation. In: Handbook of Unmanned Aerial Vehicles, pp 2755–2785. Springer (2015)
Lindsey, Q., Mellinger, D., Kumar, V.: Construction of cubic structures with quadrotor teams. In: Proc. Robotics, Science & Systems VII (2011)
Kim, S., Seo, H., Choi, S., Kim, H.J.: Vision-guided aerial manipulation using a multirotor with a robotic arm. IEEE/ASME Trans. Mechatron. 21(4), 1912 (2016)
Seo, H., Kim, S., Kim, H.J.: Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp 6362–6368. IEEE (2017)
Santamaria-Navarro, A., Grosch, P., Lippiello, V., Sola, J., Andrade-Cetto, J.: Uncalibrated visual servo for unmanned aerial manipulation. IEEE/ASME Transactions on Mechatronics (2017)
Steich, K., Kamel, M., Beardsley, P., Obrist, M.K., Siegwart, R., Lachat, T.: Tree cavity inspection using aerial robots. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 4856–4862. IEEE (2016)
Ramon Soria, P., Arrue, B.C., Ollero, A.: Detection, location and grasping objects using a stereo sensor on uav in outdoor environments. Sensors 17(1), 103 (2017)
Lippiello, V., Cacace, J., Santamaria-Navarro, A., Andrade-Cetto, J., Trujillo, M.A., Esteves, Y.R., Viguria, A.: Hybrid visual servoing with hierarchical task composition for aerial manipulation. IEEE Robot. Autom. Lett. 1(1), 259 (2016)
Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2014.2345390(2015)
Compact AeRial MAnipulator (CARMA) Assembly overwiew. https://www.youtube.com/watch?v=rOV5K43cpho
Wuthier, D., Kominiak, D., Kanellakis, C., Andrikopoulos, G., Fumagalli, M., Schipper, G., Nikolakopoulos, G.: On the design, modeling and control of a novel compact aerial manipulator. In: 2016 24th Mediterranean Conference on Control and Automation (MED), pp 665–670. IEEE (2016)
Lynen, S., Achtelik, M., Weiss, S., Chli, M., Siegwart, R.: A robust and modular multi-sensor fusion approach applied to mav navigation. In: Proc. of the IEEE/RSJ Conference on Intelligent Robots and Systems (IROS) (2013)
Alexis, K., Nikolakopoulos, G., Tzes, A.: Switching model predictive attitude control for a quadrotor helicopter subject to atmospheric disturbances. Control. Eng. Pract. 19(10), 1195 (2011)
Alexis, K., Nikolakopoulos, G., Tzes, A.: Model predictive quadrotor control: Attitude, altitude and position experimental studies. IET Control Theory Appl. 6(12), 1812 (2012)
Kamel, M., Stastny, T., Alexis, K., Siegwart, R.: Model Predictive Control for Trajectory Tracking of Unmanned Aerial Vehicles Using Robot Operating System, pp 3–39. Springer International Publishing, Cham (2017), https://doi.org/10.1007/978-3-319-54927-9_1
Piccinini, P., Prati, A., Cucchiara, R.: Real-time object detection and localization with sift-based clustering. Image Vis. Comput. 30(8), 573 (2012)
Smeulders, A.W., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: an experimental survey. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1442 (2014)
Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328 (2008)
Zhan, Q., Liang, Y., Xiao, Y.: Color-based segmentation of point clouds. Laser Scan. 38(3), 155 (2009)
Rusu, R.B., Cousins, S.: 3d is here: Point cloud library (pcl). In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp 1–4. IEEE (2011)
Oehler, B., Stueckler, J., Welle, J., Schulz, D., Behnke, S.: Efficient multi-resolution plane segmentation of 3d point clouds. Intell Robot Appl, 145–156 (2011)
Acknowledgements
This work has received funding from the European Unions Horizon 2020 Research and Innovation Programs under the Grant Agreements No.644128, AEROWORKS and No.730302, SIMS
Funding
Open access funding provided by Lulea University of Technology.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Kanellakis, C., Nikolakopoulos, G. Guidance for Autonomous Aerial Manipulator Using Stereo Vision. J Intell Robot Syst 100, 1545–1557 (2020). https://doi.org/10.1007/s10846-019-01060-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-019-01060-8