Skip to main content

FPGA-Based Fast Response Image Analysis for Orientational Control in Aerial Manipulation Tasks

Journal of Signal Processing Systems volume 90, pages 901–911 (2018)Cite this article

Abstract

In this paper an FPGA-based on-board control system for autonomous orientation of an aerial robot to assist in aerial manipulation tasks is introduced. The system is able to apply yaw control to aid an operator in precisely positioning a drone when it is nearby a bar-like object. This is achieved by applying parallel Hough transform combined with a novel image space separation method, enabling highly reliable results in various circumstances combined with high performance. The feasibility of this approach is shown by applying the system to a multi-rotor aerial robot equipped with an upward directed robotic hand on top of the airframe developed for high altitude manipulation tasks. In order to grasp a bar-like object, the object’s orientation is derived from the image data obtained by a monocular camera mounted on the robot. This data is then analyzed by the on-board FPGA system to control yaw angle of the aerial robot. Our experiments show that use of this control system achieves reliable yaw-orientation control.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17

References

  1. Fukushima, A., & Kawaguchi, Y. (2015). Insect leg inspired friction attachment for miniature quadcopter. In Proceedings of the 12th International Conference on Advances in Computer Entertainment Technology (p. 34): ACM.

  2. Honegger, D., Greisen, P., Meier, L., Tanskanen, P., & Pollefeys, M. (2012). Real-time velocity estimation based on optical flow and disparity matching. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5177–5182): IEEE.

  3. Honegger, D., Oleynikova, H., & Pollefeys, M. (2014). Real-time and low latency embedded computer vision hardware based on a combination of fpga and mobile cpu. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014) (pp. 4930–4935): IEEE.

  4. Jiang, H., Pope, M.T., Hawkes, E.W., Christensen, D.L., Estrada, M.A., Parlier, A., Tran, R., & Cutkosky, M.R. (2014). Modeling the dynamics of perching with opposed-grip mechanisms. In 2014 IEEE International Conference on Robotics and Automation (ICRA) (pp. 3102–3108): IEEE.

  5. Kim, S., Choi, S., & Kim, H. (2013). Aerial manipulation using a quadrotor with a two dof robotic arm. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 4990–4995).

  6. Konomura, R., & Hori, K. (2014). Visual 3d self localization with 8 gram circuit board for very compact and fully autonomous unmanned aerial vehicles. In 2014 IEEE International Conference on Robotics and Automation (ICRA) (pp. 5215–5220): IEEE.

  7. Koo, V.C., Chan, Y.K., Vetharatnam, G., Chua, M.Y., Lim, C.H., Lim, C.S., Thum, C., Lim, T.S., bin Ahmad, Z., Mahmood, K.A., & et al. (2012). A new unmanned aerial vehicle synthetic aperture radar for environmental monitoring. Progress In Electromagnetics Research, 122, 245–268.

    Article  Google Scholar 

  8. Koshimizu, H., & Numada, M. (1992). Fiht2 algorithm: a fast incremental hough transform. In IEICE Transactions on Information and Systems (pp. 3389–3393).

  9. Ladig, R., Shimonomuram, K. (2014). Fpga-based fast response image analysis for autonomous or semi-autonomous indoor flight. In 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 682–687).

  10. Lucia, S.D., Tipaldi, G.D., & Burgard, W. (2015). Attitude stabilization control of an aerial manipulator using a quaternion-based backstepping approach. In 2015 European Conference Mobile Robots (ECMR) (pp. 1–6).

  11. Mohta, K., Kumar, V., & Daniilidis, K. (2014). Vision-based control of a quadrotor for perching on lines. In 2014 IEEE International Conference on Robotics and Automation (ICRA) (pp. 3130–3136): IEEE.

  12. Murphy, R.R., Tadokoro, S., Nardi, D., Jacoff, A., Fiorini, P., Choset, H., & Erkmen, A.M. (2008). Search and rescue robotics. In Springer Handbook of Robotics (pp. 1151–1173): Springer.

  13. Orsag, M., Korpela, C., Pekala, M., & Oh, P. (2013). Stability control in aerial manipulation. In American Control Conference (ACC) (pp. 5581–5586).

  14. Schmid, K., & Hirschmüller, H. (2013). Stereo vision and imu based real-time ego-motion and depth image computation on a handheld device. In 2013 IEEE International Conference on Robotics and Automation (ICRA) (pp. 4671–4678): IEEE.

  15. Semsch, E., Jakob, M., Pavlicek, D., & Pechoucek, M. (2009). Autonomous uav surveillance in complex urban environments. In 2009. WI-IAT’09. IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technologies, (Vol. 2 pp. 82–85): IEEE.

  16. Shimahara, S., Ladig, R., Suphachart, L., Hirai, S., & Shimonomura, K. (2015). Aerial manipulation for the workspace above the airframe. In 2015 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS) (pp. 1453–1458).

  17. Tagzout, S., Achour, K., & Djekoune, O. (2000). Hough transform algorithm for fpga implementation. In Signal Processing Systems, 2000. SiPS 2000 IEEE Workshop (pp. 384–393).

  18. Thomas, J., Loianno, G., Daniilidis, K., & Kumar, V. (2016). Visual servoing of quadrotors for perching by hanging from cylindrical objects. IEEE Robotics and Automation Letters, 1(1), 57–64.

    Article  Google Scholar 

  19. Thomas, J., Loianno, G., Sreenath, K., & Kumar, V. (2014). Toward image based visual servoing for aerial grasping and perching. In 2014 IEEE International Conference on Robotics and Automation (ICRA) (pp. 2113–2118): IEEE.

Download references

Author information

Authors and Affiliations

  1. Department of Robotics, Ritsumeikan University, Kusatsu, Shiga, 525-8577, Japan

    Robert Ladig, Suphachart Leewiwatwong & Kazuhiro Shimonomura

Authors
  1. Robert Ladig
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suphachart Leewiwatwong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kazuhiro Shimonomura
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert Ladig.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ladig, R., Leewiwatwong, S. & Shimonomura, K. FPGA-Based Fast Response Image Analysis for Orientational Control in Aerial Manipulation Tasks. J Sign Process Syst 90, 901–911 (2018). https://doi.org/10.1007/s11265-017-1286-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-017-1286-y

Keywords

  • FPGA
  • Embedded image processing
  • Parallel hough transform
  • Aerial manipulation
  • UAV