Visual Motion Tracking and Sensor Fusion for Kite Power Systems

  • Henrik Hesse
  • Max Polzin
  • Tony A. Wood
  • Roy S. Smith
Part of the Green Energy and Technology book series (GREEN)


An estimation approach is presented for kite power systems with groundbased actuation and generation. Line-based estimation of the kite state, including position and heading, limits the achievable cycle efficiency of such airborne wind energy systems due to significant estimation delay and line sag. We propose a filtering scheme to fuse onboard inertial measurements with ground-based line data for ground-based systems in pumping operation. Estimates are computed using an extended Kalman filtering scheme with a sensor-driven kinematic process model which propagates and corrects for inertial sensor biases. We further propose a visual motion tracking approach to extract estimates of the kite position from ground-based video streams. The approach combines accurate object detection with fast motion tracking to ensure long-term object tracking in real time. We present experimental results of the visual motion tracking and inertial sensor fusion on a ground-based kite power system in pumping operation and compare both methods to an existing estimation scheme based on line measurements.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This research was support by the Swiss National Science Foundation (Synergia) No. 141836 and the Swiss Commission for Technology and Innovation (CTI). We further acknowledge Corey Houle, Damian Aregger, and Jannis Heilmann from the Fachhochschule Nordwestschweiz (FHNW) for their test support and providing all the hardware used in the experiments. Development of the hardware architecture enabling onboard measurements was done by Martin Rudin and Alexander Millane (ETH Zurich). We are grateful for their support. The authors acknowledge the SpeedGoat Greengoat program.


  1. 1.
    A.,W. T., Hesse, H., Zgraggen, A. U., Smith, R. S.: Model-based Identification and Control of the Velocity Vector Orientation for Autonomous Kites. In: Proceedings of the 2015 American Control Conference, Chicago, IL, USA, 1–3 July 2015.
  2. 2.
    Anderson, B. D. O., Moore, J. B.: Optimal Filtering. English. Dover Publications, Mineola, N.Y., USA (2005)Google Scholar
  3. 3.
    Appel, R., Fuchs, T., Dollár, P.: Quickly Boosting Decision Trees – Pruning Underachieving Features Early. In: International Conference on Machine Learning (ICML), vol. 28, pp. 594–602, Atlanta, USA, June 2013Google Scholar
  4. 4.
    Autonomous Airborne Wind Energy Project (A2WE). Accessed 31 Oct 2015
  5. 5.
    Bormann, A., Ranneberg, M., Kövesdi, P., Gebhardt, C., Skutnik, S.: Development of a Three-Line Ground-Actuated Airborne Wind Energy Converter. In: Ahrens, U., Diehl, M., Schmehl, R. (eds.) Airborne Wind Energy, Green Energy and Technology, Chap. 24, pp. 427–437. Springer, Berlin Heidelberg (2013).
  6. 6.
    Bourdev, L., Brandt, J.: Robust object detection via soft cascade. In: Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 236–243, San Diego, CA, USA, June 2005.
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 886–893, San Diego, CA, USA (2005).
  8. 8.
    Dollár, P., Appel, R., Belongie, S., Perona, P.: Fast Feature Pyramids for Object Detection. 36(8), 1532–1545 (2014).
  9. 9.
    Dollár, P., Appel, R., Kienzle, W.: Crosstalk Cascades for Frame-Rate Pedestrian Detection. In: European Conference on Computer Vision, pp. 645–659, Florence, Italy (2012).
  10. 10.
    Erhard, M., Strauch, H.: Theory and Experimental Validation of a Simple Comprehensible Model of Tethered Kite Dynamics Used for Controller Design. In: Ahrens, U., Diehl, M., Schmehl, R. (eds.) Airborne Wind Energy, Green Energy and Technology, Chap. 8, pp. 141–165. Springer, Berlin Heidelberg (2013).
  11. 11.
    Erhard, M., Strauch, H.: Sensors and Navigation Algorithms for Flight Control of Tethered Kites. In: Proceedings of the European Control Conference (ECC13), Zurich, Switzerland, 17–19 July 2013. arXiv:1304.2233 [cs.SY]
  12. 12.
    Ess, A., Leibe, B., Gool, L. V.: Depth and appearance for mobile scene analysis. In: International Conference on Computer Vision (ICCV), pp. 1–8, Rio de Janeiro, Brazil (2007).
  13. 13.
    Fagiano, L., Zgraggen, A. U., Morari, M., Khammash, M.: Automatic crosswind flight of tethered wings for airborne wind energy:modeling, control design and experimental results. IEEE Transactions on Control System Technology 22(4), 1433–1447 (2014).
  14. 14.
    Fagiano, L., Huynh, K., Bamieh, B., Khammash, M.: On sensor fusion for airborne wind energy systems. IEEE Transactions on Control Systems Technology 22(3), 930–943 (2014).
  15. 15.
    Freund, Y., Schapire, R. E.: Experiments with a New Boosting Algorithm. In: International Conference on Machine Learning (ICML), pp. 148–156, Bari, Italy, July 1996Google Scholar
  16. 16.
    Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. The Annals of Statistics 28(2), 337–407 (2000)Google Scholar
  17. 17.
    Gray, R. M.: Toeplitz and Circulant Matrices: A Review. Foundations and Trends in Communications and Information Theory 2(3), 155–239 (2006).
  18. 18.
    Heikkila, J., Silvén, O.: A four-step camera calibration procedure with implicit image correction. In: Conference on Computer Vision and Pattern Recognition, pp. 1106–1112, IEEE, San Juan, Puerto Rico, June 1997Google Scholar
  19. 19.
    Henriques, J. F., Caseiro, R., Martins, P., Batista, J.: High-Speed Tracking with Kernelized Correlation Filters. 37(3), 583–596 (2015).
  20. 20.
    Lefferts, E. J., Markley, F. L., Shuster, M. D.: Kalman filtering for spacecraft attitude estimation. Journal of Guidance, Control, and Dynamics 5(5), 417–429 (1982)Google Scholar
  21. 21.
    MATLAB® Computer Vision System ToolboxTM Reference, MathWorks, Inc., Natick, MA, USA, 2015.
  22. 22.
    Millane, A.,Wood, T. A., Hesse, H., Zgraggen, A. U., Smith, R. S.: Range-Inertial Estimation for Airborne Wind Energy. In: Conference on Decision and Control (CDC), pp. 455–460, Osaka, Japan, Dec 2015.
  23. 23.
    Pixhawk Autopilot. Accessed 31. October 2015.
  24. 24.
    Polzin, M., Hesse, H., Wood, T. A., Smith, R. S.: Visual Motion Tracking for Estimation of Kite Dynamics. In: Schmehl, R. (ed.). Book of Abstracts of the International Airborne Wind Energy Conference 2015, p. 110, Delft, The Netherlands, 15–16 June 2015. Poster available from:
  25. 25.
    Sabatini, A. M.: Kalman-filter-based orientation determination using inertial/magnetic sensors: Observability analysis and performance evaluation. Sensors 11(10), 9182–9206 (2011)Google Scholar
  26. 26.
    Savage, P. G.: Strapdown Inertial Navigation Integration Algorithm Design Part 2: Velocity and Position Algorithms. Journal of Guidance, Control, and Dynamics 21(2), 208–221 (1998)Google Scholar
  27. 27.
    Schölkopf, B., Herbrich, R., Smola, A. J.: A Generalized Representer Theorem. In: Computational Learning Theory, pp. 416–426. Springer-Verlag, Berlin, Germany (2001)Google Scholar
  28. 28.
    Shawe-Taylor, J., Cristianini, N.: Kernel methods for pattern analysis. Cambridge University Press (2004)Google Scholar
  29. 29.
    Smeulders, A.W. M., Chu, D. M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: An experimental survey. 36(7), 1442–1468 (2014).
  30. 30.
    Speedgoat User Story: Efficiently harnessing wind power high above the ground using autonomous kites, Speedgoat GmbH, Liebefeld, Switzerland, 2015.
  31. 31.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 511–518, (2001).
  32. 32.
    Wood, T. A., Hesse, H., Zgraggen, A. U., Smith, R. S.: Model-Based Flight Path Planning and Tracking for Tethered Wings. In: Conference on Decision and Control (CDC), pp. 6712–6717, Osaka, Japan, Dec 2015.
  33. 33.
    Wu, Y., Lim, J., Yang, M. H.: Online object tracking: A benchmark. In: Conference on Computer Vision and Pattern Recognition, pp. 2411–2418, (2013).
  34. 34.
    Wu, Y., Lim, J., Yang, M.-H.: Object Tracking Benchmark. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, 9, pp. 1834–1848, (2015).
  35. 35.
    Zgraggen, A. U., Fagiano, L., Morari, M.: Automatic Retraction and Full-Cycle Operation for a Class of Airborne Wind Energy Generators. IEEE Transactions on Control Systems Technology 24(2), 594–608 (2015).

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Henrik Hesse
    • 1
  • Max Polzin
    • 1
  • Tony A. Wood
    • 1
  • Roy S. Smith
    • 1
  1. 1.Automatic Control LaboratoryETH ZurichZurichSwitzerland

Personalised recommendations