A Testbed for Vision-Based Networked Control Systems

  • Christoph BachhuberEmail author
  • Simon Conrady
  • Michael Schütz
  • Eckehard Steinbach
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10528)


With the availability of low-latency wireless communication and low-delay video communication solutions, vision-based networked control systems (NCS) become feasible. In this paper, we describe an NCS testbed which is suitable for the evaluation of the interplay of computer vision algorithms, network protocols and control algorithms under a delay constraint. The system comprises an inverted pendulum which is monitored by a video camera. The h.264-encoded video is sent over a network to an image processing computer. This computer extracts the angle of inclination of the pendulum from the decoded video and sends it over a wireless link to the pendulum. The pendulum uses the angle in a control algorithm to keep itself in a vertical position. We provide a detailed description of the system including the control algorithm and the image processing algorithms, and analyse the latency contributors of the system. The build instructions and source code of the testbed are publicly available. As the testbed is based on standard low-priced components, it is particularly suitable for educational purposes.


Vision-based control Networked control systems Computer vision systems Low delay video communication 


  1. 1.
    x264 - a free open-source h.264 encoder. 30 Jan 2017
  2. 2.
    Alt, N., Steinbach, E.: Visuo-haptic sensor for force measurement and contact shape estimation. In: Proceedings of IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE), pp. 24–28 (2013)Google Scholar
  3. 3.
    Bachhuber, C., Steinbach, E.: A system for high precision glass-to-glass delay measurements in video communication. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 2132–2136 (2016)Google Scholar
  4. 4.
    Bachhuber, C., Steinbach, E.: Are today’s video communication solutions ready for the tactile internet? In: IEEE Wireless Communications and Networking Conference Workshops (WCNCW) (2017)Google Scholar
  5. 5.
    Boccardi, F., et al.: Five disruptive technology directions for 5G. IEEE Commun. Mag. 52(2), 74–80 (2014)CrossRefGoogle Scholar
  6. 6.
    Bonafilia, B., et al.: Self-Balancing Two-wheeled Robot (2015)Google Scholar
  7. 7.
    Bradski, G.: Open source CV library. Dr. Dobb’s J. Softw. Tools (2000)Google Scholar
  8. 8.
    Chan, R.P.M., Stol, K.A., Halkyard, C.R.: Review of modelling and control of two-wheeled robots. Ann. Rev. Control 37(1), 89–103 (2013)CrossRefGoogle Scholar
  9. 9.
    Chen, C.C., Wu, H., Kühnlenz, K., Hirche, S.: Switching control for a networked vision-based control system. Automatisierungstechnik Methoden und Anwendungen der Steuerungs-, Regelungs-und Informationstechnik 59(2), 124–133 (2011)Google Scholar
  10. 10.
    Eide, R., et al.: LQG control design for balancing an inverted pendulum mobile robot. Intell. Control Autom. 2(02), 160 (2011)CrossRefGoogle Scholar
  11. 11.
    Eker, J., Cervin, A., Hörjel, A.: Distributed wireless control using bluetooth. In: Proceedings of the IFAC Conference on New Technologies for Computer Control (2001)Google Scholar
  12. 12.
    Espinoza-Quesada, E., Ramos-Velasco, L.: Visual servoing for an inverted pendulum using a digital signal processor. In: 2006 IEEE International Symposium on Signal Processing and Information Technology (2006)Google Scholar
  13. 13.
    Hirata, K., Kimura, Y., Sugimoto, K.: Visual feedback control of cart-pendulum systems with webcam. In: IEEE International Conference on Mechatronics (ICM), pp. 1–6. IEEE (2007)Google Scholar
  14. 14.
    Kizir, S., Ocak, H., Bingul, Z., Oysu, C.: Time delay compensated vision based stabilization control of an inverted pendulum. Int. J. Innov. Comput. Inf. Control 8(12), 8133–8145 (2012)Google Scholar
  15. 15.
    Milton, J., Cabrera, J.L., Ohira, T., Tajima, S., Tonosaki, Y., Eurich, C.W., Campbell, S.A.: The time-delayed inverted pendulum: implications for human balance control. Chaos: Interdisc. J. Nonlinear Sci. 19(2), 026110 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Ooi, R.C.: Balancing a two-wheeled autonomous robot. University of Western Australia 3 (2003)Google Scholar
  17. 17.
    Tu, Y.W., Ho, M.T.: Design and implementation of robust visual servoing control of an inverted pendulum with an fpga-based image co-processor. Mechatronics 21(7), 1170–1182 (2011)CrossRefGoogle Scholar
  18. 18.
    Wang, H., et al.: Hybrid control for vision based cart-inverted pendulum system. In: American Control Conference, pp. 3845–3850. IEEE (2008)Google Scholar
  19. 19.
    Wenzel, L., Vazquez, N., Nair, D., Jamal, R.: Computer vision based inverted pendulum. In: Proceedings of the 17th IEEE Instrumentation and Measurement Technology Conference, IMTC 2000, vol. 3, pp. 1319–1323. IEEE (2000)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Chair of Media TechnologyTechnical University of MunichMunichGermany

Personalised recommendations