Image-based supervision of a periodically working machine


Most industrial robots perform a periodically repeating choreography. Our aim is to detect disturbances of such a periodic process by a visual inspection system that can be trained with a minimum of human effort and interaction. We present a solution that monitors the robot with a time-of-flight 3D camera. Our system can be trained using a few unperturbed cycles of the periodic process. More specifically, principal components are used to find a low-dimensional approximation of each frame, and a One-Class Support Vector Machine is used for one-class learning. We propose a novel scheme for automatic parameter tuning, which exploits the fact that successive images of the training class should be close in feature space. We present exemplary results for a miniature robot setup. The proposed strategy does not require prior information on the dimensions of the machine or its maneuvering range. The entire system is appearance-based and hence does not need access to the robot’s internal coordinates.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8


  1. 1.


  1. 1.

    Novak JL, Feddema JT (1992) A capacitance-based proximity sensor for whole arm obstacle avoidance. In: IEEE Proceedings of the international conference on robotics and automation, pp 1307–11314

  2. 2.

    Feddema JT, Novak JL (1994) Whole arm obstacle avoidance for teleoperated robots. In: IEEE proceedings of the international conference on robotics and automation, pp 3303–3309

  3. 3.

    Yu Y, Gupta K (1999) Sensor based roadmaps for motion-planning for articulated robots in unknown environment: some experiments with an eye-in-hand system. In: IEEE proceedings of the international conference on intelligent robots and systems, pp 1707–1714

  4. 4.

    Noborio H, Nishino Y (2001) Image-based path-planning algorithm on the joint space. In: IEEE proceedings of the international conference on robotics and automation, Seoul, pp 1180–1187

  5. 5.

    Lumelsky V, Cheung E (1993) Real-time collision avoidance in teleoperated whole-sensitive robot arm manipulators. IEEE Trans Syst Man Cybern 23(1):194–203

    Article  Google Scholar 

  6. 6.

    Goujou E, Miteran J, Laligant O, Truchetet F, Gorria P (1995) Human detection with video surveillance system. In: Proceedings of the IEEE international conference on industrial electronics, control, and instrumentation, vol 2, pp 1179–1184

  7. 7.

    Morioka K, Lee J-H, Hashimoto H (2004) Human-following mobile robot in a distributed intelligent sensor network. IEEE Trans Ind Electron 51(1):229–237

    Article  Google Scholar 

  8. 8.

    Vadakkepat P, Lim P, De Silva LC, Jing L, Ling LL (2008) Multimodal approach to human-face detection and tracking. IEEE Trans Ind Electron 55(3):1385–1393

    Article  Google Scholar 

  9. 9.

    Eckert G (2000) Automatic shape reconstruction of rigid 3D objects from multiple calibrated images. In: Proceedings on Eusipco 2000, Tampere

  10. 10.

    Meisel A (1994) 3D-Bildverarbeitung für feste und bewegte Kameras. Vieweg Publishing, Fortschritte der Robotik Nr.21

  11. 11.

    Gecks T, Henrich D (2006) Multi-camera collision detection allowing for object occlusions. In: 37th international symposium on robotics (ISR 2006)/4th German conference on robotics (Robotik 2006)

  12. 12.

    Henrich D, Gecks T (2008) Multi-camera collision detection between known and unknown objects. In: 2nd ACM/IEEE international conference on distributed smart cameras

  13. 13.

    Ebert DM, Henrich D (2002) Human–robot-cooperation: image-based collision detection for industrial robots. In: IEEE international conference on intelligent robots and systems

  14. 14.

    Kumar A (2008) Computer-vision-based fabric defect detection: a survey. IEEE Trans Ind Electron 55(1):348–363

    Article  Google Scholar 

  15. 15.

    Sarmis T, Argyros AA, Lourakis MIA, Hatzopoulos K (2006) Robust and efficient event detection for the monitoring of automated processes. In: VIE 2006 proceedings of the international conference on visual information engineering, pp 454–459

  16. 16.

    Jäger M, Hamprecht FA (2009) Principal component imagery for the quality monitoring of dynamic laser welding processes. IEEE Trans Ind Electron 56(4):1307–1313

    Article  Google Scholar 

  17. 17.

    Schwarte R, Heinol HG, Xu Z, Hartmann K (1995) A new active 3D-Vision system based on RF-modulation interferometry of incoherent light. In: Photonics east—intelligent systems and advanced manufacturing, Proceedings of the SPIE, vol 2588, Philadelphia

  18. 18.

    Spirig T, Seitz P, Vietze O, Heitger F (1995) The lock-in CCD—twodimensional synchronous detection of light. IEEE J Quantum Electron 31:1705–1708

    Article  Google Scholar 

  19. 19.

    Frank M, Plaue M, Rapp H, Köthe U, Jähne B, Hamprecht FA (2009) Theoretical and experimental error analysis of continuous-wave time-of-flight range cameras. Opt Eng 48(1)

  20. 20.

    Frank M, Plaue M, Hamprecht FA (2009) Denoising of continuous-wave time-of-flight depth images using confidence measures. Opt Eng 48(7)

  21. 21.

    Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  22. 22.

    Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Article  Google Scholar 

  23. 23.

    Schölkopf B, Smola AJ, Müller KR (1997) Kernel principal component analysis. Lecture notes in computer science. Springer, Berlin, vol 1327, pp 583–588

  24. 24.

    Schölkopf B, Platt JC, Shawe-Taylor J, Smola AJ (1999) Estimating the support of a high-dimensional distribution. Technical report, Microsoft research

  25. 25.

    Schölkopf B, Smola AJ (2002) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge

  26. 26.

    Vapnik V (1995) The nature of statistical learning theory. Springer, New York

    Google Scholar 

  27. 27.

    Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural computation. MIT Press, Cambridge vol 12, no. 5, pp 1207–1245

  28. 28.

    Weston J, Elisseeff A, Bakir G, Sinz F (2006) The spider—machine learning toolbox for Matlab. Department of Empirical Inference, Max-Planck Institute for biological Cybernetics, Tübingen.

  29. 29.

    Zhou T, Tao D, Wu X (2010) Manifold elastic net: a unified framework for sparse dimension reduction. Data Min Knowl Discov. doi:10.1007/s10618-010-0182-x

Download references

Author information



Corresponding author

Correspondence to Mario Frank.

Electronic supplementary material

Below is the link to the electronic supplementary material.

wmv (4.16 MB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Frank, M., Hamprecht, F.A. Image-based supervision of a periodically working machine. Pattern Anal Applic 16, 407–416 (2013).

Download citation


  • Robot
  • Security
  • TOF
  • Monitoring
  • Novelty detection
  • 3D camera