Pattern Analysis and Applications

, Volume 16, Issue 3, pp 407–416 | Cite as

Image-based supervision of a periodically working machine

  • Mario FrankEmail author
  • Fred A. Hamprecht
Short Paper


Most industrial robots perform a periodically repeating choreography. Our aim is to detect disturbances of such a periodic process by a visual inspection system that can be trained with a minimum of human effort and interaction. We present a solution that monitors the robot with a time-of-flight 3D camera. Our system can be trained using a few unperturbed cycles of the periodic process. More specifically, principal components are used to find a low-dimensional approximation of each frame, and a One-Class Support Vector Machine is used for one-class learning. We propose a novel scheme for automatic parameter tuning, which exploits the fact that successive images of the training class should be close in feature space. We present exemplary results for a miniature robot setup. The proposed strategy does not require prior information on the dimensions of the machine or its maneuvering range. The entire system is appearance-based and hence does not need access to the robot’s internal coordinates.


Robot Security TOF Monitoring Novelty detection 3D camera 

Supplementary material

wmv (4.16 MB)


  1. 1.
    Novak JL, Feddema JT (1992) A capacitance-based proximity sensor for whole arm obstacle avoidance. In: IEEE Proceedings of the international conference on robotics and automation, pp 1307–11314Google Scholar
  2. 2.
    Feddema JT, Novak JL (1994) Whole arm obstacle avoidance for teleoperated robots. In: IEEE proceedings of the international conference on robotics and automation, pp 3303–3309Google Scholar
  3. 3.
    Yu Y, Gupta K (1999) Sensor based roadmaps for motion-planning for articulated robots in unknown environment: some experiments with an eye-in-hand system. In: IEEE proceedings of the international conference on intelligent robots and systems, pp 1707–1714Google Scholar
  4. 4.
    Noborio H, Nishino Y (2001) Image-based path-planning algorithm on the joint space. In: IEEE proceedings of the international conference on robotics and automation, Seoul, pp 1180–1187Google Scholar
  5. 5.
    Lumelsky V, Cheung E (1993) Real-time collision avoidance in teleoperated whole-sensitive robot arm manipulators. IEEE Trans Syst Man Cybern 23(1):194–203CrossRefGoogle Scholar
  6. 6.
    Goujou E, Miteran J, Laligant O, Truchetet F, Gorria P (1995) Human detection with video surveillance system. In: Proceedings of the IEEE international conference on industrial electronics, control, and instrumentation, vol 2, pp 1179–1184Google Scholar
  7. 7.
    Morioka K, Lee J-H, Hashimoto H (2004) Human-following mobile robot in a distributed intelligent sensor network. IEEE Trans Ind Electron 51(1):229–237CrossRefGoogle Scholar
  8. 8.
    Vadakkepat P, Lim P, De Silva LC, Jing L, Ling LL (2008) Multimodal approach to human-face detection and tracking. IEEE Trans Ind Electron 55(3):1385–1393CrossRefGoogle Scholar
  9. 9.
    Eckert G (2000) Automatic shape reconstruction of rigid 3D objects from multiple calibrated images. In: Proceedings on Eusipco 2000, TampereGoogle Scholar
  10. 10.
    Meisel A (1994) 3D-Bildverarbeitung für feste und bewegte Kameras. Vieweg Publishing, Fortschritte der Robotik Nr.21Google Scholar
  11. 11.
    Gecks T, Henrich D (2006) Multi-camera collision detection allowing for object occlusions. In: 37th international symposium on robotics (ISR 2006)/4th German conference on robotics (Robotik 2006)Google Scholar
  12. 12.
    Henrich D, Gecks T (2008) Multi-camera collision detection between known and unknown objects. In: 2nd ACM/IEEE international conference on distributed smart camerasGoogle Scholar
  13. 13.
    Ebert DM, Henrich D (2002) Human–robot-cooperation: image-based collision detection for industrial robots. In: IEEE international conference on intelligent robots and systemsGoogle Scholar
  14. 14.
    Kumar A (2008) Computer-vision-based fabric defect detection: a survey. IEEE Trans Ind Electron 55(1):348–363CrossRefGoogle Scholar
  15. 15.
    Sarmis T, Argyros AA, Lourakis MIA, Hatzopoulos K (2006) Robust and efficient event detection for the monitoring of automated processes. In: VIE 2006 proceedings of the international conference on visual information engineering, pp 454–459Google Scholar
  16. 16.
    Jäger M, Hamprecht FA (2009) Principal component imagery for the quality monitoring of dynamic laser welding processes. IEEE Trans Ind Electron 56(4):1307–1313CrossRefGoogle Scholar
  17. 17.
    Schwarte R, Heinol HG, Xu Z, Hartmann K (1995) A new active 3D-Vision system based on RF-modulation interferometry of incoherent light. In: Photonics east—intelligent systems and advanced manufacturing, Proceedings of the SPIE, vol 2588, PhiladelphiaGoogle Scholar
  18. 18.
    Spirig T, Seitz P, Vietze O, Heitger F (1995) The lock-in CCD—twodimensional synchronous detection of light. IEEE J Quantum Electron 31:1705–1708CrossRefGoogle Scholar
  19. 19.
    Frank M, Plaue M, Rapp H, Köthe U, Jähne B, Hamprecht FA (2009) Theoretical and experimental error analysis of continuous-wave time-of-flight range cameras. Opt Eng 48(1)Google Scholar
  20. 20.
    Frank M, Plaue M, Hamprecht FA (2009) Denoising of continuous-wave time-of-flight depth images using confidence measures. Opt Eng 48(7)Google Scholar
  21. 21.
    Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRefGoogle Scholar
  22. 22.
    Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323CrossRefGoogle Scholar
  23. 23.
    Schölkopf B, Smola AJ, Müller KR (1997) Kernel principal component analysis. Lecture notes in computer science. Springer, Berlin, vol 1327, pp 583–588Google Scholar
  24. 24.
    Schölkopf B, Platt JC, Shawe-Taylor J, Smola AJ (1999) Estimating the support of a high-dimensional distribution. Technical report, Microsoft researchGoogle Scholar
  25. 25.
    Schölkopf B, Smola AJ (2002) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, CambridgeGoogle Scholar
  26. 26.
    Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkzbMATHCrossRefGoogle Scholar
  27. 27.
    Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural computation. MIT Press, Cambridge vol 12, no. 5, pp 1207–1245Google Scholar
  28. 28.
    Weston J, Elisseeff A, Bakir G, Sinz F (2006) The spider—machine learning toolbox for Matlab. Department of Empirical Inference, Max-Planck Institute for biological Cybernetics, Tübingen.
  29. 29.
    Zhou T, Tao D, Wu X (2010) Manifold elastic net: a unified framework for sparse dimension reduction. Data Min Knowl Discov. doi: 10.1007/s10618-010-0182-x

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Department of Computer ScienceETH ZurichZurichSwitzerland
  2. 2.Heidelberg Collaboratory for Image ProcessingUniversity of HeidelbergHeidelbergGermany

Personalised recommendations