Contact-state monitoring of force-guided robotic assembly tasks using expectation maximization-based Gaussian mixtures models

  • Ibrahim F. JasimEmail author
  • Peter W. Plapper


This article addresses the problem of contact-state (CS) monitoring for peg-in-hole force-controlled robotic assembly tasks. In order to perform such a monitoring target, the wrench (Cartesian forces and torques) and pose (Cartesian position and orientation) signals of the manipulated object are firstly captured for different CS’s of the object (peg) with respect to the environment including the hole. The captured signals are employed in building a model (a recognizer) for each CS, and in the framework of pattern classification, the CS monitoring would be addressed. It will be shown that the captured signals are nonstationary, i.e., they have non-normal distribution that would result in performance degradation if using the available monitoring approaches. In this article, the concept of the Gaussian mixtures models (GMM) is used in building the likelihood of each signal and the expectation maximization (EM) algorithm is employed in finding the GMM parameters. The use of the GMM would accommodate the signals nonstationary behavior and the EM algorithm would guarantee the estimation of the optimal parameters set of the GMM for each signal, and hence the modeling accuracy would be significantly enhanced. In order to see the performance of the suggested CS monitoring scheme, we installed a test stand that is composed of a KUKA lightweight robot (LWR) doing peg-in-hole tasks. Two experiments are considered; in the first experiment, we use the EM-GMM in monitoring a typical peg-in-hole robotic assembly process, and in the second experiment, we consider the robotic assembly of camshaft caps assembly of an automotive powertrain and use the EM-GMM in monitoring its CS’s. For both experiments, the excellent monitoring performance will be shown. Furthermore, we compare the performance of the EM-GMM with that obtained when using available CS monitoring approaches. Classification success rate (CSR) and computational time will be considered as comparison indices, and the EM-GMM will be shown to have a superior CSR performance with reduced a computational time.


Assembly monitoring Expectation maximization Gaussian mixtures Peg-in-hole Robotic assembly 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hirai S, Iwata K (1992) Recognition of contact state based on geometric model. In: Proceedings 1992 IEEE international conference robotics automation, Nice-France, pp 1507–1512Google Scholar
  2. 2.
    McCarragher BJ (1994) Petri net modeling for robotic assembly and trajectory planning. IEEE Trans Indust Elect 41(6):631–640CrossRefGoogle Scholar
  3. 3.
    Cao T, Sanderson AC (1994) Task decomposition and analysis of robotic assembly task plans using Petri nets. IEEE Trans Indust Elect 41(6):620–630CrossRefGoogle Scholar
  4. 4.
    Farahat AO, Graves BS, Trinkle JC (1995) Identifying contact formations in the presence of uncertainty. In: Proceedings 1995 IEEE/RSJ international conference intelligent robotics systems, Pittsburgh-PA, pp 59–64Google Scholar
  5. 5.
    McCarragher B J (1996) Task primitives for the discrete event modeling and control of 6-DOF assembly tasks. IEEE Trans Robot Autom 12(2):280–289CrossRefGoogle Scholar
  6. 6.
    Hovland GE, McCarragher BJ (1997) Combining force and position measurements for the monitoring of robotic assembly. In: Proceedings 1997 IEEE/RSJ international conference intelligent robotics systems, Grenoble-France, pp 654–660Google Scholar
  7. 7.
    Skubic M, Volz RA (1997) Learning force sensory patterns and skills from human demonstration. In: Proceedings 1997 IEEE international conference robotics automation, NM-USA, pp 284–290Google Scholar
  8. 8.
    Skubic M, Castrianni SP, Volz RA (1997) Identifying contact formations from force signals: a comparison of fuzzy and neural network classifiers. In: Proceedings 1997 international conference neur. net., Houston-TX, pp 1623–1628Google Scholar
  9. 9.
    Mosemann H, Raue A, Wahl F (1998) Classification and recognition of contact states for force guided assembly. In: Proceedings IEEE international conference system manual, cybernetics, San Diego-CA, pp 3400–3405Google Scholar
  10. 10.
    Hovland GE, McCarragher BJ (1998) Hidden Markov models as a process monitor in robotic assembly. In: International journal of robotics research, vol. 17, no. 2, pp 153–168Google Scholar
  11. 11.
    Xiao J, Liu L (1998) Contact states: representation and recognizability in the presence of uncertainties. In: Proceedings IEEE international conference robotics automation, Victoria-Canada, pp 1151–1156Google Scholar
  12. 12.
    De Schutter J, Bruyninckx H, Dutré S, De Geeter J, Katupitiya J, Demey S, Lefenvre T (1999) Estimating first-order geometric parameters and monitoring contact transitions during force controlled compliant motions. Int J Robot Res 18(12):1161–1184CrossRefGoogle Scholar
  13. 13.
    Everett LJ, Ravuri R, Volz RA, Skubic M (1999) Generalized recognition of single-ended contact formations. IEEE Trans Robot Autom 15(5):829–836CrossRefGoogle Scholar
  14. 14.
    Skubic M, Volz RA (2000) Identifying single-ended contact formations from force sensor patterns. IEEE Trans Robot Autom 16(5):597–603CrossRefGoogle Scholar
  15. 15.
    Son C (2001) A neural/fuzzy optimal process model for robotic part assembly. Int J Mach Manuf 41(12):1783–1794CrossRefGoogle Scholar
  16. 16.
    Chung S-Y, Lee DY (2001) Discrete event systems approach to fixtureless peg-in-hole assembly. In: Proceedings 2001 American control conference, Arlington-VA, pp 4962–4967Google Scholar
  17. 17.
    Fei Y, Zhao X (2003) An assembly process modeling and analysis for robotic multiple peg-in-hole. J Intell Robot Syst 36:175–189CrossRefGoogle Scholar
  18. 18.
    Lau HY (2003) A hidden Markov model-based assembly contact recognition system. Mechatronics 13(8–9):1001–1023CrossRefGoogle Scholar
  19. 19.
    Cortesão R, Koeppem R, Nunes U, Hirzenger G (2004) Data fusion for robotic assembly tasks based on human skills. IEEE Trans Robot Autom 20(6):941–952CrossRefGoogle Scholar
  20. 20.
    Debus TJ, Dupot PE, Howe RD (2004) Contact state estimation using multiple model estimation and Markov models. Int J Robot Res 23(4–5):399–413CrossRefGoogle Scholar
  21. 21.
    Iwata H, Sugano S (2005) Human-robot-contact-state identification based on tactile recognition. IEEE Trans Ind Elect 52(6):1468–1477CrossRefGoogle Scholar
  22. 22.
    Bishop CM (2006) Pattern recognition and machine learning. Springer, New York, pp 423–455zbMATHGoogle Scholar
  23. 23.
    Thomas U, Molkenstruck S, Iser R, Wahl FM (2007) Multi sensor fusion in robot assembly using particle filters. In: Proceedings IEEE international conference robotics automation, Rome-Italy, pp 3837–3843Google Scholar
  24. 24.
    Katsura S, Matsumoto Y, Ohnishi K (2007) Modeling of force sensing and validation of disturbance observer for force control. IEEE Trans Ind Elect 54(1):530–538CrossRefGoogle Scholar
  25. 25.
    Meeussen W, Rutgeerts J, Gadeyne K, Bruyninckx H, De Schutter J (2007) Contact-state segmentation using particle filters for programming by human demonstration in compliant-motion tasks. IEEE Trans Robot 23(2):218–231CrossRefGoogle Scholar
  26. 26.
    Okuda H, Takeuchi H, Inagaki S, Suzuki T (2008) Modeling and analysis of peg-in-hole task based on mode segmentation. In: Proceedings 2008 SICE annual conference, Tokyo-Japan, 20-22 Aug, pp 1595–1600Google Scholar
  27. 27.
    Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction. Springer, New York, pp 265–267CrossRefGoogle Scholar
  28. 28.
    Cabras S, Castellanos ME, Staffetti E (2010) Contact-state classification in human-demonstrated robot compliant motion tasks using the boosting algorithm. IEEE Trans Syst Man Cyber B 40(5):1372–1386CrossRefGoogle Scholar
  29. 29.
    Lightweight Robot 4+ Specification, Version: Spez LBR 4+ V5 en, KUKA Roboter GmbH, 22 Dec 2011Google Scholar
  30. 30.
    Hertkorn K, Rao MA, Preusche C, Borst C, Hirzinger G (2012) Identification of contact formations: resolving ambiguous force torque information. In: Proceedings of the 2012 IEEE international conference robotics automation, St. Paul-MN, 14–18 May, pp 3278–3284Google Scholar
  31. 31.
    Jasim IF, Plapper PW (2013) T-S fuzzy contact state recognition for compliant motion robotic tasks using gravitational search-based clustering algorithm. In: Proceedings IEEE International Conference Fuzzy Systems, Hyderabad-India, 7–10 Jul, pp 1–8Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  1. 1.Faculty of Science, Technology and CommunicationUniversity of LuxembourgCampus KirchbergLuxembourg

Personalised recommendations