Preventing Erroneous Operator Behavior and Supporting Optimal Operating Procedures within Industrial Product-Service Systems

Conference paper
Part of the Lecture Notes in Production Engineering book series (LNPE)


The performance of Industrial Product-Service Systems (IPS²) relies – in addition to many technical factors – on the optimal performance of the human operators. This can be supported through the detection of erroneous behavior and user-centered warnings before injury or damage occurs.

The detection of such behavior is based on the simulation of cognitive models which describe behavior as a combination of body movements combinations of the operator. A comparison of the simulation with the tracked movements leads to the identification and classification of possible dangerous actions of the operator. Error codes then trigger action-specific warnings. In addition to the warnings, the system automatically generates documentation in the form of avatar videos. These can be used for task demonstration and to ensure the operators compliance.

The paper details the integrative system setup for this erroneous behavior detection and the resulting warning generation. It also demonstrates the integration of these systems into IPS².


Operating Procedure Monitoring Preliminary Error Detection Multimodal Warning Strategies Avatar Instructions 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    ACT-R Theory and Architecture of Cognition,
  2. 2.
    Bregler, C.: Motion Capture Technology for Entertainment. Signal Processing Magazine 24(6), 160–158 (2007)Google Scholar
  3. 3.
    Hoege, B., Schlatow, S., Roetting, M.: Shared Vision - State of the Art and Future Application. In: Tagungsband des 17. World Congress on Ergonomics IEA 2009, Beijing, China, August 9-14. CD-ROM (2009)Google Scholar
  4. 4.
    Hoege, B., Schlatow, S., Roetting, M.: A Shared-Vision System for User Support in the Field of Micromanufactoring. In: Proceedings of HCI International 2009, pp. 855–859. Springer, Heidelberg (2009)Google Scholar
  5. 5.
    Junker, H., Amft, O., Lukowicz, P., Tröster, G.: Gesture Spotting with Body-Worn Inertial Sensors to Detect User Activities. Pattern Recognition 41(6), 2010–2024 (2008)zbMATHCrossRefGoogle Scholar
  6. 6.
    Oviatt, S.: Designing Robust Multimodal Interfaces for Universal Access. In: Proceedings of the Workshop on Universal Accessibility of Ubiquitous Computing 2001, Alcacer do Sal, Portugal, pp. 71–74 (2001)Google Scholar
  7. 7.
    Oviatt, S., Bernard, J., Levow, G.: Linguistic Adaptation during Error Resolution with Spoken and Multimodal System. Language and Speech, Special Issue on Prosody and Conversation 41(3-4), 415–438 (1999)Google Scholar
  8. 8.
    Rickel, J., Johnson, W.L.: Animated Agents for Procedural Training in Virtual Reality Perception. Cognition and Motor Control, Applied Artificial Intelligence 13, 343–382 (1999)CrossRefGoogle Scholar
  9. 9.
    Rickel, J.: Intelligent Virtual Agents for Education and Training: Opportunities and Challenges. In: de Antonio, A., Aylett, R., Ballin, D. (eds.) IVA 2001. LNCS (LNAI), vol. 2190, pp. 15–22. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  10. 10.
    Schmuntzsch, U., Sturm, C., Roetting, M.: How Can Multimodality be Used to Design Usable Interfaces in IPS2 for Older Employees. In: Proceedings of the 18th World Congress on Ergonomics IEA 2012, pp. 3533–3540 (2012)Google Scholar
  11. 11.
    Schmuntzsch, U., Sturm, C., Reichmuth, R., Roetting, M.: Virtual Agent Assistance for Maintenance Tasks in IPS2 – First Results of a Study. In: Proceedings of the 4th International Conference on Applied Human Factors and Ergonomics, San Francisco, USA, pp. 221–231 (2012)Google Scholar
  12. 12.
    Wewiór, J., Schmuntzsch, U., Seliger, G., Roetting, M.: Knowledge-based multimodal user support to increase availability in Industrial Product-Service Systems. To appear in the Proceedings of the 4th CIRP Conference on Industrial Product Service Systems, IPS2 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Chair of Human-Machine-SystemsBerlin Institute of TechnologyBerlinGermany

Personalised recommendations