Technology-Enhanced Process Elicitation of Worker Activities in Manufacturing

  • Sönke KnochEmail author
  • Shreeraman Ponpathirkoottam
  • Peter Fettke
  • Peter Loos
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 308)


The analysis of manufacturing processes through process mining requires meaningful log data. Regarding worker activities, this data is either sparse or costly to gather. The primary objective of this paper is the implementation and evaluation of a system that detects, monitors and logs such worker activities and generates meaningful event logs. The system is light-weight regarding its setup and convenient for instrumenting assembly workstations in job shop manufacturing for temporary observations. In a study, twelve participants assembled two different product variants in a laboratory setting. The sensor events were compared to video annotations. The optical detection of grasping material by RGB cameras delivered a Median F-score of 0.83. The RGB+D depth camera delivered only a Median F-score of 0.56 due to occlusion. The implemented activity detection proofs the concept of process elicitation and prepares process mining. In future studies we will optimize the sensor setting and focus on anomaly detection.


Process elicitation Activity recognition Manufacturing 



This research was funded in part by the German Federal Ministry of Education and Research (BMBF) under grant number 01IS16022E (project BaSys4.0). The responsibility for this publication lies with the authors.


  1. 1.
    van der Aalst, W.M.P.: Process Mining: Data Science in Action, pp. 3–23. Springer, Heidelberg (2016). CrossRefGoogle Scholar
  2. 2.
    von Ammon, R., Ertlmaier, T., Etzion, O., Kofman, A., Paulus, T.: Integrating complex events for collaborating and dynamically changing business processes. In: Dan, A., Gittler, F., Toumani, F. (eds.) ICSOC/ServiceWave -2009. LNCS, vol. 6275, pp. 370–384. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  3. 3.
    Bader, S.,et al.: Tracking assembly processes and providing assistance in smart factories. In: 6th International Conference on Agents and Articial Intelligence, ICAART, vol. 1, pp. 161–168. SCITEPRESS, Science and Technology Publications, Lda, Angers, France (2014)Google Scholar
  4. 4.
    Bruns, R., et al.: Using complex event processing to support data fusion for ambulance coordination. In: 17th International Conference on Information Fusion, FUSION, pp. 1–7, July 2014Google Scholar
  5. 5.
    Dencker, K., et al.: Proactive assembly systems - realising the potential of human collaboration with automation. Ann. Rev. Control 33(2), 230–237 (2009)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Estruch, A., Heredia Álvaro, J.A.: Event-driven manufacturing process management approach. In: Barros, A., Gal, A., Kindler, E. (eds.) BPM 2012. LNCS, vol. 7481, pp. 120–133. Springer, Heidelberg (2012). CrossRefGoogle Scholar
  7. 7.
    Funk, M., et al.: Cognitive assistance in the workplace. Pervasive Comput. 14(3), 53–55 (2015)CrossRefGoogle Scholar
  8. 8.
    Henderson, S.J., et al.: Augmented reality in the psychomotor phase of a procedural task. In: 10th International Symposium on Mixed and Augmented Reality, ISMAR, pp. 191–200. IEEE, Oct 2011Google Scholar
  9. 9.
    Herzberg, N., et al.: An event processing platform for business process management. In: 17th International Enterprise Distributed Object Computing Conference, EDOC, pp. 107–116. IEEE, September 2013Google Scholar
  10. 10.
    Houy, C., et al.: Empirical research in business process management - analysis of an emerging field of research. Bus. Process Manag. J. 16(4), 619–661 (2010)CrossRefGoogle Scholar
  11. 11.
    Knoch, S., et al.: Automatic capturing and analysis of manual manufacturing processes with minimal setup effort. In: International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp, pp. 305–308. ACM, Heidelberg, Germany, September 2016Google Scholar
  12. 12.
    Petersen, N., et al.: Real-time modeling and tracking manual workflows from first-person vision. In: International Symposium on Mixed and Augmented Reality, ISMAR, pp. 117–124. IEEE, October 2013Google Scholar
  13. 13.
    Quint, F., et al.: A system architecture for assistance in manual tasks. In: Intelligent Environments, IE. vol. 21, pp. 43–52, Ambient Intelligence and Smart Environments. IOS Press (2016)Google Scholar
  14. 14.
    Zivkovic, Z., et al.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn. Lett. 27(7), 773–780 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Sönke Knoch
    • 1
    Email author
  • Shreeraman Ponpathirkoottam
    • 1
  • Peter Fettke
    • 1
  • Peter Loos
    • 1
  1. 1.German Research Center for Artificial Intelligence (DFKI)Saarland UniversitySaarbrückenGermany

Personalised recommendations