Skip to main content
SpringerLink
Log in

A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments

  • Conference paper
  • First Online:
Intelligent Data Engineering and Automated Learning – IDEAL 2018 (IDEAL 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11315))

Abstract

Despite the increasing automation levels in an Industry 4.0 scenario, the tacit knowledge of highly skilled manufacturing workers remains of strategic importance. Retaining this knowledge by formally capturing it is a challenge for industrial organisations. This paper explores research on automatically capturing this knowledge by using methods from activity recognition and process mining on data obtained from sensorised workers and environments. Activity recognition lifts the abstraction level of sensor data to recognizable activities and process mining methods discover models of process executions. We classify the existing work, which largely neglects the possibility of applying process mining, and derive a taxonomy that identifies challenges and research gaps.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. van der Aalst, W.M.P.: Process Mining - Data Science in Action, 2nd edn. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49851-4

    Book  Google Scholar 

  2. Abdallah, Z.S., Gaber, M.M., Srinivasan, B., Krishnaswamy, S.: Activity recognition with evolving data streams. ACM Comput. Surv. 51(4), 1–36 (2018)

    Article  Google Scholar 

  3. Aehnelt, M., Gutzeit, E., Urban, B.: Using activity recognition for the tracking of assembly processes: challenges and requirements. In: WOAR 2014. Fraunhofer Verlag (2014)

    Google Scholar 

  4. Aggarwal, J., Ryoo, M.: Human activity analysis. ACM Comput. Surv. 43(3), 1–43 (2011)

    Article  Google Scholar 

  5. Al-Naser, M., et al.: Hierarchical model for zero-shot activity recognition using wearable sensors. In: ICAART (2), pp. 478–485. SciTePress (2018)

    Google Scholar 

  6. Augusto, A., et al.: Automated discovery of process models from event logs: review and benchmark. IEEE Trans. Knowl. Data Eng. (2018)

    Google Scholar 

  7. Bader, S., Aehnelt, M.: Tracking assembly processes and providing assistance in smart factories. In: ICAART 2014. SCITEPRESS (2014)

    Google Scholar 

  8. Bader, S., Krüger, F., Kirste, T.: Computational causal behaviour models for assisted manufacturing. In: iWOAR 2015. ACM Press (2015)

    Google Scholar 

  9. Bannach, D., Kunze, K., Lukowicz, P., Amft, O.: Distributed modular toolbox for multi-modal context recognition. In: Grass, W., Sick, B., Waldschmidt, K. (eds.) ARCS 2006. LNCS, vol. 3894, pp. 99–113. Springer, Heidelberg (2006). https://doi.org/10.1007/11682127_8

    Chapter  Google Scholar 

  10. Blanke, U., Schiele, B.: Remember and transfer what you have learned - recognizing composite activities based on activity spotting. In: ISWC 2010. IEEE (2010)

    Google Scholar 

  11. Böttcher, S., Scholl, P.M., Laerhoven, K.V.: Detecting process transitions from wearable sensors. In: iWOAR 2017. ACM Press (2017)

    Google Scholar 

  12. Feldhorst, S., Masoudenijad, M., ten Hompel, M., Fink, G.A.: Motion classification for analyzing the order picking process using mobile sensors - general concepts, case studies and empirical evaluation. In: ICPRAM 2016, pp. 706–713. SCITEPRESS (2016)

    Google Scholar 

  13. Gonella, P., Castellano, M., Riccardi, P., Carbone, R.: Process mining: a database of applications. Technical report, HSPI SpA - Management Consulting (2017)

    Google Scholar 

  14. Goto, H., Miura, J., Sugiyama, J.: Human-robot collaborative assembly by on-line human action recognition based on an FSM task model. In: Human-Robot Interaction 2013 Workshop on Collaborative Manipulation (2013)

    Google Scholar 

  15. Grzeszick, R., Lenk, J.M., Rueda, F.M., Fink, G.A., Feldhorst, S., ten Hompel, M.: Deep neural network based human activity recognition for the order picking process. In: iWOAR 2017. ACM Press (2017)

    Google Scholar 

  16. Janiesch, C., et al.: The Internet-of-Things meets business process management: mutual benefits and challenges (2017). arXiv:1709.03628

  17. Knoch, S., Ponpathirkoottam, S., Fettke, P., Loos, P.: Technology-enhanced process elicitation of worker activities in manufacturing. In: Teniente, E., Weidlich, M. (eds.) BPM 2017. LNBIP, vol. 308, pp. 273–284. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-74030-0_20

    Chapter  Google Scholar 

  18. Lara, O.D., Labrador, M.A.: A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor. 15(3), 1192–1209 (2013)

    Article  Google Scholar 

  19. Longstaff, B., Reddy, S., Estrin, D.: Improving activity classification for health applications on mobile devices using active and semi-supervised learning. In: ICST 2010. IEEE (2010)

    Google Scholar 

  20. Lukowicz, P., et al.: Recognizing workshop activity using body worn microphones and accelerometers. In: Ferscha, A., Mattern, F. (eds.) Pervasive 2004. LNCS, vol. 3001, pp. 18–32. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24646-6_2

    Chapter  Google Scholar 

  21. Maekawa, T., Nakai, D., Ohara, K., Namioka, Y.: Toward practical factory activity recognition. In: UbiComp 2016. ACM Press (2016)

    Google Scholar 

  22. Mannhardt, F., de Leoni, M., Reijers, H.A., van der Aalst, W.M.P.: Balanced multi-perspective checking of process conformance. Computing 98(4), 407–437 (2016)

    Article  MathSciNet  Google Scholar 

  23. Mannhardt, F., Petersen, S.A., de Oliveira, M.F.D.: Privacy challenges for process mining in human-centered industrial environments. In: Intelligent Environments (IE). IEEE Xplore (2018, to appear)

    Google Scholar 

  24. Marin-Perianu, M., Lombriser, C., Amft, O., Havinga, P., Tröster, G.: Distributed activity recognition with fuzzy-enabled wireless sensor networks. In: Nikoletseas, S.E., Chlebus, B.S., Johnson, D.B., Krishnamachari, B. (eds.) DCOSS 2008. LNCS, vol. 5067, pp. 296–313. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69170-9_20

    Chapter  Google Scholar 

  25. Mörzinger, R., et al.: Tools for semi-automatic monitoring of industrial workflows. In: ARTEMIS 2010. ACM Press (2010)

    Google Scholar 

  26. Mura, M.D., Dini, G., Failli, F.: An integrated environment based on augmented reality and sensing device for manual assembly workstations. Procedia CIRP 41, 340–345 (2016)

    Article  Google Scholar 

  27. Ogris, G., Lukowicz, P., Stiefmeier, T., Tröster, G.: Continuous activity recognition in a maintenance scenario: combining motion sensors and ultrasonic hands tracking. Pattern Anal. Appl. 15(1), 87–111 (2011)

    Article  MathSciNet  Google Scholar 

  28. Ogris, G., Stiefmeier, T., Lukowicz, P., Troster, G.: Using a complex multi-modal on-body sensor system for activity spotting. In: IWSC 2008. IEEE (2008)

    Google Scholar 

  29. Ramamurthy, S.R., Roy, N.: Recent trends in machine learning for human activity recognition-a survey. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 8(4), e1254 (2018)

    Article  Google Scholar 

  30. Raso, R., et al.: Activity monitoring using wearable sensors in manual production processes - an application of CPS for automated ergonomic assessments. In: MKWI 2018. Leuphana Universität Lüneburg (2018)

    Google Scholar 

  31. Repta, D., Moisescu, M.A., Sacala, I.S., Stanescu, A.M., Constantin, N.: Generic architecture for process mining in the context of cyber physical systems. Appl. Mech. Mater. 656, 569–577 (2014)

    Article  Google Scholar 

  32. Roitberg, A., Perzylo, A., Somani, N., Giuliani, M., Rickert, M., Knoll, A.: Human activity recognition in the context of industrial human-robot interaction. In: APSIPA 2014. IEEE (2014)

    Google Scholar 

  33. Roitberg, A., Somani, N., Perzylo, A., Rickert, M., Knoll, A.: Multimodal human activity recognition for industrial manufacturing processes in robotic workcells. In: ICMI 2015. ACM Press (2015)

    Google Scholar 

  34. Schlenoff, C., Kootbally, Z., Pietromartire, A., Franaszek, M., Foufou, S.: Intention recognition in manufacturing applications. Robot. Comput. Integr. Manuf. 33, 29–41 (2015)

    Article  Google Scholar 

  35. Stiefmeier, T., Lombriser, C., Roggen, D., Junker, H., Ogris, G., Troester, G.: Event-based activity tracking in work environments. In: IFAWC 2006, pp. 1–10 (2006)

    Google Scholar 

  36. Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., Tr, G.: Wearable activity tracking in car manufacturing. IEEE Pervasive Comput. 7(2), 42–50 (2008)

    Article  Google Scholar 

  37. Voulodimos, A., et al.: A threefold dataset for activity and workflow recognition in complex industrial environments. IEEE Multimed. 19(3), 42–52 (2012)

    Article  Google Scholar 

  38. Voulodimos, A.S., Kosmopoulos, D.I., Doulamis, N.D., Varvarigou, T.A.: A top-down event-driven approach for concurrent activity recognition. Multimed. Tools Appl. 69(2), 293–311 (2012)

    Article  Google Scholar 

  39. Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L.: Deep learning for sensor-based activity recognition: a survey. Pattern Recognit. Lett. (2018, in press)

    Google Scholar 

  40. Ward, J.A., Lukowicz, P., Troster, G., Starner, T.E.: Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1553–1567 (2006)

    Article  Google Scholar 

Download references

Acknowledgments

This research has received funding from the European Union’s H2020 research and innovation programme under grant agreement no. 723737 (HUMAN).

Author information

Authors and Affiliations

  1. SINTEF Digital, Trondheim, Norway

    Felix Mannhardt & Manuel Fradinho Oliveira

  2. Department of Computer Science, UCL, London, UK

    Riccardo Bovo & Simon Julier

Authors
  1. Felix Mannhardt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riccardo Bovo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel Fradinho Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Simon Julier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Felix Mannhardt .

Editor information

Editors and Affiliations

  1. University of Manchester, Manchester, UK

    Hujun Yin

  2. Rm 209, Building B, Autonomous University of Madrid, Madrid, Spain

    David Camacho

  3. Campus of Gualtar, University of Minho, Braga, Portugal

    Paulo Novais

  4. University of Seville, Seville, Spain

    Antonio J. Tallón-Ballesteros

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mannhardt, F., Bovo, R., Oliveira, M.F., Julier, S. (2018). A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments. In: Yin, H., Camacho, D., Novais, P., Tallón-Ballesteros, A. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2018. IDEAL 2018. Lecture Notes in Computer Science(), vol 11315. Springer, Cham. https://doi.org/10.1007/978-3-030-03496-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03496-2_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03495-5

  • Online ISBN: 978-3-030-03496-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation