Skip to main content

Chances of Interpretable Transfer Learning for Human Activity Recognition in Warehousing

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 13004)

Abstract

Human activity recognition evolves around classifying and analyzing workers’ actions quantitatively using convolutional neural networks on the time-series data provided by inertial measurement units and motion capture systems. However, this requires expensive training datasets since each warehouse scenario has slightly different settings and activities of interest. Here, transfer learning promises to shift the knowledge a deep learning method gained on existing reference data to new target data. We benchmark interpretable and non-interpretable transfer learning for human activity recognition on the LARa order-picking dataset with AndyLab and RealDisp as domain-related and domain-foreign reference datasets. We find that interpretable transfer learning via the recently proposed probabilistic rule stacking learner, which does not require any labeled data on the target dataset, is possible if the labels are sufficiently semantically related. The success depends on the proximity of the reference and target domains and labels. Non-interpretable transfer learning via fine-tuning can be applied even if there is a major domain-shift between the datasets and reduces the amount of labeled data required on the target dataset.

Keywords

  • Domain-shift
  • Few-shot learning
  • Interpretability
  • Logistics
  • Multi-label classification
  • Time-series
  • Zero-shot learning

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-87672-2_11
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-87672-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   139.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.

References

  1. Agresti, A.: An Introduction to Categorical Data Analysis. John Wiley, Hoboken (2018)

    Google Scholar 

  2. Atzmon, Y., Chechik, G.: Probabilistic AND-OR attribute grouping for zero-shot learning. In: Conference on Uncertainty in Artificial Intelligence (2018)

    Google Scholar 

  3. Avsar, H., Altermann, E., Reining, C., Rueda, F.M., Fink, G.A., ten Hompel, M.: Benchmarking annotation procedures for multi-channel time series HAR dataset. In: 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, pp. 453–458 (2021)

    Google Scholar 

  4. Banos, O., Toth, M.A., Damas, M., Pomares, H., Rojas, I.: Dealing with the effects of sensor displacement in wearable activity recognition. Sensors 14(6), 9995–10023 (2014)

    CrossRef  Google Scholar 

  5. Calzavara, M., Glock, C.H., Grosse, E.H., Persona, A., Sgarbossa, F.: Analysis of economic and ergonomic performance measures of different rack layouts in an order picking warehouse. Comput. Ind. Eng. 111, 527–536 (2017)

    Google Scholar 

  6. Chen, C., Jafari, R., Kehtarnavaz, N.: A survey of depth and inertial sensor fusion for human action recognition. Multimed. Tools Appl. 76(3), 4405–4425 (2017)

    CrossRef  Google Scholar 

  7. Cheng, H.T., Sun, F.T., Griss, M., Davis, P., Li, J., You, D.: NuActiv: recognizing unseen new activities using semantic attribute-based learning. In: 11th Annual Conference on Mobile Systems, Applications, and Services, pp. 361–374 (2013)

    Google Scholar 

  8. Daduna, J.R.: Automated and autonomous driving in freight transport - opportunities and limitations. In: Lalla-Ruiz, E., Mes, M., Voß, S. (eds.) ICCL 2020. LNCS, vol. 12433, pp. 457–475. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59747-4_30

    CrossRef  Google Scholar 

  9. Ding, N., Deng, J., Murphy, K.P., Neven, H.: probabilistic label relation graphs with Ising models. In: Proceedings of the 2015 IEEE International Conference on Computer Vision, pp. 1161–1169 (2015)

    Google Scholar 

  10. Feldhorst, S., Aniol, S., ten Hompel, M.: Human Activity Recognition in der Kommissionierung - Charakterisierung des Kommissionierprozesses als Ausgangsbasis für die Methodenentwicklung. Logistics J. 2016(10) (2016)

    Google Scholar 

  11. Grosse, E.H., Calzavara, M., Glock, C.H., Sgarbossa, F.: Incorporating human factors into decision support models for production and logistics: current state of research. IFAC-PapersOnLine 50(1), 6900–6905 (2017)

    CrossRef  Google Scholar 

  12. Grosse, E.H., Glock, C.H., Neumann, W.P.: Human factors in order picking system design: a content analysis. IFAC-PapersOnLine 48(3), 320–325 (2015)

    CrossRef  Google Scholar 

  13. Guo, L., Lei, Y., Xing, S., Yan, T., Li, N.: Deep convolutional transfer learning network: a new method for intelligent fault diagnosis of machines with unlabeled data. IEEE Trans. Industr. Electron. 66(9), 7316–7325 (2018)

    CrossRef  Google Scholar 

  14. Huang, H., Pouls, M., Meyer, A., Pauly, M.: Travel time prediction using tree-based ensembles. In: Lalla-Ruiz, E., Mes, M., Voß, S. (eds.) ICCL 2020. LNCS, vol. 12433, pp. 412–427. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59747-4_27

    CrossRef  Google Scholar 

  15. Kirchhof, M., Schmid, L., Reining, C., ten Hompel, M., Pauly, M.: pRSL: interpretable multi-label stacking by learning probabilistic rules. In: Uncertainty in Artificial Intelligence. PMLR (2021). (in press)

    Google Scholar 

  16. Kirchhof, M.: GitHub repository for this article (2021). https://github.com/mkirchhof/rslAppl

  17. de Koster, R., Le-Duc, T., Roodbergen, K.J.: Design and control of warehouse order picking: a literature review. Eur. J. Oper. Res. 182(2), 481–501 (2007)

    CrossRef  Google Scholar 

  18. Krüger, A., Feldmann, F., Pauly, M., ten Hompel, M.: Einsatzmöglichkeiten maschineller Lernverfahren in einer dezentral organisierten Lagerverwaltung auf Basis intelligenter Behälter. Logistics J. Proc. 2020(12) (2020)

    Google Scholar 

  19. Kull, M., Perello Nieto, M., Kängsepp, M., Silva Filho, T., Song, H., Flach, P.: Beyond temperature scaling: obtaining well-calibrated multi-class probabilities with Dirichlet calibration. Adv. Neural. Inf. Process. Syst. 32, 12316–12326 (2019)

    Google Scholar 

  20. Liu, L., Zhou, T., Long, G., Jiang, J., Zhang, C.: Attribute propagation network for graph zero-shot learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34(04), pp. 4868–4875 (2020)

    Google Scholar 

  21. Maurice, P., et al.: Human movement and ergonomics: an industry-oriented dataset for collaborative robotics. Int. J. Robot. Res. 38(14), 1529–1537 (2019)

    Google Scholar 

  22. Rueda, F.M., Grzeszick, R., Fink, G.A., Feldhorst, S., Ten Hompel, M.: Convolutional neural networks for human activity recognition using body-worn sensors. Informatics 5(2), 26 (2018)

    Google Scholar 

  23. Niemann, F., et al.: LARa: creating a dataset for human activity recognition in logistics using semantic attributes. Sensors 20(15), 4083 (2020)

    Google Scholar 

  24. Ordóñez, F., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 115 (2016)

    Google Scholar 

  25. Reining, C., Niemann, F., Rueda, F.M., Fink, G.A., ten Hompel, M.: Human activity recognition for production and logistics - a systematic literature review. Information 10(8), 245 (2019)

    CrossRef  Google Scholar 

  26. Reining, C., Rueda, F.M., Niemann, F., Fink, G.A., ten Hompel, M.: Annotation performance for multi-channel time series HAR dataset in logistics. In: 2020 IEEE International Conference on Pervasive Computing and Communications Workshops, pp. 1–6 (2020)

    Google Scholar 

  27. Ribeiro, P.M.S., Matos, A.C., Santos, P.H., Cardoso, J.S.: Machine learning improvements to human motion tracking with IMUs. Sensors 20(21), 6383 (2020)

    CrossRef  Google Scholar 

  28. Roggen, D., et al.: Collecting complex activity datasets in highly rich networked sensor environments. In: Seventh International Conference on Networked Sensing Systems (INSS), pp. 233–240 (2010)

    Google Scholar 

  29. Rueda, F.M., Fink, G.: From human pose to on-body devices for human-activity recognition. In: 26th International Conference on Pattern Recognition (ICPR), pp. 10066–10073 (2021)

    Google Scholar 

  30. Schaub, K., Caragnano, G., Britzke, B., Bruder, R.: The European assembly worksheet. Theor. Issues Ergon. Sci. 14(6), 616–639 (2013)

    CrossRef  Google Scholar 

  31. Vicon: Full Body Modeling with Plug-in Gate (2017). https://docs.vicon.com/display/Nexus26/Full+body+modeling+with+Plug-in+Gait. Accessed 16 Mar 2021

  32. Xian, Y., Lampert, C.H., Schiele, B., Akata, Z.: Zero-shot learning - a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans. Pattern Anal. Mach. Intell. 41(9), 2251–2265 (2018)

    CrossRef  Google Scholar 

  33. Yordanova, K., et al.: Challenges in Annotation of useR Data for UbiquitOUs Systems: Results from the 1st ARDUOUS Workshop (2018). arXiv:1803.05843

  34. Zhang, A., et al.: Transfer learning with deep recurrent neural networks for remaining useful life estimation. Appl. Sci. 8(12), 2416 (2018)

    CrossRef  Google Scholar 

  35. Zhang, A., Lipton, Z.C., Li, M., Smola, A.J.: Dive into Deep Learning (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lena Schmid .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Kirchhof, M., Schmid, L., Reining, C., Hompel, M.t., Pauly, M. (2021). Chances of Interpretable Transfer Learning for Human Activity Recognition in Warehousing. In: Mes, M., Lalla-Ruiz, E., Voß, S. (eds) Computational Logistics. ICCL 2021. Lecture Notes in Computer Science(), vol 13004. Springer, Cham. https://doi.org/10.1007/978-3-030-87672-2_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87672-2_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87671-5

  • Online ISBN: 978-3-030-87672-2

  • eBook Packages: Computer ScienceComputer Science (R0)