Abstract
Most of today’s assistance systems for the shop floor do not consistently follow a human-centered design approach. An explicit modeling of concepts for instructions, users and adaptations is required when aiming for a holistic approach. Therefore, we first define a morphological box to capture design alternatives for adaptive functionalities. Afterwards, a reference architecture is presented, consisting of several building blocks that implement an adaptation loop to continuously adapt the provided assistance and the system behavior to the current user. A selection of algorithms to realize adaptive behavior concludes our contribution. Following a human-centered design approach, three exemplary scenarios for assistance on the shop floor demonstrate the application of our approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Business process modeling and notation in version 2, is a graphical modeling language which creates executable models, standardized by the Object Management Group (OMG) https://www.omg.org/spec/BPMN/2.0/About-BPMN
References
Bläsing, D. & Bornewasser, M. (2021). Influence of increasing task complexity and use of informational assistance systems on mental workload. Brain Sciences, 11.
Büttner, S., Mucha, H., Funk, M., Kosch, T., Aehnelt, M., Robert, S., & Röcker, C. (2017). The design space of augmented and virtual reality applications for assistive environments in manufacturing: A visual approach. In Proceedings of the 10th International Conference on PErvasive technologies related to assistive environments.
Mark, B. G., Gualtieri, L., Rauch, E., Rojas, R., Buakum, D., & Matt, D. T. (2019). Analysis of user groups for assistance systems in Production 4.0. In 2019 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM).
Funk, M., Dingler, T., Cooper, J., & Schmidt, A. (2015). Stop helping me – I’m bored! Why assembly assistance needs to be adaptive. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the Symposium on Wearable Computers - UbiComp‘15. New York, USA.
Schmidt, A., Beigl, M., & Gellersen, H.-W. (1999). There is more to context than location. Computers & Graphics, 23, 893–901.
Kosch, T., Abdelrahman, Y., Funk, M., & Schmidt, A. (2017). One size does not fit all: Challenges of providing interactive worker assistance in industrial settings. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. ACM.
Apt, W., Schubert, M., & Wischmann, S. (2018). Digitale Assistenzsysteme: Perspektiven und Herausforderungen für den Einsatz in Industrie und Dienstleistungen. Institut für Innovation und Technik.
Kleineberg, T., Hinrichsen, S., Eichelberg, M., Busch, F., Brockmann, D., & Vierfuß, R. (2017). LEITFADEN: Einführung von Assistenzsystemen in der Montage. Springer.
Büttner, S., Sand, O., & Röcker, C. (2017). Exploring design opportunities for intelligent worker assistance: A new approach using projection-based AR and a novel hand-tracking algorithm (Vol. 10217). Springer.
Sochor, R., Kraus, L., Merkel, L., Braunreuther, S., & Reinhart, G. (2019). Approach to increase worker acceptance of cognitive assistance systems in manual assembly. In 52nd CIRP conference on manufacturing systems (Vol. 81, pp. 926–931). Elsevier.
Fischer, H., Senft, B., & Stahl, K. (2017). Akzeptierte Assistenzsysteme in der Arbeitswelt 4.0 durch systematisches Human-Centered Software Engineering. In Wissenschaftsforum Intelligente Technische Systeme (WInTeSys). Paderborn.
Streicher, A., & Smeddinck, J. D. (2016). Personalized and adaptive serious games. In R. Dörner, S. Göbel, M. Kickmeier-Rust, M. Masuch, & K. Zweig (Eds.), Entertainment computing and serious games (Vol. 9970, pp. 332–377). Springer.
Burgos, D., Tattersall, C., & Koper, R. (2007). How to represent adaptation in e-learning with IMS learning design. Interactive Learning Environments, 15, 161–170.
Oestreich, H., Wrede, S., & Wrede, B. (2020). Learning and performing assembly processes. In Proceedings of the 13th ACM International Conference on PErvasive technologies related to assistive environments. ACM.
Browne, D. P., Totterdell, P., & Norman, M. (Eds.). (1990). Adaptive user interfaces. Academic Press.
Holmes, W., Anastopoulou, S., Schaumburg, H., & Mavrikis, M. (2018). Technology-enhanced personalised learning: Untangling the evidence. Robert Bosch Stiftung.
Ras, E., Wild, F., Stahl, C., & Baudet, A. (2017). Bridging the skills gap of workers in Industry 4.0 by human performance augmentation tools. In Proceedings of the 10th International Conference on PErvasive technologies related to assistive environments. ACM.
Hodaie, Z., Haladjian, J., & Bruegge, B. (2018). TUMA: Towards an intelligent tutoring system for manual-procedural activities. In R. Nkambou, R. Azevedo, & J. Vassileva (Eds.), Intelligent tutoring systems (Vol. 10858, pp. 326–331). Springer.
GCSM. (2013). Learnstruments in value creation and learning centered work place design. Universitätsverlag der TU Berlin.
Heinz, M., Büttner, S., & Röcker, C. (2020). Exploring users’ eye movements when using projection-based assembly assistive systems. In International conference on human-computer interaction. Springer.
Prinz, C., Kreimeier, D., & Kuhlenkötter, B. (2017). Implementation of a learning environment for an Industrie 4.0 assistance system to improve the overall equipment effectiveness. Procedia Manufacturing, 9, 159–166.
Oestreich, H., Töniges, T., Wojtynek, M., & Wrede, S. (2019). Interactive learning of assembly processes using digital assistance. Procedia Manufacturing, 31, 14–19.
Letmathe, P., Schweitzer, M., & Zielinski, M. (2012). How to learn new tasks: Shop floor performance effects of knowledge transfer and performance feedback. Journal of Operations Management, 30, 221–236.
Wilschut, E. S., Könemann, R., Murphy, M. S., van Rhijn, G. J. W., & Bosch, T. (2019). Evaluating learning approaches for product assembly. In Proceedings of the 12th ACM International Conference on PErvasive technologies related to assistive environments – PETRA’19. ACM.
Baker, R. S. J., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68, 223–241.
Heinz, M., Büttner, S., & Röcker, C. (2019). Exploring training modes for industrial augmented reality learning. In Proceedings of the 12th ACM International Conference on PErvasive technologies related to assistive environments – PETRA’19. ACM.
Müller, B. C., Nguyen, T. D., Dang, Q.-V., Duc, B. M., Seliger, G., Krüger, J., & Kohl, H. (2016). Motion tracking applied in assembly for worker training in different locations. Procedia CIRP, 48, 460–465.
Büttner, S., Prilla, M., & Röcker, C. (2020). Augmented reality training for industrial assembly work-are projection-based AR assistive systems an appropriate tool for assembly training? In Proceedings of the 2020 CHI conference on human factors in computing systems. ACM.
Hinrichsen, S., & Bornewasser, M. (2019). How to design assembly assistance systems. In W. Karwowski & T. Ahram (Eds.), Intelligent human systems integration 2019 (Vol. 903, pp. 286–292). Springer.
Hold, P., Erol, S., Reisinger, G., & Sihn, W. (2017). Planning and evaluation of digital assistance systems. Procedia Manufacturing, 9, 143–150.
Dostert, J. & Müller, R. (2020). Motivational assistance system design for industrial production: From motivation theories to design strategies. Cognition, Technology & Work.
Schulz, A. S., Schulz, F., Gouveia, R., & Korn, O. (2018). Branded gamification in technical education. In 2018 10th international conference on virtual worlds and games for serious applications (VS-Games). IEEE.
Josifovska, K., Yigitbas, E., & Engels, G. (2019). A digital twin-based multi-modal UI adaptation framework for assistance systems in Industry 4.0. In Human-computer interaction. Design practice in contemporary societies. HCII. Springer.
Rönick, K., Kremer, T., & Wakula, J. (2018). Evaluation of an adaptive assistance system to optimize physical stress in the assembly. In Congress of the International Ergonomics Association. IEA.
Petzoldt, C., Keiser, D., Beinke, T., & Freitag, M. (2020). Functionalities and implementation of future informational assistance systems for manual assembly. In Subject-oriented business process management. The Digital Workplace - Nucleus of Transformation.
Taoum, J., Raison, A., Bevacqua, E., & Querrec, R. (2018). An adaptive tutor to promote learners’ skills acquisition during procedural learning. In WeASeL 2018 - 1st international workshop eliciting adaptive sequences for learning (Vol. 1, pp. 12–17). HAL CCSD.
Korn, O. (2018). Autonomie beim Einsatz kontextbewusster Systeme: Der Weg zum Emotionsbewusstsein. In T. Breyer-Mayländer (Ed.), Das Streben nach Autonomie (pp. 203–214). Nomos Verlagsgesellschaft mbH & Co.
Sauer, M. (1990). MTM-UAS, universelles Analysiersystem: eine programmierte Unterweisung. Dt. MTM-Vereinigung.
Nelles, J., Kuz, S., Mertens, A., & Schlick, C. M. (2016). Human-centered design of assistance systems for production planning and control: The role of the human in Industry 4.0. In 2016 IEEE International Conference on Industrial Technology (ICIT). IEEE.
Bächler, L., Bächler, A., Kölz, M., Hörz, T., & Heidenreich, T. (2015, 2015). Über die Entwicklung eines prozedural-interaktiven Assistenzsystems für leistungsgeminderte und-gewandelte Mitarbeiter in der manuellen Montage. Kognitive Systeme, 2015.
Aehnelt, M., & Bader, S. (2016). Providing and adapting information assistance for smart assembly stations. In Proceedings of SAI intelligent systems conference. Springer.
Bannat, A., Wallhoff, F., Rigoll, G., Friesdorf, F., Bubb, H., Stork, S., Müller, H. J., Schubö, A., Wiesbeck, M., Zäh, M. F., et al. (2008). Towards optimal worker assistance: A framework for adaptive selection and presentation of assembly instructions. In Proceedings of the 1st international workshop on cognition for technical systems, Cotesys. Springer.
Galaske, N., & Anderl, R. (2016). Approach for the development of an adaptive worker assistance system based on an individualized profile data model. In C. Schlick & S. Trzcieliński (Eds.), Advances in ergonomics of manufacturing: Managing the enterprise of the future (Vol. 490, pp. 543–556). Springer.
Burggräf, P., Dannapfel, M., Adlon, T., Hahn, V., Riegauf, A., Casla, P., Marguglio, A., & Fernandez, I. (2020). Virtual asset representation for enabling adaptive assembly at the example of electric vehicle production. Academic Press.
Gellert, A., & Zamfirescu, C.-B. (2020). Using two-level context-based predictors for assembly assistance in smart factories. In International conference on computers communications and control. IEEE.
IBM. (2006). An architectural blueprint for autonomic computing. IBM White Paper.
Bertram, P., Birtel, M., Quint, F., & Ruskowski, M. (2018). Intelligent manual working station through assistive systems. IFAC-PapersOnLine, 51, 170–175.
Rupp, S., & Müller, R. (2020). Worker assistance systems and assembly process maturity in the prototype and pre-series production. Procedia Manufacturing, 51, 1431–1438.
ElKomy, M., Abdelrahman, Y., Funk, M., Dingler, T., Schmidt, A., & Abdennadher, S. (2017). ABBAS: An adaptive bio-sensors based assistive system. In Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems. ACM.
Stoessel, C., Wiesbeck, M., Stork, S., Zaeh, M. F., & Schuboe, A. (2008). Towards optimal worker assistance: Investigating cognitive processes in manual assembly. In Manufacturing systems and technologies for the new frontier (pp. 245–250). Springer.
Kosch, T., Karolus, J., Ha, H., & Schmidt, A. (2019). Your skin resists: Exploring electrodermal activity as workload indicator during manual assembly. In Proceedings of the ACM SIGCHI symposium on engineering interactive computing systems. ACM.
Gollan, B., Haslgruebler, M., Ferscha, A., & Heftberger, J. (2018). Making sense: Experiences with multi-sensor fusion in industrial assistance systems. In PhyCS. ACM.
Gorecky, D., Worgan, S. F., & Meixner, G. (2011). COGNITO: A cognitive assistance and training system for manual tasks in industry. In Proceedings of the 29th annual European conference on cognitive ergonomics. IEEE.
Wolf, H., Herrmann, K., & Rothermel, K. (2009). Modeling dynamic context awareness for situated workflows. In OTM confederated international conferences: On the move to meaningful Internet systems. ACM.
Aehnelt, M., & Urban, B. (2015). The knowledge gap: Providing situation-aware information assistance on the shop floor. In International conference on HCI in business. HCI International.
Hollowell, J. C., Kollar, B., Vrbka, J., & Kovalova, E. (2019). Cognitive decision-making algorithms for sustainable manufacturing processes in Industry 4.0: Networked, smart, and responsive devices. Economics, Management and Financial Markets, 14, 9–15.
Aehnelt, M., & Bader, S. (2015). From information assistance to cognitive automation: A smart assembly use case. In International conference on agents and artificial intelligence. Springer.
Duan, Y., Edwards, J. S., & Dwivedi, Y. K. (2019). Artificial intelligence for decision making in the era of big data–evolution, challenges and research agenda. International Journal of Information Management, 48, 63–71.
Oestreich, H., da Silva Bröker, Y., & Wrede, S. (2021). An adaptive workflow architecture for digital assistance systems. In Proceedings of the 14th ACM international conference on PErvasive technologies related to assistive environments. ACM.
Ullrich, C. (2016). Rules for adaptive learning and assistance on the shop floor. In 13th international conference on cognition and exploratory learning in digital age (pp. 261–268). IADIS Press.
Funk, M., Kosch, T., & Schmidt, A. (2016). Interactive worker assistance: Comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. In Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing. ACM.
Blattgerste, J., Strenge, B., Renner, P., Pfeiffer, T., & Essig, K. (2017). Comparing conventional and augmented reality instructions for manual assembly tasks. In Proceedings of the 10th international conference on pervasive technologies related to assistive environments. Springer.
Fraga-Lamas, P., Fernandez-Carames, T. M., Blanco-Novoa, O., & Vilar-Montesinos, M. A. (2018). A review on industrial augmented reality systems for the industry 4.0 shipyard. IEEE Access, 6, 13358–13375.
Fite-Georgel, P. (2011). Is there a reality in industrial augmented reality? In 2011 10th IEEE international symposium on mixed and augmented reality. IEEE.
Kosch, T., Kettner, R., Funk, M., & Schmidt, A. (2016). Comparing tactile, auditory, and visual assembly error-feedback for workers with cognitive impairments. In Proceedings of the 18th international ACM SIGACCESS conference on computers and accessibility. ACM.
Günther, S., Kratz, S., Avrahami, D., & Mühlhäuser, M. (2018). Exploring audio, visual, and tactile cues for synchronous remote assistance. In Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference. ACM.
Wilson, C., & Scott, B. (2017). Adaptive systems in education: A review and conceptual unification. The International Journal of Information and Learning Technology, 34, 2–19.
Graessler, I., & Poehler, A. (2017). Integration of a digital twin as human representation in a scheduling procedure of a cyber-physical production system. IEEE.
Josifovska, K., Yigitbas, E., & Engels, G. (2019). Reference framework for digital twins within cyber-physical systems. In IEEE/ACM 5th International Workshop on Software Engineering for Smart Cyber-Physical Systems (SEsCPS) (Vol. 2019, pp. 25–31). IEEE.
Combemale, B., Kienzle, J. A., Mussbacher, G., Ali, H., Amyot, D., Bagherzadeh, M., Batot, E., Bencomo, N., Benni, B., Bruel, J.-M., Cabot, J., Cheng, B. H. C., Collet, P., Engels, G., Heinrich, R., Jezequel, J.-M., Koziolek, A., Mosser, S., Reussner, R., et al. (2020). A Hitchhiker’s guide to model-driven engineering for data-centric systems. IEEE Software, 2020, 0.
Berisha-Gawlowski, A., Caruso, C., & Harteis, C. (2020). The concept of a digital twin and its potential for learning organizations. In D. Ifenthaler, S. Hofhues, M. Egloffstein, & C. Helbig (Eds.), Digital transformation of learning organizations (pp. 95–114). Springer Nature.
Schwarz, J., Fuchs, S., & Flemisch, F. (2014). Towards a more holistic view on user state assessment in adaptive human-computer interaction. In 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE.
Funk, M., Lischke, L., Mayer, S., Shirazi, A. S., & Schmidt, A. (2018). Teach me how! Interactive assembly instructions using demonstration and in-situ projection. Assistive Augmentation, 2018, 49–73.
Homem de Mello, L. S., & Sanderson, A. C. (1990). AND/OR graph representation of assembly plans. IEEE Transactions on Robotics and Automation, 6, 188–199.
Moore, K. E., Güngör, A., & Gupta, S. M. (2001). Petri net approach to disassembly process planning for products with complex AND/OR precedence relationships. European Journal of Operational Research, 135, 428–449.
Bader, S., & Aehnelt, M. (2014). Tracking assembly processes and providing assistance in smart factories. In ICAART (pp. 161–168). SCITEPRESS - Science and Technology Publications.
Agrawala, M., Phan, D., Heiser, J., Haymaker, J., Klingner, J., Hanrahan, P., & Tversky, B. (2003). Designing effective step-by-step assembly instructions. ACM Transactions on Graphics, 22, 828–837.
Apoki, U. C., Al-Chalabi, H. K. M., & Crisan, G. C. (2020). From digital learning resources to adaptive learning objects: An overview. In D. Simian & L. F. Stoica (Eds.), Modelling and development of intelligent systems (Vol. 1126, pp. 18–32). Springer.
Haase, T., Weisenburger, N., Termath, W., Frosch, U., Bergmann, D., & Dick, M. (2014). The didactical design of virtual reality based learning environments for maintenance technicians. In Virtual, augmented and mixed reality. Applications of virtual and augmented reality. VAMR 2014. Lecture Notes in Computer Science (Vol. 8526). Springer.
Tsalgatidou, A., & Loucopoulos, P. (1991). Rule-based behaviour modelling: Specification and validation of information systems dynamics. Information and Software Technology, 33, 425–432.
Sehr, P., Moriz, N., Heinz, M., & Trsek, H. (2021). Model-based approach for adaptive assembly assistance. In 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA). IEEE.
Wagner, J., Kim, J., & André, E. (2005). From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. In 2005 IEEE international conference on multimedia and expo. IEEE.
Healey, J. A., & Picard, R. W. (2005). Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems, 6, 156–166.
Jo, J., Lee, S. J., Park, K. R., Kim, I.-J., & Kim, J. (2014). Detecting driver drowsiness using feature-level fusion and user-specific classification. Expert Systems with Applications, 41, 1139–1152.
Vodenčarević, A., Büning, H. K., Niggemann, O., & Maier, A. (2011). Using behavior models for anomaly detection in hybrid systems. In 2011 XXIII International Symposium on Information, Communication and Automation Technologies. IEEE.
Gellert, A., Precup, S.-A., Pirvu, B.-C., & Zamfirescu, C.-B. (2020). Prediction-based assembly assistance system. In 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA). IEEE.
Lieto, A., Bhatt, M., Oltramari, A., & Vernon, D. (2018). The role of cognitive architectures in general artificial intelligence. Elsevier.
Ye, P., Wang, T., & Wang, F.-Y. (2018). A survey of cognitive architectures in the past 20 years. IEEE Transactions on Cybernetics, 48, 3280–3290.
Kölz, M., Jordon, D., Kurtz, P., & Hörz, T. (2015). Adaptive assistance to support and promote performance-impaired people in manual assembly processes. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility. ACM.
Varadinek, B., Indenhuck, M., & Surowiecki, E. (2018). Rechtliche Anforderungen an den Datenschutz bei adaptiven Arbeitsassistenzsystemen. Bundesanstalt für Arbeitsschutz und Arbeitsmedizin.
Acknowledgments
This work was supported through the research program “Design of Flexible Work Environments - Human-Centric Use of Cyber-Physical Systems in Industry 4.0”, by the North Rhine-Westphalian funding scheme “Forschungskolleg” and the Ministry of Economic Affairs, Innovation, Digitalisation and Energy North-Rhine Westphalia (MWIDE) within the funding initiative it’s OWL managed by the project management agency Jülich (PTJ) under grant agreement number 005-2011-0241. The work was further supported through the research project „Technology-enabled inclusion through human-centred systems analysis and assistance in industry“, by the German Federal Ministry of Education and Research (BMBF) within the funding scheme „FHprofUnt“ managed by the project management agency VDI Technologiezentrum under grand number 13FH110PX6. The avatar graphics used for the personas in Sect. 4.3 were designed by macrovector_official/Freepik.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Oestreich, H., Heinz-Jakobs, M., Sehr, P., Wrede, S. (2023). Human-Centered Adaptive Assistance Systems for the Shop Floor. In: Röcker, C., Büttner, S. (eds) Human-Technology Interaction. Springer, Cham. https://doi.org/10.1007/978-3-030-99235-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-99235-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-99234-7
Online ISBN: 978-3-030-99235-4
eBook Packages: Computer ScienceComputer Science (R0)