Skip to main content
SpringerLink
Log in

Quality Characteristics of a Software Platform for Human-AI Teaming in Smart Manufacturing

  • Conference paper
  • First Online:
Quality of Information and Communications Technology (QUATIC 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1621))

Abstract

As AI-enabled software systems become more prevalent in smart manufacturing, their role shifts from a reactive to a proactive one that provides context-specific support to machine operators. In the context of an international research project, we develop an AI-based software platform that shall facilitate the collaboration between human operators and manufacturing machines.

We conducted 14 structured interviews with stakeholders of the prospective software platform in order to determine the individual relevance of selected quality characteristics for human-AI teaming in smart manufacturing. These characteristics include the ISO 25010:2011 standard for software quality and AI-specific quality characteristics such as trustworthiness, explicability, and auditability. The interviewees rated trustworthiness, functional suitability, reliability, and security as the most important quality characteristics for this context, and portability, compatibility, and maintainability as the least important. Also, we observed agreement regarding the relevance of the quality characteristics among interviewees having the same role. On the other hand, the relevance of each quality characteristics varied depending on the concrete use case of the prospective software platform.

The interviewees also were asked about the key success factors related to human-AI teaming in smart manufacturing. They identified improving the production cycle, increasing operator efficiency, reducing scrap, and reducing ergonomic risks as key success criteria. In this paper, we also discuss metrics for measuring the fulfillment of these quality characteristics, which we intend to operationalize and monitor during operation of the prospective software platform.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.teamingai-project.eu.

  2. 2.

    https://bit.ly/3lV3aFw.

References

  1. Barbacci, M.R., Ellison, R.J., Lattanze, A., Stafford, J.A., Weinstock, C.B., Wood, W.: Quality Attribute Workshops (QAWs), 3rd edn (2003)

    Google Scholar 

  2. Brooke, JDCUVK.: SUS: A ’Quick and Dirty’ Usability Scale. In: Usability Evaluation In Industry. CRC Press, Boca Raton (1996)

    Google Scholar 

  3. Brooke, J.: SUS: a retrospective. J. Usability Stud. 8(2), 29–40 (2013)

    Google Scholar 

  4. Felderer, M., Ramler, R.: Quality assurance for AI-based systems: overview and challenges (Introduction to Interactive Session). In: Winkler, D., Biffl, S., Mendez, D., Wimmer, M., Bergsmann, J. (eds.) SWQD 2021. LNBIP, vol. 404, pp. 33–42. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-65854-0_3

    Chapter  Google Scholar 

  5. Fujii, G., et al.: Guidelines for quality assurance of machine learning-based artificial intelligence. Int. J. Software Eng. Knowl. Eng. 30, 1589–1606 (2020)

    Article  Google Scholar 

  6. Glinz, M.: On non-functional requirements. In: 15th IEEE International Requirements Engineering Conference (RE 2007), pp. 21–26 (2007)

    Google Scholar 

  7. Glinz, M.: Rethinking the notion of non-functional requirements. In: Proceedings of the Third World Congress for Software Quality (3WCSQ 2005), vol. 2, pp. 55–64. Munich, Germany (2005)

    Google Scholar 

  8. Habibullah, K.M., Horkoff, J.: Non-functional requirements for machine learning: understanding current use and challenges in industry. In: 2021 IEEE 29th International Requirements Engineering Conference (RE), pp. 13–23 (2021)

    Google Scholar 

  9. Haindl, P., Buchgeher, G., Khan, M., Moser, B.: Towards a reference software architecture for human-AI teaming in smart manufacturing. In: 2022 IEEE/ACM 44th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pp. 96–100 (2022). https://doi.org/10.1109/ICSE-NIER55298.2022.9793509

  10. Hignett, S., McAtamney, L.: Rapid entire body assessment (REBA). Appl. Ergon. 31(2), 201–205 (2000)

    Article  Google Scholar 

  11. Horkoff, J.: Non-functional requirements for machine learning: challenges and new directions. In: 2019 IEEE 27th International Requirements Engineering Conference (RE), pp. 386–391 (2019)

    Google Scholar 

  12. Ishikawa, F., Matsuno, Y.: Evidence-driven requirements engineering for uncertainty of machine learning-based systems. In: 2020 IEEE 28th International Requirements Engineering Conference (RE), pp. 346–351 (2020)

    Google Scholar 

  13. ISO/IEC 25010: ISO/IEC 25010:2011, Systems and Software Engineering - Systems and Software Quality Requirements and Evaluation (SQuaRE) - System and Software Quality Models (2011)

    Google Scholar 

  14. Karhu, O., Kansi, P., Kuorinka, I.: Correcting working postures in industry: a practical method for analysis. Appl. Ergon. 8(4), 199–201 (1977)

    Article  Google Scholar 

  15. Khan, A., Siddiqui, I.F., Shaikh, M., Anwar, S., Shaikh, M.: Handling non-fuctional requirements in IoT-based machine learning systems. In: 2022 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering), pp. 477–479 (2022)

    Google Scholar 

  16. Kuwajima, H., Yasuoka, H., Nakae, T.: Engineering problems in machine learning systems. Mach. Learn. 109(5), 1103–1126 (2020). https://doi.org/10.1007/s10994-020-05872-w

    Article  MathSciNet  MATH  Google Scholar 

  17. Lenarduzzi, V., Lomio, F., Moreschini, S., Taibi, D., Tamburri, D.A.: Software quality for AI: where we are now? In: Winkler, D., Biffl, S., Mendez, D., Wimmer, M., Bergsmann, J. (eds.) SWQD 2021. LNBIP, vol. 404, pp. 43–53. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-65854-0_4

    Chapter  Google Scholar 

  18. McAtamney, L., Nigel Corlett, E.: RULA: a survey method for the investigation of work-related upper limb disorders. Appl. Ergon. 24(2), 91–99 (1993)

    Article  Google Scholar 

  19. Phuyal, S., Bista, D., Bista, R.: Challenges, Opportunities and future directions of smart manufacturing: a state of art review. Sustain. Futures 2, 100023 (2020)

    Article  Google Scholar 

  20. Pinto, A.: Requirement specification, analysis and verification for autonomous systems. In: 2021 58th ACM/IEEE Design Automation Conference (DAC), pp. 1315–1318 (2021)

    Google Scholar 

  21. Qu, Y.J., Ming, X.G., Liu, Z.W., Zhang, X.Y., Hou, Z.T.: Smart manufacturing systems: state of the art and future trends. Int. J. Adv. Manuf. Technol. 103(9), 3751–3768 (2019)

    Google Scholar 

  22. Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empirical Soft. Eng. 14(2), 131–164 (2009)

    Article  Google Scholar 

  23. Siebert, J., et al.: Towards guidelines for assessing qualities of machine learning Systems. In: Shepperd, M., Brito e Abreu, F., Rodrigues da Silva, A., Pérez-Castillo, R. (eds.) QUATIC 2020. CCIS, vol. 1266, pp. 17–31. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58793-2_2

    Chapter  Google Scholar 

  24. Sutcliffe, A.: Scenario-based requirements engineering. In: Proceedings. 11th IEEE International Requirements Engineering Conference, vol. 2003, pp. 320–329 (2003)

    Google Scholar 

  25. Sutcliffe, A., Maiden, N., Minocha, S., Manuel, D.: Supporting scenario-based requirements engineering. IEEE Trans. Software Eng. 24(12), 1072–1088 (1998)

    Article  Google Scholar 

  26. Vogelsang, A., Borg, M.: Requirements engineering for machine learning: perspectives from data scientists. In: 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW), pp. 245–251 (2019)

    Google Scholar 

  27. Washington state department of labour industries: hazard zone checklist (2022). https://lni.wa.gov/safety-health/_docs/HazardZoneChecklist.pdf. Accessed 31 May 2022

  28. Yin, R.: Case Study Research and Applications: Design and Methods. SAGE Publications Inc, Los Angeles, 6th edn. (2017)

    Google Scholar 

Download references

Acknowledgements

This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement number 957402.

Author information

Authors and Affiliations

  1. Software Competence Center Hagenberg, Hagenberg, Austria

    Philipp Haindl & Thomas Hoch

  2. IDEKO Research Center, Elgoibar, Spain

    Javier Dominguez & Julen Aperribai

  3. Istanbul Technical University, Istanbul, Turkey

    Nazim Kemal Ure & Mehmet Tunçel

Authors
  1. Philipp Haindl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Hoch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Javier Dominguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Julen Aperribai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nazim Kemal Ure
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mehmet Tunçel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philipp Haindl .

Editor information

Editors and Affiliations

  1. University of Malaga, Málaga, Spain

    Antonio Vallecillo

  2. Leiden University, Leiden, The Netherlands

    Joost Visser

  3. University of Castila-La Mancha, Ciudad Real, Spain

    Ricardo Pérez-Castillo

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Haindl, P., Hoch, T., Dominguez, J., Aperribai, J., Ure, N.K., Tunçel, M. (2022). Quality Characteristics of a Software Platform for Human-AI Teaming in Smart Manufacturing. In: Vallecillo, A., Visser, J., Pérez-Castillo, R. (eds) Quality of Information and Communications Technology. QUATIC 2022. Communications in Computer and Information Science, vol 1621. Springer, Cham. https://doi.org/10.1007/978-3-031-14179-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-14179-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-14178-2

  • Online ISBN: 978-3-031-14179-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation