Skip to main content
SpringerLink
Log in

A Conceptual Model for Labeling in Reinforcement Learning Systems: A Value Co-creation Perspective

  • Conference paper
  • First Online:
Design Science Research for a New Society: Society 5.0 (DESRIST 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13873))

Abstract

Artificial intelligence (AI) possesses the potential to augment customer service employees e.g. via decision support or solution recommendations. Still, its underlying data for training and testing the AI systems is provided by human annotators through human-in-the-loop configurations. However, due to the high effort for annotators and lack of incentives, AI systems face low underlying data quality. That in turn results in low prediction performance and limited acceptance by the targeted user group. Faced with the enormous volume and increasing complexity of service requests, IT service management (ITSM) especially, relies on high data quality for AI systems and incorporating domain-specific knowledge. By analyzing the existing labeling process in that specific case, we design a revised to-be process and develop a conceptual model from a value co-creation perspective. Finally, a functional prototype as an instantiation in the ITSM domain is implemented and evaluated through accuracy metrics and user evaluation. The results show that the new process increases the perceived value of both labeling quality and the perceived prediction quality. Thus, we contribute a conceptual model that supports the systematic design of efficient and interactive labeling processes in diverse applications of reinforcement learning systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Weighted precsion, recall and F1-score.

References

  1. Grønsund, T., Aanestad, M.: Augmenting the algorithm: emerging human-in-the-loop work configurations. J. Strateg. Inf. Syst. 29, 101614 (2020). https://doi.org/10.1016/j.jsis.2020.101614

    Article  Google Scholar 

  2. Li, M.M., Peters, C., Leimeister, J.M.: Designing a peer-based support system to support shakedown. In: International Conference on Information Systems (ICIS), Seoul, South Korea (2017)

    Google Scholar 

  3. Al-Hawari, F., Barham, H.: A machine learning based help desk system for IT service management. J. King Saud Univ. – Comput. Inf. Sci. 33, 702–718 (2021). https://doi.org/10.1016/j.jksuci.2019.04.001

    Article  Google Scholar 

  4. Dostál, M.: Service desk onboarding training environment. Acta Informatica Pragensia 11, 265–284 (2022)

    Article  Google Scholar 

  5. Rumburg, J.: Metric of the Month: Annual Agent Turnover (2018)

    Google Scholar 

  6. Schmidt, S., Li, M., Peters, C.: Requirements for an IT support system based on hybrid intelligence. In: HICSS (2022)

    Google Scholar 

  7. Li, M., Löfflad, D., Reh, C., et al.: Towards the design of hybrid intelligence frontline service technologies – a novel human-in-the-loop configuration for human-machine interactions. In: HICSS (2023)

    Google Scholar 

  8. Kubiak, P., Rass, S.: An overview of data-driven techniques for IT-service-management. IEEE Access 6, 63664–63688 (2018). https://doi.org/10.1109/ACCESS.2018.2875975

    Article  Google Scholar 

  9. Dellermann, D., Ebel, P., Söllner, M., et al.: Hybrid intelligence. Bus. Inf. Syst. Eng. 61, 637–643 (2019)

    Article  Google Scholar 

  10. Choi, M., Park, C., Yang, S., et al.: AILA: attentive interactive labeling assistant for document classification through attention-based deep neural networks. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York (2019)

    Google Scholar 

  11. Luo, X., Qin, M.S., Fang, Z., et al.: Artificial intelligence coaches for sales agents: caveats and solutions. J. Mark. 85, 14–32 (2021). https://doi.org/10.1177/0022242920956676

    Article  Google Scholar 

  12. Kim, T., Song, H.: Communicating the limitations of AI: the effect of message framing and ownership on trust in artificial intelligence. Int. J. Hum.–Comput. Interact. 1–11 (2022). https://doi.org/10.1080/10447318.2022.2049134

  13. Jacovi, A., Marasović, A., Miller, T., et al.: Formalizing trust in artificial intelligence. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. ACM, New York (2021)

    Google Scholar 

  14. Schmitt, A., Wambsganss, T., Janson, A.: Designing for conversational system trustworthiness: the impact of model transparency on trust and task performance. In: ECIS, vol. 172 (2022)

    Google Scholar 

  15. Holzinger, A.: Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inform. 3(2), 119–131 (2016). https://doi.org/10.1007/s40708-016-0042-6

    Article  Google Scholar 

  16. Wiethof, C., Bittner, E.: Hybrid intelligence - combining the human in the loop with the computer in the loop: a systematic literature review (2021)

    Google Scholar 

  17. Nadj, M., Knaeble, M., Li, M.X., Maedche, A.: Power to the Oracle? Design principles for interactive labeling systems in machine learning. KI - Künstliche Intelligenz 34(2), 131–142 (2020). https://doi.org/10.1007/s13218-020-00634-1

    Article  Google Scholar 

  18. Viana, L., Oliveira, E., Conte, T.: An interface design catalog for interactive labeling systems. In: Proceedings of the 23rd International Conference on Enterprise Information Systems. SCITEPRESS - Science and Technology Publications (2021)

    Google Scholar 

  19. Warsinsky, S., Schmidt-Kraepelin, M., Thiebes, S., et al.: Gamified expert annotation systems: meta-requirements and tentative design. In: Drechsler, A., Gerber, A., Hevner, A. (eds.) The Transdisciplinary Reach of Design Science Research. DESRIST 2022. Lecture Notes in Computer Science, vol. 13229, pp 154–166. Springer, Cham. https://doi.org/10.1007/978-3-031-06516-3_12

  20. Grönroos, C., Voima, P.: Critical service logic: making sense of value creation and co-creation. J. Acad. Mark. Sci. 41, 133–150 (2013)

    Article  Google Scholar 

  21. Vargo, S.L., Lusch, R.F.: The four service marketing myths: remnants of a goods-based, manufacturing model. J. Serv. Res. 6, 324–335 (2004)

    Article  Google Scholar 

  22. Zanzotto, F.M.: Viewpoint: human-in-the-loop artificial intelligence. JAIR 64, 243–252 (2019)

    Google Scholar 

  23. Porter, R.B., Theiler, J.P., Hush, D.R.: Interactive machine learning in data exploitation. Office of Scientific and Technical Information (OSTI) (2013)

    Google Scholar 

  24. Meza Martínez, M.A., Nadj, M., Maedche, A.: Towards an integrative theoretical framework of interactive machine learning systems. In: ECIS (2019)

    Google Scholar 

  25. Amershi, S., Cakmak, M., Knox, W.B., et al.: Power to the people: the role of humans in interactive machine learning. AIMag 35, 105–120 (2015). https://doi.org/10.1609/aimag.v35i4.2513

    Article  Google Scholar 

  26. Jiang, L., Liu, S., Chen, C.: Recent research advances on interactive machine learning. J. Vis. 22(2), 401–417 (2018). https://doi.org/10.1007/s12650-018-0531-1

    Article  Google Scholar 

  27. Kleinschmidt, S., Peters, C., Leimeister, J.M.: How to scale up contact-intensive services: ICT-enabled service innovation. JOSM 31, 793–814 (2020). https://doi.org/10.1108/JOSM-12-2017-0349

    Article  Google Scholar 

  28. Afsar, M.M., Crump, T., Far, B.: Reinforcement learning based recommender systems: a survey. CoRR (2021)

    Google Scholar 

  29. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. JAIR 4, 237–285 (1996). https://doi.org/10.1613/jair.301

    Article  Google Scholar 

  30. Elshan, E., Ebel, P.A., Söllner, M. et al.: Leveraging low code development of smart personal assistants: an integrated design approach with the SPADE method. J. Manag. Inf. Syst. (JMIS) (2022)

    Google Scholar 

  31. Bernard, J., Hutter, M., Zeppelzauer, M., et al.: Comparing visual-interactive labeling with active learning: an experimental study. IEEE Trans. Vis. Comput. Graph. 24, 298–308 (2018). https://doi.org/10.1109/tvcg.2017.2744818

    Article  Google Scholar 

  32. Schüritz, R., Farrell, K., Wixom, B., et al.: Value co-creation in data-driven services: towards a deeper understanding of the joint sphere (2019)

    Google Scholar 

  33. Blaschke, M., Riss, U., Haki, K., Aier, S.: Design principles for digital value co-creation networks: a service-dominant logic perspective. Electron. Mark. 29(3), 443–472 (2019). https://doi.org/10.1007/s12525-019-00356-9

    Article  Google Scholar 

  34. Peters, C.: Designing work and service systems. Doctoral Dissertation (2020)

    Google Scholar 

  35. Grönroos, C.: Service logic revisited: who creates value? And who co-creates? Eur. Bus. Rev. 20, 298–314 (2008). https://doi.org/10.1108/09555340810886585

    Article  Google Scholar 

  36. Vargo, S.L., Lusch, R.F.: Service-dominant logic: continuing the evolution. J. Acad. Mark. Sci. 36, 1 (2008). https://doi.org/10.1007/s11747-007-0069-6

    Article  Google Scholar 

  37. Grönroos, C.: Value co-creation in service logic: a critical analysis. Mark. Theory 11, 279–301 (2011). https://doi.org/10.1177/1470593111408177

    Article  Google Scholar 

  38. Kuechler, B., Vaishnavi, V.: On theory development in design science research: anatomy of a research project. Eur. J. Inf. Syst. 17, 489–504 (2008)

    Article  Google Scholar 

  39. Hevner, A.R., March, S.T., Park, J., et al.: Design science in information systems research. MIS Q. 28, 75 (2004). https://doi.org/10.2307/25148625

  40. Winter, R.: Design science research in Europe. Eur. J. Inf. Syst. 17, 470–475 (2008). https://doi.org/10.1057/ejis.2008.44

    Article  Google Scholar 

  41. Peffers, K., Rothenberger, M., Tuunanen, T., et al.: Design science research evaluation. In: Peffers, K., Rothenberger, M., Kuechler, B. (eds) Design Science Research in Information Systems. Advances in Theory and Practice. DESRIST 2012. Lecture Notes in Computer Science, vol. 7286, , pp. 398–410. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29863-9_29

  42. Venable, J., Pries-Heje, J., Baskerville, R.: FEDS: a framework for evaluation in design science research. Eur. J. Inf. Syst. 25, 77–89 (2016). https://doi.org/10.1057/ejis.2014.36

    Article  Google Scholar 

  43. Cao, H.-A., Wijaya, T.K., Aberer, K., et al.: A collaborative framework for annotating energy datasets. In: 2015 IEEE International Conference on Big Data (Big Data). IEEE (2015)

    Google Scholar 

  44. Yan, Yang, J., Hauptmann: Automatically labeling video data using multi-class active learning. In: Proceedings Ninth IEEE International Conference on Computer Vision. IEEE (2003)

    Google Scholar 

  45. Desmond, M., Duesterwald, E., Brimijoin, K., et al.: Semi-automated data labeling. In: NeurIPS 2020 Competition and Demonstration Track, pp. 156–169 (2021)

    Google Scholar 

  46. Eisenhardt, K.M.: Agency theory: an assessment and review. AMR 14, 57–74 (1989). https://doi.org/10.5465/amr.1989.4279003

    Article  Google Scholar 

  47. Ranjan, K.R., Read, S.: Value co-creation: concept and measurement. J. Acad. Mark. Sci. 44(3), 290–315 (2014). https://doi.org/10.1007/s11747-014-0397-2

    Article  Google Scholar 

  48. Mallick, A., Hsieh, K., Arzani, B., et al.: Matchmaker: data drift mitigation in machine learning for large-scale systems. Proc. Mach. Learn. Syst. 4, 77–94 (2022)

    Google Scholar 

  49. Pianykh, O.S., Langs, G., Dewey, M., et al.: Continuous learning AI in radiology: implementation principles and early applications. Radiology 297, 6–14 (2020). https://doi.org/10.1148/radiol.2020200038

    Article  Google Scholar 

  50. Mirbabaie, M., Brünker, F., Möllmann, F., Nicholas, R.J., et al.: The rise of artificial intelligence – understanding the AI identity threat at the workplace. Electron. Mark. 32, 73–99 (2022). https://doi.org/10.1007/s12525-021-00496-x

  51. Vorobeva, D., El Fassi, Y., Costa Pinto, D., et al.: Thinking skills don’t protect service workers from replacement by artificial intelligence. J. Serv. Res. 25, 601–613 (2022). https://doi.org/10.1177/10946705221104312

    Article  Google Scholar 

  52. Braun, M., Greve, M., Riquel, J., et al.: Meet your new colle (AI) GUE–exploring the impact of human-AI interaction designs on user performance. In: ECIS (2022)

    Google Scholar 

  53. Hemmer, P., Schemmer, M., Riefle, L., et al.: Factors that influence the adoption of human-AI collaboration in clinical decision-making. In: ECIS (2022)

    Google Scholar 

  54. Dietvorst, B.J., Simmons, J.P., Massey, C.: Overcoming algorithm aversion: people will use imperfect algorithms if they can (even slightly) modify them. Manag. Sci. 64, 1155–1170 (2018)

    Article  Google Scholar 

  55. Lockey, S., Gillespie, N., Holm, D., et al.: A review of trust in artificial intelligence: challenges, vulnerabilities and future directions. In: HICSS (2021)

    Google Scholar 

  56. Venable, J., Pries-Heje, J., Baskerville, R.: A comprehensive framework for evaluation in design science research. In: Peffers, K., Rothenberger, M., Kuechler, B. (eds.) Design Science Research in Information Systems. Advances in Theory and Practice. DESRIST 2012. Lecture Notes in Computer Science, vol. 7286, pp. 423–438. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29863-9_31

  57. Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 257–297. Springer, Boston, MA (2011). https://doi.org/10.1007/978-0-387-85820-3_8

    Chapter  Google Scholar 

  58. Brooke, J.: SUS: A ‘quick and dirty’ usability scale. In: Usability Evaluation in Industry. CRC Press, pp. 207–212 (1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Kassel, Kassel, Germany

    Philipp Reinhard, Mahei Manhai Li, Ernestine Dickhaut, Cornelius Reh, Christoph Peters & Jan Marco Leimeister

  2. University of St.Gallen, St.Gallen, Switzerland

    Christoph Peters & Jan Marco Leimeister

Authors
  1. Philipp Reinhard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahei Manhai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ernestine Dickhaut
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cornelius Reh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christoph Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jan Marco Leimeister
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philipp Reinhard .

Editor information

Editors and Affiliations

  1. University of the Western Cape, Cape Town, South Africa

    Aurona Gerber

  2. Georgia State University, Atlanta, GA, USA

    Richard Baskerville

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Reinhard, P., Li, M.M., Dickhaut, E., Reh, C., Peters, C., Leimeister, J.M. (2023). A Conceptual Model for Labeling in Reinforcement Learning Systems: A Value Co-creation Perspective. In: Gerber, A., Baskerville, R. (eds) Design Science Research for a New Society: Society 5.0. DESRIST 2023. Lecture Notes in Computer Science, vol 13873. Springer, Cham. https://doi.org/10.1007/978-3-031-32808-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-32808-4_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-32807-7

  • Online ISBN: 978-3-031-32808-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation