Skip to main content
SpringerLink
Log in

Continuous Experimentation and Human Factors

An Exploratory Study

  • Conference paper
  • First Online:
Product-Focused Software Process Improvement (PROFES 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14483))

  • 433 Accesses

Abstract

In today’s rapidly evolving technological landscape, the success of tools and systems relies heavily on their ability to meet the needs and expectations of users. User-centered design approaches, with a focus on human factors, have gained increasing attention as they prioritize the human element in the development process. With the increasing complexity of software-based systems, companies are adopting agile development methodologies and emphasizing continuous software experimentation. However, there is limited knowledge on how to effectively execute continuous experimentation with respect to human factors within this context. This research paper presents an exploratory qualitative study for integrating human factors in continuous experimentation, aiming to uncover distinctive characteristics of human factors and continuous software experiments, practical challenges for integrating human factors in continuous software experiments, and best practices associated with the management of continuous human factors experimentation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Key terms of this study are defined in Table 1.

References

  1. Abrahamsson, P., Salo, O., Ronkainen, J., Warsta, J.: Agile software development methods: review and analysis. arXiv preprint arXiv:1709.08439 (2017)

  2. Boy, G.A.: Human-centered design of complex systems: an experience-based approach. Design Sci. 3, e8 (2017)

    Article  Google Scholar 

  3. Charlton, S.G., O’Brien, T.G.: Handbook of Human Factors Testing and Evaluation. CRC Press, Boca Raton (2019)

    Book  Google Scholar 

  4. Clarke, V., Braun, V., Hayfield, N.: Thematic analysis. Qual. Psychol.: Pract. Guide Res. Methods 3, 222–248 (2015)

    Google Scholar 

  5. Fabijan, A., Dmitriev, P., Olsson, H.H., Bosch, J.: The evolution of continuous experimentation in software product development: from data to a data-driven organization at scale. In: 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), pp. 770–780. IEEE (2017)

    Google Scholar 

  6. Fagerholm, F., Guinea, A.S., Mäenpää, H., Münch, J.: The right model for continuous experimentation. J. Syst. Softw. 123, 292–305 (2017)

    Article  Google Scholar 

  7. Feitelson, D.G., Frachtenberg, E., Beck, K.L.: Development and deployment at Facebook. IEEE Internet Comput. 17(4), 8–17 (2013)

    Article  Google Scholar 

  8. Franklin, A.D.: What makes a ‘good’ experiment? Br. J. Philos. Sci. 32(4), 367–374 (1981)

    Article  MathSciNet  Google Scholar 

  9. Gandevia, S.: A human factor in ‘good’ experiments. Br. J. Philos. Sci. 37(4), 463–466 (1986)

    Article  Google Scholar 

  10. Hancock, P., Caird, J.K.: Experimental evaluation of a model of mental workload. Hum. Factors 35(3), 413–429 (1993)

    Article  Google Scholar 

  11. Hancock, P.A.: Some pitfalls in the promises of automated and autonomous vehicles. Ergonomics 62(4), 479–495 (2019)

    Article  Google Scholar 

  12. Human Factors and Ergonomics Society: Definitions of human factors and ergonomics (2023). https://www.hfes.org/About-HFES/What-is-Human-Factors-and-Ergonomics. Accessed 17 Feb 2023

  13. Kevic, K., Murphy, B., Williams, L., Beckmann, J.: Characterizing experimentation in continuous deployment: a case study on bing. In: 39th International Conference on Software Engineering (ICSE-SEIP). IEEE (2017)

    Google Scholar 

  14. Kohavi, R., Henne, R.M., Sommerfield, D.: Practical guide to controlled experiments on the web: listen to your customers not to the hippo. In: Proceedings of the 13th ACM SIGKDD, pp. 959–967 (2007)

    Google Scholar 

  15. Kohavi, R., et al.: Online experimentation at Microsoft. Data Min. Case Stud. 11(2009), 39 (2009)

    Google Scholar 

  16. Lee, J.D., Seppelt, B.D.: Human factors in automation design. In: Nof, S. (ed.) Springer Handbook of Automation. Springer Handbooks, pp. 417–436. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-540-78831-7_25

    Chapter  MATH  Google Scholar 

  17. Lindgren, E., Münch, J.: Software development as an experiment system: a qualitative survey on the state of the practice. In: Lassenius, C., Dingsøyr, T., Paasivaara, M. (eds.) XP 2015. LNBIP, vol. 212, pp. 117–128. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18612-2_10

    Chapter  Google Scholar 

  18. Madni, A.M.: Integrating humans with and within complex systems. CrossTalk 5 (2011)

    Google Scholar 

  19. Maruping, L.M., Matook, S.: The evolution of software development orchestration: current state and an agenda for future research. Eur. J. Inf. Syst. 29(5), 443–457 (2020)

    Article  Google Scholar 

  20. Maxwell, J.A.: Qualitative Research Design: An Interactive Approach. Sage Publications (2012)

    Google Scholar 

  21. Muhammad, A.P., Knauss, E., Bärgman, J.: Human factors in developing automated vehicles: a requirements engineering perspective. J. Syst. Softw. 111810 (2023)

    Google Scholar 

  22. Muhammad, A.P., Knauss, E., Bärgman, J., Knauss, A.: Towards challenges and practices with managing human factors in automated vehicle development. In: 31st IEEE International Requirements Engineering Conference (RE 2023). IEEE (2023)

    Google Scholar 

  23. Nakao, Y., Moriguchi, M., Noda, H.: Using agile software development methods to support human-centered design. NEC Tech. J. 8(3), 37–40 (2014)

    Google Scholar 

  24. Norman, D.: Design of Everyday Things, Revised and Expanded. Basic books (2013)

    Google Scholar 

  25. Olsson, H.H., Alahyari, H., Bosch, J.: Climbing the “stairway to heaven”-a mulitiple-case study exploring barriers in the transition from agile development towards continuous deployment of software. In: 2012 38th Euromicro Conference on Software Engineering and Advanced Applications, pp. 392–399. IEEE (2012)

    Google Scholar 

  26. Rissanen, O., Münch, J.: Continuous experimentation in the b2b domain: a case study. In: 2015 IEEE/ACM 2nd International Workshop on Rapid Continuous Software Engineering, pp. 12–18. IEEE (2015)

    Google Scholar 

  27. Royer, M., Houser, K., Durmus, D., Esposito, T., Wei, M.: Recommended methods for conducting human factors experiments on the subjective evaluation of colour rendition. Light. Res. Technol. 54(3), 199–236 (2022)

    Article  Google Scholar 

  28. Sætren, G.B., Hogenboom, S., Laumann, K.: A study of a technological development process: human factors-the forgotten factors? Cogn. Technol. Work 18, 595–611 (2016)

    Article  Google Scholar 

  29. Shneiderman, B.: Human factors experiments in designing interactive systems. Computer 12(12), 9–19 (1979)

    Article  Google Scholar 

  30. Steinberg, R., Grumman, N.: Human factors at the speed of relevance for agile engineering. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 66. SAGE Publications, Los Angeles (2022)

    Google Scholar 

  31. Verhoef, P.C., et al.: Digital transformation: a multidisciplinary reflection and research agenda. J. Bus. Res. 122, 889–901 (2021)

    Article  Google Scholar 

  32. Wickens, C.D., Gordon, S.E., Liu, Y., Lee, J.: An Introduction to Human Factors Engineering, vol. 2. Pearson Prentice Hall, Upper Saddle River (2004)

    Google Scholar 

  33. Williams, K.W.: Impact of aviation highway-in-the-sky displays on pilot situation awareness. Hum. Factors 44(1), 18–27 (2002)

    Article  Google Scholar 

  34. Yaman, S.G.: Initiating the transition towards continuous experimentation: empirical studies with software development teams and practitioners. Ser. Publ. A (2019)

    Google Scholar 

  35. Yaman, S.G., et al.: Introducing continuous experimentation in large software-intensive product and service organisations. J. Syst. Softw. 133, 195–211 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

The authors express their gratitude to the interviewees for their valuable time and insights. The project has received funding from the Marie Skłodowska-Curie grant agreement 860410 under the European Union’s Horizon 2020 research and innovation program.

Author information

Authors and Affiliations

  1. Dept. of Computer Science and Eng., Chalmers | University of Gothenburg, Gothenburg, Sweden

    Amna Pir Muhammad & Eric Knauss

  2. Department of Mechanics and Maritime Sciences, Chalmers University of Technology, Gothenburg, Sweden

    Jonas Bärgman

  3. Zenseact AB, Gothenburg, Sweden

    Alessia Knauss

Authors
  1. Amna Pir Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Knauss
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonas Bärgman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alessia Knauss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amna Pir Muhammad .

Editor information

Editors and Affiliations

  1. FHV Vorarlberg University of Applied Science, Dornbirn, Austria

    Regine Kadgien

  2. Fraunhofer Institute for Experimental Software Engineering, Kaiserslautern, Germany

    Andreas Jedlitschka

  3. FHV Vorarlberg University of Applied Science, Dornbirn, Austria

    Andrea Janes

  4. University of Oulu, Oulu, Finland

    Valentina Lenarduzzi

  5. University of Oulu, Oulu, Finland

    Xiaozhou Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Muhammad, A.P., Knauss, E., Bärgman, J., Knauss, A. (2024). Continuous Experimentation and Human Factors. In: Kadgien, R., Jedlitschka, A., Janes, A., Lenarduzzi, V., Li, X. (eds) Product-Focused Software Process Improvement. PROFES 2023. Lecture Notes in Computer Science, vol 14483. Springer, Cham. https://doi.org/10.1007/978-3-031-49266-2_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49266-2_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49265-5

  • Online ISBN: 978-3-031-49266-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation