Leveraging Business Transformation with Machine Learning Experiments
The deployment of production-quality ML solutions, even for simple applications, requires significant software engineering effort. Often, companies do not fully understand the consequences and the business impact of ML-based systems, prior to the development of these systems. To minimize investment risks while evaluating the potential business impact of an ML system, companies can utilize continuous experimentation techniques. Based on action research, we report on the experience of developing and deploying a business-oriented ML-based dynamic pricing system in collaboration with a home shopping e-commerce company using a continuous experimentation (CE) approach. We identified a set of generic challenges in ML development that we present together with tactics and opportunities.
KeywordsMachine learning Continuous experimentation Retail industry Dynamic pricing Business transformation
This work was partially supported by the Wallenberg Artificial Intelligence, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation and by the Software Center.
- 1.Breck, E., Cai, S., Nielsen, E., Salib, M., Sculley, D.: The ML test score: a rubric for ML production readiness and technical debt reduction. In: Proceedings - 2017 IEEE International Conference Big Data, Big Data 2017, vol. 2018, pp. 1123–1132 (2018)Google Scholar
- 4.Olsson, H.H., Bosch, J.: From opinions to data-driven software R&D: a multi-case study on how to close the ‘Open Loop’ problem. In: 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications, pp. 9–16 (2014)Google Scholar
- 9.Raj, A., Bosch, J., Olsson, H.H., Arpteg, A., Brinne, B.: Data management challenges for deep learning. In: 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 1–8 (2019)Google Scholar
- 11.Sculley, D., et al.: Machine Learning : The High-Interest Credit Card of Technical Debt, pp. 1–9 (2014)Google Scholar
- 12.Pineau, J.: Building reproducible, reusable, and robust machine learning software. In: 41st ACM/IEEE International Conference on Software Engineering (ICSE 2019) (2019)Google Scholar