A/B Testing in E-commerce Sales Processes

  • Kostantinos Koukouvis
  • Roberto Alcañiz Cubero
  • Patrizio PelliccioneEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9823)


E-commerce has traditionally been a field where online controlled experiments, such as A/B testing, take place. Most of these experiments focus on evaluating the front-end of the application and specifically different visual aspects, e.g. creating variations of the layouts, fonts, colors of the site, etc. In this paper we want to experiment whether A/B testing can be used to evaluate e-commerce sales processes and to improve the resilience of these processes. To achieve this goal we developed a tool in collaboration with a company, called Sonician AB, which is focused on marketing automation. The tool has been designed to empower business owners with a virtual assistant able to help customers understanding their needs and making decisions while purchasing products or services. The tool has been evaluated within the company and instantiated in two different business flow cases. Two experiments under real-life conditions show promising results. The paper concludes with lessons learned and a set of guidelines designed to help companies with interest of conducting similar experiments.


Business Owner Visual Aspect Sales Process Database Module Decision Assistant 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bosch, J.: Building products as innovation experiment systems. In: Cusumano, M.A., Iyer, B., Venkatraman, N. (eds.) ICSOB 2012. LNBIP, vol. 114, pp. 27–39. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  2. 2.
    Davenport, T.H.: How to design smart business experiments. Harvard Bus. Rev. 87(2), 68–76 (2009)MathSciNetGoogle Scholar
  3. 3.
    Garbarino, E.C., Edell, J.A.: Cognitive effort, affect, and choice. J. Consum. Res. 24(2), 147–158 (1997)CrossRefGoogle Scholar
  4. 4.
    Grenci, R.T., Todd, P.A.: Solutions-driven marketing. Commun. ACM 45(3), 64–71 (2002)CrossRefGoogle Scholar
  5. 5.
    Huffman, C., Kahn, B.E.: Variety for sale: Mass customization or mass confusion? J. Retail. 74(4), 491–513 (1998)CrossRefGoogle Scholar
  6. 6.
    Hynninen, P., Kauppinen, M.A.: B testing: a promising tool for customer value evaluation. In: Proceedings of RET 2014, pp. 16–17, August 2014Google Scholar
  7. 7.
    Kaufmann, E., Cappé, O., Garivier, A.: On the complexity of A/B testing. In: Proceedings of the Conference on Learning Theory, Junuary 2014Google Scholar
  8. 8.
    Kohavi, R., Deng, A., Frasca, B., Longbotham, R., Walker, T., Xu, Y.: Trustworthy online controlled experiments: Five puzzling outcomes explained. In: Proceedings of KDD 2012, pp. 786–794. ACM, New York, NY, USA (2012)Google Scholar
  9. 9.
    Kohavi, R., Henne, R.M., Sommerfield, D.: Practical guide to controlled experiments on the web: listen to your customers not to the hippo. In: Proceedings of KDD 2007, pp. 959–967. ACM, New York, NY, USA (2007)Google Scholar
  10. 10.
    Kohavi, R., Longbotham, R., Sommerfield, D., Henne, R.: Controlled experiments on the web: survey and practical guide. Data Min. Knowl. Discovery 18(1), 140–181 (2009)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Koukouvis, K., Alcañiz Cubero, R.: Towards extending A/B Testing in E-Commerce sales processes. Master thesis, Chalmers University of Technology, Department of Computer Science and Engineering, Gothenburg, Sweden (2015)Google Scholar
  12. 12.
    O’Keefe, R.M., McEachern, T.: Web-based customer decision support systems. Commun. ACM 41(3), 71–78 (1998)CrossRefGoogle Scholar
  13. 13.
    Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empirical Softw. Eng. 14(2), 131–164 (2009)CrossRefGoogle Scholar
  14. 14.
    Shenton, A.K., Hayter, S.: Strategies for gaining access to organisations and informants in qualitative studies. Educ. Inf. 22(3–4), 223–231 (2004)Google Scholar
  15. 15.
    Young, S.W.H.: Experience, improving library user with A, B testing: principles and process. Weave J. Libr. User Experience 1(1), (2014). doi:12535642.0001.101

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Kostantinos Koukouvis
    • 1
  • Roberto Alcañiz Cubero
    • 1
  • Patrizio Pelliccione
    • 1
    Email author
  1. 1.Chalmers University of TechnologyUniversity of GothenburgGothenburgSweden

Personalised recommendations