Skip to main content
SpringerLink
Log in

A/B Testing via Continuous Integration and Continuous Delivery

  • Conference paper
  • First Online:
Geoinformatics and Data Analysis (ICGDA 2022)

Part of the book series: Lecture Notes on Data Engineering and Communications Technologies ((LNDECT,volume 143))

Included in the following conference series:

  • 310 Accesses

Abstract

Software version ranking plays an important role in improved user experience and software quality. A/B testing is a technique to distinguish between the popularity and usability of two quite similar versions (A and B) of a product, marketing strategy, search ad, etc. Nowadays, A/B testing can be utilized for the selection of preferred version of two akin software versions. Continuous Integration (CI) and Continuous Delivery (CD) are essential approach in modern software engineering. These approaches aim at the automated verification of the developed software. Moreover, automatic deployment of the software also can be supported. In this paper, we analyse how an automatic A/B testing can be realized in modern software development environments. We present an approach that supports long-term execution of A/B testing. We take advantage of widely-used CI/CD solutions and present an actual implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Armenise, V.: Continuous delivery with Jenkins: Jenkins solutions to implement continuous delivery. In: 2015 IEEE/ACM 3rd International Workshop on Release Engineering, pp. 24–27 (2015). https://doi.org/10.1109/RELENG.2015.19

  2. Bernstein, D.: Containers and cloud: from LXC to Docker to Kubernetes. IEEE Cloud Comput. 1(3), 81–84 (2014)

    Article  Google Scholar 

  3. Chen, L.: Continuous delivery: huge benefits, but challenges too. IEEE Softw. 32(2), 50–54 (2015). https://doi.org/10.1109/MS.2015.27

    Article  Google Scholar 

  4. Fitzgerald, B., Stol, K.J.: Continuous software engineering and beyond: trends and challenges. In: Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering. RCoSE 2014, pp. 1–9. Association for Computing Machinery, New York (2014). https://doi.org/10.1145/2593812.2593813

  5. Lehtonen, T., Suonsyrjä, S., Kilamo, T., Mikkonen, T.: Defining metrics for continuous delivery and deployment pipeline. In: Nummednmaa, J., Sievi-Korte, O., Mäkinen, E. (eds.) Proceedings of the 14th Symposium on Programming Languages and Software Tools (SPLST), pp. 16–30. No. 1525 in CEUR Workshop Proceedings, Aachen (2015). http://ceur-ws.org/Vol-1525/paper-02

  6. Oberle, T., Szabó, C.: An architectural prototype for testware as a service. In: 2015 IEEE 13th International Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 15–19 (2015)

    Google Scholar 

  7. Pathania, N.: Declarative Pipeline Development Tools, pp. 191–209. Apress, Berkeley (2019). https://doi.org/10.1007/978-1-4842-4158-5_5

  8. Révész, Á., Pataki, N.: Containerized A/B testing. In: Budimac, Z. (ed.) Proceedings of the Sixth Workshop on Software Quality Analysis, Monitoring, Improvement, and Applications, pp. 14:1–14:8. CEUR-WS.org (2017). http://ceur-ws.org/Vol-1938/paper-rev.pdf

  9. Révész, A., Pataki, N.: Continuous A/B testing in containers. In: Proceedings of the 2019 2nd International Conference on Geoinformatics and Data Analysis, ICGDA 2019, pp. 11–14. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3318236.3318254

  10. Révész, A., Pataki, N.: Visualisation of Jenkins pipelines. Acta Cybernetica (2021). https://doi.org/10.14232/actacyb.284211, https://cyber.bibl.u-szeged.hu/index.php/actcybern/article/view/4119

  11. Schaefer, A., Reichenbach, M., Fey, D.: Continuous Integration and Automation for Devops, pp. 345–358. Springer, Dordrecht (2013). https://doi.org/10.1007/978-94-007-4786-9_28

  12. Tamburrelli, G., Margara, A.: Towards automated A/B testing. In: Le Goues, C., Yoo, S. (eds.) SSBSE 2014. LNCS, vol. 8636, pp. 184–198. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09940-8_13

    Chapter  Google Scholar 

  13. Török, M., Pataki, N.: Service monitoring agents for DevOps dashboard tool. In: Heričko, M. (ed.) Proceedings of the 21th International Multi-Conference INFORMATION SOCIETY IS 2018, Volume G: Collaboration, Software and Services in Information Society, pp. 47–50 (2018)

    Google Scholar 

  14. Xu, Y., Chen, N., Fernandez, A., Sinno, O., Bhasin, A.: From infrastructure to culture: A/B testing challenges in large scale social networks. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD 2015, pp. 2227–2236. Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2783258.2788602

Download references

Author information

Authors and Affiliations

  1. EPAM, Budapest, Hungary

    Ádám Révész

  2. Department of Programming Languages and Compilers, Faculty of Informatics, Eötvös Loránd University, Budapest, Hungary

    Norbert Pataki

Authors
  1. Ádám Révész
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Norbert Pataki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Norbert Pataki .

Editor information

Editors and Affiliations

  1. Multidimensional Signal Processing Group, Ecole Centrale Marseille, Marseille Cedex 13, France

    Salah Bourennane

  2. Department of Geography, Faculty of Science, Masaryk University, Brno, Czech Republic

    Petr Kubicek

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Révész, Á., Pataki, N. (2022). A/B Testing via Continuous Integration and Continuous Delivery. In: Bourennane, S., Kubicek, P. (eds) Geoinformatics and Data Analysis. ICGDA 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 143. Springer, Cham. https://doi.org/10.1007/978-3-031-08017-3_15

Download citation

Publish with us

Policies and ethics

Search

Navigation