Skip to main content

An Empirical Study to Quantify the SetUp and Maintenance Benefits of Adopting WebDriverManager

Part of the Communications in Computer and Information Science book series (CCIS,volume 1621)


Test automation brings several benefits but also presents significant problems that often force developers/testers to carry out tiring and costly manual tasks. Among these tiring tasks while using Selenium WebDriver for testing Web applications, driver management (i.e., version discovery, download, setup and maintenance) ranks in a top position.

Recently, an open source Java library, named WebDriverManager, that carries out automatically the driver management process for Selenium WebDriver has been proposed to alleviate the burden of the developers. This library appears to be very promising but until now no one has experimentally evaluated its effectiveness.

In this paper, we present an empirical study aimed at understanding whether the use of WebDriverManager allows to reduce both initial and long-term setup efforts of a multibrowser test suite.

The results are in favor of WebDriverManager as it allows an average saving of more than 33% of the time for an initial setup which translated annually corresponds to many hours of bare manual driver management to which must be added the hours for understanding that the outdated driver is the cause of the problems reported by the test suite. Having almost weekly, as we estimated, outdated drivers brings also to a reduction in the perceived reliability of E2E test suites. The adoption of WebDriverManager can help to drastically reduce all these problems.


  • E2E testing
  • Selenium WebDriver
  • Empirical study

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.

  2. 2.

  3. 3.

  4. 4.

  5. 5.

  6. 6. +version.

  7. 7.

  8. 8.

    Note that the setup of different drivers has approximately the same effort, so adopting different browsers/Drivers do not impact the results of our study.

  9. 9.

  10. 10.

  11. 11.

  12. 12.


  1. Ardito, L., Coppola, R., Morisio, M., Torchiano, M.: Espresso vs. eyeautomate: an experiment for the comparison of two generations of android gui testing. In: Proceedings of the Evaluation and Assessment on Software Engineering, EASE 2019, pp. 13–22. Association for Computing Machinery, New York (2019).

  2. Basili, V.R., Caldiera, G., Rombach, H.D.: The goal question metric approach. In: Encyclopedia of Software Engineering, Wiley, Hoboken (1994)

    Google Scholar 

  3. Cerioli, M., Lagorio, G., Leotta, M., Ricca, F.: Fight silent horror unit test methods by consulting a TestWizard. J. Softw. Evol. Proc. (JSEP) e2396 (2022).

  4. Cerioli, M., Leotta, M., Ricca, F.: What 5 million job advertisements tell us about testing: a preliminary empirical investigation. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 1586–1594 (2020)

    Google Scholar 

  5. Clarke, S.: Measuring API usability. Dr. Dobb’s J. Windows/.NET Suppl. 9(5), S6–S9 (2004)

    Google Scholar 

  6. Garcia, B.: Mastering Software Testing with JUnit 5: Comprehensive Guide to Develop high Quality Java Applications. Packt Publishing Ltd, Birmingham (2017)

    Google Scholar 

  7. García, B.: Hands-On Selenium WebDriver with Java. O’Reilly Media, Sebastopol (2022)

    Google Scholar 

  8. García, B., Gallego, M., Gortázar, F., Munoz-Organero, M.: A survey of the selenium ecosystem. Electronics 9(7), 1067 (2020)

    CrossRef  Google Scholar 

  9. Garcia, B., Gortazar, F., Lopez-Fernandez, L., Gallego, M., Paris, M.: WebRTC testing: challenges and practical solutions. IEEE Commun. Stand. Mag. 1(2), 36–42 (2017)

    CrossRef  Google Scholar 

  10. García, B., Kloos, C.D., Alario-Hoyos, C., Munoz-Organero, M.: Selenium-jupiter: a JUnit 5 extension for selenium WebDriver. J. Syst. Softw. 189, 111298 (2022)

    CrossRef  Google Scholar 

  11. García, B., Munoz-Organero, M., Alario-Hoyos, C., Kloos, C.D.: Automated driver management for selenium WebDriver. Empir. Softw. Eng. 26(5), 1–51 (2021).

    CrossRef  Google Scholar 

  12. Grissom, R.J., Kim, J.J.: Effect sizes for Research: A Broad Practical Approach. 2nd edn. Lawrence Earlbaum Associates, Mahwah (2005)

    Google Scholar 

  13. Kirinuki, H., Matsumoto, S., Higo, Y., Kusumoto, S.: NLP-assisted web element identification toward script-free testing. In: 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 639–643 (2021).

  14. Leotta, M., Biagiola, M., Ricca, F., Ceccato, M., Tonella, P.: A family of experiments to assess the impact of page object pattern in web test suite development. In: Proceedings of 13th IEEE International Conference on Software Testing, Verification and Validation (ICST 2020), pp. 263–273. IEEE (2020).

  15. Leotta, M., Cerioli, M., Olianas, D., Ricca, F.: Two experiments for evaluating the impact of Hamcrest and AssertJ on assertion development. Software Qual. J. 28(3), 1113–1145 (2020).

    CrossRef  Google Scholar 

  16. Motulsky, H.: Intuitive Biostatistics: a Non-Mathematical Guide to Statistical Thinking. Oxford University Press, Oxford (2010)

    Google Scholar 

  17. Naidu, T.J., Basri, N.A., Nagenthram, S.: SAHI vs. selenium: a comparative analysis. In: 2014 International Conference on Contemporary Computing and Informatics (IC3I), pp. 967–970 (2014).

  18. Olianas, D., Leotta, M., Ricca, F., Villa, L.: Reducing flakiness in end-to-end test suites: an experience report. In: Paiva, A.C.R., Cavalli, A.R., Ventura Martins, P., Pérez-Castillo, R. (eds.) QUATIC 2021. CCIS, vol. 1439, pp. 3–17. Springer, Cham (2021).

    CrossRef  Google Scholar 

  19. Phuc Nguyen, D., Maag, S.: Codeless web testing using selenium and machine learning. In: ICSOFT 2020: 15th International Conference on Software Technologies, pp. 51–60. 15th International Conference on Software and Data Technologies (ICSOFT), ScitePress, France, July 2020.

  20. Ricca, F., Torchiano, M., Leotta, M., Tiso, A., Guerrini, G., Reggio, G.: On the impact of state-based model-driven development on maintainability: a family of experiments using UniMod. Empir. Softw. Eng. 23(3), 1743–1790 (2017).

    CrossRef  Google Scholar 

  21. Sazoglu, F.B., Cambazoglu, B.B., Ozcan, R., Altingovde, I.S., Ulusoy, Ö.: Strategies for setting time-to-live values in result caches. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 1881–1884 (2013)

    Google Scholar 

  22. Stewart, S., Burns, D.: WebDriver, W3C Working Draft. (2022). Accessed 1 Apr 2022

  23. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M., Regnell, B., Wesslén, A.: Experimentation in Software Engineering - An Introduction. Kluwer Academic Publishers, Dordrecht (2000)

    Google Scholar 

Download references


This work was partially supported in part by the Ministerio de Ciencia e Innovación-Agencia Estatal de Investigación (10.13039/501100011033) through the H2O Learn project under Grant PID2020-112584RB-C31, and in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS-4307.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Maurizio Leotta .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Leotta, M., García, B., Ricca, F. (2022). An Empirical Study to Quantify the SetUp and Maintenance Benefits of Adopting WebDriverManager. In: Vallecillo, A., Visser, J., Pérez-Castillo, R. (eds) Quality of Information and Communications Technology. QUATIC 2022. Communications in Computer and Information Science, vol 1621. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-14178-2

  • Online ISBN: 978-3-031-14179-9

  • eBook Packages: Computer ScienceComputer Science (R0)