Abstract
Test smells are known as bad development practices that reflect poor design and implementation choices in software tests. Over the last decade, there are few attempts to study test smells in the context of system tests that interact with the System Under Test through a Graphical User Interface. To fill the gap, we conduct an exploratory analysis of test smells occurring in System User Interactive Tests (SUIT). We thus, compose a catalog of 35 SUIT-specific smells, identified through a multi-vocal literature review, and show how they differ from smells encountered in unit tests. We also conduct an empirical analysis to assess the diffuseness and removal of these smells in 48 industrial repositories and 12 open-source projects. Our results show that the same type of smells tends to appear in both industrial and open-source projects, but they are not addressed in the same way. We also find that smells originating from a combination of multiple code locations appear more often than those that are localized on a single line. This happens because of the difficulty to observe non-local smells without tool support. Furthermore, we find that smell-removing actions are not frequent with less than 50% of the affected tests ever undergoing a smell removal. Interestingly, while smell-removing actions are rare, some smells disappear while discarding tests, i.e., these smells do not appear in follow-up tests that replace the discarded ones.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Available at https://github.com/serval-uni-lu/ikora-evolution
Available at https://github.com/serval-uni-lu/sonar-ikora-plugin
The complete catalog is available at https://github.com/serval-uni-lu/suit-smell-catalog
References
Advolodkin N (2018) Top 17 automated testing best practices (supported by data). https://ultimateqa.com/automation-patterns-antipatterns/
Alegroth E, Steiner M, Martini A (2016) Exploring the presence of technical debt in industrial GUI-Based Testware: A case study. In: Proceedings of the 9th international conference on software testing, verification and validation workshops, IEEE, pp 257?262. https://doi.org/10.1109/ICSTW.2016.47
Arcelli FF, Mäntylä MV, Zanoni M, Marino A (2016) Comparing and experimenting machine learning techniques for code smell detection. Empir Softw Eng 21(3):1143–1191. https://doi.org/10.1007/s10664-015-9378-4https://doi.org/10.1007/s10664-015-9378-4
Archer M (2010) How test automation with selenium can fail. https://mattarcherblog.wordpress.com/2010/11/29/how-test-automation-with-selenium-or-watir-can-fail/
Athanasiou D, Nugroho A, Visser J, Zaidman A (2014) Test code quality and its relation to issue handling performance. IEEE Trans Softw Eng 40 (11):1100–1125. https://doi.org/10.1109/TSE.2014.2342227
Baker P, Dai ZR, Grabowski J, Haugen O, Schieferdecker I, Williams C (2008) Data-driven testing. In: Model-driven testing. Springer, pp 87–95
Banerjee I, Nguyen B, Garousi V, Memon A (2013) Graphical user interface (GUI) testing: Systematic mapping and repository. Inf Softw Technol 55 (10):1679–1694. https://doi.org/10.1016/j.infsof.2013.03.004
Basit W, Lodhi F, Ahmed F, Bhatti MU (2013) A metric based evaluation of unit tests as specialized clients in Refactoring. Pak J Eng Appl Sci 13:37–53
Battat M (2020) How do you simplify end-to-end test maintenance?. https://dzone.com/articles/how-do-you-simplify-end-to-end-test-maintenance-auhttps://dzone.com/articles/how-do-you-simplify-end-to-end-test-maintenance-au
Bavota G, Qusef A, Oliveto R, De Lucia A, Binkley D (2012) An empirical analysis of the distribution of unit test smells and their impact on software maintenance. In: Proceedings of the 28th IEEE international conference on software maintenance. IEEE, pp 56–65. https://doi.org/10.1109/ICSM.2012.6405253
Bavota G, Qusef A, Oliveto R, De Lucia A, Binkley D (2015) Are test smells really harmful? An empirical study. Empir Softw Eng 20(4):1052–1094. https://doi.org/10.1007/s10664-014-9313-0
Biagiola M, Stocco A, Ricca F, Tonella P (2019) Diversity-based web test generation. In: Proceedings of the 27th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, ACM Press, Tallinn, Estonia. https://doi.org/10.1145/3338906.3338970, vol 1, pp 142–153
Bowes D, Hall T, Petric J, Shippey T, Turhan B (2017) How Good Are My Tests?. In: Proceedings of the 8th workshop on emerging trends in software metrics.IEEE, pp 9–14. https://doi.org/10.1109/WETSoM.2017.2https://doi.org/10.1109/WETSoM.2017.2
Breugelmans M, Rompaey BV (2008) TestQ : Exploring structural and maintenance characteristics of unit test suites. In: Proceedings of the 1st international workshop on advanced software development tools and techniques, i, pp 1–16
Brooks P, Robinson B, Memon AM (2009) An initial characterization of industrial graphical user interface systems. In: Proceedings of the 2nd IEEE international conference on software testing, verification, and validation, pp 11–20. https://doi.org/10.1109/ICST.2009.11
Bushnev Y (2019) Top 15 ui test automation best practices. https://www.blazemeter.com/blog/top-15-ui-test-automation-best-practices-you-should-follow
Buwalda H (2015) Test design for automation: Anti-patterns. https://www.techwell.com/techwell-insights/2015/09/test-design-automation-anti-patterns
Buwalda H (2019) 8 test automation anti-patterns (and how to avoid them). https://dzone.com/articles/8-test-automation-anti-patterns-and-how-to-avoid-t
Canny A, Palanque P, Navarre D (2020) Model-based testing of GUI applications featuring dynamic Instanciation of widgets. In: Proceedings of the international conference on software testing Testing, verification and validation workshops, IEEE, pp 95–104. https://doi.org/10.1109/ICSTW50294.2020.00029
Chen WK, Wang JC (2012) Bad smells and refactoring methods for GUI test scripts. In: Proceedings of the 13th international conference on software engineering, artificial intelligence, networking, and parallel/distributed computing pp 289–294. https://doi.org/10.1109/SNPD.2012.10
Clayton J (2014) Acceptance tests at a single level of abstraction. https://thoughtbot.com/blog/acceptance-tests-at-a-single-level-of-abstraction
Coppola R, Morisio M, Torchiano M (2019) Mobile GUI testing fragility: a study on open-source android applications. IEEE Trans Reliab 68(1):67–90. https://doi.org/10.1109/TR.2018.2869227
Cripsin L (2018) Keep your automated testing simple and avoid anti-patterns. https://www.mabl.com/blog/keep-your-automated-testing-simple
Cunha M, Paiva ACR, Ferreira HS, Abreu R (2010) PETTool: A pattern-based GUI testing tool. In: Proceedings of the 2nd IEEE international conference on software technology and engineering, IEEE, vol 1, pp 202?206. https://doi.org/10.1109/ICSTE.2010.5608882
De Bleser J, Di Nucci D, De Roover C (2019) Assessing diffusion and perception of test smells in scala projects. In: Proceedings of the 16th international conference on mining software repositories, IEEE, pp 457–467 https://doi.org/10.1109/MSR.2019.00072
Delin M, Foegen K (2016) An analysis of information needs to detect test smells. full-scale software engineering/current trends in release engineering, pp 19–24
van Deursen A., Moonen, van Den BA, Kok G (2001) Refactoring test code. In: Proceedings of the 2nd international conference on extreme programming and flexible processes in software engineering, pp 92–95
Dharmender K (2017) Automation testing: Anti-patterns. https://alisterbscott.com/2015/01/20/five-automated-acceptance-test-anti-patterns/
Di Martino S, Fasolino AR, Starace LLL, Tramontana P (2021) Comparing the effectiveness of capture and replay against automatic input generation for Android graphical user interface testing. Softw Test Verif Reliab 31(3):272–285. https://doi.org/10.1002/stvr.1754
England T (2016) Cucumber anti-patterns (part one). https://cucumber.io/blog/bdd/cucumber-antipatterns-part-one/
Evangelisti A (2012) How to transform bad acceptance tests into awesome ones. https://mysoftwarequality.wordpress.com/2012/12/14/how-to-transform-bad-acceptance-tests-into-awesome-ones/
Femmer H, Fernȧndez DM, Juergens E, Klose M, Zimmer I (2014) Rapid requirements checks with requirements smells: two case studies. In: Proceedings of the 1st international workshop on rapid continuous software engineering, ACM Press, New York, USA, pp 10–19. https://doi.org/10.1145/2593812.2593817
Femmer H, Fernández DM, Wagner S, Eder S (2017) Rapid quality assurance with requirements smells. J Syst Softw 123:190–213. arXiv:1611.08847. https://doi.org/10.1016/j.jss.2016.02.047
Fowler M, Beck K, Brant J, Opdyke W, Roberts D (1999) Refactoring: Improving The Design of Existing Code. Addison-Wesley Longman Publishing Co.,Inc., USA
Gao Z, Liang Y, Cohen MB, Memon AM, Wang Z (2015) Making System User Interactive Tests Repeatable: When and What Should We Control?. In: Proceedings of the 37th International Conference on Software Engineering, IEEE, vol 1, pp 55–65, https://doi.org/10.1109/ICSE.2015.28
Garousi V, Ku̇ċu̇k B (2018) Smells in software test code: A survey of knowledge in industry and academia. J Syst Softw 138:52–81. https://doi.org/10.1016/j.jss.2017.12.013
Gawinecki M (2016) Anti-patterns in test automation. http://nomoretesting.com/blog/2016/05/23/anti-patterns/
Goldberg Y (2019) Node.js and javascript testing best practices 2020. https://yonigoldberg.medium.com/yoni-goldberg-javascript-nodejs-testing-best-practices-2b98924c934
Gomez L, Neamtiu I, Azim T, Millstein T (2013) RERAN: Timing- and touch-sensitive record and replay for android. In: Proceedings of the 35th international conference on software engineering, IEEE, pp 72–81. https://doi.org/10.1109/ICSE.2013.6606553
Grano G, Palomba F, Di Nucci D, De Lucia A, Gall HC (2019) Scented since the beginning: On the diffuseness of test smells in automatically generated test code. J Syst Softw 156:312–327. https://doi.org/10.1016/j.jss.2019.07.016
Greiler M, Van Deursen A, Storey MA (2013) Automated detection of test fixture strategies and smells. In: 2013 IEEE Sixth international conference on software testing,verification and validation, IEEE, pp 322–331. https://doi.org/10.1109/ICST.2013.45
Gupta P, Surve P (2011). In: Proceedings of the 1st ACM international workshop on end-to-end test script engineering, ACM Press, New York, USA, pp 1–7.https://doi.org/10.1145/2002931.2002932
Hall MW, Kennedy K (1992) Efficient call graph analysis. ACM Trans Program Lang Syst 1(3):227–242. https://doi.org/10.1145/151640.151643
Hammoudi M, Rothermel G, Stocco A (2016a) WATERFALL: an incremental approach for repairing record-replay tests of web applications. In: Proceedings of the 24th ACM SIGSOFT international symposium on foundations of software engineering, ACM, New York, NY, USA, vol 13-18-Nove, pp 751–762. https://doi.org/10.1145/2950290.2950294
Hammoudi M, Rothermel G, Tonella P (2016b) Why do Record/Replay Tests of Web Applications Break?. In: Proceedings of the international conference on software testing, verification and validation, IEEE, pp 180–190
Hanssen G, Yamashita AF, Conradi R, Moonen L (2010) Software entropy in agile product evolution. In: Proceedings of the 43rd Hawaii international conference on system sciences, IEEE, 2, pp 1–10. https://doi.org/10.1109/HICSS.2010.344. http://ieeexplore.ieee.org/document/5428534/
Hauptmann B, Heinemann L, Vaas R, Braun P (2013) Hunting for smells in natural language tests. In: Proceedings of the 35th international conference on software engineering, IEEE, San Francisco, CA, USA, pp 1217-1220. https://doi.org/10.1109/ICSE.2013.6606682
Hauptmann B, Eder S, Junker M, Juergens E, Woinke V (2015) Generating Refactoring Proposals to remove clones from automated system tests. In: Proceedings of the 23rd international conference on program comprehension, IEEE, vol 2015-August, pp 115–124. https://doi.org/10.1109/ICPC.2015.20
Humble J, Farley DG (2010) Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Addison-Wesley Upper Saddle River, NJ
Hurdugaci V, Zaidman A (2012) Aiding software developers to maintain developer tests. In: Proceedings of 16th european conference on software maintenance and Reengineering, IEEE, pp 11–20
Issa A, Sillito J, Garousi V (2012) Visual testing of graphical user interfaces: an exploratory study towards systematic definitions and approaches. In: Proceedings of the international symposium on web systems evolution, IEEE, pp 11–15. https://doi.org/10.1109/WSE.2012.6320526
Jain N (2007) Patterns and anti-patterns: Acceptance testing with fitnesse. https://blogs.agilefaqs.com/2007/08/25/patterns-and-anti-patterns-acceptance-testing-with-fitnesse/
Tang J, Cao X, Ma A (2008) Towards adaptive framework of keyword driven automation testing. In: Proceedings of the IEEE international conference on automation and logistics, IEEE, September, pp 1631–1636. https://doi.org/10.1109/ICAL.2008.4636415
Kapelonis K (2018) Software testing anti-patterns. http://blog.codepipes.com/testing/software-testing-antipatterns.htmlhttp://blog.codepipes.com/testing/software-testing-antipatterns.html
Katalon (2018) The most striking problems in test automation : A survey. Tech Rep May, Katalon
Khomh F, Vaucher S, Guéhéneuc YG, Sahraoui H (2009) A bayesian approach for the detection of code and design smells. In: Proceedings of the 9th international conference on quality software. IEEE Comput Soc, USA, pp 305–314. https://doi.org/10.1109/QSIC.2009.47
Kim DJ (2020) An empirical study on the evolution of test smell. In: Proceedings of the 42nd international conference on software engineering: companion proceedings, ACM, New York, NY, USA, i, pp 149–151.https://doi.org/10.1145/3377812.3382176
Kirinuki H, Tanno H, Natsukawa K (2019) COLOR: correct locator recommender for broken test scripts using various clues in web application. In: Proceedings of the 26th international conference on software analysis, evolution and Reengineering, IEEE, vol 36, pp 310–320
Kirkbride J (2014) Testing anti-patterns. https://medium.com/jameskbride/testing-anti-patterns-b5ffc1612b8b
Kitchenham BA, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. Tech. Rep. EBSE 2007-001, Keele University and Durham University Joint Report
Klarck P (2014) Robot framework dos and don’ts. https://slideshare.net/pekkaklarck/robot-framework-dos-and-donts
Knight A (2017a) Bdd 101: Writing good gherkin. https://automationpanda.com/2017/01/30/bdd-101-writing-good-gherkin/
Knight A (2017b) Should gherkin steps use first-person or third-person? https://automationpanda.com/2017/01/18/should-gherkin-steps-use-first-person-or-third-person/
Knight A (2019) Bdd 101: Writing good gherkin. https://techbeacon.com/app-dev-testing/7-ways-tidy-your-test-code
Labuschagne A, Inozemtseva L, Holmes R (2017) Measuring the cost of regression testing in practice: a study of Java projects using continuous integration. In: proceedings of the 11th Joint Meeting on Foundations of Software Engineering, pp 821–830. https://doi.org/10.1145/3106237.3106288
Lanubile F, Mallardo T (2007) Inspecting automated test code: a preliminary study. In: Agile processes in software engineering and extreme programming, vol 4536, LNCS, Springer Berlin Heidelberg, pp 115–122. https://doi.org/10.1007/978-3-540-73101-6_16
Lelli V, Blouin A, Baudry B, Coulon F (2015) On model-based testing advanced GUIs. In: Proceedings of the 8th international conference on software testing, verification and validation workshops, IEEE, pp 1–10. https://doi.org/10.1109/ICSTW.2015.7107403
Leotta M, Stocco A, Ricca F, Tonella P (2014) Reducing web test cases aging by means of robust XPath locators. In: Proceedings of the IEEE international symposium on software reliability engineering workshops, IEEE, pp 449–454
Leotta M, Stocco A, Ricca F, Tonella P (2016) Robula+: an algorithm for generating robust XPath locators for web testing. J softw: Evol Process 28(3):177–204
Mabl (2021) Benchmark report : the state of testing in DeVops
Machiry A, Tahiliani R, Naik M (2013) Dynodroid: an input generation system for Android apps. In: Proceedings of the 9th joint meeting on foundations of software engineering, ACM Press, New York, USA, pp 224. https://doi.org/10.1145/2491411.2491450
Mao K, Harman M, Jia Y (2016) Sapienz: multi-objective automated testing for android applications. In: Proceedings of the 25th international symposium on software testing and analysis, ACM, New York, NY, USA, pp 94–105. https://doi.org/10.1145/2931037.2931054
Marinescu R (2004) Detection strategies: metrics-based rules for detecting design flaws. In: Proceedings of the 20th IEEE international conference on software maintenance, IEEE, pp 350–359. https://doi.org/10.1109/ICSM.2004.1357820
Memon AM, Nguyen BN (2010) Advances in automated model-based system testing of software applications with a GUI Front-End. In: Advances in computers, vol 80, 1st edn, Elsevier Inc., pp 121–162. https://doi.org/10.1016/S0065-2458(10)80003-8
Mesbah A, Van DA (2009) Invariant-based automatic testing of AJAX user interfaces. In: Proceedings of the 31st international conference on software engineering, IEEE, pp 210–220. https://doi.org/10.1109/ICSE.2009.5070522
Meszaros G (2007) xUnit Test Patterns: Refactoring Test Code, 1st edn. Addison-Wesley
Morlion P (2019) Software testing anti patterns. https://www.enov8.com/blog/software-testing-anti-patterns/
Myers B (1994) Challenges of HCI design and implementation. Interactions 1(1):73–83. https://doi.org/10.1145/174800.174808
Myers BA (1995) User Interface Software Tools. ACM Trans Comput Hum Interact 2(1):64–103. https://doi.org/10.1145/200968.200971
Myers BA, Rosson MB (1992) Survey on user interface programming. In: Proceedings of the conference on Human factors in computing systems, ACM Press, New York, USA, pp 195–202. https://doi.org/10.1145/142750.142789
Nguyen BN, Robbins B, Banerjee I, Memon A (2014) GUITAR: an innovative tool for automated testing of GUI-driven software. Autom Softw Eng 21(1):65–105. https://doi.org/10.1007/s10515-013-0128-9
Nguyen V, To T, Diep GH (2021) Generating and selecting resilient and maintainable locators for Web automated testing. Softw Test Verif Reliab 31(3):19. https://doi.org/10.1002/stvr.1760
Pandit P, Tahiliani S (2015) Agile UAT: a framework for user acceptance testing based on user stories and acceptance criteria. Int J Comput Appl 120 (10):16–21. https://doi.org/10.5120/21262-3533
Peruma A, Almalki K, Newman C D, Mkaouer M W, Ouni A, Palomba F (2020) tsDetect: an open-source test smells detection tool. In: Proceedings of the 28th joint meeting on European software engineering conference and symposium on the foundations of software engineering, ACM, New York, NY, USA, pp 1650-1654. https://doi.org/10.1145/3368089.3417921
Pezzė M, Rondena P, Zuddas D (2018) Automatic GUI testing of desktop applications. In: Companion proceedings for the ISSTA/ECOOP 2018 Workshops, ACM, New York, NY, USA, i, pp 54–62. https://doi.org/10.1145/3236454.3236489
Reichhart S, Gi̇rba T, Ducasse S (2007) Rule-based assessment of test quality. J Object Technol 6(9):231. https://doi.org/10.5381/jot.2007.6.9.a12
Renaudin J (2016) Software testing anti patterns. https://www.slideshare.net/JosiahRenaudin/antipatterns-for-automated-testing
Ricca F, Stocco A (2021) Web test automation: insights from the grey literature. In: Proceedings of the 47th international conference on current trends in theory and practice of computer science, Bolzano,Italy, pp 472-485. https://doi.org/10.1007/978-3-030-67731-2_35
Ronsse M, De Bosschere K (1999) RecPlay: A fully integrated practical record/replay system. ACM Trans Comput Syst 17(2):133–152. https://doi.org/10.1145/312203.312214,
Rwemalika R, Kintis M, Papadakis M, Le Traon Y, Lorrach P (2019a) On the evolution of keyword-driven test suites. In: Proceedings of the 12th international conference on software testing, verification and validation, New York, NY, USA, pp 335-345
Rwemalika R, Kintis M, Papadakis M, Le Traon Y, Lorrach P (2019b) Ukwikora: continuous inspection for keyword-driven testing. In: Proceedings of the 28th international symposium on software testing and analysis, association for computing machinery, New York, NY, USA, pp 402–405. https://doi.org/10.1145/3293882.3339003
Saddler JA, Cohen MB (2017) EventFlowSlicer: A tool for generating realistic goal-driven GUI tests. In: Proceedings of the 32nd international conference on automated software engineering, ASE, IEEE, pp 955–960. https://doi.org/10.1109/ASE.2017.8115711
Salihu IA, Ibrahim R, Ahmed BS, Zamli KZ, Usman A (2019) AMOGA: A static-dynamic model generation strategy for mobile Apps testing. IEEE Access 7(c):17158–17173. https://doi.org/10.1109/ACCESS.2019.2895504
Satopaa V, Albrecht J, Irwin D, Raghavan B (2011) Finding a kneedle in a haystack: detecting knee points in system behavior. In: Proceedings of the 31st IEEE international conference on distributed computing systems workshops, IEEE, pp 166–171. https://doi.org/10.1109/ICDCSW.2011.20
Sciamanna A (2019) What are the anti patterns of automation with selenium? https://anthonysciamanna.com/2019/10/20/avoiding-automated-testing-pitfalls.html
Scott A (2015) Five automated acceptance test anti-patterns. https://alisterbscott.com/2015/01/20/five-automated-acceptance-test-anti-patterns/
Shay L (2019) Bdd cucumber features best practices. https://www.linkedin.com/pulse/bdd-cucumber-features-best-practices-liraz-shay/
Sheth H (2020) 16 selenium best practices for efficient test automation. https://www.lambdatest.com/blog/selenium-best-practices-for-web-testing/
Silva D, Valente MT (2017) RefDiff: detecting Refactorings in version histories. In: Proceedings of the 14th international conference on mining software repositories, IEEE, pp 269–279. arXiv:1704.01544. https://doi.org/10.1109/MSR.2017.14
Siminiuc A (2019) What are the anti patterns of automation with selenium? https://www.quora.com/What-are-the-anti-patterns-of-automation-with-selenium
Spadini D, Palomba F, Zaidman A, Bruntink M, Bacchelli A (2018) On the relation of test smells to software code quality. In: Proceedings of the international conference on software maintenance and evolution, IEEE, pp 1–12. https://doi.org/10.1109/ICSME.2018.00010
Spadini D, Schvarcbacher M, Oprescu AM, Bruntink M, Bacchelli A (2020) Investigating severity thresholds for test smells. In: Proceedings of the 17th international conference on mining software repositories, ACM, New York, NY, USA, pp 311–321. https://doi.org/10.1145/3379597.3387453
StackExchange (2017) What are anti-patterns in test automation? https://sqa.stackexchange.com/questions/8508/what-are-anti-patterns-in-test-automation
Temov J (2020) Want to speed end-to-end testing? don’t send in the clones. https://techbeacon.com/app-dev-testing/want-speed-end-end-testing-dont-send-clones
Tsantalis N, Guana V, Stroulia E, Hindle A (2013) A multidimensional empirical study on Refactoring cctivity. In: Proceedings of the conference of the center for advanced studies on collaborative research, IBM Corp, Ontario, Canada, pp 132–146
Tufano M, Palomba F, Bavota G, Di Penta M, Oliveto R, De Lucia A, Poshyvanyk D (2016) An empirical investigation into the nature of test smells. In: Proceedings of the 31st international conference on automated software engineering, ACM Press, Singapore, pp 4–15. https://doi.org/10.1145/2970276.2970340
Utting M, Pretschner A, Legeard B (2012) A taxonomy of model-based testing approaches. Software Testing Verification and Reliability 22(5):297–312. https://doi.org/10.1002/stvr.456
Van RB, Du Bois B, Demeyer S (2006) Characterizing the relative significance of a test smell. In: Proceedings of the 22nd IEEE international conference on software maintenance, IEEE, pp 391–400. https://doi.org/10.1109/ICSM.2006.18
Van RB, Du Bois B, Demeyer S, Rieger M (2007) On the detection of test smells: a metrics-based approach for general fixture and eager test. IEEE Trans Soft Eng 33(12):800–817. https://doi.org/10.1109/TSE.2007.70745
Virgi̇nio T, Martins LA, Soares LR, Santana R, Costa H, Machado I (2020) An empirical study of automatically-generated tests from the perspective of test smells. In: Proceedings of the 34th Brazilian symposium on software engineering, ACM, New York, NY, USA, pp 92–96.https://doi.org/10.1145/3422392.3422412
Yu S, Fang C, Feng Y, Zhao W, Chen Z (2019) LIRAT: layout and image recognition driving automated mobile testing of cross-platform. In: Proceedings of the 34th international conference on automated software engineering, IEEE, pp 1066–1069 https://doi.org/10.1109/ASE.2019.00103
Yuan X, Cohen M, Memon AM (2007) Covering array sampling of input event sequences for automated gui testing. In: Proceedings of the 22nd international conference on automated software engineering, ACM Press, New York, USA, pp 405
Zhang S, Jalali D, Wuttke J, Muşlu K, Lam W, Ernst MD, Notkin D (2014) Empirically revisiting the test independence assumption. In: Proceedings of the international symposium on software testing and analysis, ACM Press, New York,USA, pp 385–396. https://doi.org/10.1145/2610384.2610404
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Jinqiu Yang
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rwemalika, R., Habchi, S., Papadakis, M. et al. Smells in system user interactive tests. Empir Software Eng 28, 20 (2023). https://doi.org/10.1007/s10664-022-10251-1
Accepted:
Published:
DOI: https://doi.org/10.1007/s10664-022-10251-1