Skip to main content
SpringerLink
Log in

A reinforcement learning-based approach to testing GUI of moblie applications

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

With the popularity of mobile devices, the software market of mobile applications has been booming in recent years. Android applications occupy a vast market share. However, the applications inevitably contain defects. Defects may affect the user experience and even cause severe economic losses. This paper proposes ATAC and ATPPO, which apply reinforcement learning to Android GUI testing to mitigate the state explosion problem. The article designs a new reward function and a new state representation. It also constructs two GUI testing models (ATAC and ATPPO) based on A2C and PPO algorithms to save memory space and accelerate training speed. Empirical studies on twenty open-source applications from GitHub demonstrate that: (1) ATAC performs best in 16 of 20 apps in code coverage and defects more exceptions; (2) ATPPO can get higher code coverage in 15 of 20 apps and defects more exceptions; (3) Compared with state-of-art tools Monkey and ARES, ATAC, and ATPPO shows higher code coverage and detects more errors. ATAC and ATPPO can not only cover more code coverage but also can effectively detect more exceptions. This paper also introduces Finite-State Machine into the reinforcement learning framework to avoid falling into the local optimal state, which provides high-level guidance for further improving the test efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

Notes

  1. https://github.com/RL-ATAC/AUT-of-ATAC

References

  1. Google: UI/Application Exerciser Monkey. https://developer.android.com/studio/test/monkey

  2. Amalfitano, D., Fasolino, A.R., Tramontana, P., Carmine, S.D., Memon, A.M.: Using GUI ripping for automated testing of android applications. In: Goedicke M., Menzies T., Saeki M. (eds.) Proceedings of the 12th IEEE/ACM International Conference on Automated Software Engineering, pp. 258–261. ACM, Essen, Germany (2012)

  3. Collins, E., Dias-Neto, A.C., Vincenzi, A., Maldonado, J.C.: Deep reinforcement learning based android application GUI testing. In: Proceedings of the SBES ’21: 35th Brazilian Symposium on Software Engineering, pp. 186–194. ACM, Joinville, Santa Catarina, Brazil (2021)

  4. Romdhana, A., Merlo, A.: Keynote: ARES: A deep reinforcement learning tool for black-box testing of android apps. In: Proceedings of the 19th IEEE International Conference on Pervasive Computing and Communications Workshops and Other Affiliated Events, p. 173. IEEE, Kassel, Germany (2021)

  5. Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992)

    Article  Google Scholar 

  6. Sutton, R.S., Barto, A.G.: Reinforcement Learning - an Introduction. Adaptive computation and machine learning. MIT Press (1998)

  7. Degris, T., Pilarski, P.M., Sutton, R.S.: Model-free reinforcement learning with continuous action in practice. In: Proceedings of the 2022 American Control Conference, pp. 2177–2182. IEEE, Montreal, QC, Canada (2012)

  8. Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T.P., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: Proceedings of the 33nd International Conference on Machine Learning. JMLR.org, New York City, NY, USA (2016)

  9. Köroglu, Y., Sen, A., Muslu, O., Mete, Y., Ulker, C., Tanriverdi, T., Donmez, Y.: QBE: QLearning-based exploration of android applications. In: Proceedings of the 11th IEEE International Conference on Software Testing, Verification and Validation, pp. 105–115. IEEE Computer Society, Sweden (2018)

  10. Pan, M., Huang, A., Wang, G., Zhang, T., Li, X.: Reinforcement learning based curiosity-driven testing of android applications. In: Proceedings of the ISSTA ’20: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 153–164. ACM, USA (2020)

  11. Adamo, D., Khan, M.K., Koppula, S., Bryce, R.C.: Reinforcement learning for android GUI testing. In: Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, pp. 2–8. ACM, Lake Buena Vista, FL, USA (2018)

  12. Vuong, T.A.T., Takada, S.: A reinforcement learning based approach to automated testing of android applications. In: Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation, pp. 31–37. ACM, Lake Buena Vista, FL, USA (2018)

  13. Vuong, T.A.T., Takada, S.: Semantic analysis for deep Q-network in android GUI testing. In: Proceedings of the 31st International Conference on Software Engineering and Knowledge Engineering, pp. 123–170. KSI Research Inc. and Knowledge Systems Institute Graduate School, Hotel Tivoli, Lisbon, Portugal (2019)

  14. EclEmma: JaCoCo Java Code Coverage Library. https://www.eclemma.org/jacoco/index.html

  15. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai Gym. CoRR arXiv:1606.01540 (2016)

  16. [n.d.]: Appium: Mobile App Automation Made Awesome. http://appium.io/

  17. Developers, A.: Android Debug Bridge (ADB). https://developer.android.com/studio/command-line/adb

  18. Machiry, A., Tahiliani, R., Naik, M.: Dynodroid: an input generation system for android apps. In: Proceedings of the Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pp. 224–234. ACM, Saint Petersburg, Russian Federation (2013)

  19. Group, N.: Intent Fuzzer. https://www.nccgroup.trust/us/our-research/intent-fuzzer/

  20. Sasnauskas, R., Regehr, J.: Intent fuzzer: crafting intents of death. In: Proceedings of the 2014 Joint International Workshop on Dynamic Analysis (WODA) and Software and System Performance Testing, pp. 1–5. ACM, San Jose, CA, USA (2014)

  21. Ye, H., Cheng, S., Zhang, L., Jiang, F.: Droidfuzzer: Fuzzing the android apps with intent-filter tag. In: Proceedings of the 11th International Conference on Advances in Mobile Computing & Multimedia, p. 68. ACM, Vienna, Austria (2013)

  22. Espada, A.R., Gallardo, M., Salmer’on, A., Merino, P.: Using model checking to generate test cases for android applications. In: Proceedings Tenth Workshop on Model Based Testing. EPTCS, vol. 180, pp. 7–21. London, UK (2015)

  23. Hao, S., Liu, B., Nath, S., Halfond, W.G.J., Govindan, R.: PUMA: programmable UI-automation for large-scale dynamic analysis of mobile apps. In: Proceedings of 12th Annual International Conference on Mobile Systems, Applications, and Services, pp. 204–217. ACM, Bretton Woods, NH, US (2014)

  24. Su, T., Meng, G., Chen, Y., Wu, K., Yang, W., Yao, Y., Pu, G., Liu, Y., Su, Z.: Guided, stochastic model-based GUI testing of android apps. In: Proceedings of the 11th Joint Meeting on Foundations of Software Engineering, pp. 245–256. ACM, Paderborn, Germany (2017)

  25. Mahmood, R., Mirzaei, N., Malek, S.: Evodroid: segmented evolutionary testing of android apps. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 599–609. ACM, Hong Kong, China (2014)

  26. Anand, S., Naik, M., Harrold, M.J., Yang, H.: Automated concolic testing of smartphone apps. In: Proceedings of the 20th ACM SIGSOFT Symposium on the Foundations of Software Engineering, p. 59. ACM, Cary, NC, USA (2012)

  27. Mao, K., Harman, M., Jia, Y.: Sapienz: multi-objective automated testing for android applications. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, pp. 94–105. ACM, Saarbrücken, Germany (2016)

  28. Choi, W., Necula, G.C., Sen, K.: Guided GUI testing of android apps with minimal restart and approximate learning. In: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, pp. 623–640. ACM, Indianapolis, IN, USA (2013)

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Chuanqi Tao, Fengyu Wang, Yuemeng Gao & Hongjing Guo

  2. Ministry Key Laboratory for Safety-Critical Software Development and Verification, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Chuanqi Tao

  3. Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing, China

    Chuanqi Tao

  4. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

    Chuanqi Tao

  5. Computer Engineering Department, San Jose State University, San Jose, USA

    Jerry Gao

Authors
  1. Chuanqi Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fengyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuemeng Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongjing Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jerry Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Chuanqi Tao, Fengyu Wang and Yuemeng Gao wrote the main manuscript text, Hongjing Guo and Jerry Gao prepared Figures 1-5 and Tables 1-10. All authors reviewed the manuscript.

Corresponding author

Correspondence to Chuanqi Tao.

Ethics declarations

Consent to participate

Informed consent was obtained from all individual participants.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tao, C., Wang, F., Gao, Y. et al. A reinforcement learning-based approach to testing GUI of moblie applications. World Wide Web 27, 16 (2024). https://doi.org/10.1007/s11280-024-01252-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11280-024-01252-9

Keywords

Search

Navigation