Advertisement

Model-based testing of apps in real network scenarios

  • Laura PanizoEmail author
  • Almudena Díaz
  • Bruno García
SPIN 2018
  • 13 Downloads

Abstract

Traditional testing methods for mobile apps focus on detecting execution errors. However, the evolution of mobile networks toward 5G will require additional support for app developers to also ensure good performance and user experience. Manual testing in a number of scenarios is not enough to satisfy the expectations of the apps’ end users. This paper presents the testing framework developed in the TRIANGLE project (https://www.triangle-project.eu/), which integrates a complete mobile network testbed to test, benchmark and certify mobile apps. In this paper, we focus on a recent extension of the TRIANGLE framework that uses model-based testing based on model checking to support the automatic generation of user interactions. We introduce the complete testing framework and the basis of the model-based extension. Finally, we use the testing framework to evaluate the performance of the ExoPlayer app in different network scenarios. ExoPlayer is a video streaming app for Android that implements different adaptive streaming protocols.

Keywords

Model-based testing Mobile network testbed Model checking 

Notes

References

  1. 1.
    3GPP: TR37.901:User Equipment (UE) application layer data throughput (Rel. 15). Technical report (2018)Google Scholar
  2. 2.
    5GENESIS project consortium: Deliverable D2.3: Initial planning of tests and experimentation. Pu (2018)Google Scholar
  3. 3.
    Amalfitano, D., Amatucci, N., Memon, A.M., Tramontana, P., Fasolino, A.R.: A general framework for comparing automatic testing techniques of Android mobile apps. J. Syst. Softw. 125, 322–343 (2017).  https://doi.org/10.1016/j.jss.2016.12.017 CrossRefGoogle Scholar
  4. 4.
    Amalfitano, D., Fasolino, A.R., Tramontana, P., Ta, B.D., Memon, A.M.: MobiGUITAR: automated model-based testing of mobile apps. IEEE Softw. 32(5), 53–59 (2015).  https://doi.org/10.1109/MS.2014.55 CrossRefGoogle Scholar
  5. 5.
    Baek, Y.M., Bae, D.H.: Automated model-based android GUI testing using multi-level GUI comparison criteria. In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering, ASE 2016, pp. 238–249. ACM (2016).  https://doi.org/10.1145/2970276.2970313
  6. 6.
    Broy, M., Jonsson, B., Katoen, J.P., Leucker, M., Pretschner, A.: Model-Based Testing of Reactive Systems: Advanced Lectures. Springer, Berlin (2005)CrossRefzbMATHGoogle Scholar
  7. 7.
    Cattoni, A.F., Corrales-Madueño, G., Dieudonne, M., Merino, P., Díaz-Zayas, A., Salmerón, A., Carlier, F., Saint-Germain, B., Morris, D., Figueiredo, R., Caffrey, J., Baos, J., Cárdenas, C., Roche, N., Moore, A.: An end-to-end testing ecosystem for 5G. In: European conference on networks and communications (EuCNC 2016), pp. 307–312 (2016).  https://doi.org/10.1109/EuCNC.2016.7561053
  8. 8.
    Cellular Telecommunications Industry Association (CTIA): Battery Life Test Plan v.1.2. Technical report (2018). https://api.ctia.org/wp-content/uploads/2018/04/CTIA-Battery-Life-Test-Plan-Version-1.2.pdf
  9. 9.
    Díaz-Zayas, A., Panizo, L., Baños, J., Cárdenas, C., Dieudonne, M.: QoE Evaluation: The TRIANGLE testbed approach. Wirel. Commun. Mob. Comput. p. 12 (2018).  https://doi.org/10.1155/2018/6202854
  10. 10.
    Espada, A.R., Gallardo, M.M., Salmerón, A., Merino, P.: Performance analysis of spotify® for android with model based testing. Mob. Inf. Syst. 2017, 14 (2017).  https://doi.org/10.1155/2017/2012696 Google Scholar
  11. 11.
    Global System for Mobile Communications Association (GSMA): Smartphone performance test case guideline v3.0. Technical report (2017). https://www.gsma.com/newsroom/wp-content/uploads//TS.29_v3.0.pdf
  12. 12.
    Holzmann, G.: The SPIN Model Checker: Primer and Reference Manual. Addison-Wesley Professional, Boston (2003)Google Scholar
  13. 13.
    ISO/IEC: Information technology Dynamic adaptive streaming over HTTP (DASH).http://standards.iso.org/ittf/PubliclyAvailableStandards/c065274_ISO_IEC_23009-1_2014.zip (2014)
  14. 14.
    Koumaras, H., Tsolkas, D., Gardikis, G., Merino-Gómez, P., Frascolla, V., Triantafyllopoulou, D., Emmelmann, M., Koumaras, V., Garcia-Osma, M.L., Munaretto, D., Atxutegi, E., de Puga, J.S., Alay, O., Brunstrom, A., Bosneag, A.M.C.: 5GENESIS: The genesis of a flexible 5G Facility. In: IEEE international workshop on computer-aided modeling analysis and design of communication links and networks (CAMAD-2018 ) (2018)Google Scholar
  15. 15.
    Kozamernik, F., Steinmann, V., Sunna, P., Wyckens, E.: SAMVIQA new EBU methodology for video quality evaluations in multimedia. SMPTE Motion Imaging J. 114(4), 152–160 (2005).  https://doi.org/10.5594/J11535 CrossRefGoogle Scholar
  16. 16.
    Liu, Y., Xu, C., Cheung, S.: Characterizing and detecting performance bugs for smartphone applications. In: Proceedings of the 36th international conference on software engineering, ICSE 2014, pp. 1013–1024. ACM (2014).  https://doi.org/10.1145/2568225.2568229
  17. 17.
    Massol, V., Husted, T.: JUnit in Action. Manning Publications Co., Greenwich (2003)Google Scholar
  18. 18.
    Mehmood, M.A., Wundsam, A., Uhlig, S., Levin, D., Sarrar, N., Feldmann, A.: QoE-Lab: towards evaluating quality of experience for future internet conditions. In: Korakis, T., Li, H., Tran-Gia, P., Park, H.S. (eds.) Testbeds and Research Infrastructure. Development of Networks and Communities, pp. 286–301. Springer, Berlin (2012)CrossRefGoogle Scholar
  19. 19.
  20. 20.
    NeoLoad by Neotys: The load testing platform accelerating DevOps. http://www.neotys.com/. Last accessed: 10/26/2018
  21. 21.
    NGMN Alliance: Definition of the testing framework for the NGMN 5G pre-commecial networks trails v1.0. Technical report (2018).https://www.ngmn.org/fileadmin/ngmn/content/downloads/Technical/2018/180220__PreCommTrials_Framework_definition_v1_0.pdf
  22. 22.
    Panizo, L., Díaz-Zayas, A., García, B.: An extension of TRIANGLE testbed with model-based testing. In: M.d.M. Gallardo, P. Merino (eds.) Proceedings of the 25th international symposium on model checking software (SPIN2018), pp. 190–195. Springer International Publishing (2018). https://doi.org/10.1007/978-3-319-94111-0_11
  23. 23.
    Panizo, L., Salmerón, A., Gallardo, M.M., Merino, P.: Guided Test Case Generation for Mobile Apps in the TRIANGLE Project: Work in Progress. In: Proc. of the 24th International SPIN Symposium on Model Checking of Software, pp. 192–195. ACM (2017).  https://doi.org/10.1145/3092282.3092298
  24. 24.
    Pantos, R., May, W.: HTTP Live Streaming. RFC 8216 (2017). https://doi.org/10.17487/RFC8216
  25. 25.
    Perfecto: The cloud-based platform for continuous testing in a DevOps environment. https://www.perfecto.io/. Last accessed: 10/26/2018
  26. 26.
    Solera, M., Toril, M., Palomo, I., Gomez, G., Poncela, J.: A testbed for evaluating video streaming services in LTE. Wirel. Pers. Commun. 98(3), 2753–2773 (2018).  https://doi.org/10.1007/s11277-017-4999-0 CrossRefGoogle Scholar
  27. 27.
    TRIANGLE project consortium: deliverable D2.1: initial report on the testing scenarios, requirements and use cases. Public (2016)Google Scholar
  28. 28.
    TRIANGLE project consortium: Deliverable D2.6: Final test scenario and test specifications. Public (2018)Google Scholar
  29. 29.
    Yang, S., Yan, D., Rountev, A.: Testing for poor responsiveness in android applications. In: Proceedings of the 1st international workshop on the engineering of mobile-enabled systems (MOBS), pp. 1–6 (2013).  https://doi.org/10.1109/MOBS.2013.6614215
  30. 30.
    Yang, W., Prasad, M.R., Xie, T.: A grey-box approach for automated GUI-model generation of mobile applications. In: V. Cortellessa, D. Varró (eds.) Proceedings of the 16th international conference on fundamental approaches to software engineering (FASE 2013), pp. 250–265. Springer,Berlin (2013).  https://doi.org/10.1007/978-3-642-37057-1_19
  31. 31.
    Zein, S., Salleh, N., Grundy, J.: A systematic mapping study of mobile application testing techniques. J. Syst. Softw. 117, 334–356 (2016).  https://doi.org/10.1016/j.jss.2016.03.065 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Departamento de Lenguajes y Ciencias de la ComputaciónUniversidad de Málaga, Andalucía TechMálagaSpain
  2. 2.Universidad de Málaga, Andalucía TechMálagaSpain

Personalised recommendations