Testing-as-a-Service for Mobile Applications: State-of-the-Art Survey

  • Oleksii Starov
  • Sergiy Vilkomir
  • Anatoliy Gorbenko
  • Vyacheslav Kharchenko
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 307)

Abstract

The paper provides an introduction to the main challenges in mobile applications testing. In the paper we investigate the state-of-the-art mobile testing technologies and overview related research works in the area. We discuss general questions of cloud testing and examine a set of existing cloud services and testing-as-a-service resources facilitating testing of mobile applications and covering a large range of the specific mobile testing features.

Keywords

Mobile application software testing cloud services 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    White, J., Clarke, S., Doughtery, B., Thompson, C., Shmidt, D.: R&D Challenges and Solutions for Mobile Cyber-Physical Applications and Supporting Internet Services. Springer Journal of Internet Services and Applications 1(1), 45–56 (2010)CrossRefGoogle Scholar
  2. 2.
    Work, D.B., Bayen, A.M.: Impacts of the Mobile Internet on Transportation Cyberphysical Systems: Traffic Monitoring using Smartphones. In: National Workshop for Research on High-Confidence Transportation Cyber-Physical Systems: Automotive, Aviation and Rail, Washington, DC, USA (2008)Google Scholar
  3. 3.
    Leijdekkers, P., Gay, V.: Personal Heart Monitoring and Rehabilitation System using Smart Phones. In: Intern. Conf. on Mobile Business, Copenhagen, Denmark (2006)Google Scholar
  4. 4.
    Moser, K.: Improving Work Processes for Nuclear Plants. In: American Nuclear Society Utility Working Conf., Hollywood, Florida, USA (2012)Google Scholar
  5. 5.
    Wasserman, A.: Software engineering issues for mobile application development. In: Workshop on Future of Software Engineering Research at the 18th Int. Symposium on Foundations of Software Engineering (ACM SIGSOFT), Santa Fe, USA, pp. 397–400 (2010)Google Scholar
  6. 6.
  7. 7.
    Holler, R.: Mobile Application Development: A Natural Fit with Agile Methodologies, http://www.versionone.com/pdf/mobiledevelopment.pdf
  8. 8.
    Vilkomir, S.: Cloud Testing: A State-of-the-Art Review. Information & Security: An International Journal 28(2(17), 213–222 (2012)Google Scholar
  9. 9.
    Tilley, S., Parveen, T.: Software Testing in the Cloud: Perspectives on an Emerging Discipline. IGI Global (2012)Google Scholar
  10. 10.
    Tsai, W., Chen, X., Liu, L., Zhao, Y., Tang, L., Zhao, W.: Testing as a service over cloud. In: 5th IEEE Int. Symposium on Service Oriented System Engineering, pp. 181–188 (2010)Google Scholar
  11. 11.
    Kalliosaari, L., Taipale, O., Smolander, K.: Testing in the Cloud: Exploring the Practice. IEEE Software 29(2), 46–51 (2012)CrossRefGoogle Scholar
  12. 12.
    Weidong, F., Yong, X.: Cloud testing: The next generation test technology. In: 10th Int. Conf. Electronic Measurement & Instruments, Chengdu, China, pp. 291–295 (2011)Google Scholar
  13. 13.
    Inçki, K., Ari, I., Soze, H.: A Survey of Software Testing in the Cloud. In: IEEE 6th Int. Conf. on Software Security and Reliability Companion, pp. 18–23 (2012)Google Scholar
  14. 14.
    Priyanka, C.I., Rana, A.: Empirical evaluation of cloud-based testing techniques: a systematic review. ACM SIGSOFT Software Engineering Notes Archive 37(3), 1–9 (2012)CrossRefGoogle Scholar
  15. 15.
    Mote, D.: Cloud based Testing Mobile Apps. In: 2nd IndicThreads.com Conference on Software Quality, Pune, India (2011)Google Scholar
  16. 16.
    Cloud Testing: Database of Cyber Security and Information Systems Information Analysis Center, https://sw.thecsiac.com/databases/url/key/7848/8764/8765#.USGPb-h8vDm
  17. 17.
    Tilley, S., Parveen, T.: Software Testing in the Cloud: Migration & Execution. Springer Briefs in Computer Science (2012)Google Scholar
  18. 18.
    Riungu, L.M., Taipale, O., Smolander, K.: Research Issues for Software Testing in the Cloud. In: IEEE 2nd Int. Conf. Cloud Computing Technology and Science, pp. 557–564 (2010)Google Scholar
  19. 19.
    Rhoton, J., Haukioja, R.: Cloud Computing Architected: Solution Design Handbook. Recursive (2011)Google Scholar
  20. 20.
    Coulouris, G., Dollimore, J., Kindberg, T., Blair, G.: Distributed Systems: Concepts and Design. Addison-Wesley (2011)Google Scholar
  21. 21.
    Muccini, H., Francesco, A., Esposito, P.: Software testing of mobile applications: Challenges and future research directions. In: 7th Int. Workshop on Automation of Software Test (2012)Google Scholar
  22. 22.
    Franke, D., Weise, C.: Providing a Software Quality Framework for Testing of Mobile Applications. In: IEEE 4th Int. Conf. on Software Testing, Verification and Validation, Berlin, Germany, pp. 431–434 (2011)Google Scholar
  23. 23.
    Milano, D.: Android Application Testing Guide. Publishing Ltd. (2011)Google Scholar
  24. 24.
    Frederick, G., Lal, R.: Testing a Mobile Web Site. Beginning Smartphone App Development – Part IV. Apress (2009)Google Scholar
  25. 25.
    Dantas, V., Marinho, F., Da Costa, A., Andrade, R.: Testing requirements for mobile applications. In: 24th Int. Symposium on Computer and Information Sciences (2009)Google Scholar
  26. 26.
  27. 27.
    Make Your Mobile Testing Solution Enterprise-Ready, http://www.perfectomobile.com/portal/cms/resources/enterprise-ready_white-paper
  28. 28.
    Konstantinidis, A., Costa, C., Larkou, G., Zeinalipour-Yazti, D.: Demo: a programming cloud of smartphones. In: 10th Int. Conf. on Mobile Systems, Applications, and Services, pp. 465–466 (2012)Google Scholar
  29. 29.
    Turner, H., White, J., Reed, J., Galindo, J., Porter, A., Marathe, M., Vullikanti, A., Gokhale, A.: Building a Cloud-Based Mobile Application Testbed. IGI Global (2012)Google Scholar
  30. 30.
    She, S., Sivapalan, S., Warren, I.: Hermes: A Tool for Testing Mobile Device Applications. In: Software Engineering Conf., Queensland, Australia (2009l)Google Scholar
  31. 31.
    Mahmood, R., Esfahani, N., Kacem, T., Mirzaei, N., Malek, S., Stavrou, A.: A whitebox approach for automated security testing of Android applications on the cloud. In: 7th Int. Workshop on Automation of Software Test, pp. 22–28 (2012)Google Scholar
  32. 32.
    Nagowah, L., Sowamber, G.: A Novel Approach of Automation Testing on Mobile Devices. Int. Conf. on Computer & Information Science 2, 924–930 (2012)Google Scholar
  33. 33.
    Sanz, B., Santos, I., Laorden, C., Ugarte-Pedrero, X., Bringas, P.: On the Automatic Categorisation of Android Applications. In: 9th Annual IEEE Consumer Communications and Networking Conf. – Security and Content Protections (2012)Google Scholar
  34. 34.
    Allevato, A., Edwards, S.: RoboLIFT: simple GUI-based unit testing of student-written android applications. In: 43rd ACM Technical Symposium on Computer Science Education, p. 670 (2012)Google Scholar
  35. 35.
    Mirzaei, N., Malek, S., Păsăreanu, C., Esfahani, N., Mahmood, R.: Testing Android Apps Through Symbolic Execution. ACM SIGSOFT Software Engineering Notes Archive 37(6), 1–5 (2012)CrossRefGoogle Scholar
  36. 36.
    Zivkov, D.: Touch screen mobile application as part of testing and verification system. In: 35th Int. Convention MIPRO, pp. 892–895 (2012)Google Scholar
  37. 37.
    Amalfitano, D., Fasolino, A., Tramontana, P., De Carmine, S.: Using GUI ripping for automated testing of Android application. In: 27th IEEE/ACM Int. Conf. on Automated Software Engineering, Germany (2012)Google Scholar
  38. 38.
    Amalfitano, D., Fasolino, A.R., Tramontana, P.: A GUI Crawling-Based Technique for Android Mobile Application Testing. In: Software Testing, Verification and Validation Workshops, pp. 252–261 (2011)Google Scholar
  39. 39.
    Ridene, Y., Barbier, F.: A model-driven approach for automating mobile applications testing. In: 5th European Conf. on Software Architecture: Companion (2011)Google Scholar
  40. 40.
    Kropp, M., Morales, P.: Automated GUI Testing on the Android Platform. IMVS Fokus Report 4(1) (2010)Google Scholar
  41. 41.
    Hu, C., Neamtiu, I.: Automating GUI testing for Android applications. In: 6th Int. Workshop on Automation of Software Test, pp. 77–83 (2011)Google Scholar
  42. 42.
    Gilbert, P., Chun, B., Cox, L., Jung, J.: Automating Privacy Testing of Smartphone Applications. Technical Report CS-2011-02 (2011)Google Scholar
  43. 43.
    Edmondson, J., Gokhale, A., Neema, S.: Automating Testing of Service-oriented Mobile Applications with Distributed Knowledge and Reasoning. In: Service-Oriented Computing and Applications, pp. 1–4 (2011)Google Scholar
  44. 44.
    Takala, T., Katara, M., Harty, J.: Experiences of System-Level Model-Based GUI Testing of an Android Application. In: Software Testing, Verification and Validation, pp. 377–386 (2011)Google Scholar
  45. 45.
    Selvam, R., Karthikeyani, V.: Mobile Software Testing – Automated Test Case Design Strategies. Int. J. on Computer Science and Engineering (2011)Google Scholar
  46. 46.
    Payet, É., Spoto, F.: Static Analysis of Android Programs. In: Bjørner, N., Sofronie-Stokkermans, V. (eds.) CADE 2011. LNCS, vol. 6803, pp. 439–445. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  47. 47.
    Sadeh, B., Ørbekk, K., Eide, M.M., Gjerde, N.C.A., Tønnesland, T.A., Gopalakrishnan, S.: Towards Unit Testing of User Interface Code for Android Mobile Applications. In: Zain, J.M., Wan Mohd, W.M.b., El-Qawasmeh, E. (eds.) ICSECS 2011, Part III. CCIS, vol. 181, pp. 163–175. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  48. 48.
    Liu, Z., Gao, X., Long, X.: Adaptive random testing of mobile application. In: Computer Engineering and Technology, pp. 297–301 (2010)Google Scholar
  49. 49.
    Abdallah, N., Ramakrishnan, S.: Automated Stress Testing of Windows Mobile GUI Applications. In: 20th Int. Symposium on Software Reliability Engineering (2009)Google Scholar
  50. 50.
    Sivapalan, S., Warren, I.: Hermes: A Tool for Testing Mobile Device Applications. In: Software Engineering Conference, Australia (2009)Google Scholar
  51. 51.
    Liu, Z.-F., Liu, B., Gao, X.-P.: SOA based mobile application software test framework. In: 8th Int. Conf. Reliability, Maintainability and Safety, pp. 765–769 (2009)Google Scholar
  52. 52.
    Combinatorial Methods in Software Testing, http://csrc.nist.gov/groups/SNS/acts/
  53. 53.
    Grindal, M., Offutt, J., Andler, S.F.: Combination Testing Strategies: a Survey. Software Testing, Verification and Reliability 15(3), 167–199 (2005)CrossRefGoogle Scholar
  54. 54.
    Kuhn, D.R., Lei, Y., Kacker, R.: Practical Combinatorial Testing - Beyond Pairwise. IEEE IT Professional 6, 19–23 (2008)CrossRefGoogle Scholar
  55. 55.
    Maximoff, J.R., Trela, M.D., Kuhn, D.R., Kacker, R.: A Method for Analyzing System State-space Coverage within a t-Wise Testing Framework. In: IEEE Int. Systems Conf., San Diego (2010)Google Scholar
  56. 56.
    Kuhn, D.R., Kacker, R.N., Lei, Y.: Practical Combinatorial Testing. NIST Special Publication 10, 13–15 (2010)Google Scholar
  57. 57.
    Lei, Y., Kacker, R., Kuhn, D.R., Okun, V., Lawrence, J.: IPOG: A General Strategy for T-Way Software Testing. In: IEEE Engineering of Computer Based Systems Conf., pp. 549–556 (2007)Google Scholar
  58. 58.
    Chilenski, J.J., Miller, S.: Applicability of Modified Condition/Decision Coverage to Software Testing. Software Engineering J. 9, 193–200 (1994)CrossRefGoogle Scholar
  59. 59.
    Vilkomir, S., Bowen, J.P.: From MC/DC to RC/DC: Formalization and Analysis of Control-flow Testing Criteria. Formal Aspects of Computing 18(1), 42–62 (2006)CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Oleksii Starov
    • 1
  • Sergiy Vilkomir
    • 2
  • Anatoliy Gorbenko
    • 3
  • Vyacheslav Kharchenko
    • 3
  1. 1.Computer Science DepartmentState University of New York at Stony BrookStony BrookUSA
  2. 2.Department of Computer ScienceEast Carolina UniversityHaywardUSA
  3. 3.Department of Computer Systems and Networks (503)National Aerospace UniversityKharkivUkraine

Personalised recommendations