JSART: JavaScript Assertion-Based Regression Testing

  • Shabnam Mirshokraie
  • Ali Mesbah
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7387)


Web 2.0 applications rely heavily on JavaScript and client-side runtime manipulation of the DOM tree. One way to provide assurance about the correctness of such highly evolving and dynamic applications is through regression testing. However, JavaScript is loosely typed, dynamic, and notoriously challenging to analyze and test. We propose an automated technique for JavaScript regression testing, which is based on on-the-fly JavaScript source code instrumentation and dynamic analysis to infer invariant assertions. These obtained assertions are injected back into the JavaScript code to uncover regression faults in subsequent revisions of the web application under test. Our approach is implemented in a tool called Jsart. We present our case study conducted on nine open source web applications to evaluate the proposed approach. The results show that our approach is able to effectively generate stable assertions and detect JavaScript regression faults with a high degree of accuracy and minimal performance overhead.


JavaScript web regression testing assertions dynamic analysis 


  1. 1.
    Alshahwan, N., Harman, M.: Automated session data repair for web application regression testing. In: Proceedings of the Int. Conf. on Software Testing, Verification, and Validation (ICST 2008), pp. 298–307. IEEE Computer Society (2008)Google Scholar
  2. 2.
    Artzi, S., Dolby, J., Jensen, S., Møller, A., Tip, F.: A framework for automated testing of JavaScript web applications. In: Proceedings of the Intl. Conference on Software Engineering (ICSE), pp. 571–580. ACM (2011)Google Scholar
  3. 3.
    Bezemer, C.-P., Mesbah, A., van Deursen, A.: Automated security testing of web widget interactions. In: Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC-FSE 2009), pp. 81–91. ACM (2009)Google Scholar
  4. 4.
    Binder, R.: Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley (2000)Google Scholar
  5. 5.
    Boshernitsan, M., Doong, R., Savoia, A.: From Daikon to Agitator: lessons and challenges in building a commercial tool for developer testing. In: Proc. Int. Sym. on Software Testing and Analysis (ISSTA 2006), pp. 169–180. ACM (2006)Google Scholar
  6. 6.
    Clarke, L.A., Rosenblum, D.S.: A historical perspective on runtime assertion checking in software development. ACM SIGSOFT Software Engineering Notes 31(3), 25–37 (2006)CrossRefGoogle Scholar
  7. 7.
    Csallner, C., Tillmann, N., Smaragdakis, Y.: DySy: Dynamic symbolic execution for invariant inference. In: Proceedings of the 30th International Conference on Software Engineering (ICSE 2008), pp. 281–290. ACM (2008)Google Scholar
  8. 8.
    Elbaum, S., Rothermel, G., Karre, S., Fisher, M.: Leveraging user-session data to support web application testing. IEEE Trans. Softw. Eng. 31, 187–202 (2005)CrossRefGoogle Scholar
  9. 9.
    Ernst, M., Perkins, J., Guo, P., McCamant, S., Pacheco, C., Tschantz, M., Xiao, C.: The Daikon system for dynamic detection of likely invariants. Science of Computer Programming 69(1-3), 35–45 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Groeneveld, F., Mesbah, A., van Deursen, A.: Automatic invariant detection in dynamic web applications. Technical Report TUD-SERG-2010-037, TUDelft (2010)Google Scholar
  11. 11.
    Guarnieri, S., Livshits, B.: Gatekeeper: mostly static enforcement of security and reliability policies for JavaScript code. In: Conference on USENIX Security Symposium, SSYM 2009, pp. 151–168 (2009)Google Scholar
  12. 12.
    Guha, A., Krishnamurthi, S., Jim, T.: Using static analysis for Ajax intrusion detection. In: Intl. Conference on World Wide Web (WWW), pp. 561–570 (2009)Google Scholar
  13. 13.
    Hangal, S., Lam, M.S.: Tracking down software bugs using automatic anomaly detection. In: Proceedings of the 24th International Conference on Software Engineering (ICSE 2002), pp. 291–301. ACM Press (2002)Google Scholar
  14. 14.
    Marchetto, A., Tonella, P., Ricca, F.: State-based testing of Ajax web applications. In: Proc. 1st Int. Conference on Sw. Testing Verification and Validation (ICST 2008), pp. 121–130. IEEE Computer Society (2008)Google Scholar
  15. 15.
    Mesbah, A., van Deursen, A., Lenselink, S.: Crawling Ajax-based web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web (TWEB) 6(1), 3:1–3:30 (2012)Google Scholar
  16. 16.
    Mesbah, A., van Deursen, A., Roest, D.: Invariant-based automatic testing of modern web applications. IEEE Transactions on Software Engineering (TSE) 38(1), 35–53 (2012)CrossRefGoogle Scholar
  17. 17.
    Meyer, B.: Applying design by contract. Computer 25(10), 40–51 (1992)CrossRefGoogle Scholar
  18. 18.
    Ocariza, F.J., Pattabiraman, K., Mesbah, A.: AutoFLox: An automatic fault localizer for client-side JavaScript. In: Proceedings of the 5th IEEE International Conference on Software Testing, Verification and Validation (ICST 2012), pp. 31–40. IEEE Computer Society (2012)Google Scholar
  19. 19.
    Pattabiraman, K., Zorn, B.: DoDOM: Leveraging DOM invariants for Web 2.0 application robustness testing. In: Proc. Int. Conf. Sw. Reliability Engineering (ISSRE 2010), pp. 191–200. IEEE Computer Society (2010)Google Scholar
  20. 20.
    Ratcliff, S., White, D., Clark, J.: Searching for invariants using genetic programming and mutation testing. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (GECCO). ACM (2011)Google Scholar
  21. 21.
    Rodríguez-Carbonell, E., Kapur, D.: Program Verification Using Automatic Generation of Invariants,. In: Liu, Z., Araki, K. (eds.) ICTAC 2004. LNCS, vol. 3407, pp. 325–340. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  22. 22.
    Roest, D., Mesbah, A., van Deursen, A.: Regression testing Ajax applications: Coping with dynamism. In: Proc. 3rd Int. Conf. on Sw. Testing, Verification and Validation (ICST 2010), pp. 128–136. IEEE Computer Society (2010)Google Scholar
  23. 23.
    Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering 14(2), 131–164 (2009)CrossRefGoogle Scholar
  24. 24.
    Sprenkle, S., Gibson, E., Sampath, S., Pollock, L.: Automated replay and failure detection for web applications. In: ASE 2005: Proc. 20th IEEE/ACM Int. Conf. on Automated Sw. Eng., pp. 253–262. ACM (2005)Google Scholar
  25. 25.
    Tarhini, A., Ismail, Z., Mansour, N.: Regression testing web applications. In: Int. Conf. on Advanced Comp. Theory and Eng., pp. 902–906. IEEE Computer Society (2008)Google Scholar
  26. 26.
    Xu, L., Xu, B., Chen, Z., Jiang, J., Chen, H.: Regression testing for web applications based on slicing. In: Proc. of Int. Conf. on Computer Software and Applications (COMPSAC), pp. 652–656. IEEE Computer Society (2003)Google Scholar
  27. 27.
    Zheng, Y., Bao, T., Zhang, X.: Statically locating web application bugs caused by asynchronous calls. In: Proceedings of the Intl. Conference on the World-Wide Web (WWW), pp. 805–814. ACM (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Shabnam Mirshokraie
    • 1
  • Ali Mesbah
    • 1
  1. 1.University of British ColumbiaVancouverCanada

Personalised recommendations