Contract-Driven Testing of JavaScript Code

  • Phillip Heidegger
  • Peter Thiemann
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6141)

Abstract

JSConTest is a tool that enhances JavaScript with simple, type-like contracts and provides a framework for monitoring and guided random testing of programs against these contracts at the same time. Function contracts in JSConTest serve a dual role as specifications of the input/output behavior and as test case generators. Generation of test data for a contract is generally random, but it can be guided by annotations on the contract to achieve higher coverage. Annotations may indicate dependencies among parameters and the result or they may select lightweight program analyses, the results of which influence the choice of test data. A case study substantiates that JSConTest finds type-related errors with high probability.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Andrews, J.H., Briand, L.C., Labiche, Y.: Is mutation an appropriate tool for testing experiments? In: ICSE ’05: Proc. 27th International Conference on Software Engineering, pp. 402–411. ACM, New York (2005)Google Scholar
  2. 2.
    Andrews, J.H., Haldar, S., Lei, Y., Li, F.C.H.: Tool support for randomized unit testing. In: RT ’06: Proceedings of the 1st International Workshop on Random Testing, Portland, Maine, pp. 36–45. ACM, New York (2006)CrossRefGoogle Scholar
  3. 3.
    Bird, D.L., Munoz, C.U.: Automatic generation of random self-checking test cases. IBM Syst. J. 22(3), 229–245 (1983)CrossRefGoogle Scholar
  4. 4.
    Bloom, B., Field, J., Nystrom, N., Östlund, J., Richards, G., Strniša, R., Vitek, J., Wrigstad, T.: Thorn: Robust, concurrent, extensible scripting on the JVM. In: Arora, S., Leavens, G.T. (eds.) Proc. 24th ACM Conf. OOPSLA, Orlando, Florida, USA, pp. 117–136. ACM, New York (2009)Google Scholar
  5. 5.
    Boyapati, C., Khurshid, S., Marinov, D.: Korat: Automated testing based on Java predicates. In: ISSTA ’02: Proc. 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis, Roma, Italy, pp. 123–133. ACM, New York (2002)Google Scholar
  6. 6.
    Chen, T.Y., Kuo, F.-C., Merkel, R.G., Tse, T.H.: Adaptive random testing: The ART of test case diversity. J. Systems and Software 83(1), 60–66 (2010)CrossRefGoogle Scholar
  7. 7.
    Ciupa, I., Leitner, A., Oriol, M., Meyer, B.: ARTOO: Adaptive random testing for object-oriented software. In: Schäfer, W., Dwyer, M.B., Gruhn, V. (eds.) ICSE 2008, pp. 71–80. ACM, New York (2008)CrossRefGoogle Scholar
  8. 8.
    Claessen, K., Hughes, J.: QuickCheck: A lightweight tool for random testing of Haskell programs. In: Wadler, P. (ed.) Proc. ICFP 2000, Montreal, Canada, September 2000, pp. 268–279. ACM Press, New York (2000)Google Scholar
  9. 9.
    Csallner, C., Smaragdakis, Y.: JCrasher: An automatic robustness tester for Java. Software—Practice & Experience 34(11), 1025–1050 (2004)CrossRefGoogle Scholar
  10. 10.
    Duran, J., Ntafos, S.: An evaluation of random testing. Transactions on Software Engineering 10(4), 438–444 (1984)CrossRefGoogle Scholar
  11. 11.
    Eastlund, C.: DoubleCheck your theorems. In: ACL2 2009, Boston, MA (2009)Google Scholar
  12. 12.
    ECMAScript Language Specification, ECMA International, ECMA-262, 5th edn. (December 2009), http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf
  13. 13.
    Elkarablieh, B., Zayour, Y., Khurshid, S.: Efficiently generating structurally complex inputs with thousands of objects. In: Ernst [14], pp. 248–272Google Scholar
  14. 14.
    Ernst, E. (ed.): ECOOP 2007. LNCS, vol. 4609. Springer, Heidelberg (2007)Google Scholar
  15. 15.
    Findler, R.B., Felleisen, M.: Contracts for higher-order functions. In: Peyton-Jones, S. (ed.) Proc. ICFP 2002, Pittsburgh, PA, USA, October 2002, pp. 48–59. ACM Press, New York (2002)Google Scholar
  16. 16.
    Findler, R.B., Flanagan, C., Flatt, M., Krishnamurthi, S., Felleisen, M.: DrScheme: A pedagogic programming environment for Scheme. In: Hartel, P.H., Kuchen, H. (eds.) PLILP 1997. LNCS, vol. 1292, pp. 369–388. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  17. 17.
    Findler, R.B., Flatt, M., Felleisen, M.: Semantic casts: Contracts and structural subtyping in a nominal world. In: Odersky, M. (ed.) ECOOP 2004. LNCS, vol. 3086, pp. 364–388. Springer, Heidelberg (2004)Google Scholar
  18. 18.
    Forrester, J.E., Miller, B.P.: An empirical study of the robustness of Windows NT applications using random testing. In: 4th USENIX Windows System Symposium, Seattle (August 2000)Google Scholar
  19. 19.
    Godefroid, P., Klarlund, N., Sen, K.: Dart: Directed automated random testing. In: Proc. 2005 ACM Conf. PLDI, Chicago, IL, USA, June 2005, pp. 213–223. ACM Press, New York (2005)Google Scholar
  20. 20.
    Gray, K.E., Findler, R.B., Flatt, M.: Fine-grained interoperability through mirrors and contracts. In: Proc. 20th ACM Conf. OOPSLA, San Diego, CA, USA, pp. 231–245. ACM Press, New York (2005)Google Scholar
  21. 21.
    Hamlet, R.G.: Random testing. In: Marciniak, J. (ed.) Encyclopedia of Software Engineering, pp. 970–978. Wiley, Chichester (1994)Google Scholar
  22. 22.
    Heidegger, P., Thiemann, P.: Recency types for dynamically-typed object-based languages. In: 2009 International Workshop on Foundations of Object-Oriented Languages (FOOL’09), Savannah, Georgia, USA (January 2009), http://www.cs.cmu.edu/~aldrich/FOOL09/heidegger.pdf
  23. 23.
    Jensen, S.H., Møller, A., Thiemann, P.: Type analysis for JavaScript. In: Palsberg, J., Su, Z. (eds.) SAS 2009. LNCS, vol. 5673, pp. 238–255. Springer, Heidelberg (2009)Google Scholar
  24. 24.
    Klein, C., Findler, R.B.: Randomized testing in PLT Redex. In: Workshop on Scheme and Functional Programming 2009, Boston, MA, USA (2009)Google Scholar
  25. 25.
    Leitner, A., Oriol, M., Zeller, A., Ciupa, I., Meyer, B.: Efficient unit test case minimization. In: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, Atlanta, Georgia, USA, November 2007, pp. 417–420 (2007)Google Scholar
  26. 26.
    Matthews, J., Findler, R.B.: Operational semantics for multi-language programs. ACM TOPLAS 31, 12:1–12:44 (2009)CrossRefMathSciNetGoogle Scholar
  27. 27.
    Meyer, B.: Object-Oriented Software Construction, 2nd edn. Prentice-Hall, Englewood Cliffs (1997)MATHGoogle Scholar
  28. 28.
    Myers, G.J., Sandler, C.: The Art of Software Testing. John Wiley & Sons, Chichester (2004)Google Scholar
  29. 29.
    Offutt, A.J., Hayes, J.H.: A semantic model of program faults. In: 1996 ACM SIGSOFT International Symposium on Software Testing and Analysis, San Diego, CA, USA, January 1996, pp. 195–200 (1996)Google Scholar
  30. 30.
    Pacheco, C.: Directed Random Testing. Ph.D., MIT Department of Electrical Engineering and Computer Science, Cambridge, MA, USA (June 2009)Google Scholar
  31. 31.
    Runciman, C., Naylor, M., Lindblad, F.: Smallcheck and Lazy Smallcheck: Automatic exhaustive testing for small values. In: Haskell ’08: Proc. of the first ACM SIGPLAN Symposium on Haskell, Victoria, BC, Canada, pp. 37–48. ACM, New York (2008)CrossRefGoogle Scholar
  32. 32.
    Siek, J., Taha, W.: Gradual typing for objects. In: Ernst [14], pp. 2–27Google Scholar
  33. 33.
    Tobin-Hochstadt, S., Felleisen, M.: Interlanguage migration: From scripts to programs. In: Dynamic Languages Symposium, DLS 2006, Portland, Oregon, USA, pp. 964–974. ACM, New York (2006)Google Scholar
  34. 34.
    Tobin-Hochstadt, S., Felleisen, M.: The design and implementation of typed scheme. In: Wadler, P. (ed.) Proc. 35th ACM Symp. POPL, San Francisco, CA, USA, January 2008, pp. 395–406. ACM Press, New York (2008)Google Scholar
  35. 35.
    Wadler, P., Findler, R.B.: Well-typed programs can’t be blamed. In: Castagna, G. (ed.) ESOP 2009. LNCS, vol. 5502, pp. 1–16. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  36. 36.
    Zeller, A., Hildebrandt, R.: Simplifying and isolating failure-inducing input. IEEE Trans. Softw. Eng. 28(2), 183–200 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Phillip Heidegger
    • 1
  • Peter Thiemann
    • 1
  1. 1.Albert-Ludwigs-Universität FreiburgGermany

Personalised recommendations