ICISS 2010: Information Systems Security pp 111-124 | Cite as

Coverage Criteria for Automatic Security Testing of Web Applications

  • Thanh Binh Dao
  • Etsuya Shibayama
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6503)

Abstract

In security testing of web applications, the selection of coverage criteria for adequacy evaluation of test cases is based on the trade off between test cost and vulnerability detection effectiveness. Coverage criteria used in traditional software testing such as branch coverage and statement coverage are commonly used but they are not originally defined for security testing purpose. In this paper, we present an overview of the limitations of those common coverage criteria and propose wrapper coverage, vulnerability-aware sink coverage and vulnerability-aware wrapper coverage as other options that are more appropriate for security testing. We conduct an experiment of security testing of real-world web applications to evaluate the usefulness and discuss about the usage of these proposed coverage criteria.

Keywords

automatic security testing web application coverage criteria 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    The Open Web Application Security Project: Vulnerability Category, http://www.owasp.org/index.php/Category:Vulnerability
  2. [2]
    The Open Web Application Security Project: SQL Injection Prevention Cheat Sheet, http://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheet
  3. [3]
    Symantec Corporation: Five common Web application vulnerabilities, http://www.symantec.com/connect/articles/five-common-web-application-vulnerabilities
  4. [4]
    Chinotec Technologies Company: Paros, http://www.parosproxy.org
  5. [5]
    Acunetix. Acunetix Web Vulnerability Scanner (2008), http://www.acunetix.com/
  6. [6]
    Hewlett-Packard Development Company. HP WebInspect softwareGoogle Scholar
  7. [7]
    Jovanovic, N., Kruegel, C., Kirda, E.: Pixy: A Static Analysis Tool for Detecting Web Application Vulnerabilities (Short Paper). In: Proceedings of the 2006 IEEE Symposium on Security and Privacy, SP, pp. 258–263. IEEE Computer Society, Washinton (2006)Google Scholar
  8. [8]
    Dao, T.-B., Shibayama, E.: Idea: Automatic Security Testing for Web Applications. In: Massacci, F., Redwine Jr., S.T., Zannone, N. (eds.) ESSoS 2009. LNCS, vol. 5429. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  9. [9]
    Zhao, R., Lyu, M.R.: Character String Predicate Based Automatic Software Test Data Generation. In: Proceedings of the Third International Conference on Quality Software (QSIC 2003), p. 255. IEEE Computer Society, Washington (2003)CrossRefGoogle Scholar
  10. [10]
    Huang, Y., Huang, S., Lin, T., Tsai, C.: Web application security assessment by fault injection and behavior monitoring. In: Proceedings of the 12th International Conference on World Wide Web, WWW 2003, Budapest, Hungary, May 20-24, pp. 148–159. ACM, New York (2003)Google Scholar
  11. [11]
    Livshits, V.B., Lam, M.S.: Finding security vulnerabilities in java applications with static analysis. In: Proceedings of the 14th Conference on USENIX Security Symposium, Baltimore, MD, July 31-August 05, vol. 14, p. 18. USENIX Association, Berkeley (2005)Google Scholar
  12. [12]
    Smith, B., Shin, Y., Williams, L.: Proposing SQL statement coverage metrics. In: Proceedings of the Fourth International Workshop on Software Engineering For Secure Systems, SESS 2008, Leipzig, Germany, May 17-18, pp. 49–56. ACM, New York (2008)CrossRefGoogle Scholar
  13. [13]
    Halfond, W.G., Orso, A.: Command-Form Coverage for Testing Database Applications. In: Proceedings of the 21st IEEE/ACM International Conference on Automated Software Engineering, September 18-22, pp. 69–80. IEEE Computer Society, Washington (2006)CrossRefGoogle Scholar
  14. [14]
    Surez-Cabal, M.J., Tuya, J.: Using an SQL coverage measurement for testing database applications. In: Proceedings of the 12th ACM SIGSOFT Twelfth International Symposium on Foundations of Software Engineering, SIGSOFT 2004/FSE-12, Newport Beach, CA, USA, October 31-November 06, pp. 253–262. ACM, New York (2004)CrossRefGoogle Scholar
  15. [15]
    Kapfhammer, G.M., Soffa, M.L.: A family of test adequacy criteria for database-driven applications. In: Proceedings of the 9th European Software Engineering Conference Held Jointly with 11th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-11, Helsinki, Finland, September 01-05, pp. 98–107. ACM, New York (2003)Google Scholar
  16. [16]
    Kieyzun, A., Guo, P.J., Jayaraman, K., Ernst, M.D.: Automatic creation of SQL Injection and cross-site scripting attacks. In: Proceedings of the 31st International Conference on Software Engineering, May 16-24, pp. 199–209. IEEE Computer Society, Washington (2009)Google Scholar
  17. [17]
    Zhu, H., Hall, P.A., May, J.H.: Software unit test coverage and adequacy. ACM Comput. Surv. 29(4), 366–427 (1997)CrossRefGoogle Scholar
  18. [18]
    Balzarotti, D., Cova, M., Felmetsger, V., Jovanov, N., Kirda, E., Kruegel, C., Vigna, G.: Saner: Composing Static and Dynamic Analysis to Validate Sanitization in Web Applications. In: IEEE Security and Privacy Symposium (2008)Google Scholar
  19. [19]
    Cyber Security Bulletins, US-Cert, http://www.us-cert.gov/cas/bulletins/
  20. [20]
    Nguyen-Tuong, A., Guarnieri, S., Greene, D., Shirley, J., Evans, D.: Automatically hardening web applications using precise tainting. In: Twentieth IFIP International Information Security Conference, SEC 2005 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Thanh Binh Dao
    • 1
  • Etsuya Shibayama
    • 2
  1. 1.Dept. of Mathematical and Computing SciencesTokyo Institute of TechnologyTokyoJapan
  2. 2.Information Technology CenterThe University of TokyoTokyoJapan

Personalised recommendations