A Combinatorial Approach to Analyzing Cross-Site Scripting (XSS) Vulnerabilities in Web Application Security Testing

  • Dimitris E. Simos
  • Kristoffer Kleine
  • Laleh Shikh Gholamhossein Ghandehari
  • Bernhard Garn
  • Yu Lei
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9976)


Web applications typically employ sanitization functions to sanitize user inputs, independently whether this input is assumed to be legitimate, invalid or malicious. When such functions do not work correctly, a web application immediately becomes vulnerable to security attacks such as XSS. In this paper, we report a combinatorial approach to analyze XSS vulnerabilities in web applications. Our approach first performs combinatorial testing where a set of test vectors is executed against a subject application. If one or more XSS vulnerabilities are triggered during testing, we analyze the structure of each test vector to identify XSS-inducing combinations of its parameter model. If an attack vector contains an XSS-inducing combination, then the execution of this vector will successfully exploit an XSS vulnerability. Identification of XSS-inducing combinations provides insights about which kinds of user input might still be leverageable for XSS attacks and how to correct the function to provide better security guarantees. We conducted an experiment in which our approach was applied to four sanitization functions from the Web Application Vulnerability Scanner Evaluation Project (WAVSEP). The experimental results show that our approach can effectively identify XSS-inducing combinations for these sanitization functions.


Combinatorial testing XSS Fault localization Security testing 



This work has been funded by the Austrian Research Promotion Agency (FFG) under grant 851205 and the Austrian COMET Program (FFG).


  1. 1.
    Argyros, G., Stais, I., Kiayias, A., Keromytis, A.G.: Back in black: towards formal, black box analysis of sanitizers and filters. In: Proceedings of the 37th IEEE Symposium on Security and Privacy (2016)Google Scholar
  2. 2.
    Bozic, J., Garn, B., Kapsalis, I., Simos, D., Winkler, S., Wotawa, F.: Attack pattern-based combinatorial testing with constraints for web security testing. In: Proceedings of the 2015 IEEE International Conference on Software Quality, Reliability and Security, QRS 2015, pp. 207–212 (2015)Google Scholar
  3. 3.
    Bozic, J., Garn, B., Simos, D.E., Wotawa, F.: Evaluation of the IPO-family algorithms for test case generation in web security testing. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–10 (2015)Google Scholar
  4. 4.
    Brcic, M., Kalpic, D.: Combinatorial testing in software projects. In: Proceedings of the 35th International Convention, MIPRO, 2012 , pp. 1508–1513 (2012)Google Scholar
  5. 5.
    Duchene, F., Groz, R., Rawat, S., Richier, J.L.: XSS vulnerability detection using model inference assisted evolutionary fuzzing. In: Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, ICST 2012, pp. 815–817. IEEE Computer Society, Washington (2012)Google Scholar
  6. 6.
    Duchene, F., Rawat, S., Richier, J.L., Groz, R.: KameleonFuzz: evolutionary fuzzing for black-box XSS detection. In: CODASPY. ACM (2014)Google Scholar
  7. 7.
    Garn, B., Kapsalis, I., Simos, D., Winkler, S.: On the applicability of combinatorial testing to web application security testing: a case study. In: Proceedings of the 2014 Workshop on Joining AcadeMiA and Industry Contributions to Test Automation and Model-Based Testing, pp. 16–21. ACM (2014)Google Scholar
  8. 8.
    Ghandehari, L.S., Lei, Y., Kung, D., Kacker, R., Kuhn, R.: Fault localization based on failure-inducing combinations. In: 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE), pp. 168–177. IEEE (2013)Google Scholar
  9. 9.
    Ghandehari, L.S.G., Lei, Y., Xie, T., Kuhn, R., Kacker, R.: Identifying failure-inducing combinations in a combinatorial test set. In: 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation (ICST), pp. 370–379. IEEE (2012)Google Scholar
  10. 10.
    Grindal, M., Offutt, J.: Input parameter modeling for combination strategies. In: Proceedings of the 25th Conference on IASTED International Multi-Conference: Software Engineering SE 2007, pp. 255–260. ACTA Press, Anaheim (2007)Google Scholar
  11. 11.
    Hagar, J.D., Wissink, T.L., Kuhn, D., Kacker, R.N.: Introducing combinatorial testing in a large organization. Computer 48(4), 64–72 (2015)CrossRefGoogle Scholar
  12. 12.
    Hydara, I., Sultan, A.B.M., Zulzalil, H., Admodisastro, N.: Current state of research on cross-site scripting (XSS) a systematic literature review. Inf. Softw. Technol. 58, 170–186 (2015)CrossRefGoogle Scholar
  13. 13.
    Kuhn, D.R., Okun, V.: Pseudo-exhaustive testing for software. In: 30th Annual IEEE/NASA Software Engineering Workshop, SEW 2006, pp. 153–158. IEEE (2006)Google Scholar
  14. 14.
    Kuhn, D., Kacker, R., Lei, Y.: Introduction to Combinatorial Testing. Chapman & Hall/CRC Innovations in Software Engineering and Software Development Series. Taylor & Francis (2013)Google Scholar
  15. 15.
    van der Loo, F.: Comparison of penetration testing tools for web applications. Master’s thesis, University of Radboud, Netherlands (2011)Google Scholar
  16. 16.
    Mohammadi, M., Chu, B., Lipford, H.R., Murphy-Hill, E.: Automatic web security unit testing: XSS vulnerability detection. In: Proceedings of the 11th International Workshop on Automation of Software Test, AST 2016, pp. 78–84. ACM, New York (2016)Google Scholar
  17. 17.
    Nie, C., Leung, H.: A survey of combinatorial testing. ACM Comput. Surv. 43(2), 11: 1–11: 29 (2011)CrossRefzbMATHGoogle Scholar
  18. 18.
    Shi, L., Nie, C., Xu, B.: A software debugging method based on pairwise testing. In: Sunderam, V.S., Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2005. LNCS, vol. 3516, pp. 1088–1091. Springer, Heidelberg (2005). doi: 10.1007/11428862_179 CrossRefGoogle Scholar
  19. 19.
    Sudhodanan, A., Armando, A., Carbone, R., Compagna, L.: Attack patterns for black-box security testing of multi-party web applications. In: Proceedings of the Network and Distributed system Security Symposium (NDSS) (2016)Google Scholar
  20. 20.
    Tripp, O., Weisman, O., Guy, L.: Finding your way in the testing jungle: a learning approach to web security testing. In: Proceedings of the 2013 International Symposium on Software Testing and Analysis, ISSTA 2013, pp. 347–357. ACM, New York (2013)Google Scholar
  21. 21.
    Wang, Z., Xu, B., Chen, L., Xu, L.: Adaptive interaction fault location based on combinatorial testing. In: 2010 10th International Conference on Quality Software (QSIC), pp. 495–502. IEEE (2010)Google Scholar
  22. 22.
    Williams, J., Wichers, D.: OWASP Top 10 2013 (2013).
  23. 23.
    Yu, L., Lei, Y., Kacker, R., Kuhn, D.: Acts: a combinatorial test generation tool. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation (ICST), pp. 370–375 (2013)Google Scholar
  24. 24.
    Zhang, Z., Zhang, J.: Characterizing failure-causing parameter interactions by adaptive testing. In: Proceedings of the 2011 International Symposium on Software Testing and Analysis, pp. 331–341. ACM (2011)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2016

Authors and Affiliations

  • Dimitris E. Simos
    • 1
  • Kristoffer Kleine
    • 1
  • Laleh Shikh Gholamhossein Ghandehari
    • 2
  • Bernhard Garn
    • 1
  • Yu Lei
    • 2
  1. 1.SBA ResearchViennaAustria
  2. 2.Department of Computer Science and EngineeringUniversity of Texas at ArlingtonArlingtonUSA

Personalised recommendations