Abstract
Black-box web vulnerability scanners are a class of tools that can be used to identify security issues in web applications. These tools are often marketed as “point-and-click pentesting” tools that automatically evaluate the security of web applications with little or no human support. These tools access a web application in the same way users do, and, therefore, have the advantage of being independent of the particular technology used to implement the web application. However, these tools need to be able to access and test the application’s various components, which are often hidden behind forms, JavaScript-generated links, and Flash applications.
This paper presents an evaluation of eleven black-box web vulnerability scanners, both commercial and open-source. The evaluation composes different types of vulnerabilities with different challenges to the crawling capabilities of the tools. These tests are integrated in a realistic web application. The results of the evaluation show that crawling is a task that is as critical and challenging to the overall ability to detect vulnerabilities as the vulnerability detection techniques themselves, and that many classes of vulnerabilities are completely overlooked by these tools, and thus research is required to improve the automated detection of these flaws.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
AnantaSec: Web Vulnerability Scanners Evaluation (January 2009), http://anantasec.blogspot.com/2009/01/web-vulnerability-scanners-comparison.html
Balzarotti, D., Cova, M., Felmetsger, V., Vigna, G.: Multi-module Vulnerability Analysis of Web-based Applications. In: Proceedings of the ACM conference on Computer and Communications Security (CCS), pp. 25–35 (2007)
Curphey, M., Araujo, R.: Web Application Security Assessment Tools. IEEE Security and Privacy 4(4), 32–41 (2006)
CVE: Common Vulnerabilities and Exposures, http://www.cve.mitre.org
Foundstone: Hacme Bank v2.0 (May 2006), http://www.foundstone.com/us/resources/proddesc/hacmebank.htm
Grossman, J.: Challenges of Automated Web Application Scanning. In: BlackHat Windows Security Conference (2004)
Kals, S., Kirda, E., Kruegel, C., Jovanovic, N.: SecuBat: A Web Vulnerability Scanner. In: Proceedings of the International World Wide Web Conference (2006)
McAllister, S., Kruegel, C., Kirda, E.: Leveraging User Interactions for In-Depth Testing of Web Applications. In: Proceedings of the Symposium on Recent Advances in Intrusion Detection (2008)
Open Security Foundation: OSF DataLossDB: Data Loss News, Statistics, and Research, http://datalossdb.org/
Open Web Application Security Project (OWASP): OWASP SiteGenerator, http://www.owasp.org/index.php/OWASP_SiteGenerator
Open Web Application Security Project (OWASP): OWASP WebGoat Project, http://www.owasp.org/index.php/Category:OWASP_WebGoat_Project
Open Web Application Security Project (OWASP): Web Input Vector Extractor Teaser, http://code.google.com/p/wivet/
Open Web Application Security Project (OWASP): OWASP Top Ten Project (2010), http://www.owasp.org/index.php/Top_10
OpenID Foundation: OpenID, http://openid.net/
PCI Security Standards Council: PCI DDS Requirements and Security Assessment Procedures, v1.2 (October 2008)
Peine, H.: Security Test Tools for Web Applications. Tech. Rep. 048.06, Fraunhofer IESE (January 2006)
Provos, N., Mavrommatis, P., Rajab, M., Monrose, F.: All Your iFRAMEs Point to Us. In: Proceedings of the USENIX Security Symposium, pp. 1–16 (2008)
RSnake: Sql injection cheat sheet, http://ha.ckers.org/sqlinjection/
RSnake: XSS (Cross Site Scripting) Cheat Sheet, http://ha.ckers.org/xss.html
Small, S., Mason, J., Monrose, F., Provos, N., Stubblefield, A.: To Catch a Predator: A Natural Language Approach for Eliciting Malicious Payloads. In: Proceedings of the USENIX Security Symposium (2008)
Suto, L.: Analyzing the Effectiveness and Coverage of Web Application Security Scanners (October 2007) (case Study)
Suto, L.: Analyzing the Accuracy and Time Costs of Web Application Security Scanners (Feburary 2010)
Vieira, M., Antunes, N., Madeira, H.: Using Web Security Scanners to Detect Vulnerabilities in Web Services. In: Proceedings of the Conference on Dependable Systems and Networks (2009)
Wiegenstein, A., Weidemann, F., Schumacher, M., Schinzel, S.: Web Application Vulnerability Scanners—a Benchmark. Tech. rep., Virtual Forge GmbH (October 2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Doupé, A., Cova, M., Vigna, G. (2010). Why Johnny Can’t Pentest: An Analysis of Black-Box Web Vulnerability Scanners. In: Kreibich, C., Jahnke, M. (eds) Detection of Intrusions and Malware, and Vulnerability Assessment. DIMVA 2010. Lecture Notes in Computer Science, vol 6201. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14215-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-14215-4_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14214-7
Online ISBN: 978-3-642-14215-4
eBook Packages: Computer ScienceComputer Science (R0)