Keywords

1 Introduction

Nowadays, there is a fundamental imbalance between attackers and defenders, because the system designers and developers cannot work as security experts at the same time. Thus, to ensure the security of the system, professional test is necessary. However, traditional security test is expensive and with limited coverage. Fortunately, the emergence of crowdsourced test levels the playing field [10]. Crowdsourced test is a kind of new service pattern, where companies can publish their test projects on the crowdsourced platform and hackers can attend the projects they interested. Hackers are paid by the company if and only if they submit vulnerabilities. With this kind of pattern, both companies and hackers obtain a benefit. At present, various crowdsourced test platforms have arisen. In China, the mainstream platforms include Wooyun [16], VULBOX [15], Testin [14], 360 [1], Sobug [13], CNVD [5] and ICS-CERT [8] etc. In foreign countries, HackerOne [7] and BugCrowd [4] are widely accepted crowdsourced platforms.

Although crowdsourced tests have so many advantages, the concern about vulnerability leaks seriously limits the widespread of crowdsourced tests [17]. For some systems or applications, the security is mainly based on their privacy. So, exposing these systems or applications under the crowdsourced tests scenario will raise the concerns of the companies. For some hackers, concealing the vulnerabilities they find may bring more benefits instead of submitting them to the companies. Therefore, how to avoid vulnerability leaks is the primary issue that all crowdsourced platforms have to deal with.

Existing crowdsourced test platforms have adopt various technical and management approaches to avoid vulnerability leaks. The commonly used approaches can be summarized as three aspects: legal approaches, management approaches and technical approaches [11]. The legal approaches mainly include signing confidentiality agreements, conducting legal training, declaring authorized test boundaries, and real name certification etc. The management approaches include forming credible hacker teams, setting test projects with different secret levels, allowing companies choose hackers they trust etc. [18]. The technical approaches mainly include test traffic monitoring, VPN based accessing control for application/system under test (A/SUTs), accessing control for A/SUTs based on bastion host, test behavior auditing and screen recording etc. It is no doubt that the above-mentioned approaches considerably reduce the concerns about vulnerability leaks, but none of them is able to thoroughly remove the concerns.

Different from the indirect approaches above, this paper design a generic architecture to detect from the test traffic whether the white hats have found a vulnerability. The architecture do not need to install monitors on the various heterogeneous A/SUTs, thus having good operability. In the architecture, the rule-matching method based on Snort [3] is adopted to detect public vulnerabilities, and abnormal behavior monitoring method based on deep learning is adopted to detect unknown vulnerabilities. Finaly, the proposed architecture is applied to the ICS-CERT crowdsourced test platform [8]. The results show that, with proper rules, most of the concealing behaviors can be detected.

2 The System Architecture

The system architecture is shown in Fig. 1. The architecture includes three parts: the white hats, the A/SUT and the crowdsourced test platform. They are connected through Internet. The dotted lines in the figure denote the data flow.

Fig. 1.
figure 1

The hardware architecture and data flow (Colopr figure online)

Full size image

2.1 The Crowdsourced Test Platform Architecture

The crowdsourced test platform acts as a bridge between A/SUTs and white hats. White hats apply for test projects on the platform and companies publish test projects on the platform. In order to achieve the monitor ability, the platform have to contain the web portal module, the VPN module and the monitor module at least.

  1. (1)

    The web portal module provides access to the platform, where white hats can apply for test projects and companies can publish test projects.

  2. (2)

    The VPN module provides white hats the only access to the A/SUT. In other words, without the VPN, white hats will not able to access the A/SUT.

  3. (3)

    The monitor module gets the test traffic of VPN servers from the switch’s monitor port. The traffic-based monitor software is deployed on the monitor servers. The architecture of the software will be given in the next section.

2.2 The A/SUT Configuration

To make sure that all the test traffic is under surveillance, we require the VPN to be the unique access to the A/SUT. So, a firewall is placed before the A/SUT and only the IP address of VPN is allowed to access the A/SUT. As shown in Fig. 1, the A/SUT part should contain two modules at least: the A/SUT itself and the firewall. The A/SUT is connected to Internet through the firewall.

Obviously, the crowdsourced test platform can be connected with more than one A/SUT. But every A/SUT should keep this kind of configuration.

2.3 The Data Flow

The dotted lines in the Fig. 1 denote the data flows. The red line denotes the white hats’ access path to the web portal. It is obviously that the web portal can be accessed directly without the VPN. The green line denotes the white hats’ access path to the A/SUT. If white hats want to test a A/SUT, they have to get through the VPN servers, because the firewall before the A/SUT only allows the access from the VPN servers. The yellow line denotes the data flow between the VPN servers and the monitor servers. All the test traffic flow through the VPN servers is completely copied to the monitor servers.

3 The Software Architecture

The vulnerability detection software is deployed on the monitor servers shown in Fig. 1. In the software, the rule-matching method based on Snort [3] is adopted to detect public vulnerabilities, and Recurrent Neural Network (RNN) model based on TensorFlow [2] is adopted to detect unknown vulnerabilities. The detailed software architecture is shown in Fig. 2.

Fig. 2.
figure 2

The software architecture

Full size image

The software mainly includes three layers: the test traffic collection layer (the bottom layer), the vulnerability monitor layer (the middle layer) and the result display layer (the top layer).

  1. (1)

    In the bottom layer, the main module is the traffic sniffer which collects the test traffic from the mirror port of switch.

  2. (2)

    In the middle layer, the data preprocessing module is mainly responsible for the protocol decoding, message fragmenting and port scanning etc. After the preprocessing, the raw test traffic is converted to the format that the monitor algorithms can recognize.

Above the data preprocessing module is the test behavior monitor module. This module contains two algorithms: the rule-matching method based on Snort to detect public vulnerabilities, and RNN model based on TensorFlow to detect unknown vulnerabilities. The key technology of the rule-matching method is the rule base. In this paper, we obtain the rule base from EVERSEC [9]. The rule base is a kind long-term accumulated knowledge, which is important in engineering but plain in theory. So we do not illustrate the rule base in this paper. What we want to tell readers is that the rule matching based method is really effective in the detecting of concealing behaviors. The RNN model is trained with normal traffic of the A/SUT, and learned the normal behavior of the A/SUT. Then we use the test traffic as the input of the trained RNN model. If there is abnormal behaviors that the model cannot identify and no rule is matched, then we think a unknown vulnerability is detected.

Above the test behavior monitor module is the storage module. This module stores the processing result of test behavior monitor module. The stored content include alarms, logs and evidence.

  1. (3)

    In the result display layer, the display module read the result data from the storage module and shows to administrators.

4 The Evaluation

The proposed architecture has been realized in the ICS-CERT crowdsourced test platform. To evaluate the effectiveness of the architecture, we test it with SQL injection vulnerability, XSS vulnerability, file upload vulnerability, file inclusion vulnerabilities and command execution vulnerability. For the space limitation, we just illustrate the detailed test of SQL injection and XSS.

4.1 SQL Injection Vulnerability

We use CVE-2017-8917 [6] and Joomla 3.7 as the test case. We configure a A/SUT with the vulnerabilities and the access address of the A/SUT is http://47.94.131.119/index.php. Four POCs shown in Fig. 3 are used to evaluate the effectiveness. Except the POC3, the other three POCs are valid.

Fig. 3.
figure 3

The four POCs for SQL injection

Full size image

The results show that all the four POCs are successfully detected. The detection result for POC1 is shown in Fig. 4. Especially, for POC3 shown in Fig. 5, we identify that it is a invalid SQL injection. For the space limitation, we do not show the detection results of the other POCs in the paper.

Fig. 4.
figure 4

The detection result for POC1

Full size image
Fig. 5.
figure 5

The detection result for POC3

Full size image

4.2 XSS Vulnerability

We configure a A/SUT with XSS vulnerabilities and the accessing address is: http://104.225.151.194/cshajx/xss/easyxss/.

The two payloads we used are shown in Fig. 6. The Payload1 will trigger an alarm window, as shown in Fig. 7. The detection result of Payload1 is shown in Fig. 8. For the space limitation, we do not show the running screenshot of Payload2.

Fig. 6.
figure 6

The two XSS payloads used in the evaluation

Full size image
Fig. 7.
figure 7

The running screenshot of Payload1

Full size image
Fig. 8.
figure 8

The detection result for Payload1

Full size image

The results show that reflective XSS vulnerabilities such as Payload1 can be successfully detected. For invalid XSS vulnerabilities such as Payload2, we can identify that it is a failed XSS attack.

4.3 Command Execution Vulnerabilities

For the command execution vulnerabilities, we use the ShellShock vulnerability [12] to test the detection ability. The result shows that we can identity that whether the vulnerability is successfully exploited.

4.4 Some Unknown Vulnerabilities

For the space limitation, we do not show the running screenshots for the tests of unknown vulnerabilities, but give the test results here directly. For an unknown vulnerability, we cannot find a rule that matches the vulnerability. So we can only detect it by the RNN model.

For some unknown vulnerabilities that can cause file upload behaviors, though it is difficult to distinguish whether the file is harmful, but if the white hats try to connect or execute the file, we can successfully detect the connection and execution behaviors.

For some unknown vulnerabilities that can cause the file inclusion attacks, though we cannot identify the category of the vulnerability, but we can give alarms successfully once the vulnerability is exploited.

5 Conclusion

This paper proposes a generic architecture for crowdsourced test platforms to detect whether the white hats have found a vulnerability. In the architecture, the rule-matching method based on Snort is adopted to detect public vulnerabilities, and RNN model based on TensorFlow is adopted to detect unknown vulnerabilities. The proposed architecture is applied to the ICS-CERT crowdsourced test platform. We test it with SQL injection vulnerability, file upload vulnerability, file inclusion vulnerabilities, command execution vulnerability and XSS vulnerability. The results shows that, for most of vulnerabilities, the architecture can detect them successfully. With the technology, we can remove companies’ concerns about vulnerability leaks, thus accelerating the application of crowdsourced test.