Keywords

1 Introduction

JavaScript (JS), as the most popular web frond-end programming language, is used by 95.1% of websites [23]. Many of such websites handle sensitive information such as financial transactions and private conversions. Errors in these websites not only affect user experiences, but also endanger safety, security, and privacy of users. Therefore, these websites, particularly their dynamic functions that are often implemented in JS, must be thoroughly tested to detect software bugs. There have been many testing frameworks for JS applications, such as Jest and Puppeteer. These frameworks provide a systematic way to test JS applications and reduce the tedious testing setup, particularly for unit testing. However, although these testing frameworks simplify the execution of testing, they do not provide test data for web applications. Such test data still needs to be provided manually by application developers, which is often very time-consuming and laborious. And achieving high code and functional coverage on web applications with high-quality test data still remains a challenge [34].

Symbolic execution has shown great promises in software testing, particularly in test data generation [29]. It exercises software with symbolic inputs, explores its execution paths systematically, and generates test data for the paths explored symbolically. However, symbolic execution may suffer from path explosions when the software has too many paths to explore [26]. Concolic testing addresses path explosion by combining concrete execution with symbolic execution. The software is first exercised with a concrete input and the resulting concrete execution trace is then analyzed symbolically to explore paths that are adjacent to the concrete trace. Concolic testing has achieved many successes in software testing [25]. It is strongly desirable to apply concolic testing to front-end JS web application to generate high-quality test data automatically, so manually efforts can be reduced and test coverage can be improved. However, front-end JS applications pose major challenges to concolic testing. These applications typically execute in the contexts of web browsers, which tends to be complex, and they are usually event-driven, user-interactive, and string-intensive [35].

In this paper, we present a novel approach to concolic testing of front-end JS web application. This approach leverages widely used JS testing frameworks such as Jest and Puppeteer and conducts concolic execution on JS web functions for unit testing [39]. These testing frameworks isolate the web function under test from the context of its embedding web page by mocking the environment and provide the test data that drives the function. This isolation of web function provides an ideal target for application of concolic testing. We integrate concolic testing APIs into these testing frameworks. The seamless integration of concolic testing allows injection of symbolic variables within the native execution context of a JS web function and precise capture of concrete execution traces of this function. As the testing framework executes the function under test with test data, parts or all of the test data can be made symbolic and the resulting execution traces of the function are captured for later symbolic analysis. Concise execution traces greatly improve the effectiveness and efficiency of the subsequent symbolic analysis for test generation. The new test data generated by the symbolic analysis is again fed back to the testing frameworks to drive further concolic testing.

We have implemented our approach on Jest and Puppeteer. The application of our Jest implementation to Metamask, one of the most popular Crypto wallets, has uncovered 3 bugs and 1 test suite improvement, whose bug reports have been accepted by Metamask developers on Github. We have also applied our Puppeteer implementation to 21 Github projects and detected 4 bugs.

2 Background

2.1 Front-end JavaScript Testing Frameworks

In a general software testing framework, a test case is designed to exercise a single, logical unit of behavior in an application and ensure the targeted unit operates as expected [21]. Typically, it is structured as a tuple {P, C, Q}:

  • P are the preconditions that must be met so that the function under test can be executed.

  • C is the function under test, containing the logic to be tested.

  • Q are the post assertions of the test case that are expected to be true.

As shown in Figure 1, a front-end JS testing framework inspects the web application in the browser for JS functions to test. It utilizes testing libraries to obtain the web pages, parses them and stores page functions and their context information individually so that test runners can run the functions browser-less [4]. The test runner sets up the three parts of a test case for each JS function under test and then executes the test case. The front-end JS testing framework helps isolate the JS function under test and provides the execution context for testing the function, which is an ideal entry for our application of the concolic testing to front-end JS.

Fig. 1.
figure 1

Front-end JS testing framework workflow

Full size image

2.2 In-situ Concolic Testing of Backend JavaScript

In [9], a new approach has been introduced to applying concolic testing to backend JS in-situ, i.e., scripts are executed in their native environments (e.g., Node.js) as part of concolic execution and test cases generated are directly replayed in these environments [13]. As illustrated in Figure 2, the concrete execution step of concolic testing as indicated by the dashed box on top is conducted in the native execution environment for JS, where the trace of this concrete execution is captured. The trace is then analyzed in the symbolic execution step of concolic testing to generate test cases that are then fed back into the native concrete execution to drive further test case generation. This approach has been implemented on the Node.js execution environment and its V8 JS engine [24]. As a script is executed with Node.js, its binary-level execution trace is captured and later analyzed through symbolic execution for test case generation. It also offers the flexibility of customizing trace as needed. We leverage this functionality in our approach.

Fig. 2.
figure 2

Workflow for in-situ concolic testing of backend JavaScript

Full size image

3 Approach

3.1 Overview

Our approach strives to apply concolic testing on front-end JS web applications to generate effective test data for unit-testing of these applications. Below are the specific design goals for our approach:

  • Front-end JS Extraction. JS web functions need to be extracted from web pages to execute independently to reduce complexity for concolic testing.

  • Execution Context Construction. JS web functions under test need to have the same execution environments as they are executed in the web pages.

  • Non-intrusive and Effective Concolic Testing. Concolic execution on JS web applications needs to require minimal changes on both the applications and the symbolic engine and generate useful test cases effectively.

With the above goals in mind, we design an approach to concolic testing of front-end JS web application, which leverages the JS testing frameworks such as Jest and Puppeteer and conducts concolic execution on JS web functions for unit testing. The seamless integration of concolic testing with these testing frameworks is achieved through extending in-situ concolic testing of backend JS applications. Figure 3 illustrates how the integration is realized:

  1. 1.

    Workflow 1 in Figure 3a illustrates the original capability of in-situ concolic testing of backend JS applications. It tests pure JS functions from NPM JS libraries. The execution tracer captures the traces of the pure JS functions and feeds them to the symbolic execution engine to generate new test data.

  2. 2.

    Workflow 2 in Figure 3b illustrates a naïve application of in-situ concolic testing to a JS web application. However, in-situ concolic testing cannot handle web elements, e.g., <HTML> tags, without the capability of a browser.

  3. 3.

    Workflow 3 in Figure 3c illustrates how we leverage a JS testing framework to extract the front-end JS web function and its execution context from the web page. In the extraction, we encapsulate them as a pure JS function augmented with the web page information, inject symbolic values and capture execution traces for later symbolic analysis by calling the symbolic execution interface functions within the extracted execution context. We then utilize the test runner of the JS testing framework to initiate and drive concolic testing within the execution context to generate new test data.

Fig. 3.
figure 3

Overview for concolic testing of front-end JS

Full size image

This workflow allows faithful simulation of the execution context of a JS web function without the presence of a web browser. It enables injection of symbolic variables and captures of concrete execution traces within the execution context of the JS web function under test. A concise and accurate concrete execution trace can greatly improve the effectiveness and efficiency of the following symbolic analysis for test generation. We explain how to decide the starting point of tracing within the native execution context and what difference it makes in Section 3.2.

3.2 Concolic Testing of JS Web Function within Execution Context

A front-end JS web function is invoked from a web page and its execution depends on the execution context from the web page [28]. The core of our approach is to enable concolic testing on the JS web function within its native execution context from the web page in a manner same as in-situ concolic execution of back-end JS. We can achieve this by the following three steps: execution context extraction, execution context tracing customization (including symbolic value injection and tracing control), and concolic execution within execution context.

Fig. 4.
figure 4

Concolic testing of JS Web function within execution context

Full size image

Execution Context Extraction To transform a JS web function to a pure JS function without losing its context of a web page, we introduce a function interceptor to the JS testing framework to serve this purpose. As shown in Figure 4, the function interceptor completes the following tasks to finish this transformation in order to suit later in-situ concolic testing in the back-end:

  • First, the function interceptor requests the page frame detail of the web page where the targeted JS web function resides, utilizing the existing mocking data and the HTML render function. The mocking data and the HTML render function are usually created manually and included in the unit test suite.

  • Second, from the page frame detail, the function interceptor identifies the function body in a pure JS form given the function name. To preserve the JS function’s native web environment, it extracts the associated execution context of the web page. This is realized by calling helper functions provided by the testing libraries of the JS testing framework. The execution context contains everything that is needed for the pure JS function to be executed in the web page, which includes the arguments of the function, its concrete dependency objects set by mocking data and the function scope.

  • Third, the function interceptor delivers a complete function in the pure JS form encapsulated with its associated web execution context by assembling them, and then makes it accessible for the test runner of the JS testing framework so that the test runner can initiate the concolic execution in the execution context when running the test suite.

Execution Context Tracing Customization

In-situ concolic testing offers the capability of tracing inside the V8 JS engine to capture the execution trace that closely matches the JS bytecode interpretation [9, 22]. The conciseness of an execution trace determines the efficiency and the effectiveness of later symbolic analysis and test case generation. Therefore, to make the most of this capability, we pinpoint the locations of where to introduce symbolic values and start tracing during the extraction of the execution context, before we commence concolic testing on the encapsulated JS web function with its execution context. In-situ concolic testing provides interface functions for introducing the symbolic values (MarkSymbolic()) and tracing control (StartTracing()). We use these interface functions to customize execution context tracing as needed.

Fig. 5.
figure 5

How to avoid unnecessary tracing of the test runner setup by delaying injection of symbolic values and start of tracing

Full size image

Symbolic Value Injection and Tracing Control A JS testing frameworks uses a test runner to execute its test suites. As shown in Figure 5, the test runner prepares the dependencies for setting up the testing environment and loads the JS libraries the test suites need before starting run the individual function under test. In order to avoid tracing the unnecessary startup overhead of the test runner (indicated by the red box in Figure 5), we choose to inject symbolic values inside the execution context and start tracing when the test runner actually executes the encapsulated function, by calling the interface functions the in-situ concolic testing provides. This way the execution tracer only captures the execution trace of the encapsulated JS web function. The locations for injecting symbolic values and starting tracing are indicated in the “Execution Context (EC)” box in Figure 4 and the captured execution trace is indicated by the “Execution Trace” box in the right corner of Figure 4.

Fig. 6.
figure 6

How we obtain the most concise concrete execution trace

Full size image

Most Concise Execution Trace Figure 6 shows why our approach can obtain the most concise execution trace for the JS web function driven by the test runner of the JS testing framework. Apart from the overhead caused by the test runner, the extraction of the execution context for the JS web function involves calling a set of JS helper functions to collect web page information, such as helper_js_1 and JSHandle_js_1. If we directly apply symbolic execution within the test runner where the JS function is intercepted along with the execution context extraction, the execution tracer will also capture the execution traces of the test runner and the testing helper functions from the testing libraries shown as “Execution Trace 0” in the right-hand side of Figure 6. We modified the test runner to mark symbolic variables and enable tracing control within the execution context. Instead of starting tracing when the test runner starts, we defer the tracing of the execution to when and where the test runner actually executes the encapsulated function under test in the extracted execution context, indicated by the “Execution Trace 1” in the left-hand side of Figure 6. This way we minimize the extend of execution tracing needed.

Concolic Testing within Execution Context We leverage the test runner of the JS testing framework to initiate and start the in-situ concolic testing of the JS web function under test. Typically the test runner starts running the JS web function with an existing unit test. In our approach, the execution of the unit test triggers the function interceptor, which starts the process of extracting the execution context and encapsulating the target JS web function. During this process, symbolic values are injected and tracing is started in the right place as described in previous sections. The resulting pure JS application is then executed by in-situ concolic testing. Newly generated test data is fed back to the JS testing framework to drive further concolic testing.

4 Implementations

In this section, we demonstrate the feasibility of our approach to concolic testing of front-end JS functions by implementing it on two popular JS testing frameworks, namely Puppeteer and Jest assisted by the React testing library [14, 18].

4.1 Implementation on Puppeteer

Puppeteer is a testing framework developed by the Chrome team and implemented as a Node.js library [14]. It provides a high-level API to interact with headless (or full) Chrome. It can simulate browser functions using testing libraries. Puppeteer can execute JS functions residing in a web page without a browser. Puppeteer allows us to easily navigate pages and fetch information about those pages. In the implementation of our approach on Puppeteer, we augment it with the implementation of the function interceptor to identify the targeted web JS functions and extract their execution contexts from the web pages and encapsulate them for in-situ concolic testing.

Fig. 7.
figure 7

How Puppeteer executes a JS function in a web page

Full size image

Encapsulating JS Web Function with Execution Context As shown in Figure 7, Puppeteer communicates with the browser [15]. One browser instance can own multiple browser contexts. A Browser Context instance defines a browsing session and can have more than one pages. The Browser Context provides a way to operate an independent browser session [3]. A Page has at least one frame. Each frame has a default execution context. The default execution context is where the frame’s JavaScript is executed. This context is returned by frame.executionContext() method, which gives the detail about a page frame. We implement the function interceptor in the Execution Context class under the browser context to collect necessary information for encapsulating a JS function with its associated web execution context. The Execution Context class represents a context for JS execution in the web page. We modified it to identify the page function, its arguments and return value [5]. The pageFunction is the function in the HTML page to be evaluated in the execution context, which is in a pure JS form. For example, Listing 1.1 shows a front-end application example written with the Express web development framework [6]. This example contains a web page (from line 7 to line 17) with a JS web function marked by <script> tag in line 15. The ${path} points to the JS file that contains the implementation of the JS web function, as shown in Listing 1.2. Our approach is able to encapsulate the pure JS form of the web JS function (its implementation) with its associated web execution context.

figure a
figure b

Execution Context Tracing Customization We utilize the page.evaluate function of the Puppeteer testing framework to drive the JS function under test and extend it with the function interceptor. As described in Figure 8, to enable customized execution context tracing, the function interceptor introduces symbolic variables and set the starting point for tracing within the web execution context of the JS function wrapped by the <script> tag in the web page. This way, we make it possible for the test runner to initiate concolic testing when it starts running the test suites so that JS function can be tested concolically and automatically without tracing additional overheads. Since the Execution Context is triggered by the evaluate function in unit tests. We target applications from GitHub that uses Puppeteer to test front-end features and utilizes evaluate in unit testing. We will discuss the results later in Section 5.

Fig. 8.
figure 8

How we set symbolic variables in the execution context and enable customized execution context tracing in Puppeteer

Full size image

4.2 Implementation on Jest with React Testing Library

Another implementation of our approach is on the Jest testing framework assisted by the React testing library for unit testing. The React testing library is a lightweight library for testing React components that wrap the JS functions with the HTML elements [18]. As shown in Figure 9, there are three components in the application as indicated by the numbers. Components allow the splitting of a UI into independent, reusable pieces, and designing each piece in isolation. React is flexible; however, it has a strict rule: all React components must act as pure functions with respect to their inputs [16]. We refer to them as “functional components”. They accept arbitrary inputs (called “props”) and return React elements describing what should appear on the web page [17]. An individual component can be reused in different combinations. Therefore, the correctness of an individual component is important with the respect to the correctness of their compositions. In our implementation, we only consider components that have at least one input.

Fig. 9.
figure 9

Example React Components

Full size image

Jest has a test runner, which allows us to run tests from the command line. Jest also provides additional utilities such as mocks, stubs, etc., besides the utilities of test cases, assertions, and test suites. We use Jest’s mock data to set up the testing environment for the front-end components defined with React. Figure 10 shows how we leverage and extend Jest assisted by React testing library to apply the in-situ concolic testing to React component. To encapsulate the JS function in the component with its execution context, we augmented the render function, whose functionality is to render the React component function and props as an individual unit for Jest to execute from the web page, with the function interceptor. Through the render function, the function interceptor extracts a complete execution context for the functional component and intercepts the JS function wrapped in the functional component indicated by the arrows in Figure 10. To enable customized execution context tracing, the function interceptor then marks symbolic variables and starts tracing after the completion of the encapsulation. At last, we configure Jest’s test runner to run each unit test individually while initiating in-situ concolic execution so that we can obtain the most concise execution traces for later symbolic analysis.

Fig. 10.
figure 10

How to apply in-situ concolic testing on React components using Jest

Full size image

5 Evaluations

For evaluations, we apply our approach to in-situ concolic testing on front-end JS web application projects that come with unit test suites. They are utilizing Jest with React testing library and Puppeteer. In these evaluations, we target the String and Number types as symbolic variables for the functions under test.

5.1 Evaluation of Puppeteer Implementation on Github Projects

We have selected 21 GitHub projects utilizing Puppeteer. We test them using the Puppeteer framework extended with our concolic testing capability. As a result, we discovered 4 bugs triggered from their web pages and 2 of them originated from their dependency libraries.

Evaluation Setup We selected GitHub projects with the following properties as as our targets:

  1. (1)

    They use Puppeteer for unit testing of their JS web features;

  2. (2)

    They have JS functions in web pages and such functions have at least one argument whose type is string or number;

  3. (3)

    They utilize evaluate in their unit tests.

We have developed a script based on such properties and used the searching API provided by GitHub to collect applicable projects [20]. 21 projects were collected. Table 1 summarizes the demographics of the 21 GitHub projects collected by our script. We calculated the statistics using ls-files [7] combined with cloc provided by GitHub [8]. The LoC/JS is the LoC (lines of code) of all JS files, which includes the JS files of the libraries the project depends on. The LoC/HTML is the LoC of HTML files, which indicates the volume of its front-end web contents. The LoC of unit tests (LoC/unit test) includes the unit test files ending with .test.js. The test ratio is the ratio between the LoC/unit test over the LoC/JS, indicating the availability of unit tests for the projects. Before evaluation, we configure these projects to use the extended Puppeteer framework instead of the original one.

Table 1. Selected Projects that utilize Puppeteer for unit testing
Full size table

Result Analysis We ran each project with our approach for 30 minutes. On average, our implementation generates 200 to 400 test cases for each function. Table 2 summarizes the bugs detected. For polymer, our method generates two types of test cases that trigger two different bugs in user password validation functionalities of the project: 1) a generated test case induces execution to skip an if branch, which causes the password to be undefined, leading to the condition !password || this.password === password to return true, which should have returned false. We have fixed this bug by changing the operator || to &&. 2) test cases containing unicode characters fail password pattern matching using regular expression without g flag, i.e., .test(value). For InsugarTrading, a test case of a string not containing comma is generated for str.split(’,’) function. The return value of an empty array causes errors in the dependency library cookie-connoisseur. A number out-of-bound error is discovered in the changeCell() function of TicTacToe. For phantomas, function phantomas has a check for url to be the string type but does not have pattern matching for it. A generated test case with an invalid url causes an exception in function addScriptToEvaluateOnNewDocument of chromeDevTools.

Table 2. Bugs detected in web applications using Puppeteer from Github
Full size table

We identified two traits of the projects for which we did not detect bugs in. (1) A project does not fit the design of our Puppeteer implementation, i.e., evaluate is not used in the test suite. (2) The applicable JS part is small and well tested.

5.2 Evaluation of Jest Implementation on Metamask

In evaluation of the implementation of our concolic testing approach on Jest, we focus on Metamask’s browser extension for Chrome. MetaMask is a software crypto-currency wallet used to interact with the Ethereum blockchain. It allows users to access their Ethereum wallet through a browser extension or mobile app, which can then be used to interact with decentralized applications [12]. Metamask extension utilizes the render functionality for testing JS functions in React components. We focus on front-end JS web functions, React component functions in particular. They reside in the ui folder of the metamask-extension project.

Testing Coverage Statistics of Metamask We select the ui folder as our evaluation target for two reasons: (1) React components of metamask-extension are mostly defined and implemented under this folder; (2) the functions in this folder is under tested. Figure 11 shows the current testing coverage statistics of the ui folder of metamask-extension [1]. We can see that only one sub-folder of ui (which also happens to be named as ui) has a relatively high coverage of 82.03%. Most other folders have coverage under 70% or even lower coverage.

Fig. 11.
figure 11

Coverage statistics of ui folder of Metamask-extension

Full size image

Evaluation Setup In the unit testing workflow of metamask-extension, there is a global configuration for all unit test suites of UI components. This is because one component’s functionality may depend on other components. Therefore, metamask-extension needs to be executed as an instance to support unit testing. To evaluate the implementation of in-situ concolic testing for React components, we need an independent environment for each component function wrapped with a single test file. This test file only contains one function under test. Therefore, each test file is an independent in-situ concolic testing runner for a function in a component. We implement an evaluation setup script to complete this task. This script automatically prepares the evaluation environment for in-situ concolic testing of a React component. Specifically, it does the following work under the folder where the target component resides:

  • Jest Configuration. Configure Jest for the individual component test file with an independent jest.config.js

  • Babel Configuration. Configure Babel for the component test file to take JS native syntax, which is required by in-situ concolic testing. This is because metamask-extension JS source files are transformed using Babel.

  • Dependency Installation. Collect and install dependencies for the target component. Such dependencies can be components or libraries.

Result Analysis After we set up the evaluation environment, we can conduct our evaluation in a sandbox on the test network of Metamask. We have uncovered 3 bugs and 1 test suite improvement as shown in Table 3. We have filed them as bug reports through GitHub. They have been accepted by Metamask developers. Along the way, we also found some similar test cases that Metamask’s bot reported.

Table 3. Bugs Detected in Metamask under UI folder
Full size table

For the buy-eth feature as shown in Figure 12, a test network error with a respond code of 500 was triggered when testing the Ether deposit functionality. Concolic testing generates a test case of an invalid chainId for buyEth(), which is defined in the DepositEtherModal component. It is wrapped by a <Button> tag and can be triggered by onClick(). buyEth() calls into buyEthUrl(), which retrieves a url for buyEth() function. Because buyEthUrl() did not check if the url is valid or null before it calls openTab(url) with the returned url. And there is also no validation for input in the component implementation. Additionally, this process was not wrapped in a try/catch block. We caught this error in our evaluation. We tested 16 component folders and discovered that metamask-extension most likely will ignore input checking if inputs are not directly from users. chainId is retrieved from mock data in this case, which is generated by our concolic engine.

Fig. 12.
figure 12

Error trace of the bug discovered in buy-eth

Full size image

For the token-search feature, we uncovered a bug triggered by an empty string. In the TokenSearch component, function handleSearch() is wrapped by <TextField> with onChange method. It calls isEqualCaseInsensitive() with an empty string as its second argument without boundary checking. Function isEqualCaseInsensitive is defined in utils.js, which provides shared functions. We found that the unit testing for utils.js do not have test suites for that function, while the same bug is not found in the experiment conducted on the send.js file. In send.js, function validateRecipientUserInput also calls the incorrect function isEqualCaseInsensitive. However, since send.js checks for both empty string and null inputs before calling the faulty function, it avoids the potential error in utils.js.

For the ens-input feature, in the onChange method of component EnsInput’s <input/>, the function isValidDomain is called. Our approach generated test cases with unacceptable ASCII characters in the domain name, e.g., %ff.bar. We replay this test case, function isValidDomain returns true when it should return false. In Listing 1.3, function isValidDomain returns the value of the condition match !== undefined. This test case made through regex matching and returned null but null is not equal to undefined in JS.

figure d

For the advanced-gas-fee feature, we found the updateGasLimit(gasLimit) function (expecting a numeric input) in the <FormField> component has wrong behavior when given a string input containing only digits such as “908832”. The function simply sets the gas limit to 0 without emitting error. We do not consider this as a bug since component <FormField> restricted the input to be numeric in the HTML element. After we filed it, this has been marked with the area-testSuite tag on GitHub by developers as a test suite improvement.

6 Related Work

Our approach is closely related to work on symbolic execution for JS. Most of them aim at back-end/standalone JS programs, primarily target specific bug patterns and depend on whole-program analysis. Jalangi works on pure JS programs and instruments the source JS code to collect path constraints and data for replaying [38]. COSETTE is another symbolic execution engine for JS using an intermediate representation, namely JSIL, translated from JS [36]. ExpoSE applies symbolic execution on standalone JS and uses JALANGI as its symbolic execution engine. ExpoSE’s contribution is in addressing the limitation that JALANGI has, which is to support regular expressions for JS [33]. There are few symbolic analysis frameworks for JS web applications. Oblique injects symbolic JS library into the page’s HTML. When a user loads the page, it conducts a symbolic page load to explore the possible behaviors of a web browser and a web server during the page load process. It generates a list of pre-fetch url for client-side to speed up page load [30]. It is an extension of the ExpoSE concolic engine. SymJS is a framework for testing client-side JS script and mainly focus on automatically discovering and exploring web events [31]. It modifies Rhino JS engine for symbolic execution [19, 27]. Kudzu targets AJAX applications and focuses on discovering code injection vulnerabilities by implementing a dynamic symbolic interpreter that takes a simplified intermediate language for JS [37]. To the best of our knowledge, there has been no publicly available symbolic execution engines targeting JS functions embedded in front-end web pages [32].

Another related approach to JS testing is fuzzing, which typically uses code coverage as feedback to test generation. There are a few fuzzers for JS, e.g., jsfuzz [11] and js-fuzz [10], which are largely based on the fuzzing logic of AFL (American fuzzy lop) [2] and re-implemented it for JS. We view fuzzing and symbolic/concolic testing as complementing techniques: fuzzing for broader exploration of JS while symbolic/concolic testing for deeper exploration.

7 Conclusions

We have presented a novel approach to apply concolic execution to front-end JS. The approach makes use of an in-situ concolic executor for JS and leverages the functionality of JS testing frameworks as test runners and web content extractors. Our approach works in three steps: (1) extracting JS functions from web pages using with JS testing framework; (2) integrating the in-situ concolic testing interface in the execution context for the JS Web functions; (3) utilizing the testing framework’s test runner and its mock data as the driver for concolic execution to generate additional test data for the JS web function under test.

We have conducted evaluation on open-source projects from Github and on Metamask’s UI features, which are proper targets for our implementations on Puppeteer and Jest respectively. We have found bugs in each evaluation, whose bug reports have been accepted on GitHub. This contributes to both bug finding and test suite improvement for the applications tested. The results show that our approach to concolic testing frontend JS is both practically and effective.