Generalization of machine learning for problem reduction: a case study on travelling salesman problems

Abstract

Combinatorial optimization plays an important role in real-world problem solving. In the big data era, the dimensionality of a combinatorial optimization problem is usually very large, which poses a significant challenge to existing solution methods. In this paper, we examine the generalization capability of a machine learning model for problem reduction on the classic travelling salesman problems (TSP). We demonstrate that our method can greedily remove decision variables from an optimization problem that are predicted not to be part of an optimal solution. More specifically, we investigate our model’s capability to generalize on test instances that have not been seen during the training phase. We consider three scenarios where training and test instances are different in terms of: (1) problem characteristics; (2) problem sizes; and (3) problem types. Our experiments show that this machine learning-based technique can generalize reasonably well over a wide range of TSP test instances with different characteristics or sizes. Since the accuracy of predicting unused variables naturally deteriorates as a test instance is further away from the training set, we observe that, even when tested on a different TSP problem variant, the machine learning model still makes useful predictions about which variables can be eliminated without significantly impacting solution quality.

Appendix A: Random sampling method for sequential ordering problem

The main steps of our random sampling method to generate one feasible route for SOP can be summarized as follows:

• 1. Initialize a route starting from city 1;

• 2. Compute a set of candidate cities $V_c$ that do not have any precedence after removing the cities that have already been visited;

• 3. Randomly select a city from the candidates $V_c$ to visit;

• 4. Repeat Steps 2 and 3 until all cities have been visited.

To avoid redundant computation, we first iterate through the set of precedence constraints ${\mathcal {S}}$ to count the number of cities that should be visited before visiting city i ($i=1,\ldots ,n$) and store this in array A. We also store the individual cities that should be visited after city i ($i=1,\ldots ,n$) in a linked list L (lines 3–6 in Algorithm 3). Having A and L, we can efficiently update the set of candidate cities $V_c$ that can be visited in the next step after removing the cities already visited (lines 14–17 in Algorithm 3). The idea is that after removing city v in the current step, we iterate through the linked list L[v] and for every $v'$ in L[v], we decrement $A[v']$ by 1. If $A[v']$ is equal to 0, then city $v'$ can be visited in the next step since it does not have any precedence apart from the cities already visited. By doing this, we can generate one sample route in ${\mathcal {O}}\big (|{\mathcal {S}}|\big )$ time. Thus, the total time complexity of generating m samples is ${\mathcal {O}}\big (m|{\mathcal {S}}|\big )$.