Local search versus linear programming to detect monotonicity in simplicial branch and bound

Abstract

This study focuses on exhaustive global optimization algorithms over a simplicial feasible set with simplicial partition sets. Bounds on the objective function value and its partial derivative are based on interval automatic differentiation over the interval hull of a simplex. A monotonicity test may be used to decide to either reject a simplicial partition set or to reduce its simplicial dimension to a relative border (at the boundary of the feasible set) facet (or face) by removing one (or more) vertices. A monotonicity test is more complicated for a simplicial sub-set than for a box, because its orientation does not coincide with the components of the gradient. However, one can focus on directional derivatives (DD). In a previous study, we focused on either basic directions, such as centroid to vertex or vertex to vertex directions, or finding the best directional derivative by solving an LP or MIP. The research question of this paper refers to using local search (LS) based sampling of directions from vertex to facet. Results show that most of the monotonic DD found by LP are also found by LS, but with much less computational cost. Notice that finding a monotone direction does not require to find the direction in which a derivative bound is the steepest.

1 Introduction

Global optimization branch and bound methods aim at finding the set of global minimum points with a given guarantee. Specific methods are based on interval arithmetic [7]. Such methods guarantee to reach a so-called rigorous solution for problems that are typically limited to less than $n=10$ decision variables. All kinds of tools like monotonicity tests, constraint propagation, various lower bounds etc. may speed up the convergence. As the methods are based on box shaped partition sets, they do not directly fit to a simplicial feasible set.

Practical global optimization problems with a simplicial set as feasible region are among others blending and portfolio optimization problems. Optimization over a simplex is useful for many modern applications in machine learning, signal processing, control and game theory, see references in [12]. We specifically investigated the design of simplicial branch and bound for low dimensional mixture design problems to reach a guaranteed solution, [3]. For an overview of simplicial branch and bound we refer to the book [14]. One of the questions not touched in the described simplicial methods is the extension of the so-called monotonicity test of interval branch and bound methods to simplicial solution sets.

The use of a monotonicity test in global optimization branch and bound over a simplex has been addressed recently [2, 6, 8]. One of the main goals is to reject a simplicial partition set or to reduce it to a so-called border facet by finding a monotonic directional derivative (DD) from the vertex opposite to the facet towards the facet. In previous studies, naive directions, such as from vertex to the centroid of the simplex or other vertices, were used with good results [5]. Additionally, linear programming (LP) models and mixed-integer linear programming (MILP) models were formulated, which find the optimal (the steepest) DD bound in most of the cases [2]. However, it is time consuming to solve several LP-s or even just one MIP every time a monotonic DD is sought.

Our research question is whether a local search (LS) algorithm to find a monotonic DD over a simplex from the information provided by the gradient enclosure over the interval hull of the simplex may do the job more efficiently. The LS algorithm has to be computationally cheap with a stopping criterion that keeps track of convergence in order to stop if the monotonic DD will not be reached in a few steps. Moreover, we provide a LS algorithm that looks for a monotonic DD, regardless of its sign.

The rest of the paper is organized as follows. Notation is introduced in Sect. 2. Section 3 shows mathematical properties on monotonicity over simplices. Section 4 describes the evaluated DD with a direction based on the centroid, the vertices, LP and LS for a given facet. Section 5 shows numerical results for a practical simplicial branch and bound algorithm over a set of test instances, and Sect. 6 summarizes our findings.

2 Notation

We consider the minimization of a continuously differentiable function $f:\mathbb R^n\rightarrow \mathbb R$, over a feasible set $\Delta$, which is a $p-$simplex, i.e. $\Delta :={{\,\textrm{conv}\,}}(\mathcal W)$ is defined by a set of $p+1$ affine independent vectors that serve as vertices

$$\begin{aligned} \mathcal W:=\{v_0,\ldots ,v_{p}\}\subset {\mathbb {R}}^n, p\le n. \end{aligned}$$

(1)

The idea is to find or enclose all global minimum points of

$$\begin{aligned} \min f(x),x\in \Delta . \end{aligned}$$

(2)

The consideration of $n-$dimensional functions over $p\le n$ lower dimensional simplicial feasible area appears for instance in blending problems [3]. Our context is that of a branch and bound algorithm which encloses all minimum points of f on $\Delta$. The used partition sets are $m-$simplices S, where $m\le p$, i.e. their dimension can be lower than that of the feasible set, in contrast to the algorithms described in [14]. This means $S:={{\,\textrm{conv}\,}}(\mathcal V)$ with $\mathcal V$ a set of $m+1=|\mathcal V|$ vertices. The branch and bound algorithm works with a set $\Lambda$ of partition sets, which as a whole include all global minimum points.

Although we usually limit our context to the use of longest edge bisection, where the longest edge (v, w) of a partition set S is bisected using mid-point $x:=\frac{v+w}{2}$, we pose the monotonicity question in a larger context where any partition method may be used, as described in [9].

Specifically, we focus on dimension reduction due to monotonicity considerations, where a set $\mathcal V$ of vertices of $m-$simplex S is reduced to $\mathcal V\setminus \{v\}$ and S is replaced by one (or more) of its facets $F:={{\,\textrm{conv}\,}}(\mathcal V\setminus \{v\})$ for some $v \in \mathcal V$. Notice that F is an $(m-1)$-simplex. It may be clear that for $m=0$, the $0-$simplex $S={{\,\textrm{conv}\,}}(\{v\})$ is an individual point and does not have faces. Its dimension cannot be reduced.

The centroid of $m-$simplex $S= {{\,\textrm{conv}\,}}(\{v_0,v_1,\ldots ,v_m\})$ is given by $c=\frac{1}{m+1}\sum _{j=0}^m v_j$ and the relative interior is defined by

$$\begin{aligned} {{\,\textrm{rint}\,}}(S)=\{x=\sum _j\lambda _jv_j, \lambda _j>0, j=0,\ldots ,m,\sum _{j=0}^m \lambda _j=1\}. \end{aligned}$$

(3)

The relative boundary of a simplex S is defined by removing the relative interior from it. Given a simplicial partition set S, we are interested in whether its (simplicial) facets F are border with respect to the feasible set $\Delta$. In general, we can define a simplex to be border with respect to a simplicial feasible set.

Definition 1

Given p-simplex feasible area $\Delta$. An $m-$simplex S with $m<p$ is called border with respect to $\Delta$ if there exists an m-simplex face $\varphi$ of $\Delta$, such that $S\subseteq \varphi$.

Binary labelling of vertices can be used to to keep track of border facets, as discussed in [2].

Relevant information of sub-set S is an enclosure $\varvec{g}$ of the gradient $\nabla f(x)\subseteq \varvec{g}:=[\underline{\varvec{g}},\overline{\varvec{g}}], \forall x \in S$. Interval vector $\varvec{g}$ can be calculated by Interval Automatic Differentiation [13, 15] over the interval hull of a simplex S ($\Box S$) which implies under and over estimations. Enclosure $\varvec{g}$ indicates the traditional interval enclosure, but our reasoning is still valid for any other rigorous and sharper enclosure method for $\nabla f(S)$. Now consider directional vector $d=x-y$ as the difference between two points $x,y\in S$. Then, the corresponding DD, i.e. $d^T\nabla f(x)$, is also included in the interval

$$\begin{aligned} \varvec{h}=d^T\varvec{g}=\left[ \underline{\varvec{h}},\overline{\varvec{h}}\right] = \left[ \sum _{i=1}^{n} \min \{d_i\underline{\varvec{g}}_i, d_i \overline{\varvec{g}}_i\}, \sum _{i=1}^{n} \max \{d_i\underline{\varvec{g}}_i, d_i \overline{\varvec{g}}_i\}\right] . \end{aligned}$$

(4)

We consider example 6 (EX6) from [10] in Fig. 1, $f(x_1,x_2)=0.25x_1^2+x_1+x_2+0.25x_1x_2+0.5x_2^2$ over the simplex $S=\{v_0 = (-0.75,-1.5), v_1 = (-1,1), v_2 = (0.25,-0.5)\}$. A gradient inclusion over box $([-1,0.5],[-1.5,1])$ is $\varvec{g}= ([0.125, 1.375], [-0.75,2.0625])$. As the figure shows, it means that following a first order Taylor idea, we have bounds starting from the center of the box, $c=(-0.375,0.25)$. From the center, we also have bounds on the directional derivative in direction $d=x-c$. The illustration shows that the directional derivative is typically piecewise linear in the argument x, where the breaking points or planes are defined by the sign changing planes of d where $x_i=c_i$.

Fig. 1

Piecewise linear character of directional derivative bounds in x for $d=x-c$

Our study focuses on the question of monotonicity, where we can show that there exist points x, y, such that the inclusion of the directional derivative does not contain zero. Consider

$$\begin{aligned} \varvec{h}(x,y)=(x-y)^T\varvec{g}, \end{aligned}$$

(5)

as a function of $d=x-y, x,y\in S$.

Figure 2 shows an illustration of $\varvec{h}(x,y)$ using example 6 (EX6): $f(x_1,x_2)=0.25x_1^2+x_1+x_2+0.25x_1x_2+0.5x_2^2$ in [10], over the simplex $S=\{v_0 = (-0.75,-1.5), v_1 = (-1,1), v_2 = (0.25,-0.5)\}$. An inclusion of the gradient is given by $\varvec{g}= ([0.125, 1.375], [-0.75, 2.0625]$. This simplex has number 9 in Fig. 9. Thus, f is monotonically increasing to the right direction in any point of $\Box S$ and therefore also in S. As directional vector we fix y in the centroid of $\Box S$ and vary x over the unit circle around y. When varying x over the circle, the lower bound $\underline{\varvec{h}}$ is piece-wise convex and $\overline{\varvec{h}}$ is piece-wise concave.

Fig. 2

Example of $\varvec{h}(x,y)$ with y the centre of the $\Box S$ and x in $d=x-y$ varies over the unit circle with centre on y

In Fig. 3 we vary x over a polytope, specifically an octagon. It illustrates the piecewise linear character of the directional derivative bounds when we search over a hyperplane. This means it has breakpoints caused by $x_i=y_i$. Moreover, breakpoints are caused in this case by other vertices of the octagon. A minimum or maximum value will also occur in one of the breakpoints. This illustration shows that we might be looking for a monotonic direction d to approach a maximum value for $\underline{\varvec{h}}$ (positive $\varvec{h}$) or minimum for $\overline{\varvec{h}}$ (negative $\varvec{h}$).

Fig. 3

Example of $\varvec{h}(x,y)$ with y the centre of the $\Box S$ and x in $d=x-y$ varies over an octagon inscribed in the unit circle with centre on y

Following the same example, Fig. 4 shows $\varvec{h}$ with direction $d=x-v$, $x\in$ facet $F = {{\,\textrm{conv}\,}}(\mathcal V\setminus \{v\})$. The interesting values for directions d are those that correspond to $\underline{\varvec{h}}>0$ or $\overline{\varvec{h}}<0$. A monotonic direction may lead to reject S or to reduce it to a border facet(s). In this case, the only facet with a monotonic directional derivative bound $\varvec{h}$ is that leaves out $v_2$ with a minimum to the left direction from $v_2$. In such a graphical example one can find a monotonic direction over a facet by running x over the facet, as illustrated in the figure for $n=2$. In higher dimensions, finding a direction in which f is monotonic is not obvious.

Fig. 4

Example of $\varvec{h}(x,v_j)$ for each simplex vertex $v_j, j=0,1,2$. Endpoint x is varied over the opposite facet $F= {{\,\textrm{conv}\,}}(\mathcal V{\setminus }\{v_j\})$ using example 6 in [10] over the simplex $S=\{v_0 = (-0.75,-1.5), v_1 = (-1,1), v_2 = (0.25,-0.5)\}$. The gradient bounds are $\varvec{g}= ([(0.125,-.75)^T, (1.375, 2.0625)^T])$

3 Mathematical properties on monotonicity

The monotonicity is based on the directional derivative bounds $\varvec{h}$ (see (4)). Notice that condition $0\notin \varvec{g}$ is necessary to have monotonicity, but not sufficient. The question is which direction d to consider. The most general result for an $m-$simplex is the following.

Proposition 1

[2] Let $S\subseteq \Delta$ be an $m-$simplex with gradient enclosure $\varvec{g}$. If $\exists \ x,y\in S$, such that direction $d=x-y$ has corresponding directional derivative bounds (4) with $0\notin \varvec{h}$ then ${{\,\textrm{rint}\,}}(S)$ does not contain a global minimum point of (2).

For the proof, see [2].

Corollary 1

[2] Let $S\subseteq \Delta$ be an $m-$simplex as partition set in a branch and bound algorithm with corresponding gradient enclosure $\varvec{g}$. If the conditions of Prop. 1 apply and S has no border facets, then S can be rejected.

The argument is that the relative boundary of S may contain a global minimum point, but the same point is enclosed in the relative boundary of another partition set.

Given that the minimum is not in ${{\,\textrm{rint}\,}}(S)$, we have to decide which of the facets to focus on. In this paper, we focus on fixing one of the endpoints v and look for $d=(x-v)$, $x\in S$, such that $0\notin \varvec{h}$.

Proposition 2

[2] Given $m-$simplex $S={{\,\textrm{conv}\,}}(\mathcal V)$ and a facet F generated by removing vertex v from $\mathcal V$. Consider direction $d=x-v$ and $\varvec{h}=d^T\varvec{g}$. If $\overline{\varvec{h}}<0$, then the facet F contains all minimum points in S, i.e. ${{\,\textrm{argmin}\,}}_{x\in S}f(x)\subseteq F$.

Practically, Prop. 2 means that S can be replaced by F. However, a similar reasoning applies as in Corollary 1; if F is a non-border facet, then simplex S can be removed from further consideration in a branch and bound context. The idea is again that faces of F may contain the global minimum. However, because we are dealing with a partition, the same points are also included in other simplicial partition sets.

Proposition 3

Given $m-$simplex $S={{\,\textrm{conv}\,}}(\mathcal V)$ and a facet F generated by removing vertex v from $\mathcal V$. Consider direction $d=x-v$ and $\varvec{h}=d^T\varvec{g}$. If $\underline{\varvec{h}}>0$, then the facet F (even border) does not contain minimum points in S and S can be replaced by the union of its other border facets (see Prop. 1).

The argument uses the opposite direction in Prop. 2.

It is more appealing to find a negative bound $\varvec{h}$ with d to a facet because we can reduce S to that facet if it is border or reject S otherwise. However, to find a positive direction for $\varvec{h}$ to a facet is also interesting because we can reduce the simplex to all other border facets, excluding the current one.

4 Directional derivatives to evaluate

We focus on finding a monotonic DD for a direction $d=x-v$ from vertex $v\in S$ towards a simplex facet $F={{\,\textrm{conv}\,}}(\mathcal V{\setminus } \{v\})$. The first heuristic directions to consider is taking $x=c$ (centroid of the facet) and the other vertices in $\mathcal V{\setminus } \{v\}$. These are described in Sect. 4.1.

Based on these considerations, we define steps for the branch and bound algorithm with a certain order to check various directions on monotonicity. The algorithm will check the centroid (C) direction for each facet of S and checks vertex (V) directions in case a negative $\varvec{h}$ was not found. A further refinement considering LP or LS methods will be done after C+V only if a negative $\varvec{h}$ was not found for any facet. Notice that these tests are only relevant if $0\notin \varvec{g}$, which is a necessary condition for monotonicity. The order of the tests is described as follows.

• 1. S is full dimensional without border facets and $0\notin \varvec{g}$. Reject S.

• 2. For non-border facet F, $\exists d=x-v$ for which $\varvec{h}<0$. Reject S. See Prop. 2 and Corollary 1.

• 3. There are border facets F having $\varvec{h}<0$. Reduce S to the intersection of those facets. See Prop. 2.

• 4. There are facets F with $\varvec{h}>0$. Replace S by the union of its border facets. See Prop. 3. If there are no (other) border facets, reject S; Corollary 1.

• 5. We have for S that $0\notin \varvec{g}$ but for all facets, and all tested direction, $0\in \varvec{h}$:

  • (a) S is full dimensional. Replace S by the union of its border facets. Minimum can neither be in ${{\,\textrm{rint}\,}}(S)$ and in non-border facets of S.

  • (b) S is not full dimensional. Monotonicity cannot be proved; divide S.

We summarize an LP problem which finds a positive bound on all directional derivatives in Sect. 4.2 and describe an LS method in Sect. 4.3. LP or LS will continue checking monotonicity for all facets when no $\varvec{h}<0$ was found towards a facet using naive C and V directions. Using C+V (CV) we can find more than one facet with a negative $\varvec{h}$. LP or LS will stop after finding the first negative $\varvec{h}$ in order to reduce their impact in the computational burden.

We define a monotonicity measure for non-monotone DD-s, that is, when $0\in \varvec{h}$ as

$$\begin{aligned} \mu (\varvec{h})=\min \{\overline{\varvec{h}},-\underline{\varvec{h}}\}, \end{aligned}$$

(6)

which can be seen as the minimum distance in the interval $\varvec{h}$ to escape from zero.

In order to reduce the number of facets to be checked by LP and LS, facets with the lowest value of $\mu (\varvec{h}(c,v))$, where c the centroid of facet $F={{\,\textrm{conv}\,}}(\mathcal V\setminus \{v\})$, are considered first.

4.1 Naive directional derivatives using centroid and vertices

Having at least one coordinate i with $0\notin \varvec{g}_i$, the algorithm first checks monotonicity considering as directions

• 1. C: $d=(c-v)$, with c the centroid and v the opposite vertex of F, for each facet $F\in S$. If no negative value for $\varvec{h}$ is found, we check

• 2. V: directions towards vertices, $d=(w-v), v,w \in \mathcal V.$

The algorithm evaluates all $\left( {\begin{array}{c}m\\ 2\end{array}}\right)$ combinations of vertices in $\mathcal V$ without repetition for d in $\varvec{h}$ in lexicographical order. If $\varvec{h}(v,w) > 0$ for any $v,w \in \mathcal V$, obviously $\varvec{h}(w,v) < 0$. So every time a monotone direction is found, we can reduce the dimension (Prop. 1) or even reject the simplex (paragraph after Prop. 2).

4.2 The linear programming model

To prove that there exists a monotone direction in an $m-$simplex, at least we should have $0\notin \varvec{g}$. This is a necessary, but not sufficient condition for an $m-$simplex, $m<n$.

We can search for any monotone direction $d=(x-y)$, $x,y \in S$ to find $0 \not \in \varvec{h}(x,y)$. By fixing $y=c$ we reduce the degree of freedom. If any d is found with $\varvec{h}>0$, we know that the interior of the simplex can be removed. The corresponding LP is formulated in [2], as LP(6).

Regarding Prop. 2, we can focus on finding $\varvec{h}(x,v)<0$ in direction $d=(x-v)$, $x\in F$ and $F={{\,\textrm{conv}\,}}(\mathcal V\setminus \{v\})$:

$$\begin{aligned} 0 > \overline{\varvec{h}} = \sum _{i=1}^{n} \max \{d_i\underline{\varvec{g}}_i, d_i \overline{\varvec{g}}_i\}. \end{aligned}$$

(7)

Finding a direction d which solves (7) can be done by searching for the steepest monotone direction $\min _{d} \overline{d^T\varvec{g}}$. The corresponding LP is formulated in [2], as LP(7).¹ Notice that we do not necessarily need to know the minimum in (7), but whether there exists $x\in F$ for which $\overline{\varvec{h}}<0$.

As mentioned in Sect. 4, finding $\varvec{h}>0$ instead of $\varvec{h}<0$ in (7) is also useful. However, looking for $\varvec{h}>0$ requires to solve another LP and we do not consider it in the experiments due to the large computational cost of solving a large number of LP models. Nevertheless, it may happen that looking for $\varvec{h}<0$, we end up in $\varvec{h}>0$, i.e. min $\overline{\varvec{h}}$ results in $\underline{\varvec{h}}>0$. In order to save LP computations, we neither do LP(6) in the experiments, because a successful result of LP(7) makes LP(6) unnecessary.

LP can be used as the best solution to find a monotone direction, but it might not be computationally the best. This is investigated in the experiments of Sect. 5.

4.3 Local search algorithm

The LS focuses on a simplex $S={{\,\textrm{conv}\,}}(\mathcal V)$ and one of its facets $F={{\,\textrm{conv}\,}}(\mathcal V\setminus \{v\})$. Consider a set $\mathcal P$ of already evaluated points $x\in F$ with respect to $\varvec{h}(x,v)$. This means, after checking the centroid and other vertices, we have $\mathcal P=\{\mathcal V\setminus \{v\},c_0\}$. Each iteration generates a new endpoint x based on a centroid $c_k$ of several points in $\mathcal P$, which replaces one of the points in $\mathcal P$ trying to reduce the non-monotonicity measure in (6). In this way, we look for a bound $\varvec{h}=\varvec{h}(x,v)$ with a low value of $\mu (\varvec{h})$ regardless of the sign of $\varvec{h}$, see Props. 2 and 3.

Algorithm 1 shows the local search for a monotone direction towards a facet determined by removing v from $\mathcal V$.

LS $(\ S,\ \varvec{g},\ v,\ n,\ m$)

Fig. 5

Graphical example of one iteration of Algorithm 1. Facet $F={{\,\textrm{conv}\,}}(\{v_1,v_2,v_3\})$ in a 3-simplex S is shown in black. Already evaluated directions $d=(v_0-x)$, $x\in \mathcal P=\{v_1,v_2,v_3,c_0\}$ are drawn as dashed red arrows. The candidates $x_j$ for new centroid $c_1$ and their direction $(v_0-x_j)$ are drawn as dashed blue arrows. For instance, $x_2$ is the centroid of $\{v_1,v_2,c_0\}=\mathcal P\setminus \{v_3\}$. The green point shows the endpoint with the best $\mu$ among the candidates $x_j$ which becomes $c_1$. In the next iteration, $\mathcal P=\{v_1,v_2,c_0,c_1\}$ to get $c_2$. The bounds $\varvec{h}(c,v_0)$ with $c=c_0, c_1, c_2$ are used in EstRemIters() to estimate the number of remaining iterations and to check whether we have slow convergence

For the stopping criterion, Algorithm 2 estimates the remaining number of iterations based on the progress of the bound $\varvec{h}(x,v)$ for the last three centroids $x=c_{i-2},c_{i-1},c_i$ in Alg. 1, in a similar way as the box-rejection idea in [1]. If the estimate is bigger than the number of endpoints in $\mathcal P$ ($|\mathcal P|=m+1$) in Algorithm 1, the search is stopped. Figure 6 shows an example using facet 0 of the simplex in Fig. 4. The algorithm stops at iteration 3 because the estimated number of remaining iterations from the current iteration is 9. At iteration 2 that number was 3. The total number of iterations of the local search is also limited in line 12.

EstRemIters($\varvec{h}_{k-2}, \varvec{h}_{k-1}, \varvec{h}_k$)

Fig. 6

Example of estimating number of remaining iterations by Algorithm 2 on facet 0 in Fig. 4. At iteration 2 $(c_0,c_1,c_2)$, UpRemIt$_1$=11, UpRemIt$_2$=21, LoRemIt$_1$=3 and LoRemIt$_2$=5 for EstRemIters$(\varvec{h}_{0},\varvec{h}_{1},\varvec{h}_2)$ resulting in $3\le |\mathcal P|$. At iteration 3 $(c_1,c_2,c_3)$, UpRemIt$_2$=21, UpRemIt$_3$=41, LoRemIt$_2$=5 and LoRemIt$_3$=9 for EstRemIters$(\varvec{h}_{1},\varvec{h}_{2},\varvec{h}_3)$. The minimum is 5 $>|\mathcal P|=3$ and LS stops

In Fig. 6, one first checks centroid $c_0$, and then the two vertices. Then LS checks another 7 directions. So, in total 9 DD were checked. LS on facet 0 is an example were the LS stops without finding a monotonotonic direction. Figure 7 shows an example were the algorithm finds a monotonic direction for facet 2. In fact, facet 0 needs not be checked, as facet 2 will be visited first, because the smallest non-monotonicity measure among the vertex to center directions was attained for facet 2, i.e. $v_2={{\,\textrm{argmin}\,}}_{w} \mu (\varvec{h}(w,c))$ with c the centroid of facet $F={{\,\textrm{conv}\,}}(\mathcal V\setminus \{v\})$. The LS found a negative directional derivative and therefore finishes after checking facet 2 on this simplex.

Fig. 7

Example of LS on facet 2 of simplex in Fig. 4

5 Numerical results

Algorithms were run on an Intel(R) Core(TM) i7-4770K CPU and 16GB of RAM running Fedora 36 Linux distribution. The algorithm was coded with g++ (gcc version 12.1.1) and uses Kv$-$0.4.50 for Interval Arithmetic. Kv uses boost libraries. Algorithms were compiled with -O3 -DNDEBUG -DKV_FASTROUND options. For the Linear Programming, we use PNL 1.10.4 compiled with -DCMAKE_BUILD_TYPE = Release and -DWITH_MPI = OFF, as a C++ wrapper to 5.5.2.0-28. LPsolve is free but it is slower than other commercial solvers². PNL uses by default set_break_numeric_accuracy(lp, 5e-7) in lp_lib.c and sets default parameter values and tolerances to ProductionPARAM in lp_lib.h. In order to reduce the computational time of LPSolve, we set its verbosity to "SEVERE", and time out after 1 s. We use the algorithm with lower bounding +CFvs from [5], which refers to a lower bound calculated by the central form using the vertex with maximal objective function value. The termination accuracy is $\alpha =10^{-6}$.

The used test instances are similar as in [5] and are given in Table 1. Box constrained problems are transformed to simplicial constrained problems where the minimum is at the centroid of one of the facets of a simplex of size half of the bounding box. For instance, problem GP2 is box constrained on $([-2,2],[-2,2])$ and the minimum is at $(0,-1)$. The corresponding simplicial feasible set for GP2 has vertex set $\{(-0.5, -2), (1.5, -2),(0.5, 0.0)\}$, as illustrated in Fig. 8.

Instances EX6-1 and EX6-2 and UPQP have a simplicial feasible set. Specifically, EX6-1 and EX6-2 have vertex sets $\mathcal V=\{(-3,-1),(1,1),(1.5,-2)\}$ and $\mathcal V=\{(-2,0),(0,-3), (2,3)\}$, respectively (see [10]). The portfolio problem is a standard application over the standard simplex and in fact not hard to solve. However, it illustrates well the effect of the various monotonicity tests on the instance. A description of the portfolio quadratic program (UPQP) can be found in “Appendix A”.

Table 1 Test problems. An asterisk at n indicates that this is the selected dimension for a varying dimension test instance. Their formulation and minima can be found at https://www.sfu.ca/~ssurjano/optimization.html

Fig. 8

Simplicial constrained version of test instance GP2

Table 2 Colors used in Figs. 9 to 11

Fig. 9

Results for EX2-1 instance using the CV directions for monotonicity for facet oriented directions

The studied variants of the monotonicity test are the following:

Figures 9 to 11 show the graphical output of the CV, CV+LP and CV+LS methods. The colors are explained in Table 2.

Some simplices are numbered only in Fig. 9. The centroid of the simplex is used for the CV method to save computations. The centroid of the facet was used for CV+LP or CV+LS methods (see the last paragraph of Sect. 4). It depicts the state of the algorithm on the example EX2-1. Notice that this instance is full dimensional. As we check directions for the facets only when $0\notin \varvec{g}$, monotonicity on the complete simplex has already been demonstrated before.

Fig. 10

Results for EX2-1 instance using LP method

Fig. 11

Results for EX2-1 instance using LS tp find monotonicity from a vertex

Figure 9 illustrates two monotonic directions from a vertex to centroid for simplex 6. The direction to a non-border facet with a positive bound (blue) implies that S can be replaced by the only border facet. The direction with a negative directional derivative bound (light red) to a non-border facet is more interesting; we can reject the simplex, see comments after Prop. 2. Simplices 8, 9 and 42 did not reveal a monotonic direction from a vertex to the centroid. As $0\notin \varvec{g}$, they can be reduced to their border facets. Simplex 27 is rejected by the RangeUp test, as well as the simplices being border facets of simplex 42 and 9. Simplex 32 has a positive directional derivative bound $\varvec{h}$ from a vertex to a border facet. This means the simplex can be rejected.

Figure 10 illustrates the direction with negative directional derivative bounds $\varvec{h}$ found by LP for simplices 9 and 42. Based on these directions, they can be rejected.

Figure 11 illustrates the directions found by the LS method with a negative directional derivative bound $\varvec{h}$ for simplices 9 and 42. The figure shows the directions from vertex to trial points $x_j$ for which $\varvec{h}(x_j,v)$ has been evaluated. The LS method additionally found a positive directional derivative bound $\varvec{h}$ for a direction to a border facet of simplex 8 (right-top facet). This finding let us discard that border facet from the search (see Prop. 3). The LP method aims to find a negative value for $\varvec{h}(x,v)$. We did not run an opposite formulation due to computational cost of solving LPs (see Table 5).

Edge facets have two directions, the vertex to vertex directions, so only one has to be evaluated, the other one is opposite. For instance, the lower facet of simplex 8 has a positive directional derivative bound $\varvec{h}$ to the right vertex (border), which also gives a negative DD in the opposite way (to the left vertex, which is non-border). This means, we can reject the facet as a simplex (see comments after Prop. 2).

Tables 4, 5 and 6 show the numerical results using the different monotonicity test methods. The header meaning is explained in Table 3. Best results are shown in bold values.

Table 3 Meaning of headers in the following tables

Table 4 Results using Centroid-vertex and vertex-vertex directions (CV)

Table 5 Results using CV+LP

Table 6 Results using CV+LS

Regarding monotonicity, those instances with a small value for N/N2, high NSCV, NSLP or NSLS and small value for Div are those where LP and Local Search are effective.

Using LP or LS reduces the number of evaluated simplices for most of the instances apart from EX6-2, G7, SCH2 and MC2. None of the methods found a monotonic directional derivative from a vertex in G7. However, 44 full dimensional simplices were reduced to border facets due to $0\not \in \varvec{g}$. LP or LS found additional monotonic directions for instances SCH2 and MC2, but this does not lead to a reduction in the number of evaluated simplices because we could not remove the simplex.

The largest reduction in number of evaluated simplices compared with the CV method are realized for instances H6 and ST5. The CV+LP method evaluates 983,155 and 210,915 simplices less respectively and the CV+LS requires 962,718 and 203,718 simplices less. As expected, the running time using CV+LP is the much larger. For H6 and ST5 it needs minutes whereas the CV and CV+LS finishes in seconds. The CV and CV+LS method spend a similar time on the ST5 instance, but the CV+LS method requires 22 s and CV 30 s on H6 instance.

The instance with the largest number of evaluated simplices is L8 where the number of found/existing monotonic direction is small. Therefore, in such case trying to find a monotonic direction mainly implies an increase of the execution time.

5.1 Result for shifted minimum instances

In previous experiments the minimum point is reached quickly. This is an advantage for the RangeUp test. However, the symmetry of the minimum point can lead to the so-called clustering problem where several simplices share the minimum point [4, 11].

In order to study how results are affected when the minimum point is not at the centroid of an initial facet, the initial simplex was shifted towards one of the vertices by 0.1. For the GP2 instance, the resulting vertex set is $\mathcal V= \{(-0.55, -2.1), (1.45, -2.1), (0.45, -0.1)\}$. Tables 7 to 9 show the numerical results with headers as described in Table 3.

Table 7 Results for shifted minimum instances using CV

Table 8 Results for shifted minimum instances using CV+LP

Table 9 Results for shifted minimum instances using CV+LS

The number of evaluated simplices is for some instances larger and for others lower, but the overall view does not change. CV+LP usually results in a low number of evaluated simplices against a large running time. CV+LS generates a similar number of evaluated simplices as CV+LP, but requires less execution time. For instance H6, CV+LS requires 29 s, CV 40 s and CV+LP 8m33s.

For those instances where CV shows the smallest running time, CV+LP and CV+LS have a small value of NSLP, NSLS respectively in comparison with N/N2, which means that small number of monotonic direction exists or few monotonic direction was found. This implies a large running time.

More rejection/reduction tests, as those used in box partitioning, may be developed in simplicial partitioning. Additionally, over-estimations in $\varvec{g}$ can be reduced by covering the simplex by more boxes instead of just one.

6 Conclusions

From the experimental results we can conclude that the use of the studied local search for monotonic directional derivatives requires a similar number of simplex evaluations as finding monotonic directions by Linear Programming (LPsolve). However, it requires less execution time. An advantage of the local search (LS) is that it can stop when a monotonic direction is found, i.e. we do not need the steepest one, and also it may use the stopping criterion. In some cases, the number of evaluated simplices is smaller using LS than LP, because the later only looks for negative directional derivatives. Looking for positive ones requires running another LP model.

There exist instances for which the CV method results in the smallest execution time. The differences with CV+LS are small in these cases.

Additional search for monotonic directional derivatives by LP or LS, results in an additional small number of them found (they may not exist), incurring in an increase of the computational burden. However, for H6, the LS method saves 27% of the running time in comparison with using only the CV method.

A unconstrained portfolio as QP problem

Convex, $n=4$.

$$\begin{aligned} \min \ f(x)&= 10x^TAx-\mu ^Tx,\end{aligned}$$

(8)

$$\begin{aligned} \text {s.t.}&\sum x_i\le 1, x\ge 0 \end{aligned}$$

(9)

$\mu =(11.5, 9.5, 5, 4)^T,$

$A:=\frac{1}{100} \left( \begin{array}{*{5}c} 166 &{}34 &{}58 &{}-1 \\ 34 &{}64 &{}4 &{}-1 \\ 58 &{}4 &{}100 &{}1 \\ -1 &{}-1 &{}1 &{}10 \end{array} \right)$

Minimum $f^*=-5.3732$ at $x=(0.1678, 0.405, 0.0, 0.4272)^T$