# A corrector–predictor interior-point method with new search direction for linear optimization

- 217 Downloads

## Abstract

We introduce a feasible corrector–predictor interior-point algorithm (CP IPA) for solving linear optimization problems which is based on a new search direction. The search directions are obtained by using the algebraic equivalent transformation (AET) of the Newton system which defines the central path. The AET of the Newton system is based on the map that is a difference of the identity function and square root function. We prove global convergence of the method and derive the iteration bound that matches best iteration bounds known for these types of methods. Furthermore, we prove the practical efficiency of the new algorithm by presenting numerical results. This is the first CP IPA which is based on the above mentioned search direction.

## Keywords

Linear optimization Corrector–predictor methods New search directions Polynomial complexity## Mathematics Subject Classification

90C05 90C51## 1 Introduction

Karmarkar (1984) presented the first projective IPA for solving LO problems with polynomial-time complexity. After that, several results related to this theory have been published. The theory and practice of IPAs can be found in monographs written by Roos et al. (1997), Wright (1997), Ye (1997) and Nesterov and Nemirovski (1994). The IPAs for LO can be classified in multiple ways. One way to classify these algorithms is based on the step length. In this way, we can distinguish between short- and long-step IPAs. In theory, short-update algorithms give usually more efficient theoretical results with simpler analysis, while in practice, the large-step versions perform generally better. Another way to categorize the IPAs is related to feasibility of the iterates, hence, we can consider feasible and infeasible IPAs. Another type of IPAs that have proven to be efficient in practice are predictor–corrector IPAs. These algorithms consist of iterations using two types of steps, one predictor step and one or more corrector steps. For further details on classification of different IPAs see Illés and Terlaky (2002), Roos et al. (1997), Wright (1997) and Ye (1997).

The determination of the search directions plays a key role in the theory of IPAs. The most widely used technique for obtaining the search directions is based on barrier functions. By considering self-regular functions, Peng et al. (2002) reduced the theoretical complexity of large-step IPAs. Darvay (2002) introduced a new technique for finding search directions in case of these algorithms, namely the algebraic equivalent transformation of the system which defines the central path. Central path has been introduced independently by Sonnevend (1985, 1986) and by Megiddo (1989). The importance of the central path in the literature of IPAs can be highlighted by the fact that it is unique, see Roos et al. (1997), Terlaky (2001), Wright (1997) and Ye (1997). Sonnevend (1985, 1986) proved that the central path led to a unique optimal solution called the analytic center. General idea of IPA is to approximately trace the central path and to compute an interior point, called \(\varepsilon \)-optimal solution, that well approximates the analytic center. From an \(\varepsilon \)-optimal solution with small enough \(\varepsilon > 0\), using the so-called rounding procedure (Illés and Terlaky 2002; Roos et al. 1997), an optimal solution can be computed in strongly polynomial time.

The function of AET is applied to both sides of the nonlinear equation of the system that defines the unique central path. After the transformation the central path remains unique, and the Newton’s method is applied to the transformed system in order to determine the displacements. In the literature, the most widely used function for AET is the identity map, namely in most of IPAs the central path has not been transformed. In the papers of Darvay (2002, 2003) \(\psi (t)=\sqrt{t}\) is used while in Darvay et al. (2016) the authors introduced an IPA for LO based on the direction using a new function, namely \(\psi (t)=t-\sqrt{t}\), where the domain is \(D_{\psi }=\left( \frac{1}{4},\infty \right) .\) IPAs based on AET (Darvay 2002, 2003; Darvay et al. 2016) achieve usually the best known iteration bounds, alongside many other IPAs (Roos et al. 1997; Wright 1997; Ye 1997). Further research is needed to investigate whether IPAs that use AETs are more efficient than IPAs that are not. It would be also interesting to identify some class of LO problems for which application of AET based IPA may be beneficial.

The first PC IPA has been independently developed by Mehrotra (1992) and Sonnevend et al. (1990). The PC IPAs consist of a predictor and several corrector steps in a main iteration. The aim of the predictor step is to approach the optimal solution of the problem in a greedy way. The usual consequence of the greedy predictor step is that the obtained strictly feasible solution no longer belongs to the given neighborhood of the central path. The goal of the corrector steps is to return the iterate back to the designated neighborhood. Mizuno et al. (1993) proposed the first PC IPA which uses only one corrector step in a main iteration. Darvay (2005, 2009) introduced PC IPAs for LO that are based on the AET technique and he used the function \(\psi (t)=\sqrt{t}\) with domain \(D_{\psi }=\left( 0, \infty \right) \) in order to determine the transformed central path and the modified Newton system. The unique solution of this system led to a new search direction. Kheirfam (2016, 2015) proposed CP IPAs for convex quadratic symmetric cone optimization and second-order cone optimization, respectively.

Before summarizing the structure and results of this paper, it is worthwhile to mention that IPAs for LO have been extensively generalized to linear complementary problems (LCPs) (Cottle et al. 1992; Illés et al. 2010a, b; Kojima et al. 1991; Lešaja and Roos 2010; Potra and Sheng 1996; Yoshise 1996). There are several generalizations of the Mizuno-Todd-Ye PC IPA (Mizuno et al. 1993) from LO to sufficient LCPs like that of Potra (2002), and Illés and Nagy (2007). Recently, Potra (2014) published a new PC IPA for sufficient LCPs using wide neighborhood with optimal iteration complexity. The AET method for determining search directions for IPAs has been also extended to LCPs (Achache 2010; Asadi and Mansouri 2013; Kheirfam 2014; Wang et al. 2009) and LCPs over symmetric cones (Asadi et al. 2017a, b; Mohammadi et al. 2015; Wang 2012).

In this paper, a new CP IPA for LO is introduced. We use the AET method for the system which defines the central path that is based on the function \(\psi (t)=t-\sqrt{t}\). Newton’s method is then applied to the transformed system in order to find the search directions. The analysis of the algorithm is more complicated with this function. Nevertheless, we were able to prove global convergence of the method and derive the iteration bound that matches best-known iteration bound for these types of methods. We also present some numerical results and we compare our CP IPA with the classical primal-dual method, which is based on the same search direction and uses only one step in each iteration.

The paper is organized as follows. In Sect. 2, the primal-dual LO problem and the main concepts of the AET of the system defining the central path are given. In the following section the new CP IPA is presented. Section 4 contains the analysis of the proposed algorithm, while in Sect. 5 the iteration bound for the algorithm is derived. In Sect. 6, we provide some numerical results that prove the efficiency of this algorithm. in the last Section some concluding remarks are provided.

## 2 Preliminaries

*x*,

*y*,

*s*) produces the following system for search direction \((\varDelta x, \varDelta y, \varDelta s)\)

*x*,

*y*,

*s*) and \((x(\mu ), y(\mu ), s(\mu ))\). Moreover, note that if \(\psi (t)=t\) then \(p_v=v^{-1}-v\) and we obtain the standard proximity measure \(\delta (v)=\frac{1}{2}\Vert v-v^{-1}\Vert \) given in Roos et al. (1997). From \(\psi (t)=\sqrt{t}\) it follows that \(p_v=2(e-v)\), thus \(\delta (v)=\Vert e-v\Vert \), which was discussed in Darvay (2002, 2003). Let

The lower and upper bounds on the components of the vector *v* are given in the following lemma.

### Lemma 1

## 3 Corrector–predictor algorithm

*x*,

*y*,

*s*), \(n\mu >\epsilon \), then the algorithm calculates a new iterate by performing corrector and predictor steps. In the corrector step, we define \(v=\sqrt{\frac{xs}{\mu }}\), \({{\bar{A}}}=A \, \mathrm{diag}(\frac{x}{v})\) and we obtain the scaled search directions \(d_x\) and \(d_s\) by solving (6) with \(p_v\) given in (7), namely

## 4 Analysis of the algorithm

The following technical lemma introduced by Wright (1997), is a generalization of Lemma C.4 (first \(u{-}v\) Lemma) in Roos et al. (1997). We will use it to estimate the norm of the product of scaled search directions.

### Lemma 2

*u*and

*v*be two arbitrary vectors in \(\mathbb {R}^n\) with \(u^Tv\ge 0\). Then

In the following two sections, we will analyse the predictor and the corrector steps in detail, respectively. Note that the first step performed by the algorithm is in fact a corrector step.

### 4.1 The predictor step

The next lemma will prove the strict feasibility after a predictor step.

### Lemma 3

### Proof

The above inequality implies that \(x^p(\alpha )s^p(\alpha )>0\), for all \(0\le \alpha \le 1\). Therefore, \(x^p(\alpha )\) and \(s^p(\alpha )\) are not changing sign on \(0\le \alpha \le 1\). Since \(x^p(0)=x^+>0\) and \(s^p(0)=s^+>0\), thus we conclude that \(x^p(1)=x^++\theta \varDelta ^px=x^p>0\) and \(s^p(1)=s^++\theta \varDelta ^ps=s^p>0\) and the proof is complete. \(\square \)

### Lemma 4

### Proof

### 4.2 The corrector step

In this subsection, we deal with the corrector step. One can observe that the algorithm presented in Fig. 1 performs a full-Newton step as a corrector step, which can be obtained in the same way as the one presented in the primal-dual algorithm of Darvay et al. (2016). Thus, for the analysis of this case the lemmas proved in Darvay et al. (2016) can be applied. The next lemma gives a condition for strict feasibility of full-Newton step.

### Lemma 5

[Lemma 5.1 in Darvay et al. (2016)] Let \(\delta :=\delta (x, s, \mu )<1\) and assume that \(v\ge \frac{1}{2}e\). Then, \(x^+>0\) and \(s^+>0\).

In the next lemma we show local quadratic convergence of the Newton process.

### Lemma 6

The following lemma gives an upper bound of the duality gap after a corrector step.

### Lemma 7

In the next subsection, we analyse the update of the duality gap after a main iteration of the algorithm.

### 4.3 The effect on duality gap after a main iteration

The purpose of the algorithm is to reduce the produced duality gap of the primal-dual pair. Therefore, in order to measure this reduction, in the next lemma we give an upper bound for the gap obtained after performing an iteration of the algorithm.

### Lemma 8

### Proof

### 4.4 Determining appropriate values of parameters

In this section, we want to fix the parameters \(\tau \) and \(\theta \) with suitable values, which guarantee that after a main iteration, the proximity measure will not exceed \(\tau \).

*f*is decreasing, therefore

## 5 Iteration bound

The next lemma gives an upper bound for the number of iterations produced by the algorithm.

### Lemma 9

*k*iterations. Then, \(\left( x^k\right) ^T s^k \le \epsilon \) for

### Proof

Using the above result follows the main result of our paper.

### Theorem 1

*x*,

*y*,

*s*) satisfying \(x^Ts\le \epsilon \).

It is worth mentioning that this CP algorithm has advantages compared to the one-step method presented in Darvay et al. (2016), because in the case of the one-step algorithm \(\theta =\frac{1}{27\sqrt{n}}\), which is smaller than the value of \(\theta \) used by us. Hence, this paper leads to a slightly better complexity result.

## 6 Numerical results

Number of iterations of the algorithms for the randomly generated problems

Maximum size | Avg. It. CP | Avg. CPU CP | Avg. It. PD | Avg. CPU PD |
---|---|---|---|---|

\(10\times 10\) | 23.2 | 0.1649 | 269.8 | 1.6773 |

\(20\times 20\) | 24.8 | 0.4636 | 294.5 | 4.253 |

\(50\times 50\) | 28.7 | 2.824 | 317.2 | 28.1405 |

Number of iterations of the algorithms for problems from Netlib

Problem | Nr. It. CP | CPU CP | Nr. It. PD | CPU PD |
---|---|---|---|---|

afiro | 53 | 3.618 | 646 | 39.208 |

adlittle | 86 | 51.362 | >1000 | >564.819 |

blend | 72 | 58.961 | 571 | 442.728 |

sc50a | 56 | 16.006 | 529 | 134.543 |

sc50b | 56 | 15.785 | 491 | 125.298 |

sc105 | 63 | 138.865 | 555 | 1175.96 |

sc205 | 80 | 1161.56 | 507 | 7413.31 |

scagr7 | 88 | 344.113 | 640 | 2170.95 |

recipe | 92 | 1352.36 | >1000 | >14,863.5 |

Based on the obtained results for the two set of problems we conclude that our CP algorithm outperforms the classical primal-dual method which uses the same search direction.

## 7 Conclusions and future research

We proposed a CP IPA for LO. We used the AET method for the system which defines the central path that is based on the function \(\psi (t)=t-\sqrt{t}\) (Darvay et al. 2016). We then applied Newton’s method to the transformed system in order to get the new search directions. Furthermore, we presented the analysis of the proposed algorithm and proved that the method finds an \(\varepsilon \)-optimal solution in polynomial time. To our best knowledge, this is the first CP IPA where the function \(\psi (t)=t-\sqrt{t}\) is used to derive the search directions. The novelty of this paper consists of the techniques used in the analysis of the algorithm. We had to assure that the components of *v*-vectors of the scaled space are greater than \(\frac{1}{2}\). We highlighted the practical efficiency of the method by providing numerical results on the selected set of test problems from NETLIB. We made a comparison between the number of iterations performed by our algorithm and the classical one-step method. We concluded that our algorithm is more efficient than the classical one.

Future theoretical research direction includes extending our CP IPA to more general problems. The computational direction for further research includes implementation with different functions \(\psi \) used for AET. We would like to compare the effect of these functions on the computational results of the algorithms on quite large and well selected LO test problems. Moreover, this kind of computational study might give some insight into the good selection strategy of the target point of the search direction for different LO problems. Ideal situation would be, if the algorithm could be able to choose automatically among different \(\varphi \) functions depending on the structure of the problem. Furthermore, such computational study on the effect of different AETs could be based on the similar LO test set, like in the study of Illés and Nagy (2014) for pivot algorithms.

## Notes

### Acknowledgements

Open access funding provided by Budapest University of Technology and Economics (BME). This research has been partially supported by the Hungarian Research Fund, OTKA (Grant No. NKFIH 125700). The research of T. Illés and P.R. Rigó has been partially supported by the Higher Education Excellence Program of the Ministry of Human Capacities in the frame of Artificial Intelligence research area of Budapest University of Technology and Economics (BME FIKP-MI/FM). The research of Zs. Darvay and P.R. Rigó was supported by a Grant of Romanian Ministry of Research and Innovation, CNCS - UEFISCDI, Project Number PN-III-P4-ID-PCE-2016-0190, within PNCDI III.

## References

- Achache M (2010) Complexity analysis and numerical implementation of a short-step primal-dual algorithm for linear complementarity problems. Appl Math Comput 216(7):1889–1895Google Scholar
- Asadi S, Mansouri H (2013) Polynomial interior-point algorithm for \({P}_*(\kappa )\) horizontal linear complementarity problems. Numer Algorithms 63(2):385–398Google Scholar
- Asadi S, Mansouri H, Darvay Zs (2017a) An infeasible full-NT step IPM for horizontal linear complementarity problem over Cartesian product of symmetric cones. Optimization 66(2):225–250Google Scholar
- Asadi S, Zangiabadi M, Mansouri H (2017b) A predictor–corrector interior-point algorithm for \(P_*(\kappa )-HLCPs\) over Cartesian product of symmetric cones. Numer Funct Anal Optim 38:20–38Google Scholar
- Cottle RW, Pang J-S, Stone RE (1992) The linear complementarity problem, computer science and scientific computing. Academic Press Inc., BostonGoogle Scholar
- Darvay Zs (2002) A new algorithm for solving self-dual linear optimization problems. Studia Univ Babeş-Bolyai Ser Inform 47(1):15–26Google Scholar
- Darvay Zs (2003) New interior-point algorithms in linear programming. Adv Model Optim 5(1):51–92Google Scholar
- Darvay Zs (2005) A new predictor–corrector algorithm for linear programming. Alkalm Mat Lapok 22:135–161 (
**in Hungarian**)Google Scholar - Darvay Zs (2009) A predictor–corrector algorithm for linearly constrained convex optimization. Studia Univ Babeş-Bolyai Ser Inform 54(2):121–138Google Scholar
- Darvay Zs, Takó I (2012) Computational comparison of primal-dual algorithms based on a new software. Unpublished manuscript (2012)Google Scholar
- Darvay Zs, Papp IM, Takács PR (2016) Complexity analysis of a full-Newton step interior-point method for linear optimization. Period Math Hung 73(1):27–42Google Scholar
- Gay D (1985) Electronic mail distribution of linear programming test problems. Math Program Soc COAL Newsl 3:10–12Google Scholar
- Illés T, Nagy M (2007) A new variant of the Mizuno–Todd–Ye predictor–corrector algorithm for sufficient matrix linear complementarity problem. Eur J Oper Res 181(3):1097–1111Google Scholar
- Illés T, Nagy A (2014) Computational aspects of simplex and MBU-simplex algorithms using different anti-cycling pivot rules. Optimization 63(1):49–66Google Scholar
- Illés T, Terlaky T (2002) Pivot versus interior point methods: pros and cons. Eur J Oper Res 140:6–26Google Scholar
- Illés T, Nagy M, Terlaky T (2010a) A polynomial path-following interior point algorithm for general linear complementarity problems. J Glob Optim 47(3):329–342Google Scholar
- Illés T, Nagy M, Terlaky T (2010b) Polynomial interior point algorithms for general linear complementarity problems. Algorithmic Oper Res 5:1–12Google Scholar
- Karmarkar NK (1984) A new polynomial-time algorithm for linear programming. Combinatorica 4(4):373–395Google Scholar
- Kheirfam B (2014) A predictor–corrector interior-point algorithm for \(P_{*}(\kappa )\)-horizontal linear complementarity problem. Numer Algorithms 66(2):349–361Google Scholar
- Kheirfam B (2015) A corrector–predictor path-following method for convex quadratic symmetric cone optimization. J Optim Theory Appl 164(1):246–260Google Scholar
- Kheirfam B (2016) A corrector–predictor path-following method for second-order cone optimization. Int J Comput Math 93(12):2064–2078Google Scholar
- Kheirfam B (2018) An infeasible interior point method for the monotone SDLCP based on a transformation of the central path. J Appl Math Comput 57(1):685–702Google Scholar
- Kojima M, Megiddo N, Noma T, Yoshise A (1991) A unified approach to interior point algorithms for linear complementarity problems, vol 538. Lecture notes in computer science. Springer, BerlinGoogle Scholar
- Lešaja G, Roos C (2010) Unified analysis of kernel-based interior-point methods for \(P_*(\kappa )\)-linear complementarity problems. SIAM J Optim 20(6):3014–3039Google Scholar
- Mohammadi N, Mansouri H, Zangiabadi M, Asadi S (2015) A full Nesterov–Todd step infeasible-interior-point algorithm for Cartesian \(P_*(\kappa )\) horizontal linear complementarity problem over symmetric cones. Optimization 65:539–565Google Scholar
- Megiddo N (1989) Pathways to the optimal set in linear programming. In: Megiddo N (ed) Progress in mathematical programming. Interior-point and related methods. Springer, New York, pp 131–158Google Scholar
- Mehrotra S (1992) On the implementation of a primal-dual interior point method. SIAM J Optim 2(4):575–601Google Scholar
- Mizuno S, Todd MJ, Ye Y (1993) On adaptive-step primal-dual interior-point algorithms for linear programming. Math Oper Res 18:964–981Google Scholar
- Nesterov YE, Nemirovski A (1994) Interior point polynomial methods in convex programming. SIAM studies in applied mathematics. SIAM Publications, PhiladelphiaGoogle Scholar
- Peng J, Roos C, Terlaky T (2002) Self-regular functions: a new paradigm for primal-dual interior-point methods. Princeton University Press, PrincetonGoogle Scholar
- Potra FA (2002) The Mizuno-Todd-Ye algorithm in a larger neighborhood of the central path. Eur J Oper Res 143:257–267Google Scholar
- Potra FA (2014) Interior point methods for sufficient LCP in a wide neighborhood of the central path with optimal iteration complexity. SIAM J Optim 24(1):1–28Google Scholar
- Potra FA, Sheng R (1996) Predictor–corrector algorithm for solving \(P_*(\kappa )\)-matrix LCP from arbitrary positive starting points. Math Program 76(1):223–244Google Scholar
- Roos C, Terlaky T, Vial J-Ph (1997) Theory and algorithms for linear optimization, an interior-point approach. Wiley, ChichesterGoogle Scholar
- Sonnevend Gy (1985) A new method for solving a set of linear (convex) inequalities and its applications. Technical report, Department of Numerical Analysis, Institute of Mathematics, Eötvös Loránd University, BudapestGoogle Scholar
- Sonnevend Gy (1986) An ”analytic center” for polyhedrons and new classes of global algorithms for linear (smooth, convex) programming. In: Prékopa A, Szelezsán J, Strazicky B (eds) System modelling and optimization: proceedings of the 12th IFIP-conference held in Budapest, Hungary, Sept 1985. Lecture notes in control and information sciences, vol 84. Springer, Berlin, pp 866–876Google Scholar
- Sonnevend Gy, Stoer J, Zhao G (1990) On the complexity of following the central path by linear extrapolation in linear programming. Methods Oper Res 62:19–31Google Scholar
- Terlaky T (2001) An easy way to teach interior-point methods. Eur J Oper Res 130(1):1–19Google Scholar
- Wang GQ (2012) A new polynomial interior-point algorithm for the monotone linear complementarity problem over symmetric cones with full NT-steps. Asia-Pac J Oper Res 29(2)Google Scholar
- Wang GQ, Yue YJ, Cai XZ (2009) Weighted-path-following interior-point algorithm to monotone mixed linear complementarity problem. Fuzzy Inf Eng 1(4):435–445Google Scholar
- Wright SJ (1997) Primal-dual interior-point methods. SIAM, PhiladelphiaGoogle Scholar
- Ye Y (1997) Interior point algorithms, Wiley-interscience series in discrete mathematics and optimization. Theory and analysis. Wiley, New YorkGoogle Scholar
- Ye Y, Todd M, Mizuno S (1994) An \(O(\sqrt{n} L)\)-iteration homogeneous and self-dual linear programming algorithm. Math Oper Res 19(1):53–67Google Scholar
- Yoshise A (1996) Complementarity problems. In: Terlaky T (ed) Interior point methods of mathematical programming. Kluwer Academic Publishers, Dordrecht, pp 297–367Google Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.