Fast interior point solution of quadratic programming problems arising from PDEconstrained optimization
 1.2k Downloads
 3 Citations
Abstract
Interior point methods provide an attractive class of approaches for solving linear, quadratic and nonlinear programming problems, due to their excellent efficiency and wide applicability. In this paper, we consider PDEconstrained optimization problems with bound constraints on the state and control variables, and their representation on the discrete level as quadratic programming problems. To tackle complex problems and achieve high accuracy in the solution, one is required to solve matrix systems of huge scale resulting from Newton iteration, and hence fast and robust methods for these systems are required. We present preconditioned iterative techniques for solving a number of these problems using Krylov subspace methods, considering in what circumstances one may predict rapid convergence of the solvers in theory, as well as the solutions observed from practical computations.
Mathematics Subject Classification
65F08 65F10 65F50 76D55 93C201 Introduction
Computational techniques for PDEconstrained optimal control problems involve a discretization of the underlying PDE. There are two options for doing this, and the typical paradigm in PDEconstrained optimization literature is for both approaches to solve the problem in a similar manner. The first is to apply an optimizethendiscretize method, involving constructing continuous optimality conditions, and then discretizing these. However we find that this approach is inconvenient when examining the resulting discrete systems for the problems considered in this paper, specifically with regard to the reduction of the dimension of the system, as well as symmetry of the matrix involved. The alternative method, which we apply in this paper, is the discretizethenoptimize approach: here a discrete cost functional is constructed and discretized constraints are formulated. Then optimality conditions are derived for such (possibly huge) problems. Our motivation for using this approach originates from an observation that for a particular (quadratic) cost functional (2) the discretized PDEconstrained problem takes the form of a quadratic optimization problem for linear PDEs. The use of fine discretization leads to a substantial size of the resulting optimization problem. Therefore we will apply an interior point algorithm to solve it.
Interior point methods (IPMs) are very wellsuited to solving quadratic optimization problems and they excel when sizes of problems grow large [17, 52], which makes them perfect candidates for discretized PDEconstrained optimal control problems. The use of IPMs in PDEconstrained optimization is not new. There have been several developments which address theoretical aspects, including the functional analysis viewpoint, and study the convergence properties of an interior point algorithm [46, 49, 51], and many others which focus on the practical (computational) aspects. IPMs belong to a broad class of methods which rely on the use of Newton methods to compute optimizing directions. There have been several successful attempts to use Newtonbased approaches in the PDEconstrained optimization context [4, 5, 25, 28]. The main computational challenge in these approaches is the solution of the linear system which determines the Newton direction. For fine PDE discretizations such systems quickly get very large. Additionally, when IPMs are applied, the added interior point diagonal scaling matrices degrade the conditioning of such systems [17] and make them numerically challenging. Direct methods for sparse linear algebra [10] can handle the illconditioning well but struggle with excessive memory requirements when problems get larger. Inexact interior point methods [16, 18, 50] overcome this difficulty by employing iterative methods to solve the Newton equations.
Because of the unavoidable illconditioning of these equations the success of any iterative scheme for their solution depends on the ability to design efficient preconditioners which can improve spectral properties of linear systems. The development of such preconditioners is a very active research area. Preconditioners for IPMs in PDEconstrained optimization exploit the vast experience gathered for saddle point systems [2], but face an extra difficulty originating from the presence of IPM scaling. There have already been several successful attempts to design preconditioners for such systems, see [1, 3, 18] and the references therein.
In this paper, we propose a general methodology to design efficient preconditioners for such systems. Our approach is derived from the matching strategy originally developed for a particular Poisson control problem [37]. We adapt it to much more challenging circumstances of saddle point systems arising in IPMs applied to solve the PDEconstrained optimal control problems. We briefly comment on the enjoyable spectral properties of the preconditioned system, and provide computational results to demonstrate that they work well in practice.
This paper is structured as follows. In Sect. 2 we briefly recall a few basic facts about interior point methods for quadratic programming. In Sect. 3 we demonstrate how IPMs can be applied to PDEconstrained optimization problems. In Sect. 4 we introduce the preconditioners proposed for problems originating from optimal control. We consider separately two different cases of timeindependent and timedependent problems. In Sect. 5 we illustrate our findings with computational results and, finally, in Sect. 6 we give our conclusions.
2 Interior point methods for quadratic programming
3 PDEconstrained optimization
We now wish to demonstrate how interior point methods may be applied to PDEconstrained optimization problems. These are a crucial class of problems which may be used to model a range of applications in science and industry, for example fluid flow, chemical and biological processes, shape optimization, imaging problems, and mathematical finance, to name but a few. However the problems are often of complex structure, and sophisticated techniques are frequently required to achieve accurate solutions for the models being considered. We recommend the works [22, 45], which provide an excellent introduction to the field.
We will now apply the discretizethenoptimize approach to (7), commencing with the construction of a Lagrangian on the discrete space. The alternative optimizethendiscretize method will guarantee an accurate solution of the continuous first order optimality conditions, however when applied in conjunction with interior point methods the resulting matrix systems are not necessarily symmetric, nor can they be reduced to such low dimensions for these problems as the matrix systems illustrated later in this section. For these reasons, we find it is advantageous to apply the discretizethenoptimize approach for the interior point solution of PDEconstrained optimization problems—we highlight that this follows the approach used in important literature on the field such as [5, 28]. Provided reasonable choices are made for the discretization of the problem, it is frequently observed that both methods lead to very similar behaviour in the solutions, and indeed this paradigm has recently been used to derive discretization schemes for PDEconstrained optimization (see [20], for instance).
We observe that, using our equal order finite element method, the matrices \(M,K\in {\mathbb {R}}^{N\times {}N}\), where N denotes the number of finite element nodes used, and furthermore that \({\mathbf {y}},{\mathbf {u}}\in {\mathbb {R}}^{N}\).
In the next section we consider interior point methods for solving problems of structure (8), for a range of operators \({\mathcal {L}}\) and all \(\beta >0\). Although there has at this point been relatively little research into such strategies, we highlight that the paper [46] considers the numerical solution of problems of this type with control constraints only, and [1] derives effective preconditioners for large values of \(\beta \) and \({\mathcal {L}}y=\nabla ^{2}y+y\). We also point to the development of solvers of different forms to those presented in this paper: in [18] reducedspace preconditioners are considered for optimal control problems, and in [9] multigrid methods are discussed for a class of control problems.
3.1 Newton iteration
Note that, due to the fact that state and control bounds are enforced as strict inequalities at each Newton step, the diagonal matrices \(D_{y}\) and \(D_{u}\) are positive definite.
Of course, it is perfectly natural to consider a problem with only state constraints or only control constraints (or indeed only lower or upper bound constraints). For such cases we may follow exactly the same working to obtain a matrix system of the form (24), removing individual matrices corresponding to constraints that we do not apply.
3.2 Algorithm
We now present the structure of the interior point algorithm, adapted from the paper [17], that we apply to the problems considered in this paper. The essence of the method is to traverse the interior of the feasible region where solutions may arise—we do this by applying a relaxed Newton iteration, reducing the barrier parameter by a factor \(\sigma \) at each Newton step. Having computed the Newton updates \({\varvec{\delta }}{\mathbf {y}}\), \({\varvec{\delta }}{\mathbf {u}}\), \(\varvec{\delta \lambda }\), \({\varvec{\delta }}{\mathbf {z}}_{y,a}\), \({\varvec{\delta }}{\mathbf {z}}_{y,b}\), \({\varvec{\delta }}{\mathbf {z}}_{u,a}\), \({\varvec{\delta }}{\mathbf {z}}_{u,b}\), we make a step in this direction that also guarantees that the strict bounds are enforced at each iteration. Upon convergence the iterates approach the true solution of the optimization problem, with the additional state and control constraints automatically satisfied.
It is clear from the presentation of this method that the dominant computational work arises from the solution of the Newton system (24). It is therefore crucial to construct fast and robust solvers for this system, and this is what we focus on in Sect. 4.
3.3 Timedependent problems
4 Preconditioning for the Newton system
For the matrix systems considered in this paper, particularly those arising from timedependent problems, great care must be taken when seeking an appropriate scheme for obtaining an accurate solution. The dimensions of these systems mean that a direct method is often infeasible, so we find that the natural approach is to develop preconditioned Krylov subspace solvers.
A major objective within the remainder of this paper is to develop effective representations of the (1, 1)block \(\varPhi \) and Schur complement S for matrix systems arising from interior point solvers.
4.1 Timeindependent problems
As an alternative for our approximation \({\widehat{\varPhi }}\), one may apply a Chebyshev semiiteration method [14, 15, 48] to approximate the inverses of \(M+D_{y}\) and \(\beta {}M+D_{u}\). This is a slightly more expensive process to approximate this component of the entire system (in general the matrices with the most complex structure are K and \(K^{\top }\)), however due to the tight clustering of the eigenvalues of \([\text {diag}(\varPhi )]^{1}\varPhi \) we find greater accuracy in the results obtained.
Furthermore, it is possible to prove a lower bound of the preconditioned Schur complement for a very general matrix form, as demonstrated below.
Theorem 1
Proof

The Rayleigh quotient R is certainly finite, as the case \({\varvec{\chi }}+{\varvec{\omega }}=\mathbf {0}\) is disallowed by the assumption of invertibility of \({\widehat{S}}_{G}\).

Furthermore, depending on the (typically unknown) entries of \(D_{y}\), the term \({\mathbf {v}}^{\top }K(M+D_{y})^{1}K^{\top }{\mathbf {v}}\) should be large compared with the term \({\mathbf {v}}^{\top }M(\beta {}M+D_{u})^{1/2}(M+D_{y})^{1/2}K^{\top }{\mathbf {v}}\) arising in the denominator above, due to the fact that K has larger eigenvalues than M in general. The term being minimized in (35) will therefore not take a large negative value in general, and hence R will not become excessively large.

However, it is generally not possible to demonstrate a concrete upper bound unless \({\bar{X}}\) and \({\bar{Y}}\) have structures which can be exploited. The reason for this is that the diagonal matrices \(D_{y}\) and \(D_{u}\) that determine the distribution of the eigenvalues can take any positive value (including arbitrarily small or infinitely large values, in finite precision), depending on the behaviour of the Newton iterates, which is impossible to control. In practice, we find it is rare for the largest eigenvalues of the preconditioned Schur complement to exceed values of roughly \(510\).

However, using the methodology of Theorem 1, results of this form have been demonstrated for problems such as convectiondiffusion control [36] and heat equation control [35] (without additional bound constraints). We also highlight that, in [39, 42], preconditioners for problems with bound constraints^{1}, solved with active set Newton methods, are derived. In [39], parameterindependent bounds are derived for a preconditioned Schur complement, however the additional requirement is imposed that M is a lumped (i.e. diagonal) mass matrix. As we do not assume that the mass matrices are lumped in this work, we may not exploit this method to obtain an upper eigenvalue bound.

In general, the eigenvalues of \({\widehat{S}}_{G}^{1}S_{G}\) are better clustered if the term \({\bar{X}}{\bar{Y}}^{\top }+{\bar{Y}}{\bar{X}}^{\top }\) is positive semidefinite, or ‘nearly’ positive semidefinite. The worst case would arise in the setting where \({\varvec{\chi }}\approx {\varvec{\omega }}\), however for our problem the matrices \({\bar{X}}\) and \({\bar{Y}}\) do not relate closely to each other as the activities in the state and control variables do not share many common features.
Lemma 1
If \(D_{y}=D_{u}=0\), and the matrix \(K+K^{\top }\) is positive semidefinite^{2}, then the eigenvalues of \({\widehat{S}}_{G}^{1}S_{G}\) satisfy \(\lambda \le 1\).
Proof
Lemma 2
Proof
Clearly, it is valuable to have this insight that using our approximation \({\widehat{M}}_{1}\) retains the parameter independence of the lower bound for the eigenvalues of \({\widehat{S}}_{1}^{1}S\). We note that this can potentially be a weak bound, as the large diagonal entries in \(D_{y}\) and \(D_{u}\) are likely to dominate the behaviour of \(M+D_{y}\) and \(\beta {}M+D_{u}\), thus driving the eigenvalues of the preconditioned Schur complement closer to 1.
We highlight that, in practice, one may also approximate the inverses of \(K+{\widehat{M}}_{1}\) and its transpose effectively using a multigrid process. We apply the Aggregationbased Algebraic Multigrid (AGMG) software [30, 31, 32, 33] for this purpose within our iterative solvers.
It is useful to consider the distribution of eigenvalues of the preconditioned system, as this will control the convergence properties of the Minres method. The fundamental result we use for our analysis of saddle point matrices (30) is stated below [40, Lemma 2.1].
Theorem 2
So, using Theorem 2, the eigenvalues of \({\mathcal {P}}^{1}\mathcal {A}\) are contained within the interval stated below.
Lemma 3
It is therefore clear that, for our problem, a good approximation of the Schur complement will guarantee clustered eigenvalues of the preconditioned system, and therefore rapid convergence of the Minres method. As we have observed for our problem, the quantities of interest are therefore the largest eigenvalues of \({\widehat{S}}^{1}S\), which can vary at every step of a Newton method.
We now present a straightforward result concerning the eigenvectors of a preconditioned saddle point system of the form under consideration.
Proposition 1
Proof
We observe that the eigenvalues and eigenvectors of the (1, 1)block and Schur complement (along with their approximations) interact strongly with each other. This decreases the likelihood of many extreme eigenvalues of \({\widehat{S}}^{1}S\) arising in practice, as this would have implications on the numerical properties of \(\varPhi \) and \(\varPsi \) (which for our problems do not interact at all strongly). However the working provided here shows that this is very difficult to prove rigorously, due to the wide generality of the saddle point systems being examined—we must also rely on the physical properties of the PDE operators within the optimization framework. Our numerical experiments of Sect. 5 indicate that the eigenvalues of \({\widehat{S}}^{1}S\), and therefore the preconditioned system, are tightly clustered, matching some of the observations made in this section.
It is possible to carry out eigenvalue analysis for the block triangular preconditioner \({\mathcal {P}}_{2}\) in the same way as for the block diagonal preconditioner \({\mathcal {P}}_{1}\). However it is well known that the convergence of nonsymmetric solvers such as Gmres does not solely depend on the eigenvalues of the preconditioned system, and therefore such an analysis would be less useful in practice.
It is clear that to apply the preconditioner \({\mathcal {P}}_{3}\), we require a nonsymmetric solver such as Gmres, as it is not possible to construct a positive definite preconditioner with this rearrangement of the matrix system. Within such a solver, a key positive property of this strategy is that we may approximate \(\varPhi \) almost perfectly (and cheaply), and may apply \(K^{\top }\) exactly within \({\mathcal {P}}_{3}\) without a matrix inversion. An associated disadvantage is that our approximation of S is more expensive to apply than the approximation \({\widehat{S}}_{1}\) used within the preconditioners \({\mathcal {P}}_{1}\) and \({\mathcal {P}}_{2}\)—whereas Theorem 1 may again be applied^{4} to verify a lower bound for the eigenvalues of the preconditioned Schur complement, the values of the largest eigenvalues are frequently found to be higher than for the Schur complement approximation \({\widehat{S}}_{1}\) described earlier.
4.2 Timedependent problems
Remark 1
We highlight that a class of methods which is frequently utilized when solving PDEconstrained optimization problems, aside from the iterative methods discussed in this paper, is that of multigrid. We recommend [8] for an overview of such methods for PDEconstrained optimization, [7] for a convergence analysis of multigrid applied to these problems, [20, 21] for schemes derived for solving flow control problems, and [6] for a method tailored to problems with additional bound constraints. These solvers require the careful construction of prolongation/restriction operators, as well as smoothing methods, tailored to the precise problem at hand. Applying multigrid to the entire coupled matrix systems resulting from the problems considered in this paper, as opposed to employing this technology to solve subblocks of the system within an iterative method, also has the potential to be a powerful approach for solving problems with bound constraints. Similar multigrid methods have previously been applied to the interior point solution of PDEconstrained optimization problems in one article [9], and we believe that a carefully tailored scheme could be a viable alternative when solving at least some of the numerical examples considered in Sect. 5.
4.3 Alternative problem formulations
 Boundary control problems Our methodology could be readily extended to problems where the control (or state) variable is regularized on the boundary only within the cost functional, for instance whereFor such problems, we would need to take account of boundary mass matrices within the saddle point system that arises, however preconditioners have previously been designed for such problems that take into account these features (see [35], for instance).$$\begin{aligned} {\mathcal {J}}(y,u)=\frac{1}{2} \Vert y  {\widehat{y}} \Vert _{L_{2}(\varOmega )}^2 + \frac{\beta }{2} \Vert u \Vert _{L_{2}(\partial \varOmega )}^2. \end{aligned}$$
 Control variable regularized on a subdomain Analogously, problems may be considered using our preconditioning approach where the cost functional is of the formwhere \(\varOmega _1\subset \varOmega \). The matching strategy of Sect. 4.1 may be modified to account for the matrices of differing structures.$$\begin{aligned} {\mathcal {J}}(y,u)=\frac{1}{2} \Vert y  {\widehat{y}} \Vert _{L_{2}(\varOmega )}^2 + \frac{\beta }{2} \Vert u \Vert _{L_{2}(\varOmega _1)}^2, \end{aligned}$$
 Alternative regularizations A further possibility is for the control (or state) variable to be regularized using a different term, for instance an \(H^{1}\) regularization term of the following form:Upon discretization, stiffness matrices arise within the (1, 1)block in addition to mass matrices, however the preconditioning method introduced in this paper may still be applied, by accounting for the new matrices within the matching strategy for the Schur complement.$$\begin{aligned} {\mathcal {J}}(y,u)= & {} \frac{1}{2} \Vert y  {\widehat{y}} \Vert _{L_{2}(\varOmega )}^2 + \frac{\beta }{2} \Vert u \Vert _{H^{1}(\varOmega )}^2\\= & {} \frac{1}{2} \Vert y  {\widehat{y}} \Vert _{L_{2}(\varOmega )}^2 + \frac{\beta }{2} \Vert u \Vert _{L_{2}(\varOmega )}^2 + \frac{\beta }{2} \Vert \nabla {}u \Vert _{L_{2}(\varOmega )}^2. \end{aligned}$$
 Timedependent problems Finally, we highlight that modifications to the cost functional considered for timedependent problems in Sect. 3.3 may be made. For instance, one may measure the control (or state) variables at the final time only, that isOn the discrete level, this will lead to mass matrices being removed from portions of the (1, 1)block, and this information may be built into new preconditioners [35, 44].$$\begin{aligned} {\mathcal {J}}(y,u)=\frac{1}{2}\int _{0}^{T}\int _{\varOmega }\big (y({\mathbf {x}},t) {\widehat{y}}({\mathbf {x}},t)\big )^{2}~\mathrm{d}\varOmega \mathrm{d}t+\frac{\beta }{2}\int _{\varOmega }u({\mathbf {x}},T)^{2}~\mathrm{d}\varOmega . \end{aligned}$$
5 Numerical experiments
Having motivated our numerical methods for the solution of the problems considered, we now wish to test our solvers on a range of examples. These test problems are of both timeindependent and timedependent form, and are solved on a desktop with a quadcore 3.2GHz processor. For each test problem, we discretize the state, control and adjoint variables using Q1 finite elements. Within the interior point method, the value of the barrier reduction parameter \(\sigma \) is set to be 0.1, with \(\alpha _{0}=0.995\), and \(\epsilon _{p}=\epsilon _{d}=\epsilon _{c}=10^{6}\). To solve the Newton systems arising from the interior point method, we use the Ifiss software package [11, 43] to construct the relevant finite element matrices. When the symmetric block diagonal preconditioner \({\mathcal {P}}_{1}\) is used, we solve the Newton systems using the Minres algorithm to a relative preconditioned residual norm tolerance of \(10^{8}\), and the Chebyshev semiiteration method to approximate the inverse of the (1, 1)block (apart from within one experiment where we use a diagonal approximation), as well as the AGMG method to approximate the inverse Schur complement. Where the block triangular preconditioners \({\mathcal {P}}_{2}\) and \({\mathcal {P}}_{3}\) are applied, we solve the Newton systems with the preconditioned Gmres method to a tolerance of \(10^{8}\); we apply 20 steps of Chebyshev semiiteration to approximate the (1, 1)block, and once again utilize AGMG for the Schur complement approximations. We highlight that it would be feasible to relax the tolerances for Minres and Gmres in order to lower the overall CPU time for the interior point scheme [16], however we elect to solve the matrix systems relatively accurately in order to fully demonstrate the potency of our preconditioned iterative methods. All results are computed using Matlab R2015a.
Results for the Poisson control example with control constraints, for a range of values of h and \(\beta \), and preconditioner \({\mathcal {P}}_{1}\)
\(\beta =1\)  \(\beta =10^{1}\)  \(\beta =10^{2}\)  \(\beta =10^{3}\)  \(\beta =10^{4}\)  \(\beta =10^{5}\)  \(\beta =10^{6}\)  

\(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  
\(u\le 0.01\)  \(u\le 0.1\)  \(u\le 1\)  \(u\le 3\)  \(u\le 20\)  \(u\le 100\)  \(u\le 300\)  
\({\mathcal {P}}_{1}\) Chebyshev  
h  
\(2^{2}\)  10  5.6  11  6.3  13  6.2  15  6.6  18  7.5  19  7.2  20  7.4 
\(2^{3}\)  10  5.7  13  6.1  14  6.3  16  7.8  19  8.3  20  8.7  21  9.3 
\(2^{4}\)  10  5.6  13  6.1  15  6.5  19  7.4  22  8.6  22  8.5  21  8.8 
\(2^{5}\)  11  5.4  16  5.8  18  6.3  21  7.0  23  8.8  25  8.9  24  9.4 
\(2^{6}\)  11  5.5  16  5.8  20  6.2  22  6.8  26  15.5  24  8.9  30  9.4 
\(2^{7}\)  12  5.2  18  5.5  20  6.2  20  7.1  27  8.4  25  8.6  31  9.2 
\({\mathcal {P}}_{1}\) Diagonal  
h  
\(2^{2}\)  9  9.4  11  10.4  13  9.5  15  9.2  18  10.1  19  9.4  20  17.6 
\(2^{3}\)  10  15.1  12  16.7  14  16.9  16  18.4  19  17.5  20  18.5  21  19.5 
\(2^{4}\)  10  15.5  15  18.6  16  19.9  19  22.7  22  21.6  22  23.4  21  24.3 
\(2^{5}\)  11  16.3  16  16.2  19  19.5  21  21.1  23  24.7  25  25.7  24  25.8 
\(2^{6}\)  11  15.5  16  20.2  20  16.9  22  18.9  26  32.1  24  18.9  31  26.7 
\(2^{7}\)  12  14.3  18  15.7  21  16.1  20  18.5  28  28.8  25  19.3  31  23.4 
Results for the Poisson control example with state constraints, for a range of values of h and \(\beta \)
\(\beta =1\)  \(\beta =10^{2}\)  \(\beta =10^{4}\)  \(\beta =10^{6}\)  

\(0.1\le {}y\le 0.002\)  \(0.1\le {}y\le 0.175\)  \(0.1\le {}y\le 0.9\)  \(0.1\le {}y\le 1\)  
\({\mathcal {P}}_{2}\)  
h  
\(2^{2}\)  11  5.3  8  5.0  9  5.0  10  5.0 
\(2^{3}\)  12  9.9  9  10.2  10  13.3  10  10.9 
\(2^{4}\)  13  11.4  10  12.9  11  16.8  11  13.5 
\(2^{5}\)  14  12.1  11  13.3  13  27.4  12  15.0 
\(2^{6}\)  16  12.5  12  13.6  14  17.8  13  15.7 
\(2^{7}\)  17  12.7  13  14.6  16  16.9  14  16.3 
\({\mathcal {P}}_{3}\)  
h  
\(2^{2}\)  11  5.0  8  5.1  9  5.0  10  5.0 
\(2^{3}\)  12  9.6  9  9.1  10  10.5  10  10.5 
\(2^{4}\)  13  11.2  10  10.3  11  12.3  11  12.4 
\(2^{5}\)  14  12.1  11  10.8  13  12.9  12  13.5 
\(2^{6}\)  16  12.6  12  11.4  14  13.3  13  13.9 
\(2^{7}\)  17  13.1  13  13.0  16  13.5  14  14.5 
Number of interior point (Newton) iterations, average number of iterations of the Krylov subspace method per interior point step, and CPU time required to solve the Poisson control example with state constraints, when the preconditioners \({\mathcal {P}}_{1}\), \({\mathcal {P}}_{2}\) and \({\mathcal {P}}_{3}\) are used
\(\beta =10^{2}\)  \({\mathcal {P}}_{1}\)  \({\mathcal {P}}_{2}\)  \({\mathcal {P}}_{3}\)  

IPM  Krylov  CPU  IPM  Krylov  CPU  IPM  Krylov  CPU  
h  
\(2^{2}\)  8  8.0  0.13  8  5.0  0.20  8  5.1  0.22 
\(2^{3}\)  9  11.8  0.23  9  10.2  0.35  9  9.1  0.34 
\(2^{4}\)  10  14.5  0.46  10  12.9  0.63  10  10.3  0.57 
\(2^{5}\)  11  14.1  1.8  11  13.3  2.6  11  10.8  2.4 
\(2^{6}\)  13  14.8  9.1  12  13.6  11.4  12  11.4  10.1 
\(2^{7}\)  14  14.9  37.4  13  14.6  54.4  13  13.0  53.8 
Results for the Helmholtz problem with state constraints, for a range of values of h and \(\beta \), as well as values of k
\(k=20\)  \(k=50\)  

\(\beta =10^{2}\)  \(\beta =10^{4}\)  \(\beta =10^{6}\)  \(\beta =10^{2}\)  \(\beta =10^{4}\)  
\(0.0005\le {}y\le 0.0005\)  \(0.05\le {}y\le 0.05\)  \(0.6\le {}y\le 0.6\)  \(10^{5}\le {}y\le 10^{5}\)  \(0.001\le {}y\le 0.001\)  
\({\mathcal {P}}_{2}\)  
h  
\(2^{2}\)  7  4.3  10  5.3  10  4.7  5  4.0  8  4.7 
\(2^{3}\)  8  9.2  10  11.6  11  12.4  5  6.8  8  12.3 
\(2^{4}\)  8  10.6  11  17.7  12  30.8  6  10.4  8  17.9 
\(2^{5}\)  9  11.2  12  18.8  12  19.6  6  6.1  9  20.5 
\(2^{6}\)  9  10.4  12  15.9  13  22.7  7  10.3  10  23.1 
\(2^{7}\)  10  10.2  13  15.6  14  15.5  8  10.6  10  20.1 
\({\mathcal {P}}_{3}\)  
h  
\(2^{2}\)  7  4.2  10  5.2  10  3.9  5  3.7  8  5.1 
\(2^{3}\)  8  9.3  10  11.8  11  8.2  5  6.2  8  10.4 
\(2^{4}\)  8  9.9  11  13.9  12  10.0  6  9.3  8  16.8 
\(2^{5}\)  9  10.9  12  15.2  12  10.2  6  5.1  9  19.1 
\(2^{6}\)  9  10.2  12  15.1  13  10.4  7  9.5  10  21.8 
\(2^{7}\)  10  10.3  13  14.7  14  10.2  8  10.0  10  18.5 
Results for the convectiondiffusion control example with state and control constraints, for a range of values of h and \(\beta \)
\(\beta =10^{1}\)  \(\beta =10^{2}\)  \(\beta =10^{3}\)  \(\beta =10^{4}\)  \(\beta =10^{5}\)  

\(0\le {}y\le 0.2\)  \(0\le {}y\le 0.5\)  \(0\le {}y\le 0.5\)  \(0\le {}y\le 0.75\)  \(0\le {}y\le 0.75\)  
\(0.75\le {}u\le 0.75\)  \(2\le {}u\le 2\)  \(3\le {}u\le 3\)  \(5\le {}u\le 5\)  \(6\le {}u\le 6\)  
\({\mathcal {P}}_{2}\)  
h  
\(2^{2}\)  13  10.1  14  11.3  14  11.3  14  11.1  14  11.4 
\(2^{3}\)  14  20.9  15  19.8  15  24.8  15  21.8  15  22.4 
\(2^{4}\)  16  35.1  15  20.6  16  42.6  16  37.6  17  53.0 
\(2^{5}\)  17  44.1  17  40.4  16  45.6  19  64.3  19  69.3 
\(2^{6}\)  19  52.6  19  48.4  17  47.3  22  66.7  23  73.6 
\(2^{7}\)  21  50.0  21  48.2  22  63.5  26  75.0  27  81.7 
\({\mathcal {P}}_{3}\)  
h  
\(2^{2}\)  13  8.9  14  9.1  15  9.5  14  8.7  14  8.8 
\(2^{3}\)  14  11.3  15  10.9  15  12.1  15  12.2  15  11.8 
\(2^{4}\)  15  13.1  15  11.8  16  13.4  16  13.3  16  14.1 
\(2^{5}\)  17  13.9  17  13.3  16  14.7  19  13.7  19  14.9 
\(2^{6}\)  19  14.6  19  14.5  17  17.9  22  16.6  23  16.3 
\(2^{7}\)  21  23.0  21  14.9  22  17.3  26  17.7  27  18.4 
Results for the three dimensional Poisson control example with control constraints, for a range of values of h and \(\beta \), and preconditioner \({\mathcal {P}}_{1}\)
\({\mathcal {P}}_{1}\) Chebyshev  \(\beta =1\)  \(\beta =10^{1}\)  \(\beta =10^{2}\)  \(\beta =10^{3}\)  \(\beta =10^{4}\)  \(\beta =10^{5}\)  \(\beta =10^{6}\)  

\(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  \(u\ge 0\)  
\(u\le 0.01\)  \(u\le 0.1\)  \(u\le 1\)  \(u\le 3\)  \(u\le 20\)  \(u\le 100\)  \(u\le 300\)  
h  
\(2^{2}\)  7  10.8  8  10.7  9  11.1  10  11.2  11  11.2  12  11.1  12  11.4 
\(2^{3}\)  8  10.7  9  10.8  11  10.4  11  11.0  12  11.0  13  11.1  12  11.1 
\(2^{4}\)  9  10.9  10  10.8  11  10.8  12  10.9  12  11.2  13  11.1  13  11.1 
\(2^{5}\)  11  14.1  12  14.0  12  13.8  13  13.8  13  13.6  14  13.5  14  13.8 
Results for the heat equation control example with control constraints, for a range of values of h, \(\tau \), and \(\beta \), and preconditioner \({\mathcal {P}}_{1,T}\)
\(\beta =10^{1}\)  \(\beta =10^{2}\)  \(\beta =10^{3}\)  \(\beta =10^{4}\)  

\(0\le {}u\le 0.1\)  \(0\le {}u\le 1\)  \(0\le {}u\le 3\)  \(0\le {}u\le 30\)  
\({\mathcal {P}}_{1,T} \,\, (\tau =0.04)\)  
h  
\(2^{2}\)  13  13.1  15  16.5  16  19.7  21  31.3 
\(2^{3}\)  15  13.7  16  16.6  18  20.5  24  30.6 
\(2^{4}\)  16  14.0  18  17.1  20  20.8  24  28.2 
\(2^{5}\)  16  14.0  19  17.5  21  21.0  25  27.5 
\(2^{6}\)  18  14.5  19  17.5  22  21.1  27  27.2 
\({\mathcal {P}}_{1,T} \,\, (\tau =0.02)\)  
h  
\(2^{2}\)  14  13.0  16  15.9  17  19.8  23  31.9 
\(2^{3}\)  15  13.4  17  15.6  19  20.5  25  30.9 
\(2^{4}\)  16  13.7  18  16.0  21  20.9  25  27.6 
\(2^{5}\)  17  14.0  19  16.4  22  21.1  28  28.3 
\(2^{6}\)  15  13.4  19  16.2  22  20.8  27  27.6 
\({\mathcal {P}}_{1,T} \,\, (\tau =0.01)\)  
h  
\(2^{2}\)  14  12.2  16  15.4  18  19.6  24  31.0 
\(2^{3}\)  15  12.4  18  15.7  19  19.9  28  30.9 
\(2^{4}\)  16  12.8  18  15.7  21  20.2  27  28.2 
\(2^{5}\)  16  12.8  18  15.7  22  20.5  30  28.3 
\(2^{6}\)  17  13.0  19  15.8  22  20.4  29  28.5 
Results for the heat equation control example with control constraints, for a range of values of h, \(\tau \), and \(\beta \), and preconditioner \({\mathcal {P}}_{2,T}\)
\({\mathcal {P}}_{2,T}\)  \(\tau =0.04\)  \(\tau =0.02\)  

\(\beta =10^{1}\)  \(\beta =10^{2}\)  \(\beta =10^{3}\)  \(\beta =10^{4}\)  \(\beta =10^{1}\)  \(\beta =10^{2}\)  \(\beta =10^{3}\)  \(\beta =10^{4}\)  
\(0\le {}u\le 0.1\)  \(0\le {}u\le 1\)  \(0\le {}u\le 3\)  \(0\le {}u\le 30\)  \(0\le {}u\le 0.1\)  \(0\le {}u\le 1\)  \(0\le {}u\le 3\)  \(0\le {}u\le 30\)  
h  
\(2^{2}\)  13  8.1  15  9.9  16  11.7  21  18.1  14  7.9  16  9.6  17  11.8  23  18.5 
\(2^{3}\)  15  8.4  16  9.9  18  11.8  24  17.2  15  8.2  17  9.6  19  12.1  25  17.7 
\(2^{4}\)  16  8.5  18  10.3  20  12.1  24  16.3  16  8.4  18  9.8  21  12.4  25  16.2 
\(2^{5}\)  16  8.5  19  10.4  21  12.2  25  16.1  17  8.5  19  10.0  22  12.3  28  16.8 
\(2^{6}\)  18  8.8  19  10.4  22  12.7  27  16.3  15  8.2  19  9.9  22  12.6  27  16.5 
Results for the wave equation example with control constraints, for a range of values of h, \(\tau \), and \(\beta \)
\({\mathcal {P}}_{1,T},~~h=2^{4}\)  \({\mathcal {P}}_{1,T},~~h=2^{5}\)  \({\mathcal {P}}_{2,T},~~h=2^{4}\)  \({\mathcal {P}}_{2,T},~~h=2^{5}\)  

\(\beta \)  \(\beta \)  \(\beta \)  \(\beta \)  
\(10^{2}\)  \(10^{3}\)  \(10^{4}\)  \(10^{2}\)  \(10^{3}\)  \(10^{4}\)  \(10^{2}\)  \(10^{3}\)  \(10^{4}\)  \(10^{2}\)  \(10^{3}\)  \(10^{4}\)  
\(\tau \)  
0.04  13.7  17.7  13.3  14.7  18.0  13.4  10.1  12.1  9.9  11.1  12.7  10.0 
0.02  12.5  12.5  13.1  11.7  12.7  13.1  8.9  9.2  9.9  8.6  9.1  9.8 
0.01  14.7  10.9  10.6  31.6  50.8  10.9  10.9  13.5  8.1  22.8  23.7  8.5 
0.005  26.9  29.2  30.7  37.3  42.9  35.7  21.5  22.2  24.0  21.8  22.5  22.1 
Timedependent PDE constraints To demonstrate that our solvers are also able to handle matrix systems of vast dimension arising from timedependent PDEconstrained optimization problems, we present results in Table 7 for a heat equation control problem, with the PDE constraint given by \(y_{t}\nabla ^{2}y=u\) (for \(t\in (0,1]\)), and with additional control constraints imposed. The number of interior point iterations, and average Minres iteration count when \({\mathcal {P}}_{1,T}\) is applied, are provided for a range of h and \(\beta \). As mentioned earlier, the backward Euler method is used for the time discretization, and values of \(\tau =0.04\), 0.02 and 0.01 are tested for the timestep (in other words with 25, 50 and 100 time intervals). In Table 8, we present results obtained for the same problem using block triangular preconditioner \({\mathcal {P}}_{2,T}\) with Gmres. We once again observe a high degree of robustness in problem size (whether increased by refining the mesh in the spatial coordinates, or by decreasing the timestep) and regularization parameter.
Our final investigation involves the optimal control of the wave equation, which is the same problem as above, except with the PDE operator \(y_{tt}\nabla ^{2}y=u\) and with an initial condition imposed on \(y_{t}\) (which we set to be zero). The recent work [27] derives an implicit scheme for this problem, which involves averaging the Laplacian term in the PDE operator. Within the matrix \({\mathcal {K}}\), this leads to discrete approximations of the operator \(I\frac{\tau ^{2}}{2}\nabla ^{2}\) on the block diagonal entries, as well as additional entries on the two blocks below the diagonal (corresponding to the operators \(2I\) and \(I\frac{\tau ^{2}}{2}\nabla ^{2}\), respectively). The method is designed to be unconditionally convergent, while also removing the requirement of a Courant–Friedrichs–Lewy (CFL) condition of the form \(\tau \le {}h\) [27]. We investigate the potency of our preconditioners for this matrix system. In Table 9, we present the average number of Minres or Gmres iterations required to solve the systems arising from the interior point method. Although there is a larger variation in the number of steps required, due to the additional terms within the matrix system, the performance of the method is very encouraging considering the high complexity of the problem. We emphasize once again that the performance of the method is dependent somewhat on the severity of the box constraints imposed, however the numerical results obtained for a range of timeindependent and timedependent PDEconstrained optimization problems demonstrate the potency of the solvers presented in this manuscript.
6 Concluding remarks
In this paper we have presented a practical method for the interior point solution of a number of PDEconstrained optimization problems with state and control constraints, by reformulating the minimization of the discretized system as a quadratic programming problem. Having outlined the structure of the algorithm for solving these problems, we derived fast and feasible preconditioned iterative methods for solving the resulting Newton systems, which is the dominant portion of the algorithm in terms of computational work. Encouraging numerical results indicate the effectiveness and utility of our approach.
The problems we considered involved Poisson control, heat equation control, and both steady and timedependent convectiondiffusion control. A natural extension of this work would be to consider the control of systems of PDEs, for instance Stokes control and other problems in fluid flow, as well as the control of nonlinear PDEs, which arises in a wide range of practical scientific applications. The latter task would be accomplished by reformulating the discretization as a nonlinear programming problem—the robust solution of such formulations is a substantial challenge within the optimization community, but would represent significant progress in tackling realworld optimal control problems.
Footnotes
 1.
For the problems considered in [39], bounds for \(\alpha _{y}y+\alpha _{u}u\) are specified, where \(\alpha _{y}\) and \(\alpha _{u}\) are given constants.
 2.
This assumption holds for both Poisson control and convectiondiffusion control problems, for instance.
 3.
The main assumption made is that \({\widehat{S}}_{G}\) is invertible. This certainly holds unless \(({\bar{X}}+{\bar{Y}})^{\top }{\mathbf {v}}=\mathbf {0}\) for some \({\mathbf {v}}\), which in our setting implies that \(M^{1}(\beta {}M+D_{u})^{1/2}(M+D_{y})^{1/2}K^{\top }\) has an eigenvalue exactly equal to \(1\). As the matrices M, \(D_{y}\), \(D_{u}\) and K are unlikely to interact closely at any Newton step, this is extremely unlikely to occur and our assumption is therefore reasonable.
 4.
In the notation of Theorem 1, the matrices involved are \({\bar{X}}=K^{\top }M^{1}(\beta {}M+D_{u})^{1/2}\) and \({\bar{Y}}=(M+D_{y})^{1/2}\).
Notes
Acknowledgements
The authors are grateful to two anonymous referees for their careful reading of the manuscript and helpful comments. John W. Pearson was partially funded for this research by the Engineering and Physical Sciences Research Council (EPSRC) Fellowship EP/M018857/1. The work of Jacek Gondzio was supported by the EPSRC Research Grant EP/N019652/1.
References
 1.Battermann, A., Heinkenschloss, M.: Preconditioners for Karush–Kuhn–Tucker matrices arising in the optimal control of distributed systems. In: Desch, W., Kappel, F., Kunisch, K. (eds.) Control and Estimation of Distributed Parameter Systems, pp. 15–32. Birkhäuser, Basel (1998)Google Scholar
 2.Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer. 14, 1–137 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
 3.Benzi, M., Haber, E., Taralli, L.: Multilevel algorithms for largescale interior point methods. SIAM J. Sci. Comput. 31, 4152–4175 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
 4.Bergamaschi, L., Martinez, A.: RMCP: relaxed mixed constraint preconditioners for saddle point linear systems arising in geomechanics. Comput. Methods Appl. Mech. Eng. 221–222, 54–62 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
 5.Bergounioux, M., Haddou, M., Hintermüller, M., Kunisch, K.: A comparison of a Moreau–Yosida based active set strategy and interior point methods for constrained optimal control problems. SIAM J. Optim. 11, 495–521 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
 6.Borzì, A., Kunisch, K.: A multigrid scheme for elliptic constrained optimal control problems. Comput. Optim. Appl. 31(3), 309–333 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
 7.Borzì, A., Kunisch, K., Kwak, D.Y.: Accuracy and convergence properties of the finite difference multigrid solution of an optimal control optimality system. SIAM J. Control Optim. 41(5), 1477–1497 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
 8.Borzì, A., Schulz, V.: Multigrid methods for PDE optimization. SIAM Rev. 51(2), 361–395 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
 9.Drǎgǎnescu, A., Petra, C.: Multigrid preconditioning of linear systems for interior point methods applied to a class of boxconstrained optimal control problems. SIAM J. Numer. Anal. 50(1), 328–353 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
 10.Duff, I.S., Erisman, A.M., Reid, J.K.: Direct Methods for Sparse Matrices. Oxford University Press, New York (1987)zbMATHGoogle Scholar
 11.Elman, H.C., Ramage, A., Silvester, D.J.: IFISS: a computational laboratory for investigating incompressible flow problems. SIAM Rev. 56, 261–273 (2014)CrossRefzbMATHMathSciNetGoogle Scholar
 12.Ernst, O.G., Gander, M.J.: Why It is Difficult to Solve Helmholtz Problems with Classical Iterative Methods, Numerical Analysis of Multiscale Problems, Volume 83 of Lecture Notes in Computational Science and Engineering, pp. 325–363. Springer, Berlin, Heidelberg (2011)Google Scholar
 13.Gander, M.J., Graham, I.G., Spence, E.A.: Applying GMRES to the Helmholtz equation with shifted Laplacian preconditioning: what is the largest shift for which wavenumberindependent convergence is guaranteed? Numer. Math. 31(3), 567–614 (2015)CrossRefzbMATHMathSciNetGoogle Scholar
 14.Golub, G.H., Varga, R.S.: Chebyshev semiiterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods, I. Numer. Math. 3, 147–156 (1961)CrossRefzbMATHMathSciNetGoogle Scholar
 15.Golub, G.H., Varga, R.S.: Chebyshev semiiterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods, II. Numer. Math. 3, 157–168 (1961)CrossRefzbMATHMathSciNetGoogle Scholar
 16.Gondzio, J.: Convergence analysis of an inexact feasible interior point method for convex quadratic programming. SIAM J. Optim. 23, 1510–1527 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
 17.Gondzio, J.: Interior point methods 25 years later. Eur. J. Oper. Res. 218, 587–601 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
 18.Grotte, M.J., Huber, J., Kourounis, D., Schenk, O.: Inexact interiorpoint method for PDEconstrained nonlinear optimization. SIAM J. Sci. Comput. 36, A1251–A1276 (2014)CrossRefzbMATHMathSciNetGoogle Scholar
 19.Herzog, R., Kunisch, K.: Algorithms for PDEconstrained optimization. GAMM Mitt. 33(2), 163–176 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
 20.Hinze, M., Köster, M., Turek, S.: A Hierarchical Space–Time Solver for Distributed Control of the Stokes Equation. Priority Programme 1253, Preprint Number SPP12531601 (2008)Google Scholar
 21.Hinze, M., Köster, M., Turek, S.: A Space–Time Multigrid Solver for Distributed Control of the TimeDependent Navier–Stokes System. Priority Programme 1253, Preprint Number SPP12531602 (2008)Google Scholar
 22.Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE Constraints, Mathematical Modelling: Theory and Applications. Springer, New York (2009)zbMATHGoogle Scholar
 23.Ipsen, I.C.F.: A note on preconditioning nonsymmetric matrices. SIAM J. Sci. Comput. 23(3), 1050–1051 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
 24.Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications, Vol. 15 of Advances in Design and Control. SIAM, Philadelphia (2008)Google Scholar
 25.Kelley, C.T., Sachs, E.W.: Multilevel algorithms for constrained compact fixed point problems. SIAM J. Sci. Comput. 15, 645–667 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
 26.Kuznetsov, Y.A.: Efficient iterative solvers for elliptic finite element problems on nonmatching grids. Russ. J. Numer. Anal. Math. Model. 10, 187–211 (1995)zbMATHMathSciNetGoogle Scholar
 27.Li, B., Liu, J., Xiao, M.: A fast and stable preconditioned iterative method for optimal control problem of wave equations. SIAM J. Sci. Comput. 37(6), A2508–A2534 (2015)CrossRefzbMATHMathSciNetGoogle Scholar
 28.Mittelmann, H.D., Maurer, H.: Solving elliptic control problems with interior point and SQP methods: control and state constraints. J. Comput. Appl. Math. 120, 175–195 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
 29.Murphy, M.F., Golub, G.H., Wathen, A.J.: A note on preconditioning for indefinite linear systems. SIAM J. Sci. Comput. 21, 1969–1972 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
 30.Napov, A., Notay, Y.: An algebraic multigrid method with guaranteed convergence rate. SIAM J. Sci. Comput. 34(2), A1079–A1109 (2012)CrossRefzbMATHGoogle Scholar
 31.Notay, Y.: An aggregationbased algebraic multigrid method. Electron. Trans. Numer. Anal. 37, 123–146 (2010)zbMATHMathSciNetGoogle Scholar
 32.Notay, Y.: Aggregationbased algebraic multigrid for convectiondiffusion equations. SIAM J. Sci. Comput. 34(4), A2288–A2316 (2012)CrossRefzbMATHGoogle Scholar
 33.Notay, Y.: AGMG Software and Documentation (2012). http://homepages.ulb.ac.be/~ynotay/AGMG
 34.Paige, C.C., Saunders, M.A.: Solutions of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975)CrossRefzbMATHMathSciNetGoogle Scholar
 35.Pearson, J.W., Stoll, M., Wathen, A.J.: Regularizationrobust preconditioners for timedependent PDEconstrained optimization problems. SIAM J. Matrix Anal. Appl. 33(4), 1126–1152 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
 36.Pearson, J.W., Wathen, A.J.: Fast iterative solvers for convectiondiffusion control problems. Electron. Trans. Numer. Anal. 40, 294–310 (2013)zbMATHMathSciNetGoogle Scholar
 37.Pearson, J.W., Wathen, A.J.: A new approximation of the Schur complement in preconditioners for PDEconstrained optimization. Numer. Linear Algebra Appl. 19, 816–829 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
 38.Pestana, J., Rees, T.: Nullspace preconditioners for saddle point systems. SIAM J. Matrix Anal. Appl. 37(3), 1103–1128 (2016)CrossRefzbMATHMathSciNetGoogle Scholar
 39.Porcelli, M., Simoncini, V., Tani, M.: Preconditioning of activeset Newton methods for PDEconstrained optimal control problems. SIAM J. Sci. Comput. 37(5), S472–S502 (2016)CrossRefzbMATHMathSciNetGoogle Scholar
 40.Rusten, T., Winther, R.: A preconditioned iterative method for saddle point problems. SIAM J. Matrix Anal. Appl. 13, 887–904 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
 41.Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Comput. 7(3), 856–869 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
 42.Schiela, A., Ulbrich, S.: Operator preconditioning for a class of inequality constrained optimal control problems. SIAM J. Optim. 24(1), 435–466 (2014)CrossRefzbMATHMathSciNetGoogle Scholar
 43.Silvester, D., Elman, H., Ramage, A.: Incompressible Flow and Iterative Solver Software (IFISS), Version 3.3 (2014) http://www.manchester.ac.uk/ifiss
 44.Stoll, M., Wathen, A.: AllatOnce Solution of TimeDependent PDEConstrained Optimization Problems, Oxford Centre for Collaborative Applied Mathematics Technical Report 10/47 (2010)Google Scholar
 45.Tröltzsch, F.: Optimal Control of Partial Differential Equations: Theory, Methods and Applications. American Mathematical Society, Providence, RI (2010)CrossRefzbMATHGoogle Scholar
 46.Ulbrich, M., Ulbrich, S.: Primaldual interiorpoint methods for PDEconstrained optimization. Math. Program. 117(1–2), 435–485 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
 47.Wathen, A.J.: Realistic eigenvalue bounds for the Galerkin mass matrix. IMA J. Numer. Anal. 7(4), 449–457 (1987)CrossRefzbMATHMathSciNetGoogle Scholar
 48.Wathen, A.J., Rees, T.: Chebyshev semiiteration in preconditioning for problems including the mass matrix. Electron. Trans. Numer. Anal. 34, 125–135 (2009)zbMATHMathSciNetGoogle Scholar
 49.Weiser, M.: Interior point methods in function space. SIAM J. Control Optim. 44, 1766–1786 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
 50.Weiser, M., Deuflhard, P.: Inexact central path following algorithms for optimal control problems. SIAM J. Control Optim. 46, 792–815 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
 51.Weiser, M., Gänzler, T., Schiela, A.: A control reduced primal interior point method for a class of control constrained optimal control problems. Comput. Optim. Appl. 41, 127–145 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
 52.Wright, S.J.: PrimalDual InteriorPoint Methods. SIAM, Philadelphia (1997)CrossRefzbMATHGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.