# A Domain Decomposition Approach to Solve Dynamic Optimal Power Flow Problems in Parallel

## Abstract

We propose a parallel solver for linear systems of equations arising from the application of Primal Dual Interior Point methods to Dynamic Optimal Power Flow problems. Our solver is based on the Generalised Minimal Residual method in combination with an additive Schwarz domain decomposition method as preconditioner. This preconditioner exploits the structure of Dynamic Optimal Power Flow problems which, after linearization, is given as a matrix with large diagonal blocks and only a few off-diagonal elements. These elements correspond to intertemporal couplings due to ramping and energy storage constraints and are partially neglected in order to solve the problem in parallel. We test our method on a large-scale optimisation problem and show that a parallel speedup can be obtained.

## Keywords

Dynamical OPF Multi-period OPF Time domain decomposition Additive Schwarz method Preconditioner## 1 Introduction

While today’s power grid infrastructure has been designed for centralised power generation in conventional power plants, the ever increasing share of renewable energy sources (*RES*) leads to an increasingly uncertain, volatile and decentralised generation. In order to ensure a reliable grid operation and to maintain today’s security of supply, it is necessary to develop methods to simulate accurately a power grid at high temporal resolutions. This can be done by efficient methods for power grid optimisation, including an accurate consideration of the non-linear and non-convex AC power flow constraints. And these are to be improved.

The task to find the optimal operating state of a given power grid, also known as Optimal Power Flow (*OPF*), is formalized as the minimisation of a cost function with respect to a vector of continuous optimisation variables like the generated active power and the generated reactive power. Dynamic Optimal Power Flow (*DOPF*) considers several OPF problems, each corresponding to one specific point in time. In this case, the power grid’s power demand is time-dependent and additional constraints must be taken into account. These constraints couple some of the optimisation variables corresponding to different time steps. Among others, these couplings are introduced by energy storage facilities and ramping constraints for conventional power plants [20]. Since DOPF is a large-scale non-linear and non-convex optimisation problem, solving it in parallel is of crucial importance.

For this kind of problems Primal Dual Interior Point Methods (*PDIPM*) have proven to be among the most powerful algorithms. The main computational effort, when PDIPM is applied, lies in solving linear systems of equations [14]. And this a task suitable to be carried out in parallel on multi-core CPUs.

There exist a few works on solving linear systems, that arise from PDIPM applied to DOPF, in parallel. One approach is based on parallel matrix factorization by means of Schur complement techniques [6, 20]. Other strategies use Benders Decomposition to decompose the complete optimisation problem into smaller ones [1].

Our contribution is the use of a *Schwarz Domain Decomposition Method* as preconditioner for Krylov-type iterative methods for solving these linear systems of equations in parallel. The original Schwarz method was formulated in 1870 as theoretical tool for proving existence of elliptic partial differential equations (*PDEs*) on complicated domains [17]. Later on, modifications of it have been used as stand-alone iterative methods for solving PDEs and have become a standard technique for preconditioning Krylov-type methods in the context of PDEs [18]. We apply this technique to DOPF by decomposing the time domain of interest into several smaller subdomains, which allows the use of multiple processors for computation.

## 2 Dynamic Optimal Power Flow

We will now formulate the DOPF problem in a mathematically rigorous way. This formulation has already been stated in more detail by several authors, e.g. in [12, 20] and [16]. We would like to mention that we consider a continuous formulation of the DOPF problem, i.e. one without any discrete decision variables. A non-linear mixed-integer formulation of an OPF is considered, for example, in [19].

### 2.1 Formulation of the DOPF

In order to formulate the DOPF problem for a given time period of interest [0, *T*], we consider a uniform partition \(0 = T_1 < T_2 < \ldots < T_{N_T} = T\) with constant step size *τ* = *T*_{t} − *T*_{t−1} for all *t* ∈**T** ∖{1}, where **T**:= {1, …, *N*_{T}}.

*N*

_{B}nodes denoted by

**B**:= {1, …,

*N*

_{B}} and

*N*

_{E}transmission lines denoted by

**E**⊂

**B**×

**B**, is described by an admittance matrix

*Y*=

*Y*

^{T}, and furthermore

*Y*

_{lk}=

*Y*

_{kl}≠ 0 if and only if there is a branch connecting node

*k*and

*l*, i.e., (

*k*,

*l*) ∈

**E**and (

*l*,

*k*) ∈

**E**. Therefore,

*Y*is a sparse matrix for most real world power grids.

*k*∈

**B**for time step

*t*∈

**T**is given as the complex number

*active*power and energy for time step

*t*is given by with \(P^{t} \in \mathbb {R}^{N_{P}}\).

*SOC*) of storage facilities do not directly influence the power flow equations, the corresponding rows in

*C*

_{P}are equal to the zero vector.

#### 2.1.1 Constraints

*t*by \(L^{t}_{p}\) and \(L^{t}_{q}\), respectively, and assigning the loads to the nodes with the help of the matrix \(C_{L} \in \{0,1\}^{N_{\mathbf {B}} \times N_{L}}\), the AC power flow equations are given by [22]

*k*. Note that we dropped the index

*t*to improve readability.

*t*, and for generators

*k*,

*l*∈

**DC**modelling a DC line \(\overline {kl}\):

*ν*

_{DC}∈ (0, 1).

*ν*

_{l},

*ν*

_{g}∈ (0, 1) describing the efficiency of the loading process \(P_{sl,i}^{t}\) and generation process \(P_{sg,i}^{t}\), respectively.

*p*

_{kl}, must be constrained:

*α*is the ramping factor in percentage per unit time.

#### 2.1.2 Cost Function

*f*accounts for expenditure and income related to active power processes for the complete time interval [0,

*T*] and is composed of contributions from each time step

*t*∈

**T**[5]:

#### 2.1.3 Optimisation Problem

*n*

^{x, t}:= 2

*N*

_{B}+

*N*

_{P}+

*N*

_{Q}and

*n*

^{x}:=

*N*

_{T}

*n*

^{x, t}and, as of now, let

*n*

^{λ, t}be defined as the number of equality constraints corresponding to time step

*t*, i.e. \(\lambda ^{t} \in \mathbb {R}^{n^{\lambda ,t}}\) with \(\lambda ^{t} = (\lambda _I^{t}, \lambda _S^{t})\) denoting the Lagrangian multipliers corresponding to \(g_I^{t}\) and \(g_S^{t}\), respectively. Analogously, \(\mu ^{t} = (\mu _I^{t}, \mu _R^{t}) \in \mathbb {R}^{n^{\mu ,t}}\) denotes the Lagrangian multipliers for the inequality constraints \(h_I^{t}\) and \(h_R^{t}\). Consequently, we define

*n*

^{λ}:=

*N*

_{T}

*n*

^{λ, t}and

*n*

^{μ}:=

*N*

_{T}

*n*

^{μ, t}.

Note that the ramping constraints *h*_{R} given by (13) and the storage constraints *g*_{S} given by (9) establish the only couplings between the variables corresponding to different time steps. Without them, a solution to (16) could be obtained by solving *N*_{T} independent OPF problems. The optimisation problem (16) is non-linear due to the AC equations and due to the voltage and line flow inequality constraints.^{1} Moreover, the latter ones and the AC equations are the sources of non-convexity in (16).

## 3 Primal Dual Interior Point Method for DOPF

In this section, we briefly describe the application of the Primal Dual Interior Point Method (*PDIPM*) to a DOPF problem. The description is in line with the implementation found in the Matlab package MATPOWER [21].

### 3.1 The PDIPM Algorithm

*LICQ*) [14], for every local minimum

*x*

^{∗}of (18), there exist corresponding Lagrangian multipliers

*λ*

^{∗},

*μ*

^{∗}such that (

*x*

^{∗},

*λ*

^{∗},

*μ*

^{∗}) are in accord with the Karush-Kuhn-Tucker (

*KKT*) conditions [5]:

*S*

_{kk}=

*s*

_{k}.

*k*. This term keeps the slack variables

*s*away from zero and, thus, the inequality constraints inactive. This means that solutions (

*x*

^{(k)},

*s*

^{(k)},

*λ*

^{(k)},

*μ*

^{(k)}) to this modified version of Eq. (21) are inside the feasible region and

*not*on its surface, hence, the method is called interior point method. More rigorously formulated, we find

*barrier parameters*

*σ*= 0.1 is the

*centering parameter*and (

*μ*

^{(k−1)})

^{T}

*s*

^{(k−1)}is called

*complementary gap*. Additionally, \(e = (1, \ldots , 1) \in \mathbb {R}^{n^{\mu }}\). For details we refer the inclined reader to [5, 8] and [21].

### Remark

A very basic PDIPM is applied, i.e. there is no globalisation strategy. Since we focus on parallel linear algebra techniques for DOPF problems, a locally convergent algorithm is sufficient to our purpose.

### 3.2 KKT Matrix in PDIPM

*k*. Assuming that

*s*,

*μ*> 0, the Newton system (23) can be transformed into a reduced, symmetric saddle point form

The KKT matrices *A*, arising in every PDIPM iteration *k*, may become more and more ill-conditioned when a KKT point is approached. This is caused by the sequence of scaling matrices *Σ*^{(k)}. If strict complementary holds at the KKT point *x*^{∗}, if *x*^{(k)} → *x*^{∗} and if *h*_{i}(*x*^{∗}) is an active inequality, one can show that \(\varSigma ^{(k)}_{ii} \rightarrow \infty \). On the other hand, \(\varSigma ^{(k)}_{ii} \rightarrow 0\), if *h*_{i}(*x*^{∗}) is inactive [14].^{2} For this reason, many IPM software packages (like IPOPT [7]) use direct methods such as *LDL*^{T}-factorizations to solve the appearing linear systems. These methods are less sensitive to ill-conditioned matrices.

In contrast, iterative linear solvers like GMRES [15] are very sensitive to ill-conditioned matrices. A good preconditioner is needed to lower the matrices’ condition numbers. In exchange, they offer a higher potential of parallelization and allow the use of inexact Interior Point methods, see Sect. 3.3.

*K*, defined as

*Kv*=

*b*. By rule of thumb, a spectrum which is clustered away from 0 is beneficial to the rate of convergence of GMRES. [11, Th. 4.62]

*constraint preconditioner*is an example of a preconditioner. It is given by

*H*, that is easy to factorize, e.g., \(\tilde {H} = \text{diag}(H)\) [14].

One can show that \(\sigma (\tilde {K}^{-1}K) = \{1\}\, \cup \, \tilde {\sigma }\) with \(|\tilde {\sigma }| = n^{x}-n^{\lambda }\). Therefore, at most *n*^{x} − *n*^{λ} eigenvalues of \(\tilde {K}^{-1}K\) are not equal to 1. The distribution of these remaining eigenvalues depends on how well \(\tilde {H}\) approximates *H* on the nullspace of *B* [10].^{3}

In Sect. 4, we describe how to take advantage of the special structure given by (16) to construct a parallel preconditioner. Furthermore, we will see that there is a close relation between our proposed preconditioner and the constraint preconditioner defined above, see Sect. 4.4.

### 3.3 Inexact PDIPM

*Δ*

^{(k)}satisfies

*k*and computed the relative tolerance for the iterative solver as follows

^{−10}.

### 3.4 Termination Criteria for PDIPM

*x*

^{−}denoting the previous iterate.

## 4 Schwarz Domain Decomposition Method for DOPF

The key idea behind Schwarz domain decomposition methods applied to DOPF problems is the decomposition of the set of time steps **T** into several smaller subsets. Due to this decomposition, it is possible to define corresponding submatrices of the global KKT matrix defined in (24) that are smaller in size and therefore faster to factorize. Furthermore, the factorization of these submatrices can be done in parallel. After computing subsolutions corresponding to these submatrices, one can reconstruct an approximate solution to the global system (24), which is used as preconditioner within a Krylov-type iterative solver.

In Sect. 4.1 we introduce the mathematics, which describe the additive Schwarz Method (*ASM*) in the context of DOPF problems. In Sect. 4.2 we briefly describe some issues related to its implementation. Section 4.3 is intended to provide a deeper insight into the subproblems that have to be solved when the ASM is applied. Finally, we describe the relation between the ASM and the above mentioned constraint preconditioner in Sect. 4.4.

### 4.1 Mathematical Formulation of the Additive Schwarz Method

**T**= {1, …,

*N*

_{T}} into

*q*non-overlapping subdomains:

*s*

_{ol}time steps on both ends, yielding an overlapping decomposition of

**T**(see Fig. 1):

*s*

_{ol}∈{1, 2, 3}.

**T**

_{l}, we write

*x*

_{[l]}and

*λ*

_{[l]}yields the following definition:

**T**

_{l}. Moreover, let \(R_l \in \{0,1\}^{n_l \times n}\) be a

*restriction matrix*corresponding to the subset

**T**

_{l}defined by its action:

*l*≠ 1 and

*l*≠

*q*:

*l*. Analogously, we define \(\tilde {R}_l\) as restriction operator corresponding to the non-overlapping subdomain \(\tilde {\mathbf {T}}_l\).

*A*

_{l}from the KKT matrix

*A*by multiplying it with

*R*

_{l}:

On the important assumption, that all submatrices *A*_{l} are *non-singular*, we are able to formulate the multiplicative Schwarz Method (*MSM*) for approximately solving *AΔ*_{R} = *b*. We do this in the form of an algorithm [18]:

### Algorithm 1 Multiplicative Schwarz method

*l*, the current error with respect to the exact solution

*Δ*

_{R}, given by

*r*

^{l−1}to subdomain

**T**

_{l}and solving with \(A_l^{-1}\), one obtains an approximation

*R*

_{l}

*e*

^{l−1}on subdomain

**T**

_{l}. Prolongation with \(R_l^{T}\) yields a correction \(\delta _l = R_l^{T}\hat {e}^{l}\) to the current approximation \(\varDelta ^{l-1}_{R}\). Thus, the multiplicative Schwarz method can be seen as “defect correction algorithm”.

^{4}

Unfortunately, the the MSM is a sequential algorithm. In order to parallelize it, one omits residual updates in each iteration, yielding the ASM [18]:

### Algorithm 2 Additive Schwarz method *ΔRq* = ASM(*b*)

*AΔ*

_{R}=

*b*is then given by

*M*

_{ASM}is symmetric but in general not positive definite, we use the Generalized Minimal Residual (

*GMRES*) method for solving the non-symmetric system (37). In general, the ASM computes a less accurate approximation to the solution of

*AΔ*

_{R}=

*b*and therefore needs an increased number of GMRES iterations compared to the MSM given by Algorithm 1. However, the ASM offers a higher potential of parallelization, which results usually in better parallel scaling.

### 4.2 Implementation

In our implementation, we use one processor per subdomain **T**_{l} and distribute *A* such that every processor stores \(\tilde {R}_l A\) in its local memory. \(\tilde {R}_l A\) contains the non-overlapping portion of rows of \(A_l = R_l A R_l^{T}\).

To set up *M*_{ASM} once, every processor first has to form its local submatrix *A*_{l}, i.e., in this step the overlapping part of *A*_{l} has to be communicated to processor *l* by processor *l* − 1 and *l* + 1. Afterward, each processor computes an *LU*-factorization of *A*_{l}, i.e., *A*_{l} = *L*_{l}*U*_{l}. This step doesn’t involve any inter-processor communication and can be done in parallel.

The employment of the ASM as a preconditioner inside the GMRES method, requires to compute *M*_{ASM}*v*^{(k)} in each iteration *k*. *v*^{(k)} denotes the *k*-th basis vector of the Krylov subspace \(\mathcal {K}_{k}(AM_{ASM},b)\). On the assumption that the basis vector’s data layout is the same as the matrix’s one, i.e. every process stores \(v^{(k)}_{l} := \tilde {R}_{l} v^{(k)}\), communication with process *l* − 1 and *l* + 1 is required to multiply *M*_{ASM} with *v*^{(k)}.^{5} The computation of \(A_l^{-1} v^{(k)}_{l}\) is done by one forward substitution with *L*_{l} and a backward substitution with *U*, respectively. This step does not involve any communication. After this, the local solution \(A_l^{-1} v^{(k)}_{l}\) is prolonged back to obtain the global result of the matrix vector product *M*_{ASM}*v*^{(k)}. This again requires communicating the overlapping part of \(v^{(k)}_l\) to process *l* − 1 and *l* + 1. Finally, it’s necessary to multiply *A* with *M*_{ASM}*v*^{(k)}. Due to *A*’s data layout and its off-diagonal elements, corresponding to intertemporal couplings, additional communication is required.

*restricted*version of ASM [4], which is given by

To further stabilise the *LU*-factorization, a small regularisation parameter *𝜖* = 10^{−8} is added to the KKT matrix *A*, i.e. *A* + *𝜖I*.

### 4.3 Structure of Local KKT Matrices

In this section, we investigate the structure of the local submatrices *A*_{l} defined in Eq. (36). A good way to get an idea of their structure is to permute the KKT matrix *A* in such a way that all rows and columns belonging to *one* time step are grouped together. This yields a block diagonal matrix with some off-diagonal elements. Every block represents one time step and the off-diagonal elements arise due to the time-dependent constraints. Extracting the time steps belonging to the subdomain **T**_{l} results in a permuted version of the submatrix *A*_{l}.^{6} Its structure is exactly the same as the one of the original KKT matrix *A*. If the off-diagonal elements are taken into account as “boundary conditions”, it is possible to interpret the submatrix *A*_{l} as representing an optimisation problem in its own, i.e. the original dynamical OPF problem is reduced to the subdomain **T**_{l}. Figure 2 tries to demonstrate this. As a consequence the submatrices *A*_{l} tend to be ill-conditioned.

### 4.4 Relationship Between the ASM and the Constraint Preconditioner

*s*

_{ol}= 0. In this case, the corresponding ASM preconditioner,

*l*. Since there are only time-dependent inequality constraints left, the coupling between the variables belonging to different time steps has to arise in the (1, 1)-block of the KKT matrix, namely in the (∇

*h*)

^{T}

*Σ*(∇

*h*) part. This means that multiplying \(\tilde {M}_{ASM}\) with b is like solving a modified KKT system, or equivalently,

At this point, it also becomes clear that the ASM can only be interpreted as a constraint preconditioner, if there are no time-dependent equality constraints. If there existed time-dependent equality constraints, the (2, 1)- and (1, 2)-block of \({\tilde {M}_{ASM}}^{-1}\) would have changed too.

*n*

^{λ}and the remaining

*n*

^{x}−

*n*

^{λ}eigenvalues are solutions of a generalized eigenvalue problem of the following form [10]:

*q*, one can expect \(\tilde {H}\) to be a good approximation to

*H*, leading to a clustered spectrum \(\sigma (\tilde {M}_{ASM}A)\) close to one. However, the more subdomains, the more neglected couplings and the less accurate does \(\tilde {H}\) approximate

*H*. This results in a more scattered eigenvalue distribution of \(\tilde {M}_{ASM}A\), but still a large number of eigenvalues are equal to 1.

## 5 Numerical Experiments

In this section, we present some results for our proposed method applied to a DOPF problem. In its first part, we discuss the parallel speedup of our solver. In the second one, we investigate the way in which the KKT matrix’s spectrum *σ*(*A*) changes with the iterations of the interior point method. The test grid, which we used for our experiments, represents a possible variant of the German Transmission Grid in the year 2023 and consists of 1215 nodes, 2247 AC lines, 8 DC lines, 181 conventional power plants, 355 renewable energy sources and 67 storage facilities. For more details, we refer the reader to [12].

For solving (16), we use the PDIPM algorithm mips which is written in Matlab code and part of MATPOWER [21]. In this algorithm, we replace the standard Matlab backslash operator ∖ for solving linear systems by our own linear solver. Our solver applies a right preconditioned GMRES method. We use the ASM as the preconditioner. For computing the *LU*-factorizations of the local systems *A*_{l}, we use the software package Intel^{®} MKL PARDISO. Our solver is written in C++ and makes use of the KSPGMRES and PCASM methods provided by PETSc [2], which is compiled in Release mode with the Intel compiler 17.0. The computation of the eigenvalues of the KKT matrices and the preconditioned KKT matrices have been done with SLEPc [9] and Matlab. Inter-process communication is realised by means of the Message Passing Interface (MPI). The numerical experiments were carried out on the BwForCluster MLS&WISO Production at Heidelberg University with two Intel Xeon E5-2630v3 processors i.e. 16 CPU cores at 2.4 GHz per node, and 64 GB working memory.

### 5.1 Parallel Speedup

In our numerical experiments, we concatenated the 48 h load profile, which was part of the described test grid, to obtain a test case comprising 48 days with a temporal resolution of 1 h, i.e. *N*_{T} = 960 time steps. This causes the linear system (24), which has to be solved in every PDIPM iteration, to have the dimension *n*^{x} + *n*^{λ} ≈ 6.168 ⋅ 10^{6}.

Furthermore, we set *α* = 0.4, which led to ramping constraints limiting the change of power generation of conventional power plants by 40%/h of their respective maximum power generation.

We set *𝜖*_{feas} = *𝜖*_{grad} = *𝜖*_{cost} = *𝜖*_{comp} = 10^{−5} as PDIPM termination criteria and limited the number of PDIPM iterations to 300. The number of GMRES iterations was restricted to 1500. The GMRES method was allowed to restart after 300 iterations. Its relative residual tolerance rtol was determined as described in Sect. 3.3.

The ASM method was not restricted and the overlap *s*_{ol} was set to 3 for all tests.

The parallel speedup on *q* processors for the first *k* PDIPM iterations is defined as \(s_{q}(k) := \frac {T_{1}(k)}{T_{q}(k)}\). We computed the sequential reference time *T*_{1}(*k*) by solving the linear systems (24) using a *LU*-factorization and adding up the times needed to solve them up to the *k*-th PDIPM iteration. MKL PARDISO’s sequential *LU*-factorization algorithm was applied.

Analogously, *T*_{q}(*k*) was obtained by adding up the times needed to solve the same linear systems, but this time iteratively, i.e. the GMRES method and the ASM were used. In Fig. 3 we plotted the results of our speedup computations for different numbers of processors. The dashed graph represents the speedup obtained when the KKT systems (24) are solved directly with MKL PARDISO’s parallel implementation of the *LU*-factorization using 16 processors. For this amount of processors we observed the best speedup.

Besides, we scaled the *y*-axis logarithmically to ease drawing a comparison with the *LU*-solver and to emphasise the large speedups during the first iterations.^{7} We observed for our proposed iterative solver, no matter the number of processors *q*, a clear speedup in comparison with the *LU*-solver. The parallelization, due to the ASM, shows its power during the first PDIPM iterations. Unfortunately, this initial speedup decreases rapidly. After 40 iterations we end up with a total speedup of five. This decrease is accompanied by an increase in GMRES iterations. To demonstrate this, we plotted in Fig. 4 the number of GMRES iterations needed to solve the KKT system at the *k*-th PDIPM iteration. The first thing to note is, that the number of GMRES iterations does not only change with PDIPM iterations, but also with the number of processors used to solve the KKT systems. It’s likely that the increase in GMRES iterations with PDIPM iterations is caused by a change in the spectrum of the original KKT matrix *A*. The increase with the number of processors can be explained as follows: The more proccesors we use, the smaller the subproblems, described by *A*_{l}, become and, hence, the more intertemporal couplings are neglected, which reduces the “approximity” of *M*_{ASM} to *A*^{−1}. And the closer *M*_{ASM} is to *A*^{−1}, the more the spectrum of the preconditioned operator *σ*(*AM*_{ASM}) is clustered around 1.

Before we start to actual investigate the spectrum of the matrices in the next part of this section, we would like to summarise our findings. Even though we observed a rapid decrease in the speedup, it was possible to obtain a moderate speedup with only 16 processors and in comparison to a standard parallel *LU*-factorization also using 16 processors, our solver was more than a factor 2 faster.

### 5.2 Eigenvalue Distribution

Since the GMRES method converges faster if the eigenvalues of matrix *A* are clustered around 1, i.e. it needs less iterations, we would like to investigate how much the ASM, applied as preconditioner, improves the eigenvalues’ distribution. Therefore, we calculated the eigenvalues with the largest and smallest magnitude of the KKT matrix *A* and the preconditioned KKT matrix *AM*_{ASM}, respectively. We write \(\lambda ^{A}_{max} = \max \{|\lambda | \colon \lambda \in \sigma (A)\}\) and \(\lambda ^{A}_{min} = \min \{|\lambda | \colon \lambda \in \sigma (A)\}\). Seeing that it takes very long to compute, or that it is even not possible to compute the eigenvalues for the complete *N*_{T} = 960 time steps, we decided to calculate them for *N*_{T} = 96 time steps and for each PDIPM iteration. We set the overlap to *s*_{ol} = 1 and used *q* = 32 subdomains. The results are presented in Fig. 5. The red lines depict the development of \(\lambda ^{A}_{max}\) and \(\lambda ^{A}_{min}\) of the KKT matrix, respectively. Thus, they form the limits for all possible magnitudes of eigenvalues corresponding to *A*. The latter is indicated by the red shaded area. Analogously, the blue lines and the blue area represent the spectrum of the preconditioned KKT matrix.

The spectrum of the preconditioned KKT matrix, is clustered around 1 in every interior point method iteration. Looking at the magnified part of the graph shows that \(\lambda ^{AM_{ASM}}_{min}\) becomes smaller between the 10-th and the 20-th iteration. This may explain the increase in GMRES iterations observed in our numerical experiments and plotted in Fig. 4.

## 6 Conclusion

In this work we proposed a way of solving linear systems arising from Dynamic Optimal Power Flow (DOPF) problems in parallel by means of an overlapping Schwarz domain decomposition method, namely the additive Schwarz method. It was shown how to apply this method in the context of DOPF problems and that the obtained submatrices correspond to localised formulations of a DOPF problem with additional boundary conditions. Numerical tests on one large-scale DOPF problem showed that the combination of the Generalized Minimal Residual (GMRES) method and the Additive Schwarz Method (ASM) is able to solve DOPF problems. Furthermore, we were able to show that this combination yields good parallel speedup for some of the PDIPM iterations. And, in a small example, we demonstrated that the ASM is a good preconditioner in the sense, that it clusters the eigenvalues around 1.

Unfortunately, the proposed solver had problems with some of the KKT systems arising in the large-scale DOPF problem. Therefore, it is necessary to improve it and this will be one of our main goals in future work.

## Footnotes

- 1.
We have not explicitly stated an equation for the power flow

*p*_{kl}through the transmission line (*k*,*l*). Please refer to [13]. - 2.
*Proof (sketch)*The KKT conditions require that \(\mu ^{*}_{i}(h_{i}(x^{*})+s_{i}^{*}) = 0\). If

*h*_{i}(*x*^{∗}) is active, then \(h_{i}(x^{*}) = 0 \text{ and } s_{i}^{*} = 0\). Upon strict complementarity follows that \(\mu ^{*}_{i} \neq 0\). Finally, if*x*^{(k)}→*x*^{∗}, then \(\mu _{i}^{(k)} \rightarrow \mu _{i}^{*}\), \(s_{i}^{(k)} \rightarrow s_{i}^{*} \) and \(\varSigma ^{(k)}_{ii} = \frac {\mu ^{(k)}_{i}}{s^{(k)}_{i}} \rightarrow \infty \) . □ - 3.
Let

*Z*denote the nullspace of*B*, i.e.*BZ*= 0. Then \(\tilde {H}\) approximates*B*on its nullspace, if \((Z^{T}\tilde {H}Z)^{-1}(Z^{T}HZ) \approx 1 \). - 4.
In contrast to “classical defect correction” algorithms, there is no converging sequence \(\left (\varDelta ^{l}_{R}\right )_{l \in \mathbb {N}} \rightarrow \varDelta ^{R} \). Nonetheless, the algorithm’s construction implies that \(\varDelta ^{q}_{R} \approx \varDelta ^{R}\).

- 5.
The reader may wonder why we have to make an assumption about the basis vectors’ data layout: Our GMRES implementation is part of the external library PETSc and a quick glance at their source code did not reveal to us how they manage the according data.

- 6.
Note that this extraction cannot be done by multiplying

*A*with*R*_{l}and*R*_{l}^{T}. - 7.
We remark that the PDIPM needed in most of the tested cases more than 40 iterations to converge, yet, it converged after at most 45 iterations.

## Notes

### Acknowledgements

This work was carried out with the financial support of the German Research Foundation (DFG) within the project 242471205. Moreover, the authors acknowledge support by the state of Baden-Württemberg through bwHPC and the German Research Foundation (DFG) through grant INST 35/1134-1 FUGG.

## References

- 1.N. Alguacil, A.J. Conejo, Multiperiod optimal power flow using benders decomposition. IEEE Trans. Power Syst.
**15**(1), 196–201 (2000)CrossRefGoogle Scholar - 2.S. Balay, W.D. Gropp, L. Curfman McInnes, B.F. Smith, Efficient management of parallelism in object oriented numerical software libraries, in
*Modern software tools in scientific computing*, ed. by E. Arge, A.M. Bruaset, H.P. Langtangen (Birkhäuser Press, Boston 1997), pp. 163–202CrossRefGoogle Scholar - 3.S. Bellavia, Inexact interior-point method. J. Optim. Theory Appl.
**96**(1), 109–121 (1998)MathSciNetCrossRefGoogle Scholar - 4.X.-C. Cai, M. Sarkis, A restricted additive Schwarz preconditioner for general sparse linear systems. SIAM J. Sci. Comput.
**21**(2), 792–797 (1999)MathSciNetCrossRefGoogle Scholar - 5.F. Capitanescu, M. Glavic, D. Ernst, L. Wehenkel, Interior-point based algorithms for the solution of optimal power flow problems. Electr. Power Syst. Res.
**77**(5–6), 508–517 (2007)CrossRefGoogle Scholar - 6.C.Y. Chung, W. Yan, F. Liu, Decomposed predictor-corrector interior point method for dynamic optimal power flow. IEEE Trans. Power Syst.
**26**(3), 1030–1039 (2011)CrossRefGoogle Scholar - 7.F.E. Curtis, J. Huber, O. Schenk, A. Wächter, A note on the implementation of an interior-point algorithm for nonlinear optimization with inexact step computations. Math. Program.
**136**(1), 209–227 (2012)MathSciNetCrossRefGoogle Scholar - 8.P. Gerstner, V. Heuveline, M. Schick, A multilevel domain decomposition approach for solving time constrained optimal power flow problems. Preprint Ser. Eng. Math. Comput. Lab (EMCL)
**4**, 1–30 (2015)Google Scholar - 9.V. Hernandez, J.E. Roman, V. Vidal, SLEPc: a scalable and flexible toolkit for the solution of eigenvalue problems. ACM Trans. Math. Softw.
**31**(3), 351–362 (2005)MathSciNetCrossRefGoogle Scholar - 10.C. Keller, N.I.M. Gould, A.J. Wathen, Constraint preconditioning for indefinite linear systems. SIAM J. Matrix Anal. Appl.
**21**(4), 1300–1317 (2000)MathSciNetCrossRefGoogle Scholar - 11.A. Meister,
*Numerik linearer Gleichungssysteme. Eine Einführung in moderne Verfahren. Mit MATLAB-Implementierungen von C. Vömel*, 5th revised edn. (Springer Spektrum, Heidelberg, 2015)Google Scholar - 12.N. Meyer-Hübner, M. Suriyah, T. Leibfried, V. Slednev, V. Bertsch, W. Fichtner, P. Gerstner, M. Schick, V. Heuveline, Time constrained optimal power flow calculations on the German power grid, in
*International ETG Congress 2015; Die Energiewende – Blueprints for the new energy age*, Bonn, 2015, pp. 1–7Google Scholar - 13.N. Meyer-Hübner, F. Gielnik, M. Suriyah, T. Leibfried, Dynamic optimal power flow in AC networks with multi-terminal HVDC and energy storage, in
*2016 IEEE Innovative Smart Grid Technologies – Asia (ISGT-Asia)*, Nov 2016Google Scholar - 14.J. Nocedal, S.J. Wright,
*Numerical optimization*. Springer Series in Operations Research and Financial Engineering (Springer, Berlin, 2006)Google Scholar - 15.Y. Saad, M.H. Schultz, GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput.
**7**(3), 856–869 (1986)MathSciNetCrossRefGoogle Scholar - 16.M. Schönfelder, A. Eßer-Frey, M. Schick, W. Fichtner, V. Heuveline, T. Leibfried, New developments in modeling network constraints in techno-economic energy system expansion planning models. Zeitschrift für Energiewirtschaft
**36**(1), 27–35 (2012)CrossRefGoogle Scholar - 17.H.A. Schwarz,
*Ueber einen Grenzübergang durch alternirendes Verfahren*. Vierteljahrsschrift der Naturforschenden Gesellschaft in Zürich. Zürcher u. Furrer, 1870Google Scholar - 18.B.F. Smith, P.E. Bjørstad, W.D. Gropp,
*Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations*(Cambridge University Press, Cambridge, 1996)zbMATHGoogle Scholar - 19.E.M. Soler, V.A. de Sousa, G.R.M. da Costa, A modified primal-dual logarithmic-barrier method for solving the optimal power flow problem with discrete and continuous control variables. Eur. J. Oper. Res.
**222**(3), 616–622 (2012)CrossRefGoogle Scholar - 20.K. Xie, Y.H. Song, Dynamic optimal power flow by interior point methods. IEE Proc. Gener. Transm. Distrib.
**148**(1), 76–84 (2001)CrossRefGoogle Scholar - 21.R.D. Zimmerman, C.E. Murillo-Sánchez, R.J. Thomas, Matpower: steady-state operations, planning, and analysis tools for power systems research and education. IEEE Trans. Power Syst.
**26**(1), 12–19 (2011)CrossRefGoogle Scholar - 22.J. Zhu,
*Optimization of Power System Operation*, vol. 47 (John Wiley & Sons, Hoboken, 2015)Google Scholar

## Copyright information

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.