Complexity of the relaxed Peaceman–Rachford splitting method for the sum of two maximal strongly monotone operators

Abstract

This paper considers the relaxed Peaceman–Rachford (PR) splitting method for finding an approximate solution of a monotone inclusion whose underlying operator consists of the sum of two maximal strongly monotone operators. Using general results obtained in the setting of a non-Euclidean hybrid proximal extragradient framework, we extend a previous convergence result on the iterates generated by the relaxed PR splitting method, as well as establish new pointwise and ergodic convergence rate results for the method whenever an associated relaxation parameter is within a certain interval. An example is also discussed to demonstrate that the iterates may not converge when the relaxation parameter is outside this interval.

1 Introduction

In this paper, we consider the relaxed Peaceman–Rachford (PR) splitting method for solving the monotone inclusion

$$\begin{aligned} 0 \in (A+B)(u) \end{aligned}$$

(1)

where $A: \mathcal{X}\rightrightarrows \mathcal{X}$ and $B: \mathcal{X}\rightrightarrows \mathcal{X}$ are maximal $\beta$-strongly monotone (point-to-set) operators for some $\beta \ge 0$ (with the convention that 0-strongly monotone means simply monotone, and $\beta$-strongly monotone with $\beta > 0$ means strongly monotone in the usual sense). Recall that the relaxed PR splitting method is given by

$$\begin{aligned} x_{k} = x_{k-1} + \theta (J_{B}(2 J_{A}(x_{k-1}) - x_{k-1}) - J_{A}(x_{k-1})), \end{aligned}$$

(2)

where $\theta >0$ is a fixed relaxation parameter and $J_T :=(I+T)^{-1}$. The special case of the relaxed PR splitting method in which $\theta = 2$ is known as the Peaceman–Rachford (PR) splitting method and the one with $\theta =1$ is the widely-studied Douglas–Rachford (DR) splitting method. Convergence results for them are studied for example in [1,2,3,4, 8, 13, 14, 22].

The analysis of the relaxed PR splitting method for the case in which $\beta =0$ has been undertaken in a number of papers which are discussed in this paragraph. Convergence of the sequence of iterates generated by the relaxed PR splitting method is well-known when $\theta <2$ (see for example [1, 7, 14]) and, according to [16], its limiting behavior for the case in which $\theta \ge 2$ is not known. We actually show in Sect. 5.2 that the sequence (2) does not necessarily converge when $\theta \ge 2$. An ${\mathcal {O}}(1/\sqrt{k})$ (strong) pointwise convergence rate result is established in [18] for the relaxed PR splitting method when $\theta \in (0,2)$. Moreover, when $A=\partial f$ and $B=\partial g$ where f and g are proper lower semi-continuous convex functions, papers [9,10,11] derive strong pointwise (resp., ergodic) convergence rate bounds for the relaxed PR method when $\theta \in (0,2)$ (resp., $\theta \in (0,2]$) under different assumptions on the functions. Assuming only $\beta$-strong monotonicity of $A = \partial f$, where $\beta > 0$, some smoothness property on f, and maximal monotonicity of B, [16] shows that the relaxed PR splitting method has linear convergence rate for $\theta \in (0,2+\tau )$ for some $\tau >0$. Linear rate of convergence of the relaxed PR splitting method and its two special cases, namely, the DR splitting and PR splitting methods, are established in [2,3,4, 11, 15, 16, 22] under relatively strong assumptions on A and/or B (see also Table 2).

This paper assumes that $\beta \ge 0$, and hence its analysis applies to the case in which both A and B are monotone ($\beta =0$) and the case in which both A and B are strongly monotone ($\beta > 0$). This paragraph discusses papers dealing with the latter case. Paper [12] establishes convergence of the sequence generated by the relaxed PR splitting method for any $\theta \in (0, 2 + \beta )$ and, under some strong assumptions on A and B, establishes its linear convergence rate. We complement the convergence results in [12] by showing that for $\theta = 2 + \beta$, the sequence of iterates generated by the relaxed PR splitting method also converge, and describe an instance showing its nonconvergence when $\theta \ge \min \{2 + 2\beta , 2 + \beta + 1/\beta \}$. Moreover, we establish strong pointwise and ergodic convergence rate results (Theorems 4.6 and 4.8) for the relaxed PR splitting method when $\theta \in (0, 2 + \beta )$ and $\theta \in (0, 2 + \beta ]$, respectively.

Finally, by imposing strong assumptions requiring one of the operators to be strong monotone and one of them to be Lipschitz (and hence point-to-point), [11, 15, 16] establish linear convergence rate of the relaxed PR splitting method. As opposed to these papers, the assumptions in [12] and this paper do not imply the operators A or B to be point-to-point.

Our analysis of the relaxed PR splitting method for solving (1) is based on viewing it as an inexact proximal point method, more specifically, as an instance of a non-Euclidean hybrid proximal extragradient (HPE) framework for solving the monotone inclusion problem. The proximal point method, proposed by Rockafellar [29], is a classical iterative scheme for solving the latter problem. Paper [30] introduces an Euclidean version of the HPE framework which is an inexact version of the proximal point method based on a certain relative error criterion. Iteration-complexities of the latter framework are established in [25] (see also [26]). Generalizations of the HPE framework to the non-Euclidean setting are studied in [17, 21, 31]. Applications of the HPE framework can be found for example in [19, 20, 25, 26].

This paper is organized as follows. Section 2 describes basic concepts and notation used in the paper. Section 3 discusses the non-Euclidean HPE framework which is used to the study the convergence properties of the relaxed PR splitting method in Sects. 4 and 5. Section 4 derives convergence rate bounds for the relaxed Peaceman–Rachford (PR) splitting method. Section 5, which consists of two subsections, discusses a convergence result of the relaxed PR splitting method in the first subsection and provides an example showing that its iterates may not converge when $\theta \ge \min \{2 + 2\beta , 2 + \beta + 1/\beta \}$ in the second subsection. Finally, Sect. 6 discusses the numerical performance of the relaxed PR splitting method for solving the weighted Lasso minimization problem. Section 7 gives some concluding remarks.

2 Basic concepts and notation

This section presents some definitions, notation and terminology which will be used in the paper.

We denote the set of real numbers by $\mathbb {R}$ and the set of non-negative real numbers by $\mathbb {R}_+$. Let f and g be functions with the same domain and whose values are in $\mathbb {R}_+$. We write that $f(\cdot ) = {\Omega } (g(\cdot ))$ if there exists constant $K > 0$ such that $f(\cdot ) \ge K g(\cdot )$. Also, we write $f(\cdot ) = {\Theta }(g(\cdot ))$ if $f(\cdot ) = {\Omega }(g(\cdot ))$ and $g(\cdot ) = {\Omega }(f(\cdot ))$.

Let $\mathcal{Z}$ be a finite-dimensional real vector space with inner product denoted by $\langle \cdot ,\cdot \rangle$ (an example of $\mathcal{Z}$ is $\mathbb {R}^n$ endowed with the standard inner product) and let $\Vert \cdot \Vert$ denote an arbitrary seminorm in $\mathcal{Z}$. Its dual (extended) seminorm, denoted by $\Vert \cdot \Vert _*$, is defined as $\Vert \cdot \Vert _*:=\sup \{ \langle \cdot ,z\rangle :\Vert z\Vert \le 1\}$. It is easy to see that

$$\begin{aligned} \langle z,v\rangle \le \Vert z\Vert \Vert v\Vert _* \quad \forall z,v \in \mathcal{Z}. \end{aligned}$$

(3)

The following straightforward result states some basic properties of the dual seminorm associated with a matrix seminorm. Its proof can be found for example in Lemma A.1(b) of [23].

Proposition 2.1

Let $A:\mathcal{Z}\rightarrow \mathcal{Z}$ be a self-adjoint positive semidefinite linear operator and consider the seminorm $\Vert \cdot \Vert$ in $\mathcal{Z}$ given by $\Vert z\Vert = \langle A z, z\rangle ^{1/2}$ for every $z\in \mathcal{Z}$. Then, $\mathrm {dom}\,\Vert \cdot \Vert _*=\text{ Im }\;(A)$ and $\Vert Az\Vert _*=\Vert z\Vert$ for every $z\in \mathcal{Z}$.

Given a set-valued operator $T:\mathcal{Z}\rightrightarrows \mathcal{Z}$, its domain is denoted by $\text{ Dom }(T):=\{z\in \mathcal{Z}: T(z)\ne \emptyset \}$ and its inverse operator $T^{-1}:\mathcal{Z}\rightrightarrows \mathcal{Z}$ is given by $T^{-1}(v):=\{z : v\in T(z)\}$. The graph of T is defined by $\mathrm{{Gr}}(T) := \{(z,t) : t \in T(z) \}$. The operator T is said to be monotone if

$$\begin{aligned} \langle z-z',t-t'\rangle \ge 0\quad \forall (z,t), \, (z',t') \in \mathrm{{Gr}}(T). \end{aligned}$$

Moreover, T is maximal monotone if it is monotone and, additionally, if $T'$ is a monotone operator such that $T(z)\subset T'(z)$ for every $z\in \mathcal{Z}$, then $T=T'$. The sum $T+T':\mathcal{Z}\rightrightarrows \mathcal{Z}$ of two set-valued operators $T,T':\mathcal{Z}\rightrightarrows \mathcal{Z}$ is defined by $(T+T')(z):= \{t+t' \in \mathcal{Z}:t\in T(z),\; t'\in T'(z)\}$ for every $z\in \mathcal{Z}$. Given a scalar $\varepsilon \ge 0$, the $\varepsilon$-enlargement $T^{[\varepsilon ]}:\mathcal{Z}\rightrightarrows \mathcal{Z}$ of a monotone operator $T:\mathcal{Z}\rightrightarrows \mathcal{Z}$ is defined as

$$\begin{aligned} T^{[\varepsilon ]}(z) :=\{t\in \mathcal{Z}: \langle t-t',z-z'\rangle \ge -\varepsilon ,\;\forall z' \in \mathcal{Z}, \, \forall t'\in T(z')\} \quad \forall z \in \mathcal{Z}. \end{aligned}$$

(4)

3 A non-Euclidean hybrid proximal extragradient framework

This section discusses the non-Euclidean hybrid proximal extragradient (NE-HPE) framework and describes its associated convergence and iteration complexity results. The results of the section will be used in Sects. 4 and 5 to study the convergence and iteration complexity properties of the relaxed PR splitting method (2). It contains two subsections. The first one describes a class of distance generating functions introduced in [17]. The second one describes the NE-HPE framework and its corresponding convergence and iteration complexity results.

For the sake of shortness, all the results in this section are stated without proofs which, in turn, can be found in Section 3 of the technical report version of this paper (see [24]).

3.1 A class of distance generating functions

We start by introducing a class of distance generating functions (and its corresponding Bregman distances) which is needed for the presentation of the NE-HPE framework in Sect. 3.2.

Definition 3.1

For a given convex set $Z\subset \mathcal{Z}$, a seminorm $\Vert \cdot \Vert$ in $\mathcal{Z}$ and scalars $0 < m \le M$, we let ${\mathcal {D}}_Z(m,M)$ denote the class of real-valued functions w which are differentiable on Z and satisfy

$$\begin{aligned} w(z')-w(z) - \langle \nabla w(z),z'-z\rangle\ge & {} \frac{{m}}{2}\Vert z-z'\Vert ^2 \quad \forall z,z' \in Z, \end{aligned}$$

(5)

$$\begin{aligned} \Vert \nabla w(z)-\nabla w(z')\Vert _*\le & {} {M}\Vert z-z'\Vert \quad \forall z, z' \in Z. \end{aligned}$$

(6)

A function $w \in {\mathcal {D}}_Z(m,M)$ is referred to as a distance generating function with respect to the seminorm $\Vert \cdot \Vert$ and its associated Bregman distance $dw: Z \times Z \rightarrow \mathbb {R}$ is defined as

$$\begin{aligned} (dw)(z';z) = (dw)_{z}(z') := w(z')-w(z)-\langle \nabla w(z),z'-z\rangle \quad \forall z, z' \in Z. \end{aligned}$$

(7)

Throughout our presentation, we use the second notation $(dw)_{z}(z')$ instead of the first one $(dw)(z';z)$ although the latter one makes it clear that (dw) is a function of two arguments, namely, z and $z'$. Clearly, it follows from (5) that w is a convex function on Z which is in fact m-strongly convex on Z whenever $\Vert \cdot \Vert$ is a norm.

Note that if the seminorm in Definition 3.1 is a norm, then (5) implies that w is strongly convex on Z, in which case the corresponding dw is said to be nondegenerate on Z. However, since Definition 3.1 does not necessarily assume that $\Vert \cdot \Vert$ is a norm, it admits the possibility of w being not strongly convex on Z, or equivalently, dw being degenerate on Z.

Finally, some useful relations about the above class of Bregman distances can be found in Sect. 3.1 of the technical report version of this paper (see Lemmas 3.2 and 3.3 of [24]).

3.2 The NE-HPE framework

This subsection describes the NE-HPE framework and its corresponding convergence and iteration complexity results.

Throughout this subsection, we assume that scalars $0<m \le M$, convex set $Z \subset \mathcal{Z}$, seminorm $\Vert \cdot \Vert$ and distance generating function $w \in {\mathcal {D}}_Z(m,M)$ with respect to $\Vert \cdot \Vert$ are given. Our problem of interest in this section is the MIP

$$\begin{aligned} 0\in T(z) \end{aligned}$$

(8)

where $T:\mathcal{Z}\rightrightarrows \mathcal{Z}$ is a maximal monotone operator satisfying the following conditions:

(A0):

$\mathrm {Dom}\,(T)\subset Z$;

(A1):

the solution set $T^{-1}(0)$ of (8) is nonempty.

We now state a non-Euclidean HPE (NE-HPE) framework for solving the MIP (8) which generalizes its Euclidean counterparts studied in the literature (see for example in [25, 27, 30]).

We now make some remarks about Framework 1. First, it does not specify how to find $\lambda _k$ and $(\tilde{z}_k, z_k, \varepsilon _k)$ satisfying (9) and (10). The particular scheme for computing $\lambda _k$ and $(\tilde{z}_k, z_k, \varepsilon _k)$ will depend on the instance of the framework under consideration and the properties of the operator T. Second, if w is strongly convex on Z and $\sigma = 0$, then (10) implies that $\varepsilon _k= 0$ and $z_k = \tilde{z}_k$ for every k, and hence that $r_k \in T(z_k)$ in view of (9). Therefore, the HPE error conditions (9)–(10) can be viewed as a relaxation of an iteration of the exact non-Euclidean proximal point method, namely,

$$\begin{aligned} 0 \in \frac{1}{\lambda _k} \nabla (dw)_{z_{k-1}}(z_{k}) + T({z}_k). \end{aligned}$$

We observe that NE-HPE frameworks have already been studied in [17, 21] and [31]. The approach presented in this section differs from these three papers as follows. Assuming that Z is an open convex set, w is continuously differentiable on Z and continuous on its closure, [31] studies a special case of the NE-HPE framework in which $\varepsilon _k=0$ for every k, and presents results on convergence of sequences rather than iteration complexity. Paper [21] deals with distance generating functions w which do not necessarily satisfy conditions (5) and (6), and as consequence, obtains results which are more limited in scope, i.e., only an ergodic convergence rate result is obtained for operators with bounded feasible domains (or, more generally, for the case in which the sequence generated by the HPE framwework is bounded). Paper [17] introduces the class of distance generating functions ${\mathcal {D}}_Z(m,M)$ but only analyzes the behavior of a HPE framework for solving inclusions whose operators are strongly monotone with respect to a fixed $w \in {\mathcal {D}}_Z(m,M)$ (see condition A1 in Section 2 of [17]). This section on the other hand assumes that $w \in {\mathcal {D}}_Z(m,M)$ but it does not assume any strong monotonicity of T with respect to w.

Before presenting the main results about the the NE-HPE framework, namely, Theorems 3.3 and 3.4 establishing its pointwise and ergodic iteration complexities, respectively, and Propositions 3.5 and 3.6 showing that $\{z_k\}$ and/or $\{\tilde{z}_k\}$ approach $T^{-1}(0)$ in terms of the Bregman distance (dw), we have the following result.

Proposition 3.2

For every $k \ge 1$ and $z^* \in T^{-1}(0)$, we have

$$\begin{aligned} (dw)_{z_{k-1}}(z^*) - (dw)_{z_k}(z^*) - (1 - \sigma )(dw)_{z_{k-1}}(\tilde{z}_k) \ge \lambda _k \left[ \langle r_k, \tilde{z}_k - z^*\rangle \,+\, \varepsilon _k \right] \ge 0.\quad \end{aligned}$$

(11)

As a consequence, the following statements hold:

• (a) $\{(dw)_{z_k}(z^*)\}$ is non-increasing;

• (b) $\lim _{k\rightarrow \infty } {\lambda }_k \left[ \langle r_k, \tilde{z}_k - z^*\rangle + \varepsilon _k \right] = 0$;

• (c) $(1-\sigma ) \sum _{i=1}^k (dw)_{z_{i-1}}(\tilde{z}_i) \le (dw)_{z_0}(z^*)$.

Proof

See the proof of Proposition 3.5 of [24]. $\square$

For the purpose of stating the convergence rate results below, define

$$\begin{aligned} (dw)_0 := \inf \{ (dw)_{z_0}(z^*) : z^* \in T^{-1}(0)\}. \end{aligned}$$

(12)

The following pointwise convergence rate result describes the convergence rate of the sequence $\{(r_k,\varepsilon _k)\}$ of residual pairs associated to the sequence $\{\tilde{z}_k\}$. Note that its convergence rate bounds are derived on the best residual pair among $(r_i,\varepsilon _i)$ for $i=1,\ldots ,k$ rather than on the last residual pair $(r_k,\varepsilon _k)$.

Theorem 3.3

(Pointwise convergence) Let $(dw)_0$ be as in (12) and assume that $\sigma <1$. Then, the following statements hold:

• (a) if $\underline{\lambda }:=\inf \lambda _k>0$, then for every $k\in \mathbb {N}$ there exists $i\le k$ such that

  $$\begin{aligned} \left\| r_i\right\| _* \le M(1+\sqrt{\sigma }) \sqrt{\frac{2(dw)_0}{(1-\sigma )m}\; \left( \frac{\underline{\lambda }^{-1}}{\sum _{j=1}^k\lambda _j}\right) }\le & {} \frac{M(1+\sqrt{\sigma })}{\underline{\lambda }\sqrt{k}} \sqrt{\frac{2(dw)_0}{(1-\sigma )m}},\\ \varepsilon _i\le \frac{\sigma (dw)_0}{1-\sigma }\frac{1}{\sum _{i=1}^k\lambda _i}\le & {} \frac{\sigma (dw)_0}{(1-\sigma )\underline{\lambda }k}; \end{aligned}$$

• (b) for every $k \in \mathbb {N}$, there exists an index $i\le k$ such that

  $$\begin{aligned} \left\| r_i\right\| _*\le M(1+\sqrt{\sigma }) \sqrt{\frac{2(dw)_0}{(1-\sigma )m} \left( \frac{1}{ \sum _{j=1}^k \lambda _j^2}\right) } , \quad \quad \varepsilon _i\le \frac{\sigma (dw)_0 \lambda _i}{(1-\sigma )\sum _{j=1}^k \lambda _j^2}. \end{aligned}$$

  (13)

Proof

See the proof of Theorem 3.8 of [24]. $\square$

From now on, we focus on the ergodic convergence rate of the NE-HPE framework. For $k \ge 1$, define $\Lambda _{k} := \sum _{i=1}^k \lambda _i$ and the ergodic sequences

$$\begin{aligned} \tilde{z}^a_{k} = \frac{1}{\Lambda _{k}}\sum _{i=1}^k \lambda _i \tilde{z}_i, \quad r^a_{k} := \frac{1}{\Lambda _{k}}\sum _{i=1}^k \lambda _i r_i, \quad \varepsilon ^a_{k} := \frac{1}{\Lambda _{k}} \sum _{i=1}^k \lambda _i \left( \varepsilon _i + \langle r_i,\tilde{z}_i -\tilde{z}^a_{k}\rangle \right) . \end{aligned}$$

(14)

The following ergodic convergence result describes the association between the ergodic iterate $\tilde{z}_k^a$ and the residual pair $(r_k^a,\varepsilon _k^a)$, and gives a convergence rate bound on the latter residual pair.

Theorem 3.4

(Ergodic convergence) Let $(dw)_0$ be as in (12). Then, for every $k\ge 1$, we have

$$\begin{aligned} \varepsilon _k^a \ge 0, \quad r^a_k \in T^{[\varepsilon _k^a]}(\tilde{z}^a_k) \end{aligned}$$

and

$$\begin{aligned} \Vert r_k^a\Vert _* \le \frac{2M}{\Lambda _k}\sqrt{\frac{2(dw)_0}{m}}, \quad \varepsilon ^a_{k} \le \left( \frac{3M}{m} \right) \frac{ 2 (dw)_{0} + \rho _k}{\Lambda _k} \end{aligned}$$

where

$$\begin{aligned} \rho _k := \displaystyle \max _{i=1,\ldots ,k}(dw)_{z_{i}}(\tilde{z}_{i}). \end{aligned}$$

(15)

Moreover, the sequence $\{\rho _k\}$ is bounded under either one of the following situations:

• (a) $\sigma <1$, in which case

  $$\begin{aligned} \rho _k \le \frac{\sigma (dw)_0}{1-\sigma }; \end{aligned}$$

  (16)

• (b) $\mathrm {Dom}\,T$ is bounded, in which case

  $$\begin{aligned} \rho _k \le \frac{2M}{m} [ (dw)_0 + D] \end{aligned}$$

  (17)

  where $D := \sup \{ \min \{ (dw)_y(y'), (dw)_{y'}(y) \} : y,y' \in \mathrm {Dom}\,T\}$ is the diameter of $\mathrm {Dom}\,T$ with respect to dw.

Proof

See the proof of Theorem 3.9 of [24]. $\square$

In the remaining part of this subsection, we state some results about the sequence generated by an instance of the NE-HPE framework. We assume from now on that such instance generates an infinite sequence of iterates, i.e., the instance does not terminate in a finite number of steps and no termination criterion is checked. Since we are not assuming that the distance generating function w is nondegenerate on Z, it is not possible to establish convergence of the sequence $\{z_k\}$ generated by the NE-HPE framework to a solution of (8). However, under some mild assumptions, it is possible to establish that $\{z_k\}$ approaches a point $\tilde{z}\in T^{-1}(0)$ if the proximity measure used is the actual Bregman distance.

Proposition 3.5

Assume that for some infinite index set $\mathcal{K}$ and some $\tilde{z}\in \mathcal{Z}$, we have

$$\begin{aligned} \lim _{k \rightarrow \mathcal{K}} (r_k,\varepsilon _k)=(0,0), \quad \lim _{k \rightarrow \mathcal{K}} \tilde{z}_k = \tilde{z}. \end{aligned}$$

(18)

Then, $\tilde{z}\in T^{-1}(0) \subset Z$. If, in addition, $\lim _{k \in \mathcal{K}} (dw)_{z_k}(\tilde{z}_k) =0$, then $\lim _{k \rightarrow \infty } (dw)_{z_k}(\tilde{z}) =0$.

Proof

See the proof of Proposition 3.10 of [24]. $\square$

Proposition 3.6

Assume that $\sigma <1$, $\sum _{i=1}^\infty \lambda _k^2 = \infty$ and $\{\tilde{z}_k\}$ is bounded. Then, there exists $\tilde{z}\in T^{-1}(0) \subset Z$ such that

$$\begin{aligned} \lim _{k \rightarrow \infty } (dw)_{z_k}(\tilde{z}) = \lim _{k \rightarrow \infty } (dw)_{\tilde{z}_k}(\tilde{z}) = 0. \end{aligned}$$

(19)

Proof

See the proof of Proposition 3.11 of [24] $\square$

Clearly, if w is a nondegenerate distance generating function, then the results above give sufficient conditions for the sequences $\{z_k\}$ and $\{\tilde{z}_k\}$ to converge to some $\tilde{z}\in T^{-1}(0)$.

4 The relaxed Peaceman–Rachford splitting method

This section derives convergence rate bounds for the relaxed Peaceman–Rachford (PR) splitting method for solving the monotone inclusion (1) under the assumption that A and B are maximal $\beta$-strongly monotone operators for any $\beta \ge 0$. More specifically, its pointwise iteration-complexity is obtained in Theorem 4.6 and its ergodic iteration-complexity is derived in Theorem 4.8. These results are obtained as by-products of the corresponding ones (i.e, Theorems 3.3 and 3.4) in Sect. 3.2 and the fact that the relaxed Peaceman–Rachford (PR) splitting method can be viewed as a special instance of the NE-HPE framework.

Throughout this section, we assume that $\mathcal{X}$ a finite-dimensional real vector space with inner product and associated inner product norm denoted by $\langle \cdot ,\cdot \rangle _{\mathcal{X}}$ and $\Vert \cdot \Vert _{\mathcal{X}}$, respectively. For a given $\beta \ge 0$, an operator $T: \mathcal{X}\rightrightarrows \mathcal{X}$ is said to be $\beta$-strongly monotone if

$$\begin{aligned} \langle w-w',x-x'\rangle _{\mathcal{X}} \ge \beta \Vert x-x'\Vert _\mathcal{X}^2 \quad \forall (x,w), (x',w') \in \mathrm{Gr}(T). \end{aligned}$$

In what follows, we refer to monotone operators as 0-strongly monotone operators. This terminology has the benefit of allowing us to treat both the monotone and strongly monotone case simultaneously.

Throughout this section, we consider the monotone inclusion (1) where $A,B : \mathcal{X}\rightrightarrows \mathcal{X}$ satisfy the following assumptions:

• (B0) for some $\beta \ge 0$, A and B are maximal $\beta$-strongly monotone operators;

• (B1) the solution set $(A+B)^{-1}(0)$ is non-empty.

We start by observing that (1) is equivalent to solving the following augmented system of inclusions/equation

$$\begin{aligned} 0&\in \gamma A(u) + u - x, \\ 0&\in \gamma B(v) + x - v, \\ 0&= u-v \end{aligned}$$

where $\gamma > 0$ is an arbitrary scalar. Another way of writing the above system is as

$$\begin{aligned} 0&\in \gamma A(u) + u - x, \\ 0&\in \gamma B(v) + v + x - 2u, \\ 0&= u-v. \end{aligned}$$

Note that the first and second inclusions are equivalent to

$$\begin{aligned} u = u(x) := J_{\gamma A}(x), \quad v=v(x):=J_{\gamma B}(2u-x) = J_{\gamma B}(2J_{\gamma A}(x)-x) \end{aligned}$$

(20)

so that the third equation reduces to

$$\begin{aligned} 0 = u(x) -v(x) = J_{\gamma A}(x) - J_{\gamma B}(2J_{\gamma A}(x)-x). \end{aligned}$$

The Douglas–Rachford (DR) splitting method is the iterative procedure $x_k = x_{k-1} + v(x_{k-1})-u (x_{k-1})$, $k\ge 1$, started from some $x_0 \in \mathcal{X}$. It is known that the DR splitting method is an exact proximal point method for some maximal monotone operator [13, 14]. Hence, convergence of its sequence of iterates is guaranteed.

This section is concerned with a natural generalization of the DR splitting method, namely, the relaxed Peaceman–Rachford (PR) splitting method with relaxation parameter $\theta >0$, which iterates as

$$\begin{aligned} (u_k,v_k) := (u(x_{k-1}),v(x_{k-1})) \quad x_k = x_k^\theta := x_{k-1} + \theta (v_k-u_k) \quad \forall k \ge 1. \end{aligned}$$

(21)

We now make a few remarks about the above method. First, it reduces to the DR splitting method when $\theta =1$, and to the PR splitting method when $\theta =2$. Second, it reduces to (2) when $\gamma =1$ but it is not more general than (2) since (21) is equivalent to (2) with $(A,B)=(\gamma A, \gamma B)$. Third, as presented in (21), it can be viewed as an iterative process in the (u, v, x)-space rather than only in the x-space as suggested by (2).

Our analysis of the relaxed DR splitting method is based on further exploring the last remark above, i.e., viewing it as an iterative method in the (u, v, x)-space. We start by introducing an inclusion which plays an important role in our analysis. For a fixed $\tilde{\theta }>0$ and $\gamma >0$, consider the inclusion

$$\begin{aligned} 0 \in (\mathcal{L}_{{\tilde{\theta }}}+\gamma C)(z) \end{aligned}$$

(22)

where $\mathcal{L}_{\tilde{\theta }} : \mathcal{X}\times \mathcal{X}\times \mathcal{X}\rightarrow \mathcal{X}\times \mathcal{X}\times \mathcal{X}$ is the linear map defined as

$$\begin{aligned} \mathcal{L}_{\tilde{\theta }}(z) = \mathcal{L}_{\tilde{\theta }}(u,v,x) := \left[ \begin{array}{ccc} (1-{\tilde{\theta }})I &{} {\tilde{\theta }}I &{} -I \\ ({\tilde{\theta }}-2)I &{} (1-{\tilde{\theta }})I &{} I \\ I &{} - I &{} 0 \end{array} \right] \left( \begin{array}{c} u \\ v \\ x \end{array} \right) \end{aligned}$$

(23)

and $C: \mathcal{X}\times \mathcal{X}\times \mathcal{X}\rightrightarrows \mathcal{X}\times \mathcal{X}\times \mathcal{X}$ is the maximal monotone operator defined as

$$\begin{aligned} C(z) = C(u,v,x) := A(u) \times B(v) \times \{0\}. \end{aligned}$$

(24)

It is easy to verify that the inclusion (22) is equivalent to the two systems of inclusions/equation following conditions B0 and B1. Hence, it suffices to solve (22) in order to solve (1). The following simple but useful result explicitly show the relationship between the solution sets of (22) and (1).

Lemma 4.1

For any $\tilde{\theta }>0$, the solution set $(\mathcal{L}_{\tilde{\theta }}+\gamma C)^{-1}(0)$ is given by

$$\begin{aligned} (\mathcal{L}_{\tilde{\theta }}+\gamma C)^{-1}(0)&= \{(u^*,u^*,x^*) : \gamma ^{-1}(x^*-u^*) \in A(u^*) \cap (-B(u^*)) \} \\&= \{ (u^*,u^*,u^*+ \gamma a^*) :a^* \in A(u^*), -a^* \in B(u^*) \}. \end{aligned}$$

As a consequence, if $z^*=(u^*,u^*,x^*) \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)^{-1}(0)$, then $u^* \in (A+B)^{-1}(0)$ and $u^*=J_{\gamma A}(x^*)$.

Proof

The conclusion of the lemma follows immediately from the definitions of $\mathcal{L}_{\tilde{\theta }}$ and C in (23) and (24), respectively, and some simple algebraic manipulations. $\square$

The key idea of our analysis is to show that the relaxed PR splitting method is actually a special instance of the NE-HPE framework for solving inclusion (22) and then use the results discussed in Sect. 3.2 to derive convergence and iteration-complexity results for it. With this goal in mind, the next result gives a sufficient condition for (22) to be a maximal monotone inclusion.

Proposition 4.2

Assume that $A,B : \mathcal{X}\rightrightarrows \mathcal{X}$ satisfy B0 and let $\tilde{\theta }>0$ be given. Then,

• (a) for every $z = (u,v,x) \in \mathcal{X}\times \mathcal{X}\times \mathcal{X}$, $z'=(u',v',x') \in \mathcal{X}\times \mathcal{X}\times \mathcal{X}$, $r \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)(z)$ and $r' \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)(z')$, we have

  $$\begin{aligned} \langle \mathcal{L}_{\tilde{\theta }}(z - z'), z - z')&= (1 - \tilde{\theta }) \Vert (u-u') - (v-v') \Vert _{\mathcal{X}}^2 \end{aligned}$$

  (25)
  $$\begin{aligned} \langle r-r', z-z' \rangle&\ge (1 - \tilde{\theta }) \Vert (u-u') - (v-v') \Vert _{\mathcal{X}}^2\nonumber \\&\quad +\, \gamma \beta (\Vert u-u' \Vert ^2_{\mathcal{X}} + \Vert v - v'\Vert ^2_{\mathcal{X}} ); \end{aligned}$$

  (26)

• (b) $\mathcal{L}_{\tilde{\theta }}+ \gamma C$ is maximal monotone whenever $\tilde{\theta } \in (0, \theta _0]$ where

  $$\begin{aligned} \theta _0 := 1 + \frac{\gamma \beta }{2}. \end{aligned}$$

  (27)

Proof

• (a) Identity (25) follows from the definition of $\mathcal{L}_{\tilde{\theta }}$ in (23). To show inequality (26), assume that $r \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)(z)$ and $r' \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)(z')$. Then, $r = \mathcal{L}_{\tilde{\theta }}(z) + \gamma c$ and $r' = \mathcal{L}_{\tilde{\theta }}(z) + \gamma c'$ for some $c \in C(z)$ and $c' \in C(z')$. Using the definition of C and assumption B0, we easily see that

  $$\begin{aligned} \langle c'-c,z'-z\rangle \ge \beta (\Vert u-u' \Vert ^2_{\mathcal{X}} + \Vert v - v'\Vert ^2_{\mathcal{X}} ), \end{aligned}$$

  which together with (25), and the fact that $r = \mathcal{L}_{\tilde{\theta }}(z) + \gamma c$ and $r' = \mathcal{L}_{\tilde{\theta }}(z) + \gamma c'$, imply (26).

• (b) Monotonicity of $\mathcal{L}_{\tilde{\theta }} + \gamma C$ is due to the fact that the right hand side of (26) is nonnegative for every $(u,v), (u',v') \in \mathcal{X}\times \mathcal{X}$ whenever $\tilde{\theta } \in (0, \theta _0]$. To show $\mathcal{L}_{\tilde{\theta }} + \gamma C$ is maximal monotone, write $\mathcal{L}_{\tilde{\theta }}+ \gamma C = (\mathcal{L}_{\tilde{\theta }}+ \gamma \bar{C}) + \gamma (C-\bar{C})$ where $\bar{C}:=\beta (I,I,0)$. As a consequence of (a) with $(A,B)=\beta (I,I)$ and the definition of C, we conclude that $\mathcal{L}_{\tilde{\theta }}+ \gamma \bar{C}$ is a monotone linear operator for every ${\tilde{\theta }}\in (0, \theta _0]$. Moreover, Assumption B0 easily implies that $\gamma (C-\bar{C})$ is maximal monotone. The statement now follows by noting that the sum of a monotone linear map and a maximal monotone operator is a maximal monotone operator [1, 28].

$\square$

Note that $\theta _0$ in (27) depends on $\gamma$ and $\beta$ and that $\theta _0 = 1$ when $\beta =0$.

The following technical result states some useful identities and inclusions needed to analyze the the sequence generated by the relaxed PR splitting method.

Lemma 4.3

For a given $x_{k-1} \in \mathcal{X}$ and $\tilde{\theta }>0$, define

$$\begin{aligned} \tilde{x}_k = x_k^{\tilde{\theta }} := x_{k-1} + {\tilde{\theta }}(v_k-u _k), \quad \tilde{z}_k = z_k^{\tilde{\theta }} := (u_k,v_k,\tilde{x}_k) \end{aligned}$$

(28)

where $u_k, v_k$ are as in (21), and set

$$\begin{aligned} a_k := \frac{1}{\gamma } (x_{k-1}-u_k), \quad b_k := \frac{1}{\gamma }(2 u_k - v_k - x_{k-1}). \end{aligned}$$

(29)

Then, we have:

$$\begin{aligned} -\, (1-{\tilde{\theta }}) u_k - {\tilde{\theta }}v_k + \tilde{x}_k&= \gamma a_k \in \gamma A(u_k), \end{aligned}$$

(30)

$$\begin{aligned} (2-{\tilde{\theta }}) u_k - (1-{\tilde{\theta }})v_k - \tilde{x}_k&= \gamma b_k \in \gamma B(v_k). \end{aligned}$$

(31)

As a consequence, we have

$$\begin{aligned} u_k-v_k&= \gamma (a_k+b_k) \in \gamma A(u_k) + \gamma B(v_k) \end{aligned}$$

(32)

$$\begin{aligned} (0,0, u_k-v_k)&= \mathcal{L}_{\tilde{\theta }}(\tilde{z}_k) + \gamma c_k \in (\mathcal{L}_{\tilde{\theta }}+ \gamma C) (\tilde{z}_k) \end{aligned}$$

(33)

where

$$\begin{aligned} c_k := (a_k,b_k,0). \end{aligned}$$

(34)

Proof

Using the definition of $(u(\cdot ),v(\cdot ))$ in (20), the definition of $(u_k,v_k,x_k^{\tilde{\theta }})$ in (21), and the definitions of $a_k$ and $b_k$ in (29), we easily see that (30) and (31) hold. The equality and the inclusion in (32) follow by adding (30) and (31). Clearly, (33) follows as an immediate consequence of (30) and (31), definitions (23) and (24), and the definition of $c_k$. $\square$

The following result shows that the relaxed PR splitting method with $\theta \in (0,2\theta _0]$ can be viewed as an inexact instance of the NE-HPE framework for solving (22) where from now on we assume that

$$\begin{aligned} \tilde{\theta }:= \min \{ \theta , \theta _0 \}. \end{aligned}$$

(35)

Proposition 4.4

Consider the (degenerate) distance generating function given by

$$\begin{aligned} w(z) = w(u,v,x) = \frac{\left\| x\right\| ^2_\mathcal{X}}{2\theta } \quad \forall z=(u,v,x) \in \mathcal{X}\times \mathcal{X}\times \mathcal{X}\end{aligned}$$

(36)

and the sequence $\{z_k=(u_k,v_k,x_k)\}$ generated according to the relaxed PR splitting method (21) with any $\theta >0$. Also, define the sequences $\{\varepsilon _k\}$, $\{\lambda _k\}$ and $\{r_k\}$ as

$$\begin{aligned} \varepsilon _k:=0, \quad \lambda _k:=1, \quad r_k := \nabla (dw)_{z_{k}}(z_{k-1}) \ \ \ \forall k \ge 1, \end{aligned}$$

(37)

and the sequence $\{\tilde{z}_k = (u_k, v_k,\tilde{x}_k) \}$ as in (28) with $\tilde{\theta }$ given by (35). Then, for every $k \ge 1$, we have:

• (a) $r_k = (0,0,x_{k-1}-x_k)/\theta = (0,0,u_k- v_k) = \gamma (0,0, a_k + b_k)$;

• (b) $(\lambda _k,z_{k-1})$ and $(z_k,\tilde{z}_k,\varepsilon _k)$ satisfy (9) with $T=\mathcal{L}_{\tilde{\theta }}+\gamma C$, i.e., $r_k \in (\mathcal{L}_{\tilde{\theta }}+\gamma C) (\tilde{z}_k)$;

• (c) $(\lambda _k,z_{k-1})$ and $(z_k,\tilde{z}_k,\varepsilon _k)$ satisfy (10) with $\sigma = (\theta /\tilde{\theta }-1)^2$ and w as in (36).

As a consequence, the relaxed PR splitting method with $\theta \in (0,2\theta _0)$ (resp., $\theta =2\theta _0$) is an NE-HPE instance with respect to the monotone inclusion $0 \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)(z)$ in which $\sigma <1$ (resp., $\sigma =1$), $\varepsilon _k = 0$ and $\lambda _k=1$ for every k.

Proof

• (a) The first identity in (a) follows from (36) and the definition of $r_k$ in (37). The second and third equalities in (a) are due to the second identity in (21) and relation (32), respectively.

• (b) This statement follows from (a) and (33).

• (c) Using the second identity in (21), relation (36) and the definition of $\tilde{x}_k$ in (28), we conclude that for any $\theta \in (0,2\theta _0]$,

  $$\begin{aligned} (dw)_{z_k}(\tilde{z}_k)= & {} \frac{\Vert \tilde{x}_k-x_k\Vert ^2_{\mathcal{X}}}{2\theta } = \left( \frac{\theta }{\tilde{\theta }}-1\right) ^2 \frac{\Vert \tilde{x}_k-x_{k-1}\Vert ^2_{\mathcal{X}}}{2\theta }\nonumber \\= & {} \left( \frac{\theta }{\tilde{\theta }}-1\right) ^2 (dw)_{z_{k-1}}(\tilde{z}_k) \end{aligned}$$

  and hence that (10) is satisfied with $\sigma =(1-\theta /\tilde{\theta })^2$.

  The last conclusion follows from statements (b) and (c), and Proposition 4.2(b).

$\square$

We now make a remark about the special case of Proposition 4.4 in which $\theta \in (0,\theta _0]$. Indeed, in this case, $\tilde{\theta }= \theta$, and hence $\sigma =0$ and $\tilde{z}_k=z_k$ for every $k \ge 1$. Thus, the relaxed PR splitting method with $\theta \in (0,\theta _0]$ can be viewed as an exact non-Euclidean proximal point method with distance generating function w as in (36) with respect to the monotone inclusion $0 \in T(z) := (\mathcal{L}_\theta +\gamma C)(z)$. Note also that the latter inclusion depends on $\theta$.

As a consequence of Proposition 4.4, we are now ready to describe the pointwise and ergodic convergence rate for the relaxed PR splitting method. We first endow the space $\mathcal{Z}:= \mathcal{X}\times \mathcal{X}\times \mathcal{X}$ with the semi-norm $\Vert (u,v,x)\Vert : = \Vert x\Vert _\mathcal{X}$ and hence Proposition 2.1 implies that

$$\begin{aligned} \Vert (0,0,x)\Vert _* = \Vert x\Vert _\mathcal{X}. \end{aligned}$$

(38)

It is also easy to see that the distance generating function w defined in (36) is in ${\mathcal {D}}_Z(m,M)$ with respect to $\Vert \cdot \Vert$ where $M=m=1/\theta$ (see Definition 3.1).

Our next goal is to state a pointwise convergence rate bound for the relaxed PR splitting method. We start by stating a technical result which is well-known for the case where $\beta =0$ (see for example Lemma 2.4 of [18]). The proof for the general case, i.e., $\beta \ge 0$, is similar and is given in the “Appendix” for the sake of completeness.

Lemma 4.5

Assume that $\theta \in (0,2 \theta _0]$. Then, for every $k \ge 2$, we have $\Vert \Delta x_{k}\Vert _{\mathcal{X}} \le \Vert \Delta x_{k-1}\Vert _{\mathcal{X}}$ where $\Delta x_k := x_k - x_{k-1}$.

We now state the pointwise convergence rate result for the relaxed PR splitting method.

Theorem 4.6

Consider the sequence $\{z_k = (u_k, v_k, x_k)\}$ generated by the relaxed PR splitting method with $\theta \in (0,2\theta _0)$. Then, for every $k \ge 1$ and $z^*=(u^*,u^*,x^*) \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)^{-1}(0)$,

$$\begin{aligned} a_k + b_k \in A(u_k) + B(v_k), \quad \gamma \left\| a_k +b_k\right\| _\mathcal{X}=\left\| u_k - v_k\right\| _\mathcal{X}\le \frac{\sqrt{2}\Vert x_0-x^*\Vert _\mathcal{X}}{\sqrt{k} \sqrt{2 {\tilde{\theta }}-\theta }}. \end{aligned}$$

Proof

The inclusion and the equality in the theorem follows from (32). Since by Proposition 4.4, the relaxed PR splitting method with $\theta \in (0,2\theta _0)$ is an NE-HPE instance for solving the monotone inclusion $0 \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)(z)$ in which $\sigma = (\theta /\tilde{\theta }-1)^2 <1$, $\varepsilon _k = 0$ and $\lambda _k=1$ for all $k \ge 1$, it follows from Lemma 4.5, Theorem 3.3, the fact that $M=m=1/\theta$, and relation (12) that

$$\begin{aligned} \Vert r_k\Vert _* \le \frac{\sqrt{2}M}{\sqrt{m}}(1+\sqrt{\sigma }) \sqrt{\frac{(dw)_0}{1-\sigma } \left( \frac{1}{ \sum _{j=1}^k \lambda _j^2}\right) } \le \frac{\sqrt{2}\Vert x_0-x^*\Vert _\mathcal{X}}{\sqrt{k} \sqrt{2 {\tilde{\theta }}-\theta }}. \end{aligned}$$

The inequality of the theorem then follows by Proposition 4.4(a) and relation (38). $\square$

Our main goal in the remaining part of this section is to derive ergodic convergence rate bounds for the relaxed PR splitting method for any $\theta \in (0, 2\theta _0]$. We start by stating the following variation of the transportation lemma for maximal $\beta$-strongly monotone operators.

Proposition 4.7

Assume that T is a maximal $\beta$-strongly monotone operator for some $\beta \ge 0$. Assume also that $t_i \in T(u_i)$ for $i = 1, \ldots , k$, and define

$$\begin{aligned} \bar{t}_k = \frac{1}{k}\sum _{i=1}^{k} t_i, \quad \bar{u}_k = \frac{1}{k} \sum _{i=1}^{k} u_i, \quad \varepsilon _k = \frac{1}{k} \sum _{i=1}^k \langle t_i - \beta u_i, u_i - \bar{u}_k \rangle _{\mathcal{X}} \end{aligned}$$

(39)

Then, $\varepsilon _k \ge 0$ and $\bar{t}_k \in T^{[\varepsilon _k]}(\bar{u}_k)$.

Proof

The assumption that T is a maximal $\beta$-strongly monotone operator implies that $T - \beta I$ is maximal monotone. Hence, it follows from the weak transportation formula (see Theorem 2.3 of [5]) applied to $T - \beta I$ that $\varepsilon _k \ge 0$ and $\bar{t}_k - \beta \bar{u}_k \in (T - \beta I)^{[\varepsilon _k]}(\bar{u}_k)$. The result then follows by observing that $(T - \beta I)^{[\varepsilon _k]}(\bar{u}_k) + \beta \bar{u}_k \subseteq T^{[\varepsilon _k]}(\bar{u}_k)$. $\square$

In order to state the ergodic iteration complexity bound for the relaxed PR splitting method, we introduce the ergodic sequences

$$\begin{aligned} \bar{u}_k = \frac{1}{k} \sum _{i=1}^k u_i, \quad \bar{v}_k = \frac{1}{k} \sum _{i=1}^k v_i, \quad \bar{a}_k = \frac{1}{k} \sum _{i=1}^k a_i, \quad \bar{b}_k = \frac{1}{k} \sum _{i=1}^k b_i \end{aligned}$$

(40)

and the scalar sequences

$$\begin{aligned} \varepsilon _k' := \frac{1}{k} \sum _{i=1}^k \langle a_i - \beta u_i, u_i-\bar{u}_k\rangle _{\mathcal{X}}, \quad \varepsilon _k'' := \frac{1}{k} \sum _{i=1}^k \langle b_i - \beta v_i, v_i-\bar{v}_k\rangle _{\mathcal{X}}. \end{aligned}$$

(41)

Theorem 4.8

Assume that $\theta \in (0,2\theta _0]$ and consider the ergodic sequences above. Then, for every $k \ge 1$ and $z^*=(u^*,u^*,x^*) \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)^{-1}(0)$,

$$\begin{aligned}&\bar{a}_k \in A^{[\varepsilon _k']}(\bar{u}_k), \quad \bar{b}_k \in B^{[\varepsilon _k'']}(\bar{v}_k),\\&\gamma \left\| \bar{a}_k + \bar{b}_k\right\| _\mathcal{X}=\left\| \bar{u}_k - \bar{v}_k\right\| _\mathcal{X}\le \frac{2 \Vert x_0 - x^*\Vert _\mathcal{X}}{k \theta }, \quad \\&\quad \varepsilon _k' + \varepsilon _k'' \le \frac{3(1+ 2(1-\tilde{\theta }/\theta )^2) \Vert x_0-x^*\Vert ^2_\mathcal{X}}{k \gamma \theta }. \end{aligned}$$

Proof

The first two inclusions follow from the two inclusions in (30) and (31), relation (40), Assumption B0 and Proposition 4.7. We will now derive the equality and the two inequalities of the theorem using the fact that the relaxed PR splitting method with $\theta \in (0,2\theta _0]$ is an instance of the NE-HPE method. Letting $\lambda _k=1$, $\varepsilon _k = 0$ for every k and $\tilde{z}_k^a$, $r_k^a$ and $\varepsilon _k^a$ be as in (14), we easily see from Proposition 4.4(a) and (40) that

$$\begin{aligned} r^a_k&= (0,0,\bar{u}_k-\bar{v}_k) = \gamma (0,0,\bar{a}_k + \bar{b}_k), \end{aligned}$$

(42)

$$\begin{aligned} \varepsilon _k^a&= \frac{1}{k} \sum _{i=1}^k \langle r_i,\tilde{z}_i-\tilde{z}_k^a\rangle . \end{aligned}$$

(43)

We claim that

$$\begin{aligned} \varepsilon _k^a \ge \gamma (\varepsilon _k' + \varepsilon _k''). \end{aligned}$$

(44)

Before proving this claim, we will use it to complete the proof of the theorem. Indeed, using the definition of w in (36), relations (12), (38), (42) and (44), the conclusion of Proposition 4.4, and Theorem 3.4 with $T=\mathcal{L}_{\tilde{\theta }} + \gamma C$, $M=m=1/\theta$ and $\lambda _k=1$ for all k, we conclude that

$$\begin{aligned} \gamma \left\| \bar{a}_k + \bar{b}_k\right\| _\mathcal{X}= \Vert \bar{u}_k-\bar{v}_k\Vert _\mathcal{X}= \Vert r_k^a\Vert _* \le \frac{2 \sqrt{2}(dw)_0^{1/2}}{\sqrt{\theta } \Lambda _k} \le \frac{2 \Vert x_0-x^*\Vert _\mathcal{X}}{k \theta } \end{aligned}$$

and

$$\begin{aligned} \gamma (\varepsilon _k' + \varepsilon _k'') \le \varepsilon _k^a \le \frac{ 3(2 (dw)_0 + \rho _k)}{\Lambda _k} \le \frac{3 \left( \Vert x_0-x^*\Vert ^2_\mathcal{X}+ \rho _k \right) }{k \theta } \end{aligned}$$

(45)

where $\rho _k$ is defined in (15). Moreover, using (15), the definition of w in (36), the definition of ${x}_i$ and $\tilde{x}_i$ in (21) and (28), respectively, the triangle inequality, and Proposition 3.2(a), we conclude that

$$\begin{aligned} \rho _k :=&\displaystyle \max _{i=1,\ldots ,k}(dw)_{z_{i}}(\tilde{z}_{i}) = \displaystyle \max _{i=1,\ldots ,k} \frac{\Vert x_{i} - \tilde{x}_i\Vert _\mathcal{X}^2}{2\theta } \\ =&\,\,(1- \tilde{\theta }/\theta )^2 \displaystyle \max _{i=1,\ldots ,k} \frac{\Vert x_{i} - x_{i-1}\Vert _\mathcal{X}^2}{2\theta } \le \frac{2(1-\tilde{\theta }/\theta )^2 \Vert x_0-x^*\Vert ^2_\mathcal{X}}{\theta }. \end{aligned}$$

The inequalities of the theorem now follows from the above three relations.

In the remaining part of the proof, we establish our previous claim (44). By Proposition 4.4(a) and relations (33) and (43), we have

$$\begin{aligned} k \varepsilon _k^a = \sum _{i=1}^k \langle r_i,\tilde{z}_i - \tilde{z}_k^a\rangle = \sum _{i=1}^k \langle \mathcal{L}_{\tilde{\theta }}(\tilde{z}_i)+ \gamma c_i,\tilde{z}_i-\tilde{z}_k^a\rangle \end{aligned}$$

(46)

where $c_i$ is defined in (34). Moreover, we have

$$\begin{aligned} \sum _{i=1}^k \langle \mathcal{L}_{\tilde{\theta }}(\tilde{z}_i)+ \gamma c_i,\tilde{z}_i-\tilde{z}_k^a\rangle&= \sum _{i=1}^k \langle \mathcal{L}_{\tilde{\theta }}(\tilde{z}_i - \tilde{z}_k^a),\tilde{z}_i-\tilde{z}_k^a\rangle + \gamma \sum _{i=1}^{k} \langle c_i,\tilde{z}_i-\tilde{z}_k^a\rangle \nonumber \\&= \,(1 - \tilde{\theta }) \sum _{i=1}^{k} \Vert (u_i - \bar{u}_k) - (v_i - \bar{v}_k) \Vert ^2_{\mathcal{X}}\nonumber \\&\quad +\, \gamma \sum _{i=1}^k \langle c_i,\tilde{z}_i-\tilde{z}_k^a\rangle \nonumber \\&\ge \, - \frac{\gamma \beta }{2} \sum _{i=1}^{k} \Vert (u_i - \bar{u}_k) - (v_i - \bar{v}_k) \Vert ^2_{\mathcal{X}}\nonumber \\&\quad +\, \gamma \sum _{i=1}^k \langle c_i,\tilde{z}_i-\tilde{z}_k^a\rangle , \end{aligned}$$

(47)

where the second equality follows from (25) and the definitions of $\tilde{z}_k^a$ in (14), $\tilde{z}_k$ in (28), and $\bar{u}_k$ and $\bar{v}_k$ in (40), and the inequality follows from (27) and the fact that $\tilde{\theta }\le \theta _0$ in view of (35). Finally, using the definitions of $\tilde{z}_k^a$ in (14), and $\tilde{z}_i$ and $c_i$ in Lemma 4.3, and the straightforward relation

$$\begin{aligned} - \frac{1}{2} \sum _{i=1}^{k} \Vert (u_i - \bar{u}_k) - (v_i - \bar{v}_k) \Vert ^2_{\mathcal{X}} \ge - \sum _{i=1}^{k} \left( \langle u_i,u_i - \bar{u}_k\rangle _{\mathcal{X}} + \langle v_i,v_i - \bar{v}_k\rangle _{\mathcal{X}}\right) , \end{aligned}$$

we conclude from (46) and (47) that

$$\begin{aligned} \varepsilon _k^a \ge \frac{\gamma }{k} \sum _{i=1}^k ( \langle a_i - \beta u_i,u_i-\bar{u}_k\rangle _{\mathcal{X}} + \langle b_i - \beta v_i,v_i-\bar{v}_k\rangle _{\mathcal{X}} ), \end{aligned}$$

and hence that the claim holds in view of (41). $\square$

We now make some remarks about the convergence rate bounds obtained in Theorem 4.8. In view of Lemma 4.1, $x^*$ depends on $\gamma$ according to

$$\begin{aligned} x^* = \gamma a^* + u^*, \quad a^* \in A(u^*) \cap -B(u^*). \end{aligned}$$

Hence, letting

$$\begin{aligned} d_0&:= \inf \{ \Vert x_0 - u^* \Vert _\mathcal{X}: u^* \in (A+B)^{-1}(0) \}, \\ S&:= \sup \{ \Vert a^*\Vert : a^* \in A(u^*) \cap -B(u^*), \, u^* \in (A+B)^{-1}(0) \}, \end{aligned}$$

and assuming that $S< \infty$, it is easy to see that Theorem 4.8 and (27) imply that the relaxed PR splitting method with $\theta =2\theta _0$ satisfies

$$\begin{aligned} \left\| \bar{a}_k + \bar{b}_k\right\| _\mathcal{X}\le \frac{C_1(\gamma )}{\gamma k}, \quad \left\| \bar{u}_k - \bar{v}_k\right\| _\mathcal{X}\le \frac{C_1(\gamma )}{k}, \quad \varepsilon _k' + \varepsilon _k'' \le \frac{C_2(\gamma )}{k} \end{aligned}$$

where

$$\begin{aligned} C_1(\gamma )= & {} C_1(\gamma ;\beta ,d_0) = {\Theta } \left( \frac{d_0 + \gamma S}{1+ \beta \gamma } \right) , \quad \\ C_2(\gamma )= & {} C_2(\gamma ;\beta ,d_0) = {\Theta } \left( \frac{(d_0 + \gamma S)^2}{\gamma (1+ \beta \gamma )} \right) . \end{aligned}$$

When $S/\beta \ge d_0$, then $\gamma = d_0/S$ minimizes both $C_1(\cdot )$ and $C_2(\cdot )$ up to a multiplicative constant, in which case $C_1^* ={\Theta }(d_0)$, $C_1^*/\gamma = {\Theta }(S)$ and $C_2^*={\Theta }(Sd_0)$ where

$$\begin{aligned} C^*_1 = C^*_1(\beta ,d_0) := \inf \{ C_1(\gamma ) : \gamma>0\}, \quad C^*_2 = C^*_2(\beta ,d_0) := \inf \{ C_2(\gamma ) : \gamma >0\}. \end{aligned}$$

Note that this case includes the case in which $\beta =0$. On the other hand, when $S/\beta < d_0$, then both $C_1$ and $C_2$ are minimized up to a multiplicative constant by any $\gamma \ge d_0/S$, in which case $C_1^* ={\Theta }(S/\beta )$ and $C_2^*={\Theta }(S^2/\beta )$. Clearly, in this case, $C_1^*/\gamma$ converges to zero as $\gamma$ tends to infinity.

Indeed, assume first that $S/\beta \ge d_0$. Then, up to some multiplicative constants, we have

$$\begin{aligned} C_1(\gamma )\ge & {} \frac{d_0+\gamma S}{1+\beta \gamma } \ge \frac{d_0+\gamma S}{1+S \gamma /d_0} = d_0,\\ C_2(\gamma )\ge & {} \frac{(d_0+\gamma S)^2}{\gamma (1+\beta \gamma )} \ge \frac{(d_0+\gamma S)^2}{\gamma (1+S \gamma /d_0)} = \frac{d_0( d_0+\gamma S)}{\gamma } = \frac{d_0^2}{\gamma } + Sd_0, \end{aligned}$$

and hence that $C_1^*= \Omega (d_0)$ and $C_2^*={\Omega }(Sd_0)$. Moreover, if $\gamma =d_0/S$, then the assumption $S/\beta \ge d_0$ implies that $\beta \gamma \le 1$, and hence that $C_1^* = {\Theta }(d_0)$ and $C_2^*={\Theta }(Sd_0)$.

Assume now that $S/\beta < d_0$. Then, up to multiplicative constants, it is easy to see that

$$\begin{aligned} C_1(\gamma )\ge & {} \frac{d_0+\gamma S}{1+\beta \gamma } \ge \frac{S}{\beta }\\ C_2(\gamma )\ge & {} \frac{(d_0+\gamma S)^2}{\gamma (1+\beta \gamma )} \ge \frac{(S/\beta +\gamma S)^2}{\gamma (1+\beta \gamma )} = \frac{S^2}{\gamma \beta ^2} (1+\gamma \beta ), \end{aligned}$$

and hence that $C_1^*= \Omega (S/\beta )$ and $C_2^*={\Omega }(S^2/\beta )$. Moreover, if $\gamma \ge d_0/S$, then it is easy to see that $C_1^*={\Theta }(S/\beta )$ and $C_2^* = {\Theta }(S^2/\beta )$.

Based on the above discussion, the choice $\gamma =d_0/S$ is optimal but has the disadvantage that $d_0$ is generally difficult to compute. One possibility around this difficulty is to use $\gamma = D_0/S$ where $D_0$ is an upper bound on $d_0$.

5 On the convergence of the relaxed PR splitting method

This section discusses some new convergence results about the sequence generated by the relaxed PR splitting method for the case in which $\beta >0$. It contains two subsections. As observed in the Introduction, [12] already establishes the convergence of the relaxed PR sequence for the case in which $\beta \ge 0$ and $\theta <2\theta _0$. The first subsection establishes convergence of the relaxed PR sequence for the case in which $\beta >0$ and $\theta =2\theta _0$. The second subsection describes an instance showing that the relaxed PR spliting method may diverge when $\beta \ge 0$ and $\theta \ge \min \{ 2(1+\gamma \beta ), 2 + \gamma \beta + 1/(\gamma \beta ) \}$. (Here, we assume that $1/0=\infty$.) Note that this instance, specialized to the case $\beta = 0$, shows that the sequence $\{ z_k = (u_k, v_k, x_k) \}$ generated by the relaxed PR splitting method with $\beta =0$ may diverge for any $\theta \ge 2$, and hence that the convergence result obtained for any $\theta \in (0,2)$ in [12] cannot be improved.

5.1 Convergence result about the relaxed PR sequence

It is known that the sequence $\{z_k = (u_k, v_k, x_k)\}$ generated by the relaxed PR splitting method with $\theta \in (0,2\theta _0)$ and $\beta \ge 0$ converges [12]. The main result of this subsection, namely Theorem 5.2, establishes convergence of this sequence for $\theta = 2 \theta _0$ when $\beta > 0$.

We start by giving a lemma which is used in the proof of Theorem 5.2.

Lemma 5.1

Consider the sequence $\{z_k = (u_k, v_k, x_k)\}$ generated by the relaxed PR splitting method with $\theta \in (0,2\theta _0]$ and the sequence $\{\tilde{z}_k=(u_k,v_k,\tilde{x}_k)\}$ defined in (28). Then, the sequences $\{ z_k \}$ and $\{ \tilde{z}_k \}$ are bounded.

Proof

The assumption that $\theta \in (0, 2\theta _0]$ together with the last conclusion of Proposition 4.4 imply that the relaxed PR splitting method is an NE-HPE instance with $\sigma \le 1$. Hence, for any $z^* \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)^{-1}(0)$, it follows from Proposition 3.2(a) that the sequence $\{(dw)_{z_k}(z^*)\}$ is non-increasing where w is the distance generating function given by (36). Clearly, this observation implies that $\{x_k\}$ is bounded. This conclusion together with (20) and the nonexpansiveness of $J_{\gamma A}, J_{\gamma B}$ imply that $\{ u_k \}$ and $\{ v_k \}$ are also bounded. Finally, $\{ \tilde{x}_k \}$ is bounded due to the definition of $\tilde{x}_k$ in (28), and the boundedness of $\{x_k\}$, $\{ u_k \}$ and $\{ v_k \}$. $\square$

As mentioned at the beginning of this subsection, the convergence of $\{(u_k,v_k)\}$ to some pair $(u^*,u^*)$ where $u^* \in (A+B)^{-1}(0)$ has been established in [12] for the case in which $\beta > 0$ and $\theta < 2 \theta _0$. The following result shows that the latter conclusion can also be extended to $\theta = 2 \theta _0$.

Theorem 5.2

In addition to Assumption B1, assume that Assumption B0 holds with $\beta >0$. Then, the sequence $\{z_k = (u_k, v_k, x_k)\}$ generated by the relaxed PR splitting method with $\theta =2\theta _0$ converges to some point lying in $(\mathcal{L}_{\theta _0}+\gamma C)^{-1}(0)$.

Proof

We assume that $\theta =2\theta _0$ and without any loss of generality that $\gamma =1$. In view of (35), we have $\tilde{\theta } = \theta _0$. Let $z^*\in (\mathcal{L}_{\theta _0} + C)^{-1}(0)$. Then, by Lemma 4.1, we have $z^*= (u^*, u^*, x^*)$ where

$$\begin{aligned} u^*= (A+B)^{-1}(0), \quad x^*- u^*\in A(u^*), \quad -x^*+ u^*\in B(u^*). \end{aligned}$$

(48)

Since $\tilde{\theta } = \theta _0$, it follows from Proposition 4.4(b) that $r_k \in (\mathcal{L}_{\theta _0} + C)(\tilde{z}_k)$. This together with the fact that $0 \in (\mathcal{L}_{\theta _0} + C)(z^*)$, inequality (26) with $(z,r)=(\tilde{z}_k,r_k)$ and $(z',r')=(z^*,0)$, and the fact that $\theta _0 = 1 + \beta /2$, then imply that

$$\begin{aligned} \langle r_k, \tilde{z}_k - z^*\rangle&\ge (1 - \theta _0) \Vert (u_k - u^*) - (v_k - u^*) \Vert _{\mathcal{X}}^2 + \beta (\Vert u_k - u^*\Vert ^2_{\mathcal{X}} + \Vert v_k - u^*\Vert ^2_{\mathcal{X}} ) \nonumber \\&= \frac{\beta }{2} \Vert u_k + v_k - 2u^*\Vert ^2_{\mathcal{X}} \ge 0. \end{aligned}$$

(49)

Since the last conclusion of Proposition 4.4 states that the relaxed PR splitting method with $\theta =2\theta _0$ is an NE-HPE instance with respect to the monotone inclusion $0 \in (\mathcal{L}_{\tilde{\theta }}+\gamma C)(z)$ in which $\sigma =1$, $\lambda _k = 1$ and $\varepsilon _k = 0$ for every k, it follows from Proposition 3.2(b), (49) and the assumption that $\beta >0$ that

$$\begin{aligned} \lim _{k \rightarrow \infty } \langle r_{k}, \tilde{z}_{k} - z^*\rangle = \lim _{k \rightarrow \infty } \Vert u_{k} + v_{k} - 2u^*\Vert _\mathcal{X}= 0. \end{aligned}$$

(50)

By Lemma 5.1, $\{z_k\}$, and hence $\{x_k\}$, is bounded. Therefore, there exist an infinite index set $\mathcal{K}$ and $\bar{x} \in {\mathcal{X}}$ such that $\lim _{k \in \mathcal{K}} x_{k-1} = \bar{x}$, from which we conclude that

$$\begin{aligned} \lim _{k \in \mathcal{K}} u_k = \bar{u} := J_A(\bar{x}) , \quad \lim _{k \in \mathcal{K}} v_k = \bar{v} := J_B( 2 \bar{u} - \bar{x}) \end{aligned}$$

(51)

in view of (20), (21) and the continuity of the point-to-point maps $J_A$ and $J_B$. Clearly, relations (21), (50) and (51), Proposition 4.4(a), the definitions of $J_B$ following (2) and $\tilde{z}_k$ in (28), and the fact that $\tilde{\theta } = \theta _0$, imply that

$$\begin{aligned} \bar{u} + \bar{v}&= 2 u^*, \quad 2 \bar{u} - \bar{v} - \bar{x} \in B(\bar{v}), \quad \lim _{k \in \mathcal{K}} z_k = (\bar{u}, \bar{v}, \bar{x}+ \theta (\bar{v}-\bar{u})), \end{aligned}$$

(52)

$$\begin{aligned} \lim _{k \in \mathcal{K}} r_k&= (0,0,\bar{u}-\bar{v}), \quad \lim _{k \in \mathcal{K}} \tilde{z}_k = \tilde{z}:= (\bar{u}, \bar{v}, \bar{x}+ \theta _0 (\bar{v}-\bar{u})). \end{aligned}$$

(53)

Clearly, (50) and (53) imply that

$$\begin{aligned} 0 = \lim _{k \in \mathcal{K}} \langle r_{k}, \tilde{z}_{k} - z^*\rangle = \langle \bar{u} - \bar{v}, \bar{x}\,+\, \theta _0 (\bar{v}-\bar{u}) - x^*\rangle _{\mathcal{X}} = - \theta _0 \Vert \bar{u}- \bar{v}\Vert ^2_{\mathcal{X}}\, +\, \langle \bar{u} - \bar{v} , \bar{x} - x^* \rangle _{\mathcal{X}}.\nonumber \\ \end{aligned}$$

(54)

Using the second inclusion in (48), the identity and the inclusion in (52), the $\beta$-strong monotonicity of B, and relation (54), we then conclude that

$$\begin{aligned} \frac{\beta }{4} \Vert \bar{v} - \bar{u}\Vert ^2_{\mathcal{X}}&= \beta \Vert \bar{v} - u^*\Vert ^2_{\mathcal{X}} \le \left\langle (2 \bar{u} - \bar{v} - \bar{x}) - ( u^*- x^*), \bar{v} - u^*\right\rangle _{\mathcal{X}}\\&= \frac{1}{2} \left\langle \frac{3}{2} (\bar{u} - \bar{v}) - \bar{x} + x^*, \bar{v} - \bar{u} \right\rangle _{\mathcal{X}} \\&= \frac{1}{2} \left\langle \bar{x} - x^*, \bar{u} - \bar{v} \right\rangle _{\mathcal{X}} - \frac{3}{4} \Vert \bar{v} - \bar{u}\Vert ^2_{\mathcal{X}} = \left( \frac{\theta _0}{2} - \frac{3}{4} \right) \Vert \bar{v} - \bar{u}\Vert ^2_{\mathcal{X}}. \end{aligned}$$

The latter inequality together with the fact that $\theta _0=1+(\beta /2)$ then imply that $\bar{u}=\bar{v} = u^*$ where the last equality is due to the identity in (52). We have thus shown that $\{u_k\}_{k \in \mathcal{K}}$ and $\{v_k\}_{k \in \mathcal{K}}$ both converge to $u^* = (A+B)^{-1}(0)$. Since $\bar{u}=\bar{v}=u^*$, it follows from (52) and (53) that

$$\begin{aligned} \lim _{k \in \mathcal{K}}r_k=0, \quad \lim _{k \in \mathcal{K}} z_k&= \lim _{k \in \mathcal{K}} \tilde{z}_k = \tilde{z}= (u^*,u^*,\bar{x}), \\ \lim _{k \in \mathcal{K}} (dw)_{z_k}(\tilde{z}_k)&= \lim _{k \in \mathcal{K}} \Vert x_k-\tilde{x}_k\Vert _\mathcal{X}^2/(2\theta ) = \Vert \bar{x}-\bar{x}\Vert _\mathcal{X}^2/(2\theta )=0. \end{aligned}$$

Hence, Proposition 3.5 with $T=(\mathcal{L}_{\theta _0} + \gamma C)$ implies that $\tilde{z}\in (\mathcal{L}_{\theta _0} + \gamma C)^{-1}(0)$ and $0=\lim _{k \rightarrow \infty } (dw)_{z_k}(\tilde{z}) = \lim _{k \rightarrow \infty } \Vert \bar{x}-x_k\Vert ^2_\mathcal{X}/(2\theta )$. We thus conclude that $\{z_k\}$ converges to $(u^*,u^*, \bar{x})=\tilde{z}\in (\mathcal{L}_{\theta _0} + \gamma C)^{-1}(0)$. $\square$

Before ending this subsection, we make two remarks. First, for a fixed $\tau >0$, consider the set

$$\begin{aligned} R(\tau ):= \{ (\beta ,\theta ) \in \mathbb {R}^2 : \beta >0, \, 0 < \theta \le 2 + \tau \beta \}. \end{aligned}$$

Then, it follows from Theorem 5.2 and the observation in the paragraph preceding it that the sequence generated by the relaxed PR splitting method with relaxation parameter $\theta$ to solve (1) with A, B maximal $\beta$-strongly monotone converges for any $(\beta ,\theta ) \in R(1)$. Second, it follows from the example presented in the next subsection that the above conclusion fails if R(1) is enlarged to the region $R(\tau )$ for any $\tau >1$.

5.2 Non-convergent instances for $\theta \ge \min \{ 2+2\gamma \beta , 2 + \gamma \beta + 1/(\gamma \beta ) \}$

By [12] and Theorem 5.2, the sequence $\{ x_k \}$ generated by the relaxed PR splitting method converges whenever either $\theta \in (0, 2 + \gamma \beta )$ or $\theta = 2 + \gamma \beta$ and $\beta > 0$. This subsection gives an instance of (1), where A, B are maximal $\beta$-strongly monotone, for which the sequence $\{x_k\}$ generated by the relaxed PR splitting method with relaxation parameter $\theta$ does not converge when $\beta \ge 0$ and $\theta \ge \min \{ 2(1+\gamma \beta ), 2 + \gamma \beta + 1/(\gamma \beta ) \}$.

Recall from (20) and (21) that the relaxed PR splitting method iterates as

$$\begin{aligned} x_{k+1} = x_{k} + \theta (J_{\gamma B}(2J_{\gamma A}(x_{k}) - x_{k}) - J_{\gamma A}(x_{k})) \end{aligned}$$

(55)

where $\theta > 0$. Without any loss of generality, we assume that $\gamma = 1$ in (55).

We now describe our instance. First, let $\mathcal{X}:= \mathcal {\tilde{X}} \times \mathcal {\tilde{X}}$ where $\mathcal {\tilde{X}}$ is a finite-dimensional real vector space, and let $A_0, B_0: \mathcal{X}\rightrightarrows \mathcal{X}$ be defined as

$$\begin{aligned} A_0(\tilde{x}_1,\tilde{x}_2) = (0,0), \quad B_0(\tilde{x}_1,\tilde{x}_2)= N_{\{0\}}(\tilde{x}_1) \times \{0\}, \quad \forall x=(\tilde{x}_1,\tilde{x}_2) \in \mathcal {\tilde{X}} \times \mathcal {\tilde{X}} \end{aligned}$$

where $N_{\{0\}}(\cdot )$ denotes the normal cone operator of the set $\{0\}$. Clearly, $A_0$ and $B_0$ are both maximal monotone operators and

$$\begin{aligned} J_{A_0}(\tilde{x}_1, \tilde{x}_2) = (\tilde{x}_1,\tilde{x}_2), \quad J_{B_0}(\tilde{x}_1,\tilde{x}_2) = (0,\tilde{x}_2), \quad \forall x=(\tilde{x}_1, \tilde{x}_2) \in \mathcal {\tilde{X}} \times \mathcal {\tilde{X}}. \end{aligned}$$

Now define $A := A_0 + \beta I$ and $B:=B_0+\bar{\beta } I$ where $\bar{\beta } \ge \beta \ge 0$. It follows that A is a $\beta$-strongly maximal monotone operator and B is a $\bar{\beta }$-strongly maximal monotone operator. Hence, the instance we are describing is slightly more general in that A and B have different strong monotonicity parameters.

Moreover, for any $x =({\tilde{x}}_1,{\tilde{x}}_2) \in \mathcal{X}$, it is easy to see that

$$\begin{aligned} J_A(x)= & {} J_{\frac{1}{1+\beta }A_0}\left( \frac{1}{1 + \beta }({\tilde{x}}_1,{\tilde{x}}_2)\right) = \frac{1}{1 + \beta }({\tilde{x}}_1,{\tilde{x}}_2),\\ J_B(x)= & {} J_{\frac{1}{1+\beta }B_0}\left( \frac{1}{1 + \bar{\beta }}({\tilde{x}}_1,{\tilde{x}}_2)\right) = \frac{1}{1+\bar{\beta }} (0, {\tilde{x}}_2). \end{aligned}$$

and hence that

$$\begin{aligned} x + \theta [J_{B}(2J_{A}(x) - x) - J_{A}(x)] = \left( \frac{(1 + \beta - \theta )}{1 + \beta }\tilde{x}_1, \left[ 1 - \frac{(\beta + \bar{\beta }) \theta }{(1+\beta )(1+ \bar{\beta })}\right] \tilde{x}_2 \right) .\nonumber \\ \end{aligned}$$

(56)

From (55) and (56), we easily see that the sequence $\{x_k\}$ generated by the relaxed PR splitting method diverges whenever

$$\begin{aligned} \frac{(1 + \beta - \theta )}{1 + \beta } \le -1, \ \ \text{ or } \ \ 1 - \frac{(\beta + \bar{\beta }) \theta }{(1+\beta )(1+ \bar{\beta })} \le -1, \end{aligned}$$

or equivalently, whenever

$$\begin{aligned} \theta \ge \min \left\{ 2(1+\beta ) \,,\, 2 + \frac{2(1+\beta \bar{\beta })}{\beta + \bar{\beta }} \right\} . \end{aligned}$$

Note that when $\beta =\bar{\beta }$, the above inequality reduces to $\theta \ge \min \{2(1+\beta ), 2 + \beta + 1/\beta \}$.

Before ending this subsection, we make two remarks. First, when $\beta =0$ and hence A is not strongly monotone, the sequence $\{ x_k \}$ for the above example diverges for any $\theta \ge 2$ even if B is strongly monotone, i.e., $\bar{\beta }>0$. Second, the above example specialized to the case in which $\beta =\bar{\beta }$ easily shows that the sequence generated by the relaxed PR splitting method does not necessarily converge for any $(\beta ,\theta ) \in R(\tau )$ if $\tau >1$ where $R(\tau )$ is defined at the end of Sect. 5.1.

6 Numerical study

This section illustrates the behavior of the relaxed PR splitting method for solving the weighted Lasso minimization problem [16] (see also [6])

$$\begin{aligned} \min _{u \in \mathbb {R}^n} f(u) + g(u), \end{aligned}$$

(57)

where $f(u) := \frac{1}{2} \Vert Cu - b \Vert _{\mathcal{X}}^2$ and $g(u) := \Vert Wu \Vert _1$ for every $u \in \mathbb {R}^n$. Our numerical experiments consider instances where $n = 200$, $b \in \mathbb {R}^{300}$ and $C \in \mathbb {R}^{300 \times 200}$ is a sparse data matrix with an average of 10 nonzero entries per row. Each component of b and each nonzero element of C is drawn from a Gaussian distribution with zero mean and unit variance, while $W \in \mathbb {R}^{200 \times 200}$ is a diagonal matrix with positive diagonal elements drawn from a uniform distribution on the interval [0, 1]. This setup follows that of [16]. Note that $\mathcal{X}= \mathbb {R}^{300}$ and $\Vert \cdot \Vert _1$ is the 1-norm on $\mathbb {R}^{200}$. Observe that f is $\alpha$-strongly convex¹ on $\mathbb {R}^{200}$ where $\alpha = \lambda _{\min }(C^TC)$ is the minimum eigenvalue of $C^T C$. Also, f is differentiable and its gradient is $\kappa$-Lipschitz continuous on $\mathbb {R}^{200}$ where $\kappa = \lambda _{\max }(C^TC)$ is the maximum eigenvalue of $C^TC$. The function g is clearly convex on $\mathbb {R}^{200}$.

We consider solving (57) by apply the relaxed PR splitting method (55) to solve the inclusion (1) with

$$\begin{aligned} A = \partial f - \alpha ^\prime I, \quad B = \partial g + \alpha ^\prime I, \end{aligned}$$

(58)

where $0 \le \alpha ^\prime \le \alpha = \lambda _{\min }(C^TC)$. Since A (resp., B) is $(\alpha -\alpha ')$-strongly (resp., $\alpha '$-strongly) maximal monotone, the results developed in Sects. 4 and 5 for the relaxed PR splitting method with (A, B) as above applies with $\beta = \min \{ \alpha - \alpha ^\prime , \alpha ^\prime \}$. Our goal in this section is to gain some intuition of how the relaxed PR splitting method performs as $\alpha ^\prime$ (and hence $\beta$), $\gamma$ and $\theta$ change. In our numerical experiments, we start the relaxed PR splitting algorithm with $x_0 = 0$ and terminate it when $\Vert x_{k+1} - x_{k} \Vert _{\mathcal{X}} \le 10^{-5}$. The paragraphs below report the results of three experiments.

In the first experiment, we generate 100 random instances of (C, W, b) and we observed that the condition $\lambda _{\min }(C^T C) > 0$ holds for all instances. The relaxed PR splitting method is used to solve these instances of (57) for various values of $\theta$ and with the pair $(\gamma ,\alpha ')$ taking on the values (1, 0), $(1,\alpha /2)$, $(1/\sqrt{\alpha \kappa },0)$ and $(1/\sqrt{\alpha \kappa },\alpha /2)$. Note that it follows from Proposition 3 of [16] that when $\alpha '=0$ and $\theta =2$, the choice of $\gamma =1/\sqrt{\alpha \kappa }$ has been shown to be optimal for the relaxed PR splitting method. Our results are shown in Table 1. We see from the table that, except when $\theta = 2$ and $\theta = 2 + \gamma \alpha /2$, the average number of iterations for $\alpha ^\prime = 0$ and $\alpha ^\prime = \alpha /2$ are similar. However, when $\theta = 2$ and $\theta = 2 + \gamma \alpha /2$, the choice $\alpha ^\prime = \alpha /2$ outperforms the one with $\alpha ^\prime = 0$. One possible explanation for this behavior is due to the fact that when $\theta = 2$ and $\theta = 2 + \gamma \alpha /2$, the relaxed PR sequence converges when both operators are strongly monotone, while it does not necessary converge when either one of the operators is only monotone. Note also that the results in the last row of the table confirm the convergence result of the relaxed PR splitting method (see Theorem 5.2) for the case in which A and B are $\beta$-strongly maximal monotone operators with $\beta > 0$ and $\theta = 2 + \gamma \beta$. Finally, our results (the last two rows of table) suggest that, if A is maximal $\alpha$-strongly monotone and B is only maximal monotone, it might be advantageous to use the relaxed PR splitting method with $0< \alpha ^\prime < \alpha$ (and hence $\beta >0$) instead of $\alpha '= 0$ (and hence $\beta =0$).

Table 1 Average number of iterations performed by the relaxed PR splitting method (55) to solve (57) based on the partition (A, B) given by (58) for 4 pairs $(\gamma ,\alpha ')$ and 6 different values of $\theta$

In our second experiment, we use relaxed PR splitting method with $(\theta ,\gamma )$ equal to (2, 1) and $(2,1/\sqrt{\alpha \kappa })$, and with $\alpha ^\prime$ varying from 0 to $\alpha$, to solve (57) for a randomly generated (C, W, b). In this instance, $\alpha = \lambda _{\min }(C^T C) = 0.3792$ and $\kappa = \lambda _{\max }(C^T C) = 57.6624$. Our results are shown in Fig. 1. We see from Fig. 1 that the number of iterations decreases as $\alpha ^\prime$ increases in both cases. These graphs again suggest that it might be advantageous to have A and B maximal $\beta$-strongly monotone with $\beta > 0$. We also observe that as $\alpha ^\prime$ approaches $\alpha$, the number of iterations does not increase even though the operator A is losing its strong monotonicity.

Fig. 1

Two graphs showing how the number of iterations performed by (55) changes with varying $\alpha ^\prime$ using the partition (A, B) given (58)

In our third experiment, we performed the same numerical experiments as the ones mentioned above but with $(A,B) = (\partial g + \alpha ^\prime I, \partial f - \alpha ^\prime I)$ instead of $(A,B) = (\partial f - \alpha ^\prime I, \partial g + \alpha ^\prime I)$ and note that the results obtained were very similar to the ones reported above. Hence interchanging A and B in the implementation of the relaxed PR splitting method have little impact on its performance.

Table 2 Summary of convergence rate results for the relaxed PR splitting method on the sum of two convex functions f, g

7 Concluding remarks

This paper establishes convergence of the sequence of iterates and an $\mathcal{O}(1/{k})$ ergodic convergence rate bound for the relaxed PR splitting method for any $\theta \in (0, 2 + \gamma \beta ]$ by viewing it as an instance of a non-Euclidean HPE framework. It also establishes an $\mathcal{O}(1/\sqrt{k})$ pointwise convergence rate bound for it for any $\theta \in (0, 2 + \gamma \beta )$. Furthermore, an example showing that PR iterates do not necessarily converge for $\beta \ge 0$ and $\theta \ge \min \{ 2(1+\gamma \beta ), 2 + \gamma \beta + 1/(\gamma \beta ) \}$ is given.

Table 2 (resp., Table 3) for the case in which $\beta =0$ (resp., $\beta >0$) provides a summary of the convergence rate results known so far for the relaxed PR splitting method when $(A,B)=( \partial f, \partial g)$ for some convex functions f and g. However, we observe that some of these results also hold for pairs (A, B) of maximal monotone operators which are not subdifferentials. The term “R-linear” in the tables below stands for linear convergence of the sequence $\{ x_k \}$ generated by the relaxed PR splitting method.

Table 3 Summary of convergence rate results for the relaxed PR splitting method on the sum of two $\beta$-strongly convex functions f, g, with $\beta > 0$

We observe that our analysis in Sects. 4 and 5, in contrast to the ones in [11, 15, 16, 22], does not impose any regularity condition on A and B such as assuming one of them to be a Lipschitz, and hence point-to-point, operator. Also, if only one of the operators, say A, is assumed to be maximal $\beta$-strongly monotone, (1) is equivalent to $0\in (A'+B')(u)$ where $A':=A-(\beta /2)I$ and $B':=B+(\beta /2)I$ are now both $(\beta /2)$-strongly monotone. Thus, to solve (1), the relaxed PR method with (A, B) replaced by $(A',B')$ can be applied, thereby ensuring convergence of the iterates, as well as pointwise and ergodic convergence rate bounds, for values of $\theta \ge 2$. This idea was tested in our computational results of Sect. 6 where a weighted Lasso minimization problem [6] is solved using the partitions (A, B) and $(A',B')$. Our conclusion is that the partition $(A',B')$ substantially outperforms the other one for values of $\theta \ge 2$.

Appendix

Proof of Lemma 4.5

To simplify notation, let

$$\begin{aligned} \Delta x= & {} \Delta x_k, \quad \Delta x^- := \Delta x_{k-1}, \quad \Delta u = u_k-u_{k-1}, \quad \\ \Delta v= & {} v_k-v_{k-1}, \quad \Delta a = a_k-a_{k-1}, \quad \Delta b = b_k-b_{k-1}. \end{aligned}$$

Then, it follows from the second identity in (21) and relation (29) that

$$\begin{aligned} \Delta x = \Delta x^- + \theta (\Delta v - \Delta u), \quad \gamma \Delta a = \Delta x^- - \Delta u, \quad \gamma \Delta b = 2 \Delta u - \Delta v - \Delta x^-.\nonumber \\ \end{aligned}$$

(59)

Also, the two inclusions in (30) and (31) together with the $\beta$-strong monotonicity of A and B imply that

$$\begin{aligned} \langle \Delta a,\Delta u\rangle _{\mathcal{X}} \ge \beta \Vert \Delta u\Vert ^2_{\mathcal{X}}, \quad \langle \Delta b,\Delta v\rangle _{\mathcal{X}} \ge \beta \Vert \Delta v\Vert ^2_{\mathcal{X}}. \end{aligned}$$

Combining the last two identities in (59) with the above inequalities, we obtain

$$\begin{aligned} \langle \Delta x^- - \Delta u,\Delta u\rangle _{\mathcal{X}} \ge \gamma \beta \Vert \Delta u\Vert ^2_{\mathcal{X}}, \quad \langle 2 \Delta u - \Delta v - \Delta x^-,\Delta v\rangle _{\mathcal{X}} \ge \gamma \beta \Vert \Delta v\Vert ^2_{\mathcal{X}}. \end{aligned}$$

Adding these two last inequalities and simplifying the resulting expression, we obtain

$$\begin{aligned} \langle \Delta x^-,\Delta u - \Delta v\rangle _{\mathcal{X}} + 2 \langle \Delta u,\Delta v\rangle _{\mathcal{X}} \ge (1+ \gamma \beta ) [ \Vert \Delta u\Vert ^2_{\mathcal{X}} + \Vert \Delta v\Vert ^2_{\mathcal{X}} ] \end{aligned}$$

(60)

From the first equality in (59), we have

$$\begin{aligned} 2\theta \langle \Delta x^-,\Delta u - \Delta v\rangle _{\mathcal{X}} = \Vert \Delta x^- \Vert ^2_{\mathcal{X}} - \Vert \Delta x \Vert ^2_{\mathcal{X}} + \theta ^2 \Vert \Delta v - \Delta u \Vert ^2_{\mathcal{X}}, \end{aligned}$$

which upon substituting into (60), the following is true:

$$\begin{aligned} \Vert \Delta x^- \Vert ^2_{\mathcal{X}} - \Vert \Delta x \Vert ^2_{\mathcal{X}}\ge & {} 2 \theta \left( \left[ 1 + \gamma \beta - \frac{\theta }{2}\right] ( \Vert \Delta u \Vert ^2_{\mathcal{X}} + \Vert \Delta v \Vert ^2_{\mathcal{X}} )\right. \\&\left. +\, 2 \left[ \frac{\theta }{2} - 1 \right] \langle \Delta u,\Delta v\rangle _{\mathcal{X}} \right) . \end{aligned}$$

Note that the right-hand side in the above inequality is greater than or equal to zero if $\theta \in (0, 2\theta _0]$. Hence, we have if $\theta \in (0, 2\theta _0]$,

$$\begin{aligned} \Vert \Delta x \Vert _{\mathcal{X}} \le \Vert \Delta x^- \Vert _{\mathcal{X}}. \end{aligned}$$

$\square$