1 Introduction

Let \(\theta_{i}:\mathcal{R}^{n_{i}}\rightarrow(-\infty, +\infty]\ (i=1,2)\) be two lower semicontinuous proper (not necessarily smooth) functions. This work aims to solve the following two-block separable convex minimization problem:

$$ \min \bigl\{ \theta_{1}(x_{1})+ \theta_{2}(x_{2})|A_{1}x_{1}+A_{2}x_{2}=b \bigr\} , $$
(1)

where \(A_{i}\in\mathcal{R}^{l\times n_{i}}\ (i=1,2)\), \(b\in\mathcal{R}^{l}\). If there are convex set constraints \(x_{i}\in\mathcal{X}_{i}\ (i=1,2)\), where \(\mathcal{X}_{i}\subseteq\mathcal{R}^{n_{i}}\ (i=1,2)\) are some simple convex set, such as the nonnegative cones or positive semi-definite cones, etc. Then, we can define the indicator function as \(I_{\mathcal {X}_{i}}(\cdot)\) (\(I_{\mathcal{X}_{i}}(x_{i})=0\) if \(x_{i}\in\mathcal{X}_{i}\); otherwise, \(I_{\mathcal{X}_{i}}(x_{i})=+\infty\)), by which we can incorporate the constraints \(x_{i}\in\mathcal{X}_{i}\ (i=1,2)\) into the objective function of (1), and get the following equivalent form:

$$\min \bigl\{ \theta_{1}(x_{1})+I_{\mathcal{X}_{1}}(x_{1})+ \theta _{2}(x_{2})+I_{\mathcal{X}_{2}}(x_{2})|A_{1}x_{1}+A_{2}x_{2}=b \bigr\} . $$

Then, we can further introduce some auxiliary variables and functions to rewrite the above problem as problem (1) (Please refer to [1] for more details). Therefore, problem (1) is quite general, and in fact problems like (1) come from diverse applications, such as the latent variable graphical model selection [2], the sparse inverse covariance selection [3], stable principal component pursuit with nonnegative constraint [4], and robust alignment for linearly correlated images [5], etc.

As one of the first-order methods, the following Algorithm 1, that is proximal alternating direction method of multipliers (P-ADMM) [6–8] is quite efficient for solving (1) or related problems, especially for large scale case.

Algorithm 1
figure a

The P-ADMM for (1)

Full size image

The parameter γ in the P-ADMM is called the Glowinski relaxation factor in the literature, and \(\gamma>1\) can often accelerate the P-ADMM [9]. Due to its high efficiency, the P-ADMM has been intensively studied during the past few decades, and many scholars presented a lot of customized variants of the P-ADMM for some concrete separable minimization problems [10–12].

In this paper, we only focus our attention on the P-ADMM. In fact, the theory developed in this work can easily be extended to its various variants. Now, let us briefly analyze the structure advantages of the P-ADMM. Obviously, the P-ADMM fully utilizes the separable structure inherent to the original problem (1), which decouples the primal variable \((x_{1},x_{2})\) and get two subproblems with lower-dimension. Then, at each iteration, the computation of P-ADMM is dominated by solving its two subproblems. Fortunately, the two subproblems in (2) often admit closed-form solutions provided that \(\theta_{i}(\cdot)\ (i=1,2)\) are some the functions (such as \(\theta_{i}(\cdot)=\Vert \cdot \Vert _{1}\), \(\Vert \cdot \Vert _{2}\) or \(\Vert \cdot \Vert _{*}\)) and the matrices \(A_{i}\ (i=1,2)\) are unitary (i.e. \(A_{i}^{\top}A_{i}\ (i=1,2)\) are the identity matrices). Even if \(A_{i}\ (i=1,2)\) are not unitary, we can judiciously set \(G_{i}=rI_{n_{i}}-\beta A_{i}^{\top}A_{i}\) with \(r>\beta \Vert A_{i}^{\top}A_{i}\Vert \ (i=1,2)\), and then the two subproblems in the P-ADMM also have closed-form solutions in many practical applications. The global convergence of the P-ADMM with \(\gamma=1\) has been proved in [10, 11] for some concrete models of (1), and in [13], Xu and Wu presented an elegant analysis of the global convergence of the P-ADMM with \(\gamma\in(0,\frac{1+\sqrt{5}}{2})\) for the general model (1). Quite recently, He et al. [14] have further studied the P-ADMM and get some substantial advances by relaxing the matrix \(G_{2}\) in the proximal regularization term of its second subproblem to be indefinite. This is quite preferred in practical applications since the indefinite proximal matrix can result in a larger step size for the subproblem and thus maybe accelerate the overall convergence speed of the P-ADMM.

Compared with the study of the global convergence of the P-ADMM, the research of its convergence rate is quite insubstantial in the literature. In [14, 15], under the assumption that the feasible set of (1) is bounded, He et al. have proved the worst-case \(\mathcal{O}(1/t)\) convergence rate of the P-ADMM with \(\gamma=1\), where t denotes the iteration counter. In [1], Lin et al. have presented a parallel version of the P-ADMM with the adaptive penalty β, and proved that the convergence rate of their new method is also \(\mathcal{O}(1/t)\). In addition, Goldstein et al. [16] proved a better convergence rate than \(\mathcal{O}(1/t)\) for the P-ADMM scheme with \(\gamma=1\) and \(G_{1}=0, G_{2}=0\) under the assumption that \(\theta_{i}(\cdot)\ (i=1,2)\) are both strongly convex, which is usually violated in practice, and thus excludes many practical applications of the P-ADMM. Then, by introducing some free parameters \(\alpha_{k}\) and \(\gamma_{k}\), Xu [17] developed a new variant of the P-ADMM for (1), which refined the results in [16]. In fact, only under the assumption that the function \(\theta_{2}(\cdot)\) is strongly convex, Xu [17] proved that the new method has \(\mathcal{O}(1/t)\) convergence rate with constant parameters and enjoys \(\mathcal {O}(1/t^{2})\) convergence rate with adaptive parameters.

In this paper, we aim to further improve the above results by removing the assumptions of the strong convexity of \(\theta_{2}(\cdot)\) and the boundedness of the feasible set of (1), and prove that the P-ADMM for the convex minimization problem (1) has a worst-case \(\mathcal{O}(1/t)\) convergence rate in an ergodic sense, which partially improves the results in [8, 13–15, 17].

The remaining of the paper is organized as follows. Section 2 gives some useful preliminaries. In Section 3, we prove the convergence rate of the P-ADMM in detail. In Section 4, a simple experiment on compressive sensing is conducted to demonstrate the effectiveness of the P-ADMM.

2 Preliminaries

In this section, we summarize some basic concepts and preliminaries that will be used in the later discussion.

First, we list some notation to be used in this paper. \(\langle\cdot ,\cdot\rangle\) denotes the inner product of \(\mathcal{R}^{n}\); \(G\succ0\) (or \(G\succeq0\)) denotes that the symmetric matrix G is positive definite (or positive semi-definite); If G is symmetric, we set \(\Vert x\Vert _{G}^{2}=x^{\top}Gx\) though G maybe not positive definite. The effective domain of a function \(f: \mathcal{X} \rightarrow(-\infty ,+\infty]\) is defined as \(\operatorname{dom}(f):= \{x\in\mathcal{X}|f(x) < +\infty\}\). The set of all relative interior points of a given nonempty convex set \(\mathcal{C}\) is denoted by \(\operatorname{ri}(\mathcal{C})\).

A function \(f:\mathcal{R}^{n}\rightarrow\mathcal{R}\) is convex iff

$$f \bigl(\alpha x+(1-\alpha)y \bigr)\leq\alpha f(x)+(1-\alpha)f(y), \quad\forall x,y\in \mathcal{R}^{n}, \alpha\in[0,1]. $$

Then, if \(f:\mathcal{R}^{n}\rightarrow\mathcal{R}\) is convex, we have the following first-order necessary condition:

$$ f(x)\geq f(y)+\langle\xi, x-y\rangle,\quad \forall x,y\in \mathcal{R}^{n}, \xi\in\partial f(y), $$
(3)

where \(\partial f(y)=\{\xi\in\mathcal{R}^{n}:f(\bar{y})\geq f(y)+\langle \xi,\bar{y}-y\rangle,\ \mathrm{for all}\ \bar{y}\in\mathcal{R}^{n}\}\) denotes the subdifferential of \(f(\cdot)\) at the point y.

The following equality is used frequently in the paper:

$$ \langle x-y,x-z\rangle=\frac{1}{2} \bigl(\Vert x-y\Vert ^{2}+\Vert x-z\Vert ^{2}-\Vert y-z\Vert ^{2} \bigr),\quad \forall x,y,z\in\mathcal{R}^{n}. $$
(4)

From now on, we denote

$$x=(x_{1},x_{2}),\qquad \theta(x)=\theta_{1}(x_{1})+ \theta_{2}(x_{2}),\qquad A=(A_{1},A_{2}). $$

Throughout this paper, we make the following assumptions.

Assumption 2.1

The functions \(\theta_{i}(\cdot)\ (i=1,2)\) are both convex.

Assumption 2.2

There is a point \((\hat{x}_{1},\hat{x}_{2})\in\operatorname {ri}(\operatorname{dom}(\theta_{1})\times\operatorname{dom}(\theta_{2}))\) such that \(A_{1}\hat{x}_{1}+A_{2}\hat{x}_{2}=b\).

Then, under Assumption 2.2, it follows from Corollaries 28.2.2 and 28.3.1 in [18] that \(({x}_{1}^{*},{x}_{2}^{*})\in\operatorname{ri}(\operatorname {dom}{(\theta_{1})}\times\operatorname{dom}{(\theta_{2})})\) is an optimal solution to problem (1) iff there exists a Lagrangian multiplier \(\lambda^{*}\in\mathcal{R}^{l}\) such that \(({x}_{1}^{*},{x}_{2}^{*},\lambda^{*})\) is a solution of the following KKT systems:

$$ \textstyle\begin{cases} 0\in\partial\theta_{1}(x_{1}^{*})-A_{1}^{\top}\lambda^{*},\\ 0\in\partial\theta_{2}(x_{2}^{*})-A_{2}^{\top}\lambda^{*},\\ A_{1}x_{1}^{*}+A_{2}x_{2}^{*}=b. \end{cases} $$
(5)

The set of the solutions of (5) is denoted by \(\mathcal{W}^{*}\). By Assumption 2.1, (3), and (5), for any \(({x}^{*},\lambda^{*})=({x}_{1}^{*},{x}_{2}^{*},\lambda^{*})\in\mathcal{W}^{*}\), we have the following useful inequality:

$$ \theta(x)-\theta \bigl(x^{*} \bigr)- \bigl\langle \lambda^{*}, Ax-b \bigr\rangle \geq0,\quad \forall x=(x_{1}, x_{2})\in \mathcal{R}^{n_{1}+n_{2}}. $$
(6)

Assumption 2.3

The solution set \(\mathcal{W}^{*}\) of the KKT systems (5) is nonempty, and at least one \(({x}_{1}^{*},{x}_{2}^{*},\lambda^{*})\in\mathcal{W}^{*}\) with \(\lambda^{*}\neq0\).

3 Convergence rate of the P-ADMM

In this section, we aim to prove the convergence rate of the P-ADMM, and to accomplish this, we need to make some restrictions of the matrices \(A_{i},G_{i}\ (i=1,2)\) included in the P-ADMM as follows.

Assumption 3.1

(1) The matrix \(G_{1}\succeq0\), and \(A_{1}\) is full-column rank if \(G_{1}=0\).

(2) The matrix \(G_{2}\) is set as \(G_{2}=\alpha\tau I_{n_{2}}-\beta A_{2}^{\top}A_{2}\) with \(\tau>\beta \Vert A_{2}^{\top}A_{2}\Vert \), \(\alpha\in(0,1]\), and \(\alpha \geq(5-\min\{\gamma,1+\gamma-\gamma^{2}\})/5\).

Remark 3.1

In [14], the parameter α can take any value of the interval \([0.8,1)\). Obviously, the parameter α in this paper can also obtain the lower bound 0.8 if \(\gamma=1\).

Let us introduce some matrices to simplify our notation in the subsequent analysis. More specifically, we set

$$G= \begin{pmatrix}G_{1}&0\\0&G_{2} \end{pmatrix} $$

and

$$\begin{aligned} \begin{aligned} & \bar{G}_{2}=\tau I_{n_{2}}-\beta A_{2}^{\top}A_{2},\qquad M=G_{1}+\beta A_{1}^{\top}A_{1},\\ & N=G_{2}+\beta A_{2}^{\top}A_{2},\qquad H=G_{2}+\beta\min \bigl\{ \gamma,1+\gamma-\gamma^{2} \bigr\} A_{2}^{\top}A_{2}. \end{aligned} \end{aligned}$$
(7)

Remark 3.2

From Assumption 3.1 and \(\gamma\in(0,\frac{1+\sqrt {5}}{2})\), we see that the matrices \(\bar{G}_{2}, M, N, H\) defined by (7) are all positive definite. However, the matrix \(G_{2}\) defined in Assumption 3.1 may be indefinite. For example, when \(\gamma=1, \alpha=0.8\), and \(\tau=1.01\beta \Vert A_{2}^{\top}A_{2}\Vert \), then \(G_{2}=-0.198\beta A_{2}^{\top}A_{2}\), which is obviously indefinite if the matrix \(A_{2}\) is full-column rank.

Remark 3.3

From the definitions of \(G_{2}\) and \(\bar{G}_{2}\), we have

$$ G_{2}=\alpha\bar{G}_{2}-(1-\alpha)\beta A_{2}^{\top}A_{2}. $$
(8)

Now, we start proving the convergence rate of the P-ADMM under Assumptions 2.1-2.3 and Assumption 3.1. Firstly, we prove three lemmas step by step.

Lemma 3.1

Let \(\{(x^{k},\lambda^{k})\}=\{(x_{1}^{k},x_{2}^{k},\lambda^{k})\} \) be the sequence generated by the P-ADMM. Under Assumptions 2.1-2.3, for any \((x_{1},x_{2},\lambda)\in\mathcal{R}^{n_{1}+n_{2}+l}\) such that \(A_{1}x_{1}+A_{2}x_{2}=b\), we have

$$\begin{aligned} &\theta(x)-\theta\bigl(x^{k+1}\bigr) \\ &\quad\geq\frac{1}{2}\bigl(\bigl\Vert x^{k+1}-x\bigr\Vert _{G}^{2}-\bigl\Vert x^{k}-x\bigr\Vert _{G}^{2}+\bigl\Vert x^{k+1}-x^{k}\bigr\Vert _{G}^{2}\bigr)+\bigl\langle Ax^{k+1}-b,-{ \lambda}\bigr\rangle +\frac{2-\gamma }{2\beta\gamma^{2}}\bigl\Vert \lambda^{k+1}- \lambda^{k}\bigr\Vert ^{2} \\ &\qquad{}+\frac{1}{2\beta\gamma}\bigl(\bigl\Vert \lambda^{k+1}-\lambda \bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda\bigr\Vert ^{2}\bigr)+\beta \bigl\langle Ax^{k+1}-b,A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1} \bigr\rangle \\ &\qquad{}+\frac{\beta}{2}\bigl(\bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2} \bigr\Vert ^{2}-\bigl\Vert A_{2}x_{2}^{k}-A_{2}x_{2} \bigr\Vert ^{2}+\bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2}^{k} \bigr\Vert ^{2}\bigr). \end{aligned}$$
(9)

Proof

Note that the optimality condition for the first subproblem (i.e., the subproblem with respect to \(x_{1}\)) in (2) is

$$\begin{aligned} 0&=\nabla\theta_{1}\bigl(x_{1}^{k+1} \bigr)+\beta A_{1}^{\top}\biggl(A_{1}x_{1}^{k+1}+A_{2}x_{2}^{k}-b- \frac{1}{\beta}\lambda ^{k}\biggr)+G_{1} \bigl(x_{1}^{k+1}-x_{1}^{k}\bigr) \\ & =\nabla\theta_{1}\bigl(x_{1}^{k+1} \bigr)-A_{1}^{\top}\tilde{\lambda}^{k}+\beta A_{1}^{\top}A_{2}\bigl(x_{2}^{k}-x_{2}^{k+1} \bigr)+G_{1}\bigl(x_{1}^{k+1}-x_{1}^{k} \bigr), \end{aligned}$$
(10)

where \(\nabla\theta_{1}(x_{1}^{k+1})\) is a subgradient of \(\theta_{1}(\cdot)\) at \(x_{1}^{k+1}\), \(\tilde{\lambda}^{k}=\lambda^{k}-\beta (A_{1}x_{1}^{k+1}+A_{2}x_{2}^{k+1}-b)\), and the second equality uses the updating formula for λ in (2). Then (10) can be rewritten as

$$\begin{aligned} 0={}&\bigl\langle x_{1}^{k+1}-x_{1}, \nabla\theta_{1}\bigl(x_{1}^{k+1}\bigr)\bigr\rangle +\bigl\langle x_{1}^{k+1}-x_{1}, G_{1}\bigl(x_{1}^{k+1}-x_{1}^{k} \bigr)+\beta A_{1}^{\top}A_{2}\bigl(x_{2}^{k}-x_{2}^{k+1} \bigr)\bigr\rangle \\ &{} +\bigl\langle A_{1}\bigl(x_{1}^{k+1}-x_{1} \bigr),-\tilde{\lambda}^{k}\bigr\rangle \\ \geq{}&\theta_{1}\bigl(x_{1}^{k+1}\bigr)- \theta_{1}(x_{1})+\bigl\langle x_{1}^{k+1}-x_{1}, G_{1}\bigl(x_{1}^{k+1}-x_{1}^{k} \bigr)+\beta A_{1}^{\top}A_{2}\bigl(x_{2}^{k}-x_{2}^{k+1} \bigr)\bigr\rangle \\ &{} +\bigl\langle A_{1}\bigl(x_{1}^{k+1}-x_{1} \bigr),-\tilde{\lambda}^{k}\bigr\rangle , \end{aligned}$$
(11)

where the inequality comes from the convexity of \(\theta_{1}(\cdot)\) and (3). Similarly, the optimality condition for the second subproblem (i.e., the subproblem with respect to \(x_{2}\)) in (2) gives

$$\begin{aligned} 0&=\nabla\theta_{2} \bigl(x_{2}^{k+1} \bigr)+\beta A_{2}^{\top}\biggl(A_{1}x_{1}^{k+1}+A_{2}x_{2}^{k+1}-b- \frac{1}{\beta}\lambda ^{k} \biggr)+G_{2} \bigl(x_{2}^{k+1}-x_{2}^{k} \bigr) \\ & =\nabla\theta_{1} \bigl(x_{1}^{k+1} \bigr)-A_{2}^{\top}\tilde{\lambda }^{k}+G_{2} \bigl(x_{2}^{k+1}-x_{2}^{k} \bigr){,} \end{aligned}$$

i.e.,

$$\begin{aligned} 0={}&\bigl\langle x_{2}^{k+1}-x_{2}, \nabla\theta_{2}\bigl(x_{2}^{k+1}\bigr)\bigr\rangle + \bigl\langle A_{2}\bigl(x_{2}^{k+1}-x_{2} \bigr),-\tilde{\lambda}^{k}\bigr\rangle +\bigl\langle x_{2}^{k+1}-x_{2}, G_{2} \bigl(x_{2}^{k+1}-x_{2}^{k}\bigr)\bigr\rangle \\ \geq{}&\theta_{2}\bigl(x_{2}^{k+1}\bigr)- \theta_{2}(x_{2})+\bigl\langle x_{2}^{k+1}-x_{2}, G_{2}\bigl(x_{2}^{k+1}-x_{2}^{k} \bigr)\bigr\rangle +\bigl\langle A_{2}\bigl(x_{2}^{k+1}-x_{2} \bigr),-\tilde{\lambda }^{k}\bigr\rangle , \end{aligned}$$
(12)

where the inequality follows from the convexity of \(\theta_{2}(\cdot)\) and (3). Then, adding (11) and (12), we obtain

$$\begin{aligned} &\theta(x)-\theta\bigl(x^{k+1}\bigr) \\ &\quad\geq\sum_{i=1}^{2}\bigl\langle x_{i}^{k+1}-x_{i}, G_{i} \bigl(x_{i}^{k+1}-x_{i}^{k}\bigr)\bigr\rangle +\sum_{i=1}^{2}\bigl\langle A_{i}\bigl(x_{i}^{k+1}-x_{i}\bigr),- \tilde{\lambda}^{k}\bigr\rangle \\ &\qquad{}+\beta\bigl\langle A_{1}x_{1}^{k+1}-A_{1}x_{1},A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1} \bigr\rangle \\ &\quad=\sum_{i=1}^{2}\bigl\langle x_{i}^{k+1}-x_{i}, G_{i} \bigl(x_{i}^{k+1}-x_{i}^{k}\bigr)\bigr\rangle +\bigl\langle Ax^{k+1}-b,-\tilde{\lambda }^{k}\bigr\rangle +\beta\bigl\langle A_{1}x_{1}^{k+1}-A_{1}x_{1},A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1} \bigr\rangle \\ &\quad=\sum_{i=1}^{2}\bigl\langle x_{i}^{k+1}-x_{i}, G_{i} \bigl(x_{i}^{k+1}-x_{i}^{k}\bigr)\bigr\rangle +\bigl\langle Ax^{k+1}-b,-\tilde{\lambda }^{k}\bigr\rangle \\ &\qquad{}+\beta\bigl\langle A_{1}x_{1}^{k+1}+A_{2}x_{2}^{k+1}-A_{1}x_{1}-A_{2}x_{2},A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1} \bigr\rangle \\ &\qquad{}+\beta\bigl\langle A_{2}x_{2}-A_{2}x_{2}^{k+1},A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1} \bigr\rangle \\ &\quad=\sum_{i=1}^{2}\bigl\langle x_{i}^{k+1}-x_{i}, G_{i} \bigl(x_{i}^{k+1}-x_{i}^{k}\bigr)\bigr\rangle +\bigl\langle Ax^{k+1}-b,-\tilde{\lambda }^{k}\bigr\rangle \\ &\qquad{}+\beta\bigl\langle Ax^{k+1}-b,A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1} \bigr\rangle +\beta\bigl\langle A_{2}x_{2}-A_{2}x_{2}^{k+1},A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1} \bigr\rangle \\ &\quad=\frac{1}{2}\sum_{i=1}^{2} \bigl(\bigl\Vert x_{i}^{k+1}-x_{i}\bigr\Vert _{G_{i}}^{2}-\bigl\Vert x_{i}^{k}-x_{i} \bigr\Vert _{G_{i}}^{2}+\bigl\Vert x_{i}^{k+1}-x_{i}^{k} \bigr\Vert _{G_{i}}^{2}\bigr)+\bigl\langle Ax^{k+1}-b,- \tilde{\lambda}^{k}\bigr\rangle \\ &\qquad{}+\beta\bigl\langle Ax^{k+1}-b,A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1} \bigr\rangle \\ &\qquad{}+\frac{\beta}{2}\bigl(\bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2} \bigr\Vert ^{2}-\bigl\Vert A_{2}x_{2}^{k}-A_{2}x_{2} \bigr\Vert ^{2}+\bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2}^{k} \bigr\Vert ^{2}\bigr), \end{aligned}$$
(13)

where the last equality comes from the identity (4). Now, let us deal with the term \(\langle Ax^{k+1}-b,-\tilde{\lambda }^{k}\rangle\) on the right side of (13). Specifically, from the updating formula for λ in (2) again, we can get

$$\begin{aligned} &\bigl\langle Ax^{k+1}-b,-\tilde{ \lambda}^{k}\bigr\rangle \\ &\quad=\bigl\langle Ax^{k+1}-b,-\lambda\bigr\rangle +\frac{1}{\beta} \bigl\langle \lambda^{k}-\tilde{\lambda}^{k},\lambda-\tilde{ \lambda}^{k}\bigr\rangle \\ &\quad=\bigl\langle Ax^{k+1}-b,-\lambda\bigr\rangle +\frac{1}{\beta} \biggl\langle \lambda^{k}-\frac{1}{\gamma}{\lambda}^{k+1}+ \frac{1-\gamma}{\gamma}\lambda ^{k},\lambda-\frac{1}{\gamma}{ \lambda}^{k+1}+\frac{1-\gamma}{\gamma }\lambda^{k}\biggr\rangle \\ &\quad= \bigl\langle Ax^{k+1}-b,-\lambda\bigr\rangle + \frac{1}{\beta}\biggl\langle \frac{1}{\gamma}\lambda^{k}- \frac{1}{\gamma}{\lambda}^{k+1},\lambda -\lambda^{k}+ \frac{1}{\gamma}\lambda^{k}-\frac{1}{\gamma}{\lambda }^{k+1}\biggr\rangle \\ &\quad =\bigl\langle Ax^{k+1}-b,-\lambda\bigr\rangle +\frac{1}{\beta\gamma } \bigl\langle \lambda^{k}-{\lambda}^{k+1},\lambda- \lambda^{k}\bigr\rangle +\frac {1}{\beta\gamma^{2}}\bigl\Vert \lambda^{k+1}-\lambda^{k}\bigr\Vert ^{2} \\ &\quad=\bigl\langle Ax^{k+1}-b,-\lambda\bigr\rangle +\frac{1}{2\beta\gamma } \bigl(\bigl\Vert \lambda^{k+1}-\lambda\bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda\bigr\Vert ^{2}\bigr)+ \frac{2-\gamma }{2\beta\gamma^{2}}\bigl\Vert \lambda^{k+1}-\lambda^{k}\bigr\Vert ^{2}, \end{aligned}$$
(14)

where the second equality comes from \(\lambda^{k+1}=\lambda^{k}-\gamma (\lambda^{k}-\tilde{\lambda}^{k})\), and the last equality uses the identity (4). Then, substituting (14) into (13) yields (9). This completes the proof. □

The following lemma aims to further refine the crossing term \(\beta \langle Ax^{k+1}-b,A_{2}x_{2}^{k}-A_{2}x_{2}^{k+1}\rangle\) on the right side of (9).

Lemma 3.2

Let \(\{(x^{k},\lambda^{k})\}=\{(x_{1}^{k},x_{2}^{k},\lambda^{k})\} \) be the sequence generated by the P-ADMM. Under Assumptions 2.1-2.3, for any \((x_{1},x_{2},\lambda)\in\mathcal{R}^{n_{1}+n_{2}+l}\) such that \(A_{1}x_{1}+A_{2}x_{2}=b\), we have

$$\begin{aligned} &\theta(x)-\theta\bigl(x^{k+1}\bigr) \\ &\quad\geq\frac{1}{2}\bigl(\bigl\Vert x^{k+1}-x\bigr\Vert _{G}^{2}-\bigl\Vert x^{k}-x\bigr\Vert _{G}^{2}+\bigl\Vert x^{k+1}-x^{k}\bigr\Vert _{G}^{2}\bigr)+\bigl\langle Ax^{k+1}-b,-{ \lambda}\bigr\rangle \\ &\qquad{}+\frac{1}{2\beta\gamma}\bigl(\bigl\Vert \lambda^{k+1}-\lambda \bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda\bigr\Vert ^{2}\bigr)+\frac{2-\gamma}{2\beta\gamma^{2}}\bigl\Vert \lambda ^{k+1}-\lambda^{k}\bigr\Vert ^{2} \\ &\qquad{}+(1-\gamma)\beta\bigl\langle A_{2}\bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),A_{1}x_{1}^{k}+A_{2}x_{2}^{k}-b \bigr\rangle +\bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{G_{2}}^{2} \\ &\qquad{}+\bigl\langle x_{2}^{k}-x_{2}^{k+1}, G_{2}\bigl(x_{2}^{k}-x_{2}^{k-1} \bigr)\bigr\rangle \\ &\qquad{}+\frac{\beta}{2}\bigl(\bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2} \bigr\Vert ^{2}-\bigl\Vert A_{2}x_{2}^{k}-A_{2}x_{2} \bigr\Vert ^{2}+\bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2}^{k} \bigr\Vert ^{2}\bigr). \end{aligned}$$
(15)

Proof

Setting \(x_{2}=x_{2}^{k}\) in (12), we get

$$0\geq\theta_{2} \bigl(x_{2}^{k+1} \bigr)- \theta_{2} \bigl(x_{2}^{k} \bigr)+ \bigl\langle x_{2}^{k+1}-x_{2}^{k}, G_{2} \bigl(x_{2}^{k+1}-x_{2}^{k} \bigr) \bigr\rangle + \bigl\langle A_{2} \bigl(x_{2}^{k+1}-x_{2}^{k} \bigr),-\tilde {\lambda}^{k} \bigr\rangle . $$

That is,

$$ \theta_{2} \bigl(x_{2}^{k} \bigr)- \theta_{2} \bigl(x_{2}^{k+1} \bigr)- \bigl\langle A_{2} \bigl(x_{2}^{k+1}-x_{2}^{k} \bigr),-\tilde{\lambda}^{k} \bigr\rangle \geq \bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{G_{2}}^{2}. $$
(16)

Similarly, taking \(x_{2}=x_{2}^{k+1}\) in (12) for \(k:=k-1\), and thus we have

$$0\geq\theta_{2} \bigl(x_{2}^{k} \bigr)- \theta_{2} \bigl(x_{2}^{k+1} \bigr)+ \bigl\langle x_{2}^{k}-x_{2}^{k+1}, G_{2} \bigl(x_{2}^{k}-x_{2}^{k-1} \bigr) \bigr\rangle + \bigl\langle A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),-\tilde {\lambda}^{k-1} \bigr\rangle . $$

That is,

$$ \theta_{2} \bigl(x_{2}^{k+1} \bigr)- \theta _{2} \bigl(x_{2}^{k} \bigr)- \bigl\langle A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),-\tilde{\lambda}^{k-1} \bigr\rangle \geq \bigl\langle x_{2}^{k}-x_{2}^{k+1}, G_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr) \bigr\rangle . $$
(17)

Adding (16) and (17), we obtain

$$ \bigl\langle A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),\tilde {\lambda}^{k-1}-\tilde{\lambda}^{k} \bigr\rangle \geq \bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{G_{2}}^{2}+ \bigl\langle x_{2}^{k}-x_{2}^{k+1}, G_{2} \bigl(x_{2}^{k}-x_{2}^{k-1} \bigr) \bigr\rangle . $$
(18)

We have

$$\begin{aligned} &\tilde{\lambda}^{k-1}-\tilde{\lambda}^{k} \\ &\quad=\tilde{\lambda}^{k-1}- \bigl[\lambda^{k}-\beta \bigl(A_{1}x_{1}^{k+1}+A_{2}x_{2}^{k+1}-b \bigr) \bigr] \\ &\quad=\tilde{\lambda}^{k-1}- \bigl[\lambda^{k-1}-\gamma\beta \bigl(A_{1}x_{1}^{k}+A_{2}x_{2}^{k}-b \bigr)-\beta \bigl(A_{1}x_{1}^{k+1}+A_{2}x_{2}^{k+1}-b \bigr) \bigr] \\ &\quad =-(1-\gamma)\beta \bigl(A_{1}x_{1}^{k}+A_{2}x_{2}^{k}-b \bigr)+\beta \bigl(A_{1}x_{1}^{k+1}+A_{2}x_{2}^{k+1}-b \bigr). \end{aligned}$$

Substituting the above equality into (18), we obtain

$$\begin{aligned} &\beta\bigl\langle A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),A_{1}x_{1}^{k+1}+A_{2}x_{2}^{k+1}-b \bigr\rangle \\ &\quad\geq(1-\gamma)\beta\bigl\langle A_{2}\bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),A_{1}x_{1}^{k}+A_{2}x_{2}^{k}-b \bigr\rangle +\bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{G_{2}}^{2} \\ &\qquad{}+\bigl\langle x_{2}^{k}-x_{2}^{k+1}, G_{2}\bigl(x_{2}^{k}-x_{2}^{k-1} \bigr)\bigr\rangle . \end{aligned}$$
(19)

Then, substituting (19) into (9) yields (15). The proof is completed.

Now, let us deal with the term \(\frac{2-\gamma}{2\beta\gamma^{2}}\Vert \lambda ^{k+1}-\lambda^{k}\Vert ^{2} +(1-\gamma)\beta\langle A_{2}(x_{2}^{k}-x_{2}^{k+1}),A_{1}x_{1}^{k}+A_{2}x_{2}^{k}-b\rangle+\frac{\beta}{2}\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2}^{k}\Vert ^{2}\) on the right side of (15). □

Lemma 3.3

Let \(\{(x^{k},\lambda^{k})\}=\{(x_{1}^{k},x_{2}^{k},\lambda^{k})\} \) be the sequence generated by the P-ADMM. Then, we have

$$\begin{aligned} &\frac{2-\gamma}{2\beta\gamma^{2}}\bigl\Vert \lambda^{k+1}- \lambda ^{k}\bigr\Vert ^{2}+(1-\gamma)\beta\bigl\langle A_{2}\bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),A_{1}x_{1}^{k}+A_{2}x_{2}^{k}-b \bigr\rangle +\frac{\beta}{2}\bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2}^{k} \bigr\Vert ^{2} \\ &\quad\geq\frac{1}{2\beta\gamma^{2}}\max\bigl\{ 1-\gamma,1-\gamma^{-1}\bigr\} \bigl(\bigl\Vert \lambda^{k+1}-\lambda^{k}\bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda^{k-1}\bigr\Vert ^{2}\bigr) \\ &\qquad{}+\frac{\beta\varrho}{2}\biggl(\frac{1}{\gamma^{3}\beta^{2}}\bigl\Vert \lambda^{k+1}-\lambda^{k}\bigr\Vert ^{2}+\bigl\Vert A_{2}\bigl(x_{2}^{k}-x_{2}^{k+1} \bigr)\bigr\Vert ^{2}\biggr), \end{aligned}$$
(20)

where \(\varrho=\min\{\gamma,1+\gamma-\gamma^{2}\}\).

Proof

Obviously, by the updating formula for λ in (2), we have

$$ \bigl\langle A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),A_{1}x_{1}^{k}+A_{2}x_{2}^{k}-b \bigr\rangle =\frac{1}{\beta \gamma} \bigl\langle A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr), \lambda^{k-1}-\lambda^{k} \bigr\rangle . $$
(21)

Then, applying the Cauchy-Schwartz inequality, we can get

$$\textstyle\begin{cases} (1-\gamma)\beta\langle A_{2}(x_{2}^{k}-x_{2}^{k+1}),(\lambda^{k-1}-\lambda ^{k})/(\beta\gamma)\rangle\\ \quad\geq-\frac{(1-\gamma)\beta}{2}(\Vert A_{2}(x_{2}^{k}-x_{2}^{k+1})\Vert ^{2}+\frac{1}{\gamma^{2}\beta^{2}}\Vert \lambda ^{k-1}-\lambda^{k}\Vert ^{2})\\ \qquad \mathrm{if\ } \gamma\in(0,1],\\ (1-\gamma)\beta\langle A_{2}(x_{2}^{k}-x_{2}^{k+1}),(\lambda^{k-1}-\lambda ^{k})/(\beta\gamma)\rangle\\ \quad\geq-\frac{(\gamma-1)\beta}{2}(\gamma \Vert A_{2}(x_{2}^{k}-x_{2}^{k+1})\Vert ^{2}+\frac{1}{\gamma^{3}\beta^{2}}\Vert \lambda ^{k-1}-\lambda^{k}\Vert ^{2})\\ \qquad\mathrm{if\ } \gamma\in(1,+\infty). \end{cases} $$

Then, substituting the above two inequalities into (21), and by some simple manipulations, we obtain

$$\begin{aligned} &\frac{2-\gamma}{2\beta\gamma^{2}} \bigl\Vert \lambda^{k+1}-\lambda^{k} \bigr\Vert ^{2}+(1-\gamma )\beta \bigl\langle A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr),A_{1}x_{1}^{k}+A_{2}x_{2}^{k}-b \bigr\rangle +\frac{\beta}{2} \bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2}^{k} \bigr\Vert ^{2} \\ &\quad\geq\frac{1}{2\beta\gamma^{2}}\max \bigl\{ 1-\gamma,1-\gamma^{-1} \bigr\} \bigl( \bigl\Vert \lambda ^{k+1}-\lambda^{k} \bigr\Vert ^{2}- \bigl\Vert \lambda^{k}-\lambda^{k-1} \bigr\Vert ^{2} \bigr) \\ &\qquad{}+\frac{\beta}{2}\min \bigl\{ \gamma,1+\gamma-\gamma^{2} \bigr\} \biggl(\frac{1}{\gamma ^{3}\beta^{2}} \bigl\Vert \lambda^{k+1}- \lambda^{k} \bigr\Vert ^{2}+ \bigl\Vert A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr) \bigr\Vert ^{2} \biggr), \end{aligned}$$

which is the same as the assertion (20), and the lemma is thus proved.

Substituting (20) into (15), we get the following important inequality:

$$\begin{aligned} &\theta(x)-\theta\bigl(x^{k+1}\bigr) \\ &\quad\geq\frac{1}{2}\bigl(\bigl\Vert x^{k+1}-x\bigr\Vert _{G}^{2}-\bigl\Vert x^{k}-x\bigr\Vert _{G}^{2}+\bigl\Vert x^{k+1}-x^{k}\bigr\Vert _{G}^{2}\bigr)+\bigl\langle Ax^{k+1}-b,-{ \lambda}\bigr\rangle \\ &\qquad{}+\frac{1}{2\beta\gamma}\bigl(\bigl\Vert \lambda^{k+1}-\lambda \bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda\bigr\Vert ^{2}\bigr) \\ &\qquad{}+\frac{1}{2\beta\gamma^{2}}\max\bigl\{ 1-\gamma,1-\gamma ^{-1}\bigr\} \bigl(\bigl\Vert \lambda^{k+1}- \lambda^{k}\bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda^{k-1}\bigr\Vert ^{2}\bigr) \\ &\qquad{}+\bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{G_{2}}^{2}+\bigl\langle x_{2}^{k}-x_{2}^{k+1}, G_{2}\bigl(x_{2}^{k}-x_{2}^{k-1} \bigr)\bigr\rangle \\ &\qquad{}+\frac{\beta}{2}\bigl(\bigl\Vert A_{2}x_{2}^{k+1}-A_{2}x_{2} \bigr\Vert ^{2}-\bigl\Vert A_{2}x_{2}^{k}-A_{2}x_{2} \bigr\Vert ^{2}\bigr) \\ &\qquad{}+\frac{\beta}{2}\min\bigl\{ \gamma,1+\gamma-\gamma^{2}\bigr\} \biggl(\frac {1}{\gamma^{3}\beta^{2}}\bigl\Vert \lambda^{k+1}- \lambda^{k}\bigr\Vert ^{2}+\bigl\Vert A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1}\bigr)\bigr\Vert ^{2}\biggr). \end{aligned}$$
(22)

Now, let us deal with all the terms related with the variable \(x_{2}\) on the right side of (22). From the definition of the matrices \(G_{2}, N\) and (8), we have

$$\begin{aligned} &\frac{1}{2} \bigl( \bigl\Vert x_{2}^{k+1}-x_{2} \bigr\Vert _{G_{2}+\beta A_{2}^{\top}A_{2}}^{2}- \bigl\Vert x_{2}^{k}-x_{2} \bigr\Vert _{G_{2}+\beta A_{2}^{\top}A_{2}}^{2} \bigr)\\ &\qquad{}+\frac{1}{2} \bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{G_{2}+\beta\varrho A_{2}^{\top}A_{2}}^{2} \\ &\qquad{}+ \bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{G_{2}}^{2}+ \bigl\langle x_{2}^{k}-x_{2}^{k+1}, G_{2} \bigl(x_{2}^{k}-x_{2}^{k-1} \bigr) \bigr\rangle \\ &\quad=\frac{1}{2} \bigl( \bigl\Vert x_{2}^{k+1}-x_{2} \bigr\Vert _{N}^{2}- \bigl\Vert x_{2}^{k}-x_{2} \bigr\Vert _{N}^{2} \bigr)+\frac {1}{2} \bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{3\alpha\bar{G}_{2}-(3-3\alpha-\varrho)\beta A_{2}^{\top}A_{2}}^{2} \\ &\qquad{}+\alpha \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr)^{\top}\bar{G}_{2} \bigl(x_{2}^{k}-x_{2}^{k-1} \bigr)-(1-\alpha )\beta \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr)^{\top}\bigl(A_{2}^{\top}A_{2} \bigr) \bigl(x_{2}^{k}-x_{2}^{k-1} \bigr) \\ &\quad\geq\frac{1}{2} \bigl( \bigl\Vert x_{2}^{k+1}-x_{2} \bigr\Vert _{N}^{2}- \bigl\Vert x_{2}^{k}-x_{2} \bigr\Vert _{N}^{2} \bigr)+\frac {1}{2} \bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{3\alpha\bar{G}_{2}-(3-3\alpha-\varrho)\beta A_{2}^{\top}A_{2}}^{2} \\ &\qquad{}-\frac{\alpha}{2} \bigl( \bigl\Vert x_{2}^{k}-x_{2}^{k+1} \bigr\Vert _{\bar{G}_{2}}^{2}+ \bigl\Vert x_{2}^{k}-x_{2}^{k-1} \bigr\Vert _{\bar{G}_{2}}^{2} \bigr)\\ &\qquad{}-\frac{(1-\alpha)\beta}{2} \bigl( \bigl\Vert A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr) \bigr\Vert ^{2}+ \bigl\Vert A_{2} \bigl(x_{2}^{k}-x_{2}^{k-1} \bigr) \bigr\Vert ^{2} \bigr) \\ &\quad=\frac{1}{2} \bigl( \bigl\Vert x_{2}^{k+1}-x_{2} \bigr\Vert _{N}^{2}- \bigl\Vert x_{2}^{k}-x_{2} \bigr\Vert _{N}^{2} \bigr)+\frac {\alpha}{2} \bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{\bar{G}_{2}}^{2}\\ &\qquad{}+\frac{\alpha}{2} \bigl( \bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{\bar{G}_{2}}^{2}- \bigl\Vert x_{2}^{k}-x_{2}^{k-1} \bigr\Vert _{\bar{G}_{2}}^{2} \bigr) \\ &\qquad{}+\frac{(1-\alpha)\beta}{2} \bigl( \bigl\Vert A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr) \bigr\Vert ^{2}- \bigl\Vert A_{2} \bigl(x_{2}^{k}-x_{2}^{k-1} \bigr) \bigr\Vert ^{2} \bigr)\\ &\qquad{}+\frac{\beta(\varrho-5+5\alpha)}{2} \bigl\Vert A_{2} \bigl(x_{2}^{k}-x_{2}^{k+1} \bigr) \bigr\Vert ^{2}. \end{aligned}$$

Then, substituting the above inequality into (22), we can obtain

$$\begin{aligned} &\theta(x)-\theta\bigl(x^{k+1}\bigr) \\ &\quad\geq\bigl\langle Ax^{k+1}-b,-{\lambda}\bigr\rangle + \frac{1}{2}\bigl(\bigl\Vert x_{1}^{k+1}-x_{1} \bigr\Vert _{G_{1}}^{2}-\bigl\Vert x_{1}^{k}-x_{1} \bigr\Vert _{G_{1}}^{2}+\bigl\Vert x_{1}^{k+1}-x_{1}^{k} \bigr\Vert _{G_{1}}^{2}\bigr) \\ &\qquad{}+\frac{1}{2}\bigl(\bigl\Vert x_{2}^{k+1}-x_{2} \bigr\Vert _{N}^{2}-\bigl\Vert x_{2}^{k}-x_{2} \bigr\Vert _{N}^{2}\bigr) \\ &\qquad{}+\frac{1}{2}\bigl(\bigl\Vert x_{2}^{k}-x_{2}^{k+1}\bigr\Vert _{\alpha\bar{G}_{2}+\beta(1-\alpha )A_{2}^{\top}A_{2}}^{2}-\bigl\Vert x_{2}^{k-1}-x_{2}^{k} \bigr\Vert _{\alpha\bar{G}_{2}+\beta(1-\alpha )A_{2}^{\top}A_{2}}^{2}\bigr) \\ &\qquad{}+\frac{1}{2\beta\gamma}\bigl(\bigl\Vert \lambda^{k+1}-\lambda \bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda\bigr\Vert ^{2}\bigr) \\ &\qquad{}+\frac{1}{2\beta\gamma^{2}}\max\bigl\{ 1-\gamma,1-\gamma ^{-1}\bigr\} \bigl(\bigl\Vert \lambda^{k+1}- \lambda^{k}\bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda^{k-1}\bigr\Vert ^{2}\bigr) \\ &\qquad{}+\frac{\alpha}{2}\bigl\Vert x_{2}^{k+1}-x_{2}^{k} \bigr\Vert _{\bar{G}_{2}}^{2}+\frac {\beta(\varrho-5+5\alpha)}{2}\bigl\Vert A_{2}\bigl(x_{2}^{k}-x_{2}^{k+1} \bigr)\bigr\Vert ^{2}+\frac{\varrho }{2\gamma^{3}\beta}\bigl\Vert \lambda^{k+1}-\lambda^{k}\bigr\Vert ^{2} \\ &\quad\geq\bigl\langle Ax^{k+1}-b,-{\lambda}\bigr\rangle + \frac{1}{2}\bigl(\bigl\Vert x_{1}^{k+1}-x_{1} \bigr\Vert _{G_{1}}^{2}-\bigl\Vert x_{1}^{k}-x_{1} \bigr\Vert _{G_{1}}^{2}\bigr)+\frac{1}{2}\bigl(\bigl\Vert x_{2}^{k+1}-x_{2}\bigr\Vert _{N}^{2}-\bigl\Vert x_{2}^{k}-x_{2} \bigr\Vert _{N}^{2}\bigr) \\ &\qquad{}+\frac{1}{2}\bigl(\bigl\Vert x_{2}^{k}-x_{2}^{k+1} \bigr\Vert _{\alpha\bar{G}_{2}+\beta (1-\alpha)A_{2}^{\top}A_{2}}^{2}-\bigl\Vert x_{2}^{k-1}-x_{2}^{k} \bigr\Vert _{\alpha\bar{G}_{2}+\beta (1-\alpha)A_{2}^{\top}A_{2}}^{2}\bigr) \\ &\qquad{}+\frac{1}{2\beta\gamma}\bigl(\bigl\Vert \lambda^{k+1}-\lambda \bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda\bigr\Vert ^{2}\bigr) \\ &\qquad{}+\frac{1}{2\beta\gamma^{2}}\max\bigl\{ 1-\gamma,1-\gamma ^{-1}\bigr\} \bigl(\bigl\Vert \lambda^{k+1}- \lambda^{k}\bigr\Vert ^{2}-\bigl\Vert \lambda^{k}-\lambda^{k-1}\bigr\Vert ^{2}\bigr), \end{aligned}$$
(23)

where the inequality comes from \(\alpha\in(0,1]\) and \(\alpha\geq\frac {5-\varrho}{5}\). Based on (23), we can prove the worst-case \(\mathcal {O}(1/t)\) convergence rate in an ergodic sense of the P-ADMM. □

Theorem 3.1

Suppose that Assumptions 2.1-2.3 and Assumption 3.1 hold. Let \(\{(x^{k},\lambda^{k})\}=\{(x_{1}^{k},x_{2}^{k},\lambda^{k})\}\) be the sequence generated by the P-ADMM and let \(\bar{x}^{t}=\frac{1}{t}\sum_{k=1}^{t}x^{k+1}\), where t is a positive integer. Then,

$$ \textstyle\begin{cases} \vert \theta(\bar{x}^{t})-\theta(x^{*})\vert \leq\frac{D}{t},\\ \Vert A\bar{x}^{t}-b\Vert \leq\frac{D}{t\Vert \lambda^{*}\Vert }, \end{cases} $$
(24)

where \((x^{*},\lambda^{*})=(x_{1}^{*},x_{2}^{*},\lambda^{*})\) with \(\lambda^{*}\neq0\) is a point satisfying the KKT conditions in (5), and D is a constant defined by

$$\begin{aligned} D={}& \frac{1}{2}\bigl\Vert x^{1}-x^{*}\bigr\Vert _{G_{1}}^{2}+\frac{1}{2} \bigl\Vert x_{2}^{1}-x_{2}^{*}\bigr\Vert _{N}^{2}+\frac{1}{2}\bigl\Vert x_{2}^{0}-x_{2}^{1}\bigr\Vert _{\alpha\bar {G}_{2}+\beta(1-\alpha)A_{2}^{\top}A_{2}}^{2} \\ &{} +\frac{1}{\beta\gamma}\bigl(\bigl\Vert \lambda^{1}\bigr\Vert ^{2}+\Vert \lambda \Vert ^{2}\bigr) +\frac{1}{2\beta\gamma^{2}} \max\bigl\{ 1-\gamma,1-\gamma^{-1}\bigr\} \bigl\Vert \lambda ^{1}-\lambda^{0}\bigr\Vert ^{2}. \end{aligned}$$
(25)

Proof

Setting \(x=x^{*}\) in the inequality (22) and summing it over \(k= 1, 2,\ldots,t\), we obtain

$$\begin{aligned} &\sum_{k=1}^{t} \bigl[ \theta \bigl(x^{k+1} \bigr)-\theta \bigl(x^{*} \bigr)- \bigl\langle Ax^{k+1}-b,\lambda \bigr\rangle \ \bigr] \\ &\quad\leq\frac{1}{2} \bigl\Vert x^{1}-x^{*} \bigr\Vert _{G_{1}}^{2}+\frac{1}{2} \bigl\Vert x_{2}^{1}-x_{2}^{*} \bigr\Vert _{N}^{2}+\frac{1}{2} \bigl\Vert x_{2}^{0}-x_{2}^{1} \bigr\Vert _{\alpha\bar {G}_{2}+\beta(1-\alpha)A_{2}^{\top}A_{2}}^{2}\\ &\qquad{}+\frac{1}{2\beta\gamma} \bigl\Vert \lambda ^{1}-\lambda \bigr\Vert ^{2} \\ &\qquad{}+\frac{1}{2\beta\gamma^{2}}\max \bigl\{ 1-\gamma,1- \gamma^{-1} \bigr\} \bigl\Vert \lambda^{1}- \lambda^{0} \bigr\Vert ^{2} \\ &\quad\leq\frac{1}{2} \bigl\Vert x^{1}-x^{*} \bigr\Vert _{G_{1}}^{2}+\frac{1}{2} \bigl\Vert x_{2}^{1}-x_{2}^{*} \bigr\Vert _{N}^{2}+\frac{1}{2} \bigl\Vert x_{2}^{0}-x_{2}^{1} \bigr\Vert _{\alpha\bar {G}_{2}+\beta(1-\alpha)A_{2}^{\top}A_{2}}^{2}\\ &\qquad{}+\frac{1}{\beta\gamma} \bigl( \bigl\Vert \lambda ^{1} \bigr\Vert ^{2}+\Vert \lambda \Vert ^{2} \bigr) \\ &\qquad{}+\frac{1}{2\beta\gamma^{2}}\max \bigl\{ 1-\gamma,1- \gamma^{-1} \bigr\} \bigl\Vert \lambda^{1}- \lambda^{0} \bigr\Vert ^{2} , \end{aligned}$$

which together with the convexity of the function \(\theta(\cdot)\) implies

$$\begin{aligned} &\theta \bigl(\bar{x}^{t} \bigr)-\theta \bigl(x^{*} \bigr)- \bigl\langle A \bar{x}^{t}-b,\lambda \bigr\rangle \\ &\quad\leq\frac{1}{2t} \bigl\Vert x^{1}-x^{*} \bigr\Vert _{G_{1}}^{2}+\frac{1}{2t} \bigl\Vert x_{2}^{1}-x_{2}^{*} \bigr\Vert _{N}^{2}+\frac{1}{2t} \bigl\Vert x_{2}^{0}-x_{2}^{1} \bigr\Vert _{\alpha\bar {G}_{2}+\beta(1-\alpha)A_{2}^{\top}A_{2}}^{2}\\ &\qquad{}+\frac{1}{\beta\gamma t} \bigl( \bigl\Vert \lambda ^{1} \bigr\Vert ^{2}+\Vert \lambda \Vert ^{2} \bigr) \\ &\qquad{}+\frac{1}{2\beta\gamma^{2} t}\max \bigl\{ 1-\gamma,1- \gamma^{-1} \bigr\} \bigl\Vert \lambda^{1}- \lambda^{0} \bigr\Vert ^{2} . \end{aligned}$$

Using the Lemmas 2.2 and 2.3 of [17] with \(\rho=2\Vert \lambda^{*}\Vert \) (ρ is a parameter defined in Lemmas 2.2 and 2.3 of [17]), we can get (24). This completes the proof. □

Remark 3.4

From (24) and (25), we can conclude that larger values of γ is more beneficial for accelerating the convergence of the P-ADMM, as the larger γ, the smaller D, which controls the upper bounds of \(\vert \theta(\bar{x}^{t})-\theta(x^{*})\vert \) and \(\Vert A\bar{x}^{t}-b\Vert \).

4 Numerical experiments

In this section, we apply the P-ADMM to solve the compressive sensing, a concrete problem of the general model (1). The codes were written by Matlab R2010a and conducted on a ThinkPad notebook with Pentium(R) Dual-Core CPU T4400@2.2 GHz, 2 GB of RAM using Windows 7.

Let us briefly review the compressive sensing. Compressive sensing (CS) is to recover a sparse signal \(\bar{x}\in\mathcal{R}^{n}\) from an undetermined linear system \(y=A\bar{x}\), where \(A\in\mathcal {R}^{m\times n}(m\ll n)\) is a linear mapping and \(y\in\mathcal{R}^{m}\) is an observation. An important decoding model of CS is

$$ \min_{x\in\mathcal{R}^{n}}\frac{1}{2}\Vert Ax-y\Vert ^{2}+\nu \Vert x\Vert _{1}, $$
(26)

where the parameter \(\nu>0\) is used to trade off both terms for minimization. This is a special case of the general two-block separable convex minimization model (1). In fact, setting \(x_{1}=x, x_{2}=x\), (26) can be recast as

$$\begin{aligned} &\min\frac{1}{2}\Vert Ax_{1}-y\Vert ^{2}+\nu \Vert x_{2}\Vert _{1} \\ &\mathrm{s.t.}\ x_{1}-x_{2}=0, \\ &\quad x_{1}\in\mathcal{R}^{n}, x_{2}\in \mathcal{R}^{n}, \end{aligned}$$

which is a special case of (1) with

$$\begin{aligned} &\theta_{1}(x_{1})=\frac{1}{2}\Vert Ax_{1}-y\Vert ^{2},\qquad \theta_{2}(x_{2})= \nu \Vert x_{2}\Vert _{1},\\ & A_{1}=I_{n},\qquad A_{2}=-I_{n},\qquad b=0,\qquad\mathcal{X}_{1}= \mathcal{X}_{2}=\mathcal{R}^{n}, \end{aligned}$$

and thus, the P-ADMM can be used to solve CS.

In our experiment, the stopping criterion of the P-ADMM is set as

$$\frac{\Vert f_{k}-f_{k-1}\Vert }{\Vert f_{k-1}\Vert }< 10^{-5}, $$

where \(f_{k}\) denotes the function value of (26) at the iterate \(x_{1}^{k}\). The initial points of \(x_{1},x_{2},\lambda\) are all set as \(A^{\top}y\), and due to the limit of EMS memory of our computer, we only test a medium scale of (26) with \(n=1{,}000, m=300\), \(k=60\), where k is the number of random nonzero elements contained in the original signal. In addition, we set

$$\bar{A}=\operatorname{randn}(m,n), \qquad[Q, R]=\operatorname{qr} \bigl( \bar{A}',0 \bigr), \qquad A = Q', $$

and \(\nu=0.01\), \(G_{1}=\tau I_{n}-\beta A^{\top}A\) with \(\tau=2, \beta =\operatorname{mean}(\operatorname{abs}(b))\). In the literature, the relative error (RelErr) is usually used to measure the quality of recovered signal and is defined by

$$\operatorname{RelErr}=\frac{\Vert \tilde{x}-\bar{x}\Vert }{\Vert \bar{x}\Vert }, $$

where x̃ and x̄ denote the recovered signal and the original signal, respectively.

First, let us illustrate the sensitivity of γ for the P-ADMM. We choose different values of γ in the interval [0.1, 1.6] (More specifically, we take \(\gamma =0.10,0.15,\ldots,1.60\)). The numerical results of the objective value of (26) and the CPU time in seconds requited by the P-ADMM are depicted in Figure 1, and the numerical results of the numbers of iteration and the RelErr required by P-ADMM are depicted in Figure 2.

Figure 1
figure 1

Objective value and CPU time with different γ .

Full size image
Figure 2
figure 2

Numbers of iteration and relative error with different γ .

Full size image

According to the curves in Figures 1-2, we can see that the relaxation factor γ works well for a wide range of values and, based on this experiment, the values greater than 0.5 are more preferred.

Now let us test the effectiveness of the P-ADMM with the indefinite proximal matrix \(G_{2}=(\alpha\tau-\beta)I_{n}\). Here we set \(\alpha=0.8, \tau=1.1\), \(\beta=1\), and \(\gamma=1\). The numerical results of one experiment are as follows: the objective value is 0.4291; the CPU time is 1.0920; the numbers of iteration is 378 and the RelErr is 5.75%. The original signal, the measurement and the signal recovered by the P-ADMM for this test scenario are given in Figure 3. The recovered results are marked by a red circle in the third subplot of Figure 3, which shows clearly that almost the original signal is recovered with high precision. This indicates that the P-ADMM is effective though the proximal matrix \(G_{2}\) is indefinite.

Figure 3
figure 3

The original signal, noisy measurement and recovered results.

Full size image