Regularity of Sets Under a Reformulation in a Product Space with Reduced Dimension

Different notions on regularity of sets and of collection of sets play an important role in the analysis of the convergence of projection algorithms in nonconvex scenarios. While some projection algorithms can be applied to feasibility problems defined by finitely many sets, some other require the use of a product space reformulation to construct equivalent problems with two sets. In this work we analyze how some regularity properties are preserved under a reformulation in a product space of reduced dimension. This allows us to establish local linear convergence of parallel projection methods which are constructed through this reformulation.


Introduction
The so-called feasibility problem asks for a point in the intersection of a family of sets C 1 , . . ., C r in an Euclidean space; that is, (1.1) Projection algorithms are widely employed methods for solving (1.1) whenever the individual projectors onto the sets can be easily computed.The method of alternating projections (MAP) [25] and the Douglas-Rachford algorithm (DR) [17,22] are well known projection algorithms originally devised for solving feasibility problems with two sets.While the former can be naturally extended for an arbitrary number of sets [10], it is not so obvious for the case of DR (see, e.g., [2,Section 3.3]).Although there exist some cyclic versions of DR for finitely many sets [6,9], these are not frugal in the sense that some of the projectors are employed more than once at each iteration.In contrast, one can always apply Pierra's product space reformulation [27] to derive a frugal parallel DR-type projection algorithm embedded in the product Euclidean space X r .This enlargement of the dimension of the ambient space has been called as r-fold lifting.In general, reduced lifting is preferred as this leads to computational memory savings.
In the more general context of monotone inclusions, which include feasibility problems as particular cases, the impossibility of a frugal three-operator DR algorithm without lifting was proved in [28].In addition, the author showed that the minimal lifting for three operators is 2-fold.That result has been recently generalized in [23] for an arbitrary number of operators.Further, frugal splitting algorithms with minimal (r − 1)-lifting have been independently proposed in [12,23].By now, the analysis of splitting algorithms with reduced lifting has become a very active research topic; see, e.g.[1,3,8,11,14,29].
While the convergence of projection algorithms is well understood when the sets are convex, they are also popular in nonconvex settings.In this framework, local linear convergence of the schemes is usually analyzed by assuming some regularity properties of the individual sets and of their intersection; see, e.g., [6,7,14,15,16,18,19,20,21,26].In this work we analyze how some of these properties are preserved through the product space reformulation with reduced dimension studied by the author in [12].This trick reformulates problem (1.1) as an equivalent feasibility problem defined by two sets in the product space X r−1 while keeping the computability of the projectors.Thus, it allows for devising new projection algorithms with (r − 1)-lifting from already existing two-sets methods.Although the reformulation was shown to be valid for not necessarily convex sets but rather proximinal, there was a lack of theoretical results from the perspective of the local convergence of projection algorithms.The aim of this work is to extend the analysis of the reformulation by establishing that super-regularity of the sets (see Definition 2.6), as well as linear regularity and strong regularity of their intersection (see Definition 2.7), are inherit by the new product sets in the reformulated problem.Hence, the local linear convergence of the derived algorithms can be deduced assuming that those conditions hold for the original problem.
The structure of this manuscript is as follows.We collect some preliminary notions and results in Section 2. In Section 3 we revisit the product space reformulation with reduced dimension and we include our main result regarding the preservation of regularity properties.We apply our results in Section 4 to derive the local linear convergence of a parallel projection algorithm with reduced lifting, where we also include a numerical experiment to illustrate the result.Finally, some conclusions are drawn in Section 5.

Preliminaries
Throughout this paper, X is a Euclidean space endowed with inner product ⟨•, •⟩ and induced norm ∥ • ∥.The set of nonnegative integers is denoted by N and B(x; δ) stands for the closed ball centered at x ∈ X with radius δ ≥ 0. Given a linear subspace M ⊆ X we denote by M ⊥ to its orthogonal complement, i.e., M ⊥ = {u ∈ X : ⟨x, u⟩ = 0, ∀x ∈ M } .

Projection mapping
Definition 2.1.The projection mapping (or projector ) onto C is the possibly set-valued operator P C : X ⇒ C defined at each x ∈ X by Any point p ∈ P C (x) is said to be a best approximation to x from C (or a projection of x onto C).
If C is assumed to be closed, then it is proximinal ; i.e., a projection onto C exists for every point in the space (see, e.g., [4,Corollary 3.15]).When C is in addition convex, then C is Chebyshev ; i.e., the projector P C (x) is single-valued for all x ∈ X (see, e.g., [4,Remark 3.17]).
We recall next some properties of the projector.
Fact 2.2.Let C ⊆ X be nonempty.The following hold.
(i) If C is closed and convex, then P C is continuous.
(ii) If C is a linear subspace, then P C is a linear mapping.
In general, no closed expression exists for the projector onto the intersection of two sets, in terms of the individual projectors.However, if one of the involved sets is an affine subspace with some additional intersection structure with another closed set, we can establish the following relation on the projectors.

Normal cone
Regularity notions shall be defined in terms of the (limiting) normal cone to the sets.Definition 2.4.The (limiting) normal cone to C ⊆ X at a point x ∈ C is given by For a closed and convex set C ⊆ X , the limiting normal cone coincides with the classical convex normal cone {u ∈ X : ⟨c − x, u⟩ ≤ 0, ∀c ∈ C} .
In addition, when C is a linear subspace, its normal cone turns to its orthogonal complement; i.e., N C (x) = C ⊥ for all x ∈ C (see, e.g., [4,Example 6.43]).As in the case of projectors, there is no general expression relating the normal cone to the intersection of sets to those of the individual sets.The next lemma, which is a key tool in our analysis, establishes such a relation under the same assumptions than Fact 2.3.Lemma 2.5.Let C ⊆ X be a nonempty and closed set and let D ⊆ X be a linear subspace.
(2.1) Set d k := P D (x k ), for each k ∈ N. On the one hand, by continuity of the projector P D (see Fact 2.2(i)) we get that d k → P D (x) = x.On the other hand, since z k ∈ P C∩D (x k ), by applying Fact 2.3 we obtain that Now, we can split (2.1) as provided that both limits exist.Indeed, since P D is a (continuous) linear mapping (see Fact 2.2), from (2.1) we derive that where we have used the fact that z k ∈ D for all k ∈ N, according to (2.2).This shows that the first limit in (2.3), and therefore both of them, exist.Hence, we have obtained that Since x ∈ C ∩ D was arbitrary, we have proved the direct inclusion On the other hand, by taking , which combined with the previous expression yields the reverse inclusion of (2.5) and finishes the proof.

Regularity of sets
Let us finally recall the following notions of regularity of sets and of collection of sets.Definition 2.6 (Super-regular sets).A subset C ⊆ X is said to be super-regular at a point x ∈ C if, for any ε > 0, there exists δ > 0 such that Definition 2.7 (Regularity of collection of sets).A finite family of sets C 1 , . . ., C r ⊆ X is said to be (i) linearly regular around x ∈ X if there exist κ ≥ 0 and δ > 0 such that The above conditions are usually employed to derive the local linear convergence of some projection algorithms on nonconvex problems.Recall that a sequence {x k } k∈N converges R-linearly to a point x * if there exist η ∈ [0, 1[ and M > 0 such that In the following fact, we recall the (local) linear convergence of the so-called generalized Douglas-Rachford algorithm for two sets under regularity conditions.Fact 2.8 (Linear convergence of gDR).Let A, B ⊆ X be nonempty super-regular sets of X and let Suppose that any of the following conditions holds: (i) {A, B} is strongly regular at w, (ii) min{λ, µ} < 2 and {A, B} is linearly regular around w.
If the initial point x 0 is sufficiently close to w, then, the sequence generated by (2.8) converges R-linearly to a point x * ∈ A ∩ B. When, in addition, A and B are convex sets, the R-linear convergence of the sequence is global.
3 Regularity under a product space reformulation with reduced dimension We begin this section by introducing the product space reformulation in a reduced dimensional product space proposed in [12].To this aim, consider the product space and define which is a linear subspace of X r−1 commonly known as the diagonal.We denote j r−1 : X → D r−1 the canonical embedding that maps any x ∈ X to j r−1 (x) = (x, . . ., x) ∈ D r−1 .Then, consider the product sets The equivalency, from the point of view of projection algorithms, between problem (1.1) and the one described by the sets in (3.1) is recalled in the following fact.
Fact 3.1.Let C 1 , C 2 , . . ., C r ⊆ X be closed sets and let B, K ⊆ X r−1 the product operators as defined in (3.1).Then the following hold.
(i) B is closed and If, in addition, C 1 , . . ., C r−1 are convex then so is B.
(ii) K is closed and If, in addition, C r is convex then so is K.
Remark 3.2.Recall that classical Pierra's reformulation [27] reframes problem (1.1) as where the projectors are given by x i ; for any x = (x 1 , x 2 , . . ., x r ) ∈ X r (see, e.g., [12,Proposition 4.2]).In contrast, we reformulate the same feasibility problem as with B and K being the sets in (3.1), whose projectors are given in Fact 3.1.Note that this leads to a reduction of one dimension of the ambient space in comparison to Pierra's trick.The computational advantage was shown in [12] through some numerical experiments.
The analysis of the regularity properties of the sets in Pierra's reformulation is usually employed in order to derive local linear convergence of parallel projection algorithms.See, for instance, [21,Theorem 7.3] for the method of averaged projections.In particular, superregularity of sets and linear and strong regularity of the intersection are kept after the reformulation (see, e.g., [14, Propositions 3.1(i) and 3.2]).We establish next analogous results for the reformulation in the product space with reduced dimension in Fact 3.1.Although our analysis employs similar techniques to those of [14], we need to stablish first the following technical result about the normal cones to the product sets in (3.1).Lemma 3.3.Let C 1 , . . ., C r ⊆ X be nonempty and closed sets, let B, K ⊆ X r−1 be the product sets as defined in (3.1) and consider D r−1 the diagonal of the product space X r−1 .Then, the following hold.
In particular, P S (q)∩D r−1 ̸ = ∅ for all q ∈ D r−1 .Hence, we can apply Lemma 2.5 to express Since N S (x) = r−1 i=1 N Cr (x) by item (i), the result follows.
We are now ready to derive our main result regarding the regularity of the product sets in (3.1) and of their intersection, provided that the original sets verify those conditions.Theorem 3.4.Let C 1 , . . ., C r ⊆ X be nonempty and closed sets and let B, K ⊆ X r−1 be the product sets as defined in (3.1).Then, the following statements hold.(i) If C i is super-regular at xi ∈ C i , for all i = 1, . . ., r − 1, then the product set B is super-regular at x := (x 1 , . . ., xr−1 ) ∈ B.
(ii) If C r is super-regular at x ∈ C r , then K is super-regular at x := j r−1 (x) ∈ K.
By summing up all these equations we arrive at Therefore, v = w = 0 and we get that {B, K} is strongly regular at x.

Application to projection algorithms
We finally apply our main result (Theorem 3.4) to derive (local) linear convergence of projection algorithms constructed by means of the product space reformulation in Fact 3.1.In particular, we consider the generalized Douglas-Rachford (gDR) algorithm analyzed in [15], as it includes the method of alternating projections (MAP) and the Douglas-Rachford (DR) algorithm as particular cases.
Theorem 4.1 (Linear convergence of parallel gDR algorithm with reduced lifting).
Let C 1 , C 2 , . . ., C r ⊆ X be nonempty super-regular sets with ∩ r i=1 Suppose that any of the following conditions holds: If the initial points x 1,0 , . . ., x r−1,0 are sufficiently close to x, then, for each i ∈ {1, . . ., r − 1}, the sequence {x i,k } k∈N converges R-linearly to a point x * ∈ ∩ r i=1 C i .When, in addition, C 1 , C 2 , . . ., C r are convex sets, the R-linear convergence of the sequences is global.
As previously mentioned, iteration (4.2) recovers some well-known classical projection methods.Hence, Theorem 4.1 provides local linear convergence for reduced parallel versions of these algorithms.We state next such result for the method of alternating projections, leading to what we will refer to as reduced averaged projections method.Corollary 4.2 (Linear convergence of reduced averaged projections method).Let C 1 , C 2 , . . ., C r ⊆ X be nonempty super-regular sets with linearly regular intersection around x ∈ ∩ r i=1 C i .Given x 0 ∈ X , set

Numerical experiment
In this section, we present a numerical example to illustrate the linear convergence of the reduced averaged projections method discussed in Corollary 4.2.Our objective is to replicate the signal compression problem analyzed in [21, Section 9], which was utilized to show the linear convergence of the traditional averaged projections method.
Given a "dictionary" W ∈ R n×m and a threshold α > 0, the recovery of the signal is addressed by solving the feasibility problem It is not difficult to check that the projectors onto these sets can be computed as where the maximum and minimum in P C are understood componentwise.Furthermore, as mentioned in [21], the three sets in (4.4) are super-regular, whereas the linear regularity of their intersection is expected from randomness when generating the problem, provided that α is not too small.In our experiment, we set n = 128, m = 512, d = 8 and α = 0.1.The entries of the matrix W ∈ R 128×512 , as well as those of the initial iterate U 0 ∈ R 8×512 , were randomly generated from a standard normal distribution.From that point, we run the averaged projections algorithm, which iterates as and the reduced averaged projections method in Corollary 4.2.Note that (4.5) is completely symmetric with respect to the order of the sets.However, this is not the case for the reduced averaged projections in (4.3),where the set C r acts as a "central coordinator".Thus, in our experiment we consider all three possibilities for this method depending on which of the sets L, M , or C plays the role of coordinator (indicated between brackets).We stopped each algorithm when ∥U k+1 − U k ∥ < 10 −12 .In Figure 1 we plot the norm ∥U k − U * ∥ with respect to the iteration, where U * denotes the limit of the sequence.We can clearly observe a linearly convergent behavior of all tested methods, showing the reduced versions of the method a better convergence rate than its classical version.Furthermore, the choice of the coordinator set C r in (4.3) seems to have a strong impact in the convergence rate of the method.In our experiment, the fastest convergence was achieved by selecting set C in (4.4c), followed closely by selecting L in (4.4a).Overall, the results suggest that choosing the appropriate coordinator set can significantly improve the convergence rate of the method.

Conclusions
In this manuscript we explored how some regularity properties of sets and of collections of sets are preserved under a reformulation in a product space with reduced dimension.This allows for the establishment of local linear convergence of parallel projection methods constructed through this reformulation.Specifically, the results were applied to the generalized Douglas-Rachford algorithm, which include some well-known projection algorithms as particular cases.
A numerical demonstration on a signal compression problem, replicating that of [21, Section 9], was included.This study tested the method of averaged projections and some reduced versions of this method constructed trough the analyzed reformulation.As expected, all methods showed to be linearly convergent.In addition, a better convergence rate was obtained for the reduced methods in this specific experiment.It remains open for future research to analyze the convergence rate of these methods, particularly with respect to the order of the sets and its effect on the rate.

(4. 3 )Remark 4 . 3 .
If x 0 is sufficiently close to x, then the sequence {x k } k∈N converges R-linearly to a point x * ∈ ∩ r i=1 C i .When, in addition, C 1 , C 2 , . . ., C r are convex sets, the R-linear convergence of the sequence is global.Proof.Apply Theorem 4.1(i) with λ = µ = α = 1.An analogous result can be derived for the Douglas-Rachford algorithm by taking λ = µ = 2 in (4.1).In particular, Theorem 4.1 under scenario (i) applies to the parallel DR-algorithm with reduced dimension proposed in[12, Theorem 5.1]  in the context of feasibility problems.

Figure 1 :
Figure1: Comparison of the convergence rate of averaged projections and reduced averaged projections methods for solving a signal compression problem.For each method we plot the distance to solution, in logarithmic scale, with respect to the iteration.