Global optimization in Hilbert space

We propose a complete-search algorithm for solving a class of non-convex, possibly infinite-dimensional, optimization problems to global optimality. We assume that the optimization variables are in a bounded subset of a Hilbert space, and we determine worst-case run-time bounds for the algorithm under certain regularity conditions of the cost functional and the constraint set. Because these run-time bounds are independent of the number of optimization variables and, in particular, are valid for optimization problems with infinitely many optimization variables, we prove that the algorithm converges to an \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varepsilon $$\end{document}ε-suboptimal global solution within finite run-time for any given termination tolerance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varepsilon > 0$$\end{document}ε>0. Finally, we illustrate these results for a problem of calculus of variations.


Introduction
Infinite-dimensional optimization problems arise in many research fields, including optimal control [7,8,24,54], optimization with partial differential equations (PDE) embedded [22], and shape/topology optimization [5]. In practice, these problems are often solved approximately by applying discretization techniques; the original infinitedimensional problem is replaced by a finite-dimensional approximation that can then be tackled using standard optimization techniques. However, the resulting discretized optimization problems may comprise a large number of optimization variables, which grows unbounded as the accuracy of the approximation is refined. Unfortunately, worst-case run-time bounds for complete-search algorithms in nonlinear programming (NLP) scale poorly with the number of optimization variables. For instance, the worst-case run-time of spatial branch-and-bound [17,44] scales exponentially with the number of optimization variables. In contrast, algorithms for solving convex optimization problems in polynomial run-time are known [11,40], e.g. in linear programming (LP) or convex quadratic programming (QP). While these efficient algorithms enable the solution of very large-scale convex optimization problems, such as structured or sparse problems, in general their worst-case run-time bounds also grow unbounded as the number of decision variables tends to infinity.
Existing theory and algorithms that directly analyze and exploit the infinitedimensional nature of an optimization problem are mainly found in the field of convex optimization. For the most part, these algorithms rely on duality in convex optimization in order to construct upper and lower bounds on the optimal solution value, although establishing strong duality in infinite-dimensional problems can prove difficult. In this context, infinite-dimensional linear programming problems have been analyzed thoroughly [3]. A variety of algorithms are also available for dealing with convex infinite-dimensional optimization problems, some of which have been analyzed in generic Banach spaces [14], as well as certain tailored algorithms for continuous linear programming [4,13,32].
In the field of non-convex optimization, problems with an infinite number of variables are typically studied in a local neighborhood of a stationary point. For instance, local optimality in continuous-time optimal control problems can be analyzed by using Pontryagin's maximum principle [46], and a number of local optimal control algorithms are based on this analysis [6,12,51,54]. More generally, approaches in the classical field of variational analysis [37] rely on local analysis concepts, from which information about global extrema may not be derived in general. In fact, non-convex infinite-dimensional optimization remains an open field of research and, to the best of our knowledge, there currently are no generic complete-search algorithms for solving such problems to global optimality.
This paper asks the question whether a global optimization algorithm can be constructed, whose worst-case run-time complexity is independent of the number of optimization variables thereof, such that this algorithm would remain tractable for infinite-dimensional optimization problems. Clearly, devising such an algorithm may only be possible for a certain class of optimization problems. Interestingly, the fact that the "complexity" or "hardness" of an optimization problem does not necessarily depend on the number of optimization variables has been observed-and it is in fact exploited-in state-of-the-art global optimization solvers for NLP/MINLP, although these observations are still to be analyzed in full detail. For instance, instead of applying a branch-and-bound algorithm in the original space of optimization variables, global NLP/MINLP solvers such as BARON [49,52] or ANTIGONE [34] proceed by lifting the problem to a higher-dimensional space via the introduction of auxiliary variables from the DAG decomposition of the objective and constraint functions. In a different context, the solution of a lifted problem in a higher-dimensional space has become popular in numerical optimal control, where the so-called multiple-shooting methods often outperform their single-shooting counterparts despite the fact that the former calls for the solution a larger-scale (discretized) NLP problem [7,8]. This idea that certain optimization problems become easier to solve than equivalent problems in fewer variables is also central to the work on lifted Newton methods [2]. To the best of our knowledge, such behaviors cannot be explained currently with results from the field of complexity analysis, which typically give monotonically increasing worst-case run-time bounds as the number of optimization variables increases. In this respect, these run-time bounds therefore predict the opposite behavior to what can sometimes be observed in practice.

Problem formulation
The focus of this paper is on complete-search algorithms for solving non-convex optimization problems of the form: where F : H → R and C ⊆ H denote the cost functional and the constraint set, respectively; and the domain H of this problem is a (possibly infinite-dimensional) Hilbert space with respect to the inner product ·, · : H × H → R. The theoretical considerations in the paper do not assume a separable Hilbert space, although our various illustrating examples are based on separable spaces.
Definition 1 A feasible point x * ∈ C is said to be an ε-suboptimal global solution-or ε-global optimum-of (1), with ε > 0, if We make the following assumptions regarding the geometry of C throughout the paper.

Assumption 1
The constraint set C is convex, has a nonempty relative interior, and is bounded with respect to the induced norm on H ; that is, there exists a constant γ < ∞ such that Our main objective in this paper is to develop an algorithm that can locate an εsuboptimal global optimum of Problem (1), in finite run-time for any given accuracy ε > 0, provided that F satisfies certain regularity conditions alongside Assumption 1.
Remark 1 Certain infinite-dimensional optimization problems are formulated in a Banach space (B, · ) rather than a Hilbert space, for instance in the field of optimal control of partial differential equations in order to analyze the existence of extrema [22]. The optimization problem (1) becomes withF : B → R andĈ a convex bounded subset of B. Provided that: 1. the Hilbert space H ⊆ B is convex and dense in (B, · ); 2. the functionF is upper semi-continuous inĈ; and 3. the constraint setĈ has a nonempty relative interior; we may nonetheless consider Problem (1) with C :=Ĉ ∩ H instead of (2), for any ε-suboptimal global solution of the former is also an ε-suboptimal global solution of (2), and both problems have such ε-suboptimal points. Because Conditions 1-3 are often satisfied in practical applications, it is for the purpose of this paper not restrictive to assume that the domain of the optimization variables is indeed a Hilbert space.

Outline and contributions
The paper starts by discussing several regularity conditions for sets and functionals defined in a Hilbert space in Sect. 2, based on which complete-search algorithms can be constructed whose run-time is independent of the number of optimization variables. Such an algorithm is presented in Sect. 3 and analyzed in Sect. 4, which constitutes the main contributions and novelty. A numerical case study is presented in Sect. 5 in order to illustrate the main results, before concluding the paper in Sect. 6. Although certain of these algorithmic ideas are inspired by a recent paper on global optimal control [25], we develop herein a much more general framework for optimization in Hilbert space. Besides, Sect. 4 derives novel worst-case complexity estimates for the proposed algorithm. We argue that these ideas could help lay the foundations towards new ways of analyzing the complexity of certain optimization problems based on their structural properties rather than their number of optimization variables. Although the run-time estimates for the proposed algorithm remain conservative, they indicate that complexity in numerical optimization does not necessarily depend on whether the problem at hand being small-scale, large-scale, or even infinite-dimensional.

Some regularity conditions for sets and functionals in Hilbert space
This section builds upon basic concepts in infinite-dimensional Hilbert spaces in order to arrive at certain regularity conditions for sets and functionals defined in such spaces. Our focusing on Hilbert space is motivated by the ability to construct an orthogonal basis Φ 0 , Φ 1 , . . . ∈ H such that ∀i, j ∈ N, for some scalars σ 0 , σ 1 , . . . ∈ R ++ . We make the following assumption throughout the paper:

Assumption 2
The basis functions Φ k are uniformly bounded with respect to · H .
Equipped with such a basis, we can define the associated projection functions P M : A natural question to ask at this point, is what can be said about the distance between an element x ∈ H and its projection P M (x) for a given M ∈ N. Proof For each M ∈ N, we have The result follows by Assumption 1.
Despite being uniformly bounded, the function D C (M) may not converge to zero as M → ∞ in an infinite-dimensional Hilbert space in general. Such lack of convergence is illustrated in the following example.
Example 1 Consider the case that all the basis functions Φ 0 , Φ 1 , . . . are in the constraint set C, and define the sequence {x k } k∈N with x k := Φ k+1 . For all k ∈ N, we have P k (x k ) = 0, and therefore This behavior is unfortunate because the existence of minimizers to Problem (1) cannot be ascertained without making further regularity assumptions. Moreover, for a That is, any attempt to approximate the infimum by constructing a sequence of finite parameterizations of the optimization variable x could in principle be unsuccessful. A principal aim of the following sections is to develop an optimization algorithm, whose convergence to an ε-global optimum of Problem (1) can be certified. But instead of making assumptions about the existence, or even the regularity, of the minimizers of Problem (1), we shall impose suitable regularity conditions on the objective function F in (1). In preparation for this analysis, we start by formalizing a particular notion of regularity for the elements of H .
Moreover, we call the function R C (·, g) : N → R + the convergence rate at g on C.
Theorem 1 For any g ∈ H , we have In the particular case of g being a regular element for C, we have Proof Let M ∈ N, and consider the optimization problem where we have introduced the variable w : Since the functions Φ 0 , . . . , Φ M are orthogonal to each other, we have Φ k , w = 0 for all k ∈ {0, . . . , M}, and it follows that Next, we use duality to obtain where λ ∈ R M+1 are multipliers associated with the constraints Φ k , w = 0 for k ∈ {0, . . . , M}. Applying the Cauchy-Schwarz inequality gives and with the particular choice λ * k : The optimal value of the minimization problem can be estimated analogously, giving V M ≥ −R C (M, g), and the result follows.
The following example establishes the regularity of piecewise smooth functions with a finite number of singularities in the Hilbert space of square-integrable functions with the Legendre polynomials as orthogonal basis functions. There are numerous results on approximating functions using polynomials, including convergence rate estimates [15]. One such result in [48] shows that any piecewise smooth function f : for any given α, β > 0 with either α < 1 and β ≥ α, or α = 1 and β > 1; some constants K 1 , K 2 > 0; and where d(y) denotes the distance to the nearest singularity.
In particular, the following convergence rate estimate can be derived using this result in the present example, for any piecewise smooth functions g : [0, 1] → R with a finite number of singularities: In order to establish the very last part of the above inequality, it is enough to consider a function g with a single singularity, e.g., at the mid-point y = 1 2 , and using α = β = 1 2 : Convergence rate estimates for k-times differentiable and piecewise smooth functions can be obtained in a similar way, using for instance the results in [15,48].
A useful generalization of Definition 3 and a corollary of Theorem 1 are given below.
Moreover, we call the function R C (·, G) : N → R + the worst-case convergence rate for G on C.

Corollary 1 For any regular set G ⊆ H , we have
Remark 2 While any subset of the Euclidean space R n is trivially regular for a given bounded subset C ⊂ R n , only certain subsets/subspaces of an infinite-dimensional Hilbert space happen to be regular. Consider for instance the space of square-integrable functions, H := L 2 [a, b], and let G p be any subset of p-times differentiable functions on [a, b], with uniformly Lipschitz-continuous p-th derivatives. It can be shown-e.g., 1 We have used the integration formula e from the analysis in [27] using the standard trigonometric Fourier basis, or from the results in [55] using the Legendre polynomial basis-that for any bounded constraint set C ⊂ L 2 [a, b], and G p is thereby regular for C. This leads to a rather typical situation, whereby the stronger the regularity assumptions on the function class, the faster the convergence of the associated worst-case convergence rate R(·, G p )-an increase in the convergence rate order log(M)M − p−1 with p in this instance. In the limit of smooth (C ∞ ) functions, it can even be established-e.g., using standard results from Fourier analysis [19,28]-that the convergence rate becomes exponential, Example 2 (Continued) Consider the following set of unit-step functions for which we want to establish regularity in the sense of Definition 4. Using earlier results in Example 2, it is known that the function x 0.5 can be approximated with a sequence of polynomials For every t ∈ [0, 1] likewise, we can construct the family of polynomials Since the latter satisfy the same property as x 0.5 that This example can be generalized to other classes of functions. For instance, given any smooth function f ∈ L 2 [0, 1], the subset is regular in H , and also satisfies R C (M, G f ) ≤ O M −1/2 . This result can be established by writing the elements in G f as the product between the piecewise smooth function f and the function x t , and then approximating the factors separately.
In the remainder of this section, we analyze and illustrate a regularity condition for the cost functional in Problem (1).

Definition 5
The functional F : H → R is said to be strongly Lipschitz-continuous on C if there exists a bounded subset G ⊂ H which is regular on C and a constant L < ∞ such that Remark 3 In the special case of an affine functional F, given by where F 0 ∈ H , andĝ ∈ H is a regular element for C, the condition (7) is trivially satisfied with L = 1 and G = {ĝ}. In this interpretation, the regularity condition (7) essentially provides a means of keeping the nonlinear part of F under control.

Remark 4
Consider the finite-dimensional Euclidean space R n , a bounded subset S ⊂ R n , and a continuously-differentiable function F : R n → R whose first derivative is bounded in the subset G ⊂ R n . By the mean-value theorem, F satisfies Thus, any continuously differentiable function with a bounded first derivative is strongly Lipschitz-continuous on any bounded subset of R n . This result can be generalized to certain classes of functionals in infinite-dimensional Hilbert space. For instance, let F : H → R be Fréchet differentiable, such that and let the set of Fréchet derivatives G := {D F(x) | x ∈ H } ⊆ H be both bounded and regular on C. Then, F is strongly Lipschitz-continuous on C.
The following two examples investigate strong Lipschitz continuity for certain classes of functionals in the practical space of square-integrable functions with the Legendre polynomials as orthogonal basis functions. The first one (Example 3) illustrates the case of a functional that is not strongly Lipschitz-continuous; the second one (Example 4) identifies a broad class of strongly Lipschitz-continuous functionals defined via the solution of an embedded ODE system. The intention here is to help the reader develop an intuition that strongly Lipschitz-continuous functionals occur naturally in many, although not all, problems of practical relevance.

Example 3
We consider the Hilbert space H = L 2 [0, 1] of square-integrable functions on the interval [0, 1] with the standard inner product, and select the orthogonal basis functions (Φ k ) k∈N as the Legendre polynomials on the interval [0, 1] with weighting factors σ k = 1 2k+1 . We investigate whether the functional F given below is strongly Lipschitz-continuous on the set C : Consider the family of sets defined by If the condition (7) were to hold for some bounded and regular set G, we would have by Theorem 1 that and it would follow from Corollary 1 that However, this leads to a contradiction since we also have Therefore, the regularity condition (7) may not be satisfied for any bounded and regular set G, and F is not strongly Lipschitz-continuous on C. Remark where B ∈ R n×n is a constant matrix; and f : R n → R n , a continuously-differentiable and globally Lipschitz-continuous function, so that the solution trajectory x(·, u) : [0, 1] → R n is well-defined for all u ∈ L 2 [0, 1]. For simplicity, we consider the functional F given by for some real vector c ∈ R n . Moreover, the constraint set C ⊆ H may be any uniformly bounded function subset, such as simple uniform bounds of the form The following developments aim to establish that F is strongly Lipschitz-continuous on C. By Taylor's theorem, the defect δ(t, u, e) := x(t, u + e) − x(t, u) satisfies the differential equation u, e)) dη. The right-handside function f being globally Lipschitz-continuous, we have for any given smooth matrix-valued function A : [0, 1] → R n×n , for some constant 1 < ∞. For a particular choice of A, we can decompose δ(t, u, e) into the sum δ l (t, e) + δ n (t, u, e, δ l ) corresponding to the solution of the ODE system with δ l (0, e) = δ n (0, u, e, δ l ) = 0. In this decomposition, the left-hand side of (7) satisfies (1, u, e) .
Regarding the linear term δ l first, we have ∀s ∈ [0, 1], c T δ l (s, e) = g s , e with where Γ (t, τ ) denotes the fundamental solution of the linear ODE (9) such that Since A is smooth, it follows from Example 2 that the set G := {g s | s ∈ [0, 1]} is both regular on C and bounded, and satisfies Regarding the nonlinear term δ n , since the function Λ is uniformly bounded, applying Gronwall's lemma to the ODE (10) gives for some constant < ∞. Finally, combining (11) and (12) shows that F satisfies the condition (7) with L := 1 + exp( ), thus F is strongly Lipschitz-continuous on C.

Remark 6
The functional F in the previous example is defined implicitly via the solution of an ODE. The result that such functionals are strongly Lipschitz-continuous is particularly significant insofar as the proposed optimization framework will indeed encompass a broad class of optimal control problems as well as problems in the calculus of variations. In fact, it turns out that strong Lipschitzness still holds in replacing the constant matrix B in (8) with any matrix-valued continuously differentiable and globally Lipschitz-continuous function of x(t, u), thus encompassing quite a general class of nonlinear affine-control systems. In the case of general nonlinear ODEs, however, strong Lipschitzness may be lost. Strong Lipschitzness could nevertheless be recovered by restricting condition (7) in Definition 5 as We close this section with a brief analysis of the relationship between strong and classical Lipschitzness in infinite-dimensional Hilbert space.

Lemma 2 Every strongly Lipschitz-continuous functional F : H → R on C is also Lipschitz-continuous on C.
Proof Let G be a bounded and regular subset of H on C such that the condition (7) is satisfied. Since G is bounded, there exists a constant constant α < ∞ such that sup g∈G g H ≤ α. Applying the Cauchy-Schwarz inequality to the right-hand side of (7) gives and so F is Lipschitz-continuous on C.
Remark 7 With regularity of the set G alone, i.e. without boundedness, the condition (7) may not imply Lipschitz-continuity, or even continuity of F. As a counter-example, let G := span (Φ 0 , Φ 1 , . . . , Φ N ) be the subspace spanned by the first N basis functions in the infinite-dimensional Hilbert space H . It is clear that G is regular on any bounded set C ⊂ H since R C (M, G) = 0 for all M ≥ N . Now, let the functional F : H → R given by Therefore, despite being discontinuous, the condition (7) is indeed satisfied.

Remark 8
In general, Lipschitz-continuity does not imply strong Lipschitz-continuity in an infinite-dimensional Hilbert space. A counter-example is easily contrived for the functional F : L 2 [0, 1] → R given by Although this functional is Lipschitz-continuous, it can be shown by a similar argument as in Example 3 that it is not strongly Lipschitz-continuous.

Global optimization in Hilbert space using complete search
The application of complete-search strategies to infinite-dimensional optimization problems such as (1) calls for an extension of the (spatial) branch-and-bound principle [23] to general Hilbert space. The approach presented in this section differs from branch-and-bound in that the dimension M of the search space is adjusted, as necessary, during the iterations of the algorithm, by using a so-called lifting operation-hence the name branch-and-lift algorithm. The basic idea is to bracket the optimal solution value of Problem (1) and progressively refine these bounds via this lifting mechanism, combined with traditional branching and fathoming. Based on the developments in Sect. 2, the following subsections describe methods for exhaustive partitioning in infinite-dimensional Hilbert space (Sect. 3.1) and for computing rigorous upper and lower bounds on given subsets of the variable domain (Sect. 3.2), before presenting the proposed branch-and-lift algorithm (Sect. 3.3).

Partitioning in infinite-dimensional Hilbert space
Similar to branch-and-bound search, the proposed branch-and-lift algorithm maintains a partition A := {A 1 , . . . , A k } of finite-dimensional sets A 1 , . . . , A k . This partition is updated through the repeated application of certain operations, including branching and lifting, in order to close the gap between an upper and a lower bound on the global solution value of the optimization problem (1). The following definition is useful in order to formalize these operations:

Moreover, we say that the set
Notice that each subregion X M (A) is a convex set if the sets C and A are themselves convex. For practical reasons, we restrict ourselves to compact subsets A ∈ S M+1 ⊆ P(R M+1 ) herein, where the class of sets S M+1 is easily stored and manipulated by a computer. For example, S M+1 could be a set of interval boxes, polytopes, ellipsoids, etc.
The ability to detect infeasibility of a set A ∈ S M+1 is pivotal for complete search. Under the assumption that the constraint set C is convex (Assumption 1), a certificate of infeasibility can be obtained by considering the convex optimization problem It readily follows from the Cauchy-Schwarz inequality that for any (normalized) basis function Φ k , and so x − y H = 0 implies x, Φ k = y, Φ k . Consequently, a set A is infeasible if and only if d C (A) > 0. Because Slater's constraint qualification holds for Problem (13) under Assumption 1, one approach to checking infeasibility to within high numerical accuracy relies on duality for computing lower bounds on the optimal solution value d C (A)-similar in essence to the infinite-dimensional convex optimization techniques in [4,14]. For the purpose of this paper, our focus is on a general class of non-convex objective functionals F, whereas the constraint set C is assumed to be convex and to have a simple geometry in order to avoid numerical issues in solving feasibility problems of the form (13). We shall therefore assume, from this point onwards, that infeasibility can be verified with high numerical accuracy for any set A ∈ S M+1 .
A branching operation subdivides any set A ∈ S M+1 in the partition A into two compact subsets A l , A r ∈ S M+1 such that A l ∪ A r ⊇ A, thereby updating the partition as On the other hand, a lifting operation essentially lifts any set A ∈ S M+1 into a higher-dimensional space under the function Γ M : S M+1 → S M+2 , defined such that The question as to defining the higher-order coefficient x, Φ M+1 in such a lifting is related to the so called moment problem that asks the question under which conditions on a sequence (a k ) k∈{1,...,N } , named moment sequence, can we find an associated Classical examples of such moment problems are Stieltjes', Hamburger's, and Legendre's moment problems [1]. Here, we adopt the modern standpoint on moment problems using convex optimization [30,42], by considering the following optimization subproblems: Although both optimization problems in (14) are convex when A and C are convex, they remain infinite-dimensional, and thus intractable in general. Obtaining lower and upper bounds a M+1 (A), a M+1 (A) is nonetheless straightforward under Assumption 1.
In case no better approach is available, one can always use which follows readily from the Cauchy-Schwarz inequality and the property that Φ M+1 H = 1. As already mentioned in the introduction of the paper, a variety of algorithms are now available for tackling convex infinite dimensional problems both efficiently and reliably [4,14], which could provide tighter bounds in practical applications.
A number of remarks are in order:

Remark 9
The idea to introduce a lifting operation to enable partition in infinitedimensional function space was originally introduced by the authors in a recently publication [25], focusing on global optimization of optimal control problems. One principal contribution in the present paper is a generalization of these ideas to global optimization in any Hilbert space, by identifying a set of sufficient regularity conditions on the cost functional and constraint set for the resulting branch-and-lift algorithms to converge to an ε-global solution in finite run-time.
Remark 10 Many recent optimization techniques for global optimization are based on the theory of positive polynomials and their associated linear matrix inequality (LMI) approximations [30,45], which are also originally inspired by moment problems. Although these LMI techniques may be applied in the practical implementation of the aforementioned lifting operation, they are not directly related to the branch-andlift algorithm that is developed in the following sections. An important motivation for moving away from the generic LMI framework is that the available implementations scale quite poorly with the number of optimization variables, due to the combinatorial increase of the number of monomials in the associated multivariate polynomial. Therefore, a direct approximation of the cost function F with multivariate polynomials would conflict with our primary objective to develop a global optimization algorithm whose worst-case run-time does not depend on the number of optimization variables.

Strategies for upper and lower bounding of functionals
Besides partitioning, the efficient construction of tight upper and lower bounds on the global solution value of (1) for given subregions of H is key in a practical implementation of branch-and-lift. Thereafter, functions L M , U M : S M+1 → R such that shall be call lower-and upper-bounding functions of the functional F, respectively. A simple approach to constructing these lower and upper bounds relies on the following two-step decomposition: 1. Compute bounds L 0 M (A) and U 0 M (A) on the finite-dimensional approximation of F as Clearly, it depends on the particular expression of F how to determine such bounds in practice. In the case that F is factorable, various arithmetics can be used to propagate bounds through a DAG of the function, including interval arithmetic [36], McCormick relaxations [9,33], and Taylor/Chebyshev model arithmetic [10,43,47]. Moreover, if the expression of F is embedding a dynamic system described by differential equations, validated bounds can be obtained by using a variety of setpropagation techniques as described, e.g., in [26,31,38,50,53]; or via hierarchies of LMI relaxations as in [21,29].

Compute a bound Δ M (A) on the approximation errors such that
In the case that F is strongly Lipschitz-continuous on C, we can always take Δ M (A) := L R C (M, G), where the constant L < ∞ and the bounded regular set G satisfy the condition (7). Naturally, better bounds may be derived by exploiting a particular structure or expression of F. We state the following assumptions in anticipation of the convergence analysis in Sect. 4. (1) is strongly Lipschitz-continuous on C, with the condition (7) holding for the constant L < ∞ and the bounded regular subset G ⊂ H .

Remark 11 Under Assumption 3, Lemma 2 implies that
for a Lipschitz constant L ≥ L sup g∈G g H . Thus, if Assumption 2 is also satisfied, any pair (M, A) ∈ N × S M+1 is such that  (15) may be computed more directly by solving the following nonconvex optimization problem to local optimality,

Remark 12 An alternative upper bound U M (A) in
Without further assumptions on the orthogonal basis functions Φ 0 , Φ 1 , . . . and on the constraint set C, however, it is not hard to contrive examples where P M (x) / ∈ C for all x ∈ C and all M ∈ N; that is, contrive examples where the upper bound (18) does not converge as M → ∞. This upper-bounding approach could nonetheless be combined with another bounding approach based on set arithmetics in order to prevent convergence issues; e.g., use the solution value of (18) as long as it provides a bound that is smaller than U 0

Branch-and-lift algorithm
The foregoing considerations on partitioning and bounding in Hilbert space can be combined in Algorithm 1 for solving infinite-dimensional optimization problems to ε-global optimality.
-Besides the branching and lifting operations introduced earlier in Sect. 3.1, fathoming in Step 4 of Algorithm 1 refers to the process of discarding a given set A ∈ A from the partition if -The main idea behind the lifting condition defined in Step 6 of Algorithm 1, namely is that a subset A should be lifted to a higher-dimensional space whenever the approximation error Δ M (A) due to the finite parameterization becomes of the same order of magnitude as the current optimality gap U M (A) − L M (A). The aim here is to apply as few lifts as possible, since it is preferable to branch in a lower dimensional space. The convergence of the branch-and-lift algorithm under this lifting condition is examined in Sect. 4 below. Notice also that a lifting operation is applied globally-that is, to all parameter subsets in the partition A -in Algorithm 1, so all the subsets in A share the same parameterization order at any iteration. In a variant of Algorithm 1, one could also imagine a family of subsets that would have different parameterization orders by applying the lifting condition locally instead. -Finally, it will be established in the following section that, upon termination and under certain assumptions, Algorithm 1 returns an ε-suboptimal solution of Problem (1). In particular, Assumption 1 rules out the possibility of an infeasible solution.

Convergence analysis of branch-and-lift
This section investigates the convergence properties of the branch-and-lift algorithm (Algorithm 1) developed previously. It is convenient to introduce the following notation in order to conduct the analysis: Definition 7 Let G ⊆ H be a regular set for C, and define the inverse function R −1 The following result is a direct consequence of the lifting condition (19) in the branchand-lift algorithm: Proof Assume that M = M in Algorithm 1, and that the termination condition is not yet satisfied; that is, for a certain feasible set A ∈ A . If the lifting condition (19) were to hold for A, then it would follow from (16)- (17) that Moreover, F being strongly Lipschitz-continuous on C by Assumption 3, we would have This is a contradiction, since R C (M, G) ≤ ε 2(ρ+1)L by Definition 7.
Besides having a finite number of lifting operations, the convergence of Algorithm 1 can be established if the elements of a partition can be made arbitrarily small after applying a finite number of subdivisions.
Proof By Lemma 3, the maximal number M of lifting operations during a run of Algorithm 1 is finite, such that M ≤ M. Therefore, the lifting condition (19) may not be satisfied for any feasible subset A ∈ A , and we have By Assumptions 2 and 3 and Remark 11, we have and the termination condition is thus satisfied if This latter condition is met after at most Σ ερ K (ρ+1) , M iterations under the assumption that the partitioning scheme is exhaustive.

Remark 13
In the case that the sets A ∈ A are simple interval boxes and the lifting process is implemented per (14) It should be clear, at this point, that the worst-case estimate Σ given in Theorem 2 may be extremely conservative, and the performance of Algorithm 1 could be much better in practice. Nonetheless, a key property of this estimate Σ is that it is independent of the actual nature or the number of optimization variables in Problem (1), be it a finite-dimensional or even an infinite-dimensional optimization problem. As already pointed in the introduction of the paper, this result is quite remarkable since available run-time estimates for standard convex and non-convex optimization algorithms do not enjoy this property. On the other hand, Σ is dependent on: -the bound γ on the constraint set C; -the Lipschitz constants K and L of the cost functional F; -the uniform bound sup k Φ k H and the scaling factors σ k of the chosen orthogonal functions Φ k ; and -the lifting parameter ρ and the termination tolerance ε in Algorithm 1.
All these dependencies are illustrated in the following example.
Example 5 Consider the space of square-integrable functions H := L 2 [− π, π], for which it has been established in Remark 2 that any subset G p of p-times differentiable functions with uniformly Lipschitz-continuous p-th derivatives on [− π, π] is regular, with convergence rate R C (M, G p ) ≤ α M − p for some constant α < ∞. On choosing the standard trigonometric Fourier basis, such that σ k = π are constant scaling factors and K := K sup k Φ k 2 = K , and doing the partitioning using simple interval boxes as in Remark 13, a worst-case iteration count can be obtained as Furthermore, if the global minimizer of Problem (1) happens to be a smooth (C ∞ ) function, the convergence rate can be expected to be of the form R(M, G ∞ ) = α exp(− β M), and Theorem 2 then predicts a worst-case iteration count as which is much more favorable.

Numerical case study
We consider the Hilbert space H := L 2 [0, T ] of square-integrable functions on the interval [0, T ], here with T = 10. Our focus is on the following nonconvex, infinitedimensional optimization problem with the functions f 1 and f 2 given by ∀t ∈ R, f 1 (t) = t 2 sin π t 2T + 1 and f 2 = ∂ f 1 ∂t .
Notice the symmetry in the optimization problem (21), as F(x) = F(− x) and x ∈ C if and only if −x ∈ C. Thus, if x * is a global solution point of (21), then − x * is also a global solution point. for interval boxes A ⊆ R M+1 , we use standard LMI relaxation techniques [20] here. At this point, we have all the elements needed for implementing Algorithm 1 for Problem (21). On selecting the termination tolerance ε = 10 −5 and the lifting parameter ρ = 1, Algorithm 1 terminates after less than 100 iterations and applies 8 lifting operations (starting with M = 1). The corresponding decrease in the gap between upper and lower bounds as a function of the lifted subspace dimension Mimmediately after each lifting operation-is shown on the left plot of Fig. 1. Upon convergence, the infimum of (21) is bracketed as − 0.16812 ≤ inf x∈C F(x) ≤ −0.16811 , and a corresponding ε-global solution x is reported on the right plot of Fig. 1; the symmetric function (− x) provides another ε-global solution for this problem. Overall, this case study demonstrates that the proposed branch-and-lift algorithm is thus capable of solving such non-convex and infinite-dimensional optimization problem to global optimality within reasonable computational effort.

Conclusions
This paper has presented a complete-search algorithm, called branch-and-lift, for global optimization of problems with a non-convex cost functional and a bounded and convex constraint sets defined on a Hilbert space. A key contribution is the determination of run-time complexity bounds for branch-and-lift that are independent of the number of variables in the optimization problem, provided that the cost functional is strongly Lipschitz-continuous with respect to a regular and bounded subset of that Hilbert space. The corresponding convergence conditions are satisfied for a large class of practically relevant problems in calculus of variations and optimal control. In particular, the complexity analysis in this paper implies that branch-and-lift can be applied to solve potentially non-convex and infinite-dimensional optimization problems with-out needing a-priori knowledge about the existence or regularity of minimizers, as the run-time bounds solely depend on the structural and regularity properties of the cost functional as well as the underlying Hilbert space and the geometry of the constraint set. This could pave the way for a new complexity analysis of optimization problems, whereby the "complexity" or "hardness" of a problem does not necessarily depend on their number of optimization variables. In order to demonstrate that these algorithmic ideas and complexity analysis are not of pure theoretical interest only, the practical applicability of branch-and-lift has been illustrated with a numerical case study for a problem of calculus of variations. The case study of an optimal control problem in [25] provides another illustration.