Abstract
We present a rigorous convergence analysis for cylindrical approximations of nonlinear functionals, functional derivatives, and functional differential equations (FDEs). The purpose of this analysis is twofold: First, we prove that continuous nonlinear functionals, functional derivatives, and FDEs can be approximated uniformly on any compact subset of a real Banach space admitting a basis by highdimensional multivariate functions and highdimensional partial differential equations (PDEs), respectively. Second, we show that the convergence rate of such functional approximations can be exponential, depending on the regularity of the functional (in particular its Fréchet differentiability), and its domain. We also provide necessary and sufficient conditions for consistency, stability and convergence of cylindrical approximations to linear FDEs. These results open the possibility to utilize numerical techniques for highdimensional systems such as deep neural networks and numerical tensor methods to approximate nonlinear functionals in terms of highdimensional functions, and compute approximate solutions to FDEs by solving highdimensional PDEs. Numerical examples are presented and discussed for prototype nonlinear functionals and for an initial value problem involving a linear FDE.
Introduction
A nonlinear functional is a map from a space of functions into the real line or the complex plane. Such map, which seems a rather abstract mathematical concept, plays a fundamental role in many areas of mathematical physics and applied sciences. In fact, nonlinear functionals were used, for example, by Wiener to describe Brownian motion mathematically [100], by Hohenberg and Kohn [47] to reduce the dimensionality of the Schrödinger equation in manybody quantum systems [56, 67], by Hopf to describe the statistical properties of turbulence [2, 48, 63], and by Bogoliubov to model systems of interacting bosons in superfluid liquid helium [12, 84]. Applications of nonlinear functionals to other areas of mathematical physics can be found in [3, 37, 52, 54, 59, 92].
Nonlinear functionals have also appeared in evolution equations known as functional differential equations (FDEs) [93]. A classical example in fluid dynamics is the Hopf equation [48, 66, 77]
which governs the dynamics of the characteristic functional
Here, u(x, t) represents a stochastic solution to the Navier–Stokes equation corresponding to a random initial state, and \({\mathbb {E}}\{\cdot \}\) is the expectation over the probability measure of such random initial state. Remarkably, the complexvalued nonlinear functional (2) encodes all statistical information of the stochastic solution to the Navier–Stokes equation. For this reason, Eq. (1) was deemed by Monin and Yaglom ( [63, Ch. 10]) to be “the most compact formulation of the general turbulence problem”, which is the problem of determining the statistical properties of the velocity and the pressure fields of the Navier–Stokes equations given statistical information on the initial state.^{Footnote 1} Another wellknown example of functional differential equation is the Schwinger–Dyson equation of quantum field theory [68, 105]. Such equation describes the dynamics of the generating functional of the Green functions of a quantum field theory, allowing us to propagate field interactions in a perturbation setting (e.g., with Feynman diagrams), or in a strong coupling regime. The Schwinger–Dyson functional formalism is also useful in studying statistical dynamics of classical systems described in terms of stochastic ordinary or partial differential equations^{Footnote 2} [52, 59, 69]. More recently, FDEs appeared in meanfield games [17] and meanfield optimal control [31, 79]. Meanfield games are optimization problems involving a very large (potentially infinite) number of interacting players. In some cases, it is possible to reformulate such optimization problems in terms of a nonlinear Hamilton–Jacobi FDE in probability density space. The standard form of such equation is [20]
where \(\rho (x)\) is a ndimensional probability density function supported on \(\Omega \subseteq {\mathbb {R}}^n\), \(\delta F /\delta \rho (x)\) is the firstorder functional derivative of F relative to \(\rho (x)\), and \({\mathcal {H}}\) is the Hamilton functional
Here, \(\Psi \) is a Hamilton function, and \(G([\rho ])\) is an interaction potential. More general FDEs of the type (3) have been recently derived in the context of unnormalized optimal PDF transport [40]. Meanfield theory is also useful in optimal feedback control of nonlinear stochastic dynamical systems and in deep learning. For instance, recent work of W. E and collaborators [31] laid the mathematical foundations of the population risk minimization problem in deep learning as a meanfield optimal control problem. Such meanfield optimal control problem yields a generalized version of the Hamilton–Jacobi–Bellman equation in a Wasserstein space, which is a nonlinear FDE (see Eq. (20) in [31]).
Computing accurate approximations of the solution to FDEs such as (1) or (3) is a longstanding problem in mathematical physics. In a recent Physics Report [93], we reviewed stateoftheart methods to approximate nonlinear functionals and FDEs. In particular, we discussed an approximation method, known as “cylindrical approximation”, in which nonlinear functionals and FDEs defined on function spaces admitting a basis are approximated by multivariate functions and multivariate partial differential equations (PDEs), respectively. The idea is, if a function space admits a basis, then any function in the space can be represented uniquely by projection coefficients onto the basis. Accordingly, nonlinear functionals defined on such a function space can be represented as multivariate functions of the coefficients. The objective of this paper is to provide a rigorous mathematical foundation for cylindrical approximations to nonlinear functionals, functional derivatives, and FDEs defined on Banach spaces admitting a basis. The purpose of this analysis is twofold: first, we prove that cylindrical approximations converge uniformly on compact subsets of real Banach spaces admitting a basis. Second, we prove that the convergence rate can be exponential in the number of projection coefficients. We also provide necessary and sufficient conditions for consistency, stability and convergence of cylindrical approximations to FDEs based on the Trotter–Kato approximation theorem [33, 41].
This paper is organized as follows. In Sect. 2 we briefly review the theory of nonlinear functionals defined on a Banach space, and recall the notions of continuity, compactness and differentiability. In Sect. 3 we specialize these concepts to nonlinear functionals defined on a real separable Hilbert space H. In Sect. 4 we introduce cylindrical approximations of nonlinear functionals and functional derivatives defined on a Hilbert space. Uniform convergence for both approximations is established in Sects. 5 and 6, provided the functional (or functional derivative) is defined on a compact subset of H. We also show that cylindrical approximations can converge exponentially fast for Fréchet differentiable functionals. In Sect. 7 we develop a selfconsistent convergence analysis of cylindrical approximations to linear FDEs in compact subsets of real separable Hilbert spaces. In Sect. 8 we outline the extension of the functional approximation theory we developed in Hilbert spaces to compact subsets of real Banach spaces admitting a basis. In Sect. 9 we provide numerical examples demonstrating convergence of cylindrical approximations of nonlinear functionals and a linear FDE. In particular, we study the Hopf equation corresponding to a linear advection problem evolving from a random initial state. The main findings are summarized in Sect. 10. We also include two brief Appendices where we discuss cylindrical approximations of functional integrals in real separable Hilbert spaces, and the notion of distance between function spaces.
Nonlinear functionals in Banach spaces
Let X be a Banach space. A nonlinear functional on X is a map F from X into a field \({\mathbb {F}}\). In this paper, \({\mathbb {F}}\) will either be the real line (\({\mathbb {R}}\)) or the complex plane (\({\mathbb {C}}\)). In general, the functional F does not operate on the entire Banach space X but rather on a subset set of X, which we denote as \(D(F) \subseteq X\) (domain of the functional)
As an example, consider
where \(C^{(1)}([0,1])\) is the space of continuously differentiable realvalued functions defined on [0, 1]. The map (6) associates to each function \(\theta \in C^{(1)}([0,1])\) the real number \(F([\theta ])\). Analysis of nonlinear functionals in Banach spaces is a welldeveloped subject [35, 49, 65, 83, 90]. In particular, classical definitions of continuity and differentiability that hold for realvalued functions can be extended to functionals. For instance,
Definition 2.1
(Pointwise continuity of functionals) A nonlinear functional \(F: D(F)\subseteq X \rightarrow {\mathbb {F}}\) is continuous at a point \(\theta \in D(F)\) if for any Cauchy sequence \(\{\theta _1, \theta _2, \ldots \}\) in D(F) converging to \(\theta \) (in the metric of X) we have that the sequence \(\{F([\theta _1]), F([\theta _1]), \ldots \}\) converges to \(F([\theta ])\) (in the metric of \({\mathbb {F}}\)), i.e.,
Example 1 The functional (6) is continuous at every point \(\theta \in C^{(1)}([0,1])\) relative to the norm
Definition 2.2
(Uniform continuity of functionals) The functional \(F: D(F)\subseteq X \rightarrow {\mathbb {F}}\) is said to be uniformly continuous on the domain D(F) if for every \(\epsilon >0\) there exists \(\delta >0\) such that the inequality \( F([\theta _1])F([\theta _2])<\epsilon \) holds for all points \(\theta _1,\theta _2\in D(F)\) satisfying \(\left\ \theta _1\theta _2\right\ _X\le \delta \).
Note that the definition of continuity and uniform continuity of a functional depends on how we measure the distance between elements of Banach space X.
Definition 2.3
(Compactness and complete continuity of functionals) The functional \(F: D(F)\subseteq X \rightarrow {\mathbb {F}}\) is said to be compact on the domain D(F) if it maps every bounded subset of D(F) into a precompact subset set of \({\mathbb {F}}\), i.e., a subset whose closure is compact. The functional \(F([\theta ])\) is called completely continuous on D(F) if it is continuous and compact.
It is clear that a continuous functional \(F([\theta ])\) is completely continuous if and only if for every bounded sequence \(\{\theta _n\}\) in D(F) we have that the sequence \(\{F([\theta _n])\}\) has a convergent subsequence. As is well known, continuous functions defined on a closed and bounded subset of \({\mathbb {R}}^n\) are always uniformly continuous and bounded. This is not the case with functionals defined on Banach spaces. In fact, uniform continuity of a functional \(F([\theta ])\) on a closed and bounded set \(S\subseteq D(F)\), say the unit sphere \(S=\{\theta \in C^{(1)}([0,1]): \left\ \theta \right\ \le 1\}\),^{Footnote 3} is not sufficient to guarantee that the functional is bounded [90, p. 18]. However, if the functional \(F([\theta ])\) is uniformly continuous on D(F), then it maps compact sets into compact sets. Moreover, if F is completely continuous on a bounded set \(K\subseteq D(F)\) (open or closed), then F is bounded on K. This is obvious since the definition of complete continuity of F implies that bounded sets are mapped into relatively compact sets, which are bounded.
Example 2 Consider the nonlinear functional
in the Banach space of Lipschitz continuous periodic functions in \([0,2\pi ]\),
As is well known, the Fourier series of any element \(\theta \in D(F)\) defines a sequence of partial sums \(\{\theta _m\}\in D(F)\) that converge uniformly to \(\theta \) (see [50]). Thanks to such uniform convergence result, we have
where the last equality follows from [7, Theorem 10]. Hence, the functional (9) is continuous on \(C^{(0)}_{\text {Lip}}([0,2\pi ])\). Moreover, F sends any bounded subset of such function space into a precompact subset of the real line, and therefore, F is completely continuous.
Differentials and derivatives of nonlinear functionals
Let us consider a nonlinear functional \(F: D(F)\subseteq X \rightarrow {\mathbb {F}}\), where D(F) is an open set. We say that F is Gâteaux differentiable at a point \(\theta \in D(F)\) if the limit
exists and is finite for all \(\eta \in D(F)\). The quantity \(\mathrm{d}F_\eta ([\theta ])\) is known as Gâteaux differential of F in the direction of \(\eta \) [83, 90]. Under rather mild conditions (see, e.g., [90, p. 37]), such differential can be represented as a linear operator acting on \(\eta \) [65, 92]. Such linear operator is known as the Gâteaux derivative of F at \(\theta \), and it will be denoted by \(F'([\theta ])\)
The Fréchet differential, on the other hand, is defined as the term \(\mathrm{d}F([\theta ],\eta )\) in the series expansion
It is well known that if \(F([\theta ])\) has a continuous Gâteaux derivative on D(F), then F is Fréchet differentiable on D(F), and these two derivatives coincide [90, p. 41]. In this paper, we consider nonlinear functionals F that are continuously Gâteaux differentiable in D(F). Hence, we will not need to distinguish between Fréchet and Gâteaux derivatives.
There has been significant research activity on obtaining the minimal conditions under which a nonlinear functional is Gâteaux or Fréchet differentiable. It turns out that there are reasonably satisfactory results on Gâteaux differentiablility of Lipschitz functionals. For instance,
Theorem 2.1
(Gâteaux differentiablility of Lipschitz functionals [4, 57, 58]) Let X be a separable Banach space. Then, every real or complexvalued Lipschitz functional F from an open set \(D(F)\subseteq X\) is Gâteaux differentiable outside a Gaussnull set.^{Footnote 4}
Results on Fréchet differentiability are more rare, and usually much harder to prove [57]. For instance, we have
Theorem 2.2
(Fréchet derivatives of Lipschitz functionals [71]) Let K be a compact subset of a Hilbert space H. If \(F([\theta ])\) is realvalued and locally Lipschitz on K, then F is Fréchet differentiable on a dense subset of K.
Note that Theorem 2.2 does not imply that F is Fréchet differentiable everywhere on K. On the other hand, a continuously differentiable functional F, which is also completely continuous in the sense of Definition 2.3, has completely continuous Gâteaux and Fréchet derivatives ( [90, p.51]). As we will see in Sect. 3, continuously differentiable nonlinear functionals on compact metric spaces are also compact, and have compact Fréchet derivative. We emphasize that it is also possible to define Gâteaux and Fréchet differentiability directly in terms of bounded linear operators. For instance, Lindenstrauss and Preiss [57, p. 258] define F as Gâteaux differentiable if there exists a bounded linear operator \(F':D(F)\rightarrow {\mathbb {F}}\) such that for every \(\eta \in D(F)\)
Clearly, this definition is more strict than (12)–(13), since it does not allow for unbounded derivative operators \(F'([\theta ])\).
In the context of nonlinear functionals defined on spaces of functions, it is convenient to define another type of functional derivative, namely
provided the limit exists. The quantity \(\delta F([\theta ])/\delta \theta (x)\) is known as firstorder functional derivative of \(F([\theta ])\) with respect to \(\theta (x)\) [45, p. 309]). Functional derivatives are used extensively in many areas mathematical physics, e.g., in stochastic dynamics [37, 44, 52, 95], turbulence modeling [30, 63, 66], and quantum field theory [67, 105].
If the Fréchet derivative \(F'([\theta ])\) admits an integral representation,^{Footnote 5} then it is possible to establish a onetoone correspondence between \(\delta F([\theta ])/\delta \theta (x)\) and \(F'([\theta ])\). For instance, if \(\ell (\eta ) = F'([\theta ])\eta \) is a bounded linear functional in a Hilbert space H, and \(F'([\theta ])\) is continuous in \(D(F)\subseteq H\), then the Riesz representation theorem guarantees that there exists a unique element \(\delta F([\theta ])/\delta \theta (x)\in H\) such that
Here, \((\cdot ,\cdot )_H\) denotes the inner product in H. As we will see in Sect. 3, the Fréchet derivative of a continuous nonlinear functional defined on a compact subset of a real separable Hilbert space is a completely continuous linear operator, i.e., continuous and compact. In this case, the Riesz representation (16) holds and we have a onetoone correspondence between \(F'([\theta ])\) and \(\delta F([\theta ])/\delta \theta (x)\).
More generally, if \(\Omega \) is a locally compact Hausdorff space, e.g., an open or closed subset of \({\mathbb {R}}^n\), and \(F'([\theta ])\) is a bounded linear operator from \(C_c^{(0)}(\Omega )\) (space of compactly supported continuous functions on \(\Omega \)) into \({\mathbb {R}}\), then there exists a unique finite regular signed measure^{Footnote 6}\(\mu ([\theta ],x)\) on the Borel subsets of \(\Omega \) such that
(see [85, p. 324], [28, p. 4] or [35, 103]). In addition, if \(\mu ([\theta ],x)\) is absolutely continuous with respect to x, then there exists a Radon–Nikodym derivative, i.e., a functional density \(\delta F([\theta ])/\delta \theta (x)\), such that
Under these conditions, the Fréchet derivative of F admits the Lebesgue integral representation
We emphasize that (16) and (19) can be considered as infinitedimensional generalizations of the concept of directional derivative of a multivariate function f(x), in which the dot product between the gradient \(\nabla f\) and a vector \(\eta \) is now replaced by the inner product of \(\delta F([\theta ])/\delta \theta (x)\) and \(\eta (x)\). By analogy, the quantity \(\delta F([\theta ])/\delta \theta (x)\) can be thought of as an infinitedimensional gradient. Note that such gradient is a nonlinear functional of \(\theta \) and a function of x. Higherorder Fréchet and functional derivatives can be defined similarly [44, 93].
Example 1 The Fréchet derivative of the nonlinear functional (9) is obtained as
where both \(\theta \) and \(\eta \) are in the space (10) of Lipschitz continuous periodic functions in \([0,2\pi ]\). Clearly, equation (20) can be written as
From this expression we see that the signed measure \(\mathrm{d}\mu ([\theta ],x)\) appearing in equation (17) in this case has a density, which coincides with the firstorder functional derivative
Such derivative is a distribution in x and a continuous functional of \(\theta \). The Fréchet differential (20) is a linear functional in \(\eta \), bounded in the \(C^{(0)}\) norm. In fact,
However, such functional is unbounded in \(L^2([0,2\pi ])\), and therefore, (21) is not based on the Riesz representation (16), but rather on (19). To show this, we just need to prove that (20) admits an integral representation with kernel that is not in \(L^2([0,2\pi ])\). To this end, let us represent \(\eta \) relative to any orthonormal trigonometric basis \(\{\varphi _0,\varphi _1,\ldots \}\) of \(L_p^2([0,2\pi ])\) (space of square integrable periodic functions in \([0,2\pi ]\))
The series (24) converges in the \(L_p^2([0,2\pi ])\) sense, and also pointwise since \(\eta \) is continuous [50]. A substitution of (24) into (20) yields
It straightforward to show that
and therefore, the function v(x) in (25) is not an element of \(L_p^2([0,2\pi ])\). Indeed, v(x) is the trigonometric series expansion of the Dirac delta function \(\delta (x\pi )\), which is not in \(L_p^2([0,2\pi ])\).
Nonlinear functionals defined on compact subsets of real separable Hilbert spaces
In this section, we study the mathematical properties of nonlinear functionals defined on compact subsets of real separable Hilbert spaces. As we will see in subsequent sections, this is a very important class of functionals which allows us to build an effective approximation theory based on orthogonal projections.
Before we present such theory, let us briefly review the notion of bounded, closed, compact and precompact sets. Consider a metric space of functions X, e.g., a Hilbert or a Banach space, and a subset \(K\subseteq X\). We say that K is bounded if for all \(\theta \in K\) we have \(\left\ \theta \right\ _X \le M\), where M is a finite real number, and \(\Vert \cdot \Vert _X\) is the norm in X. The set K is said to be closed if any convergent sequence in K has a limit in K. An example of a closed and bounded subset of the Hilbert space \(L^2([0,1])\) is the unit sphere \(S=\{\theta \in L^2([0,1]): \left\ \theta \right\ _{L^2([0,1])} \le 1\}\). We say that the subset \(K\subseteq X\) is compact if every open cover of K has a finite subcover.
There are several equivalent characterizations of compactness in metric spaces. For instance, a subset K of a metric space X is compact if and only if every sequence in K has a bounded subsequence whose limit is in K [49, §1.7]. The set K is said to be precompact if its closure is compact, meaning that every sequence in K has a convergent subsequence whose limit is in X (not in K). Closed and bounded function spaces are not necessarily compact, since we can define sequences that do not have convergent subsequences. An example is the unit sphere S mentioned above. On the other hand, a compact set is always bounded and closed. A useful characterization of precompactness in real separable Hilbert spaces is the following:
Theorem 3.1
(Compact subsets of real separable Hilbert spaces) A subset E of a real separable Hilbert space H is precompact if and only if it is bounded, closed, and for any (one) orthonormal basis \(\{\varphi _1,\varphi _2,\ldots \}\) of H, and any \(\epsilon >0\) there exists a natural number m such that^{Footnote 7}
for all \(\theta \in E\). The closure of B in H, which we denote as \({\overline{E}}\), is compact.
The proof of Theorem 3.1 can be found in [62, p. 76] (Proposition 3.8). Note that neither precompact nor compact subsets of a vector space can be vector spaces, since precompactness implies boundedness. Hereafter, we provide two simple examples of compact subsets of real separable Hilbert spaces.
Example 1 Let \(H_p^{s}([0,2\pi ])\) be the Sobolev space of weakly differentiable (up to degree \(s\ge 1\)) periodic functions in \([0,2\pi ]\). The set
where \(L_p^{2}([0,2\pi ])\) is the Lebesgue space of periodic functions in \([0,2\pi ]\) and \(\rho >0\) is the radius of the Sobolev sphere, is a compact subset of \(L_p^{2}([0,2\pi ])\). Indeed, by expanding an arbitrary element \(\theta \in K\) in a Fourier series we obtain, for any \(1\le q\le s\) (see [46, p. 35])
At this point it is clear that for any given \(\epsilon >0\) there exists a natural number m such that the righthand side of (29) can be made smaller than \(\epsilon \) for any \(\theta \in K\). In other words, the equismall tail condition (27) is satisfied. Moreover, in (28) we take the closure of the Sobolev sphere in \(L_p^2([0,2\pi ])\), which makes K is a compact subset of \(L_p^2([0,2\pi ])\).
Example 2 A closed sphere with radius \(\rho \) in \(H_w^s([1,1])\) (weighted Sobolev space of degree s), is a precompact subset of \(L_w^2([1,1])\) (weighted Lebesgue space of functions in \([1,1]\)). Hence,
is a compact subset of \(L_w^2([1,1])\). This claim is based on the following wellknown spectral convergence result [46, p. 109]
where \(\varphi _k(x)\) here are ultraspherical polynomials. By combining (31) with (30), we see that the equismall tail condition
is satisfied \(\forall \theta \in K\).
The compact subsets we discussed in Example 1 and Example 2 are particular instances of a general compact embedding result known as Rellich–Kondrachov theorem [1, §6]. Such theorem states that the Sobolev space of functions \(W^{k,p}(\Omega )\) defined on a compact subset \(\Omega \subseteq {\mathbb {R}}^n\) with differentiable boundary is compactly embedded in \(W^{l,q}(\Omega )\), provided \(k>l\) and \(kp/n>lq/n\). This means that there exists a compact linear operator \(T:W^{k,p}(\Omega )\rightarrow W^{l,q}(\Omega )\) that maps bounded subsets of \(W^{k,p}(\Omega )\) into precompact subsets of \(W^{l,q}(\Omega )\). In Example 1 and Example 2, we have that \(H^s = W^{s,2}\) and \(L^2= W^{0,2}\) Hence, the Rellich–Kondrachov embedding in this case reduces to the statement that the Sobolev space \(H^s\) is compactly embedded in \(L^2\) for all \(s>0\).
Continuous nonlinear functionals defined on a compact subset K of a metric space have nice mathematical properties. First of all, they are bounded since they map compact sets into a closed and bounded subset of \({\mathbb {R}}\) or \({\mathbb {C}}\). Moreover, by the Heine–Cantor Theorem we have that any continuous functional defined on a compact set is necessarily uniformly continuous and bounded [49]. If the functional is realvalued, this means that the maximum and the minimum are attained at points within K. We also recall that closed subsets of compact sets are necessarily compact. Hence, a continuous functional on K maps any closed subset of K into a closed and bounded subset of \({\mathbb {R}}\) or \({\mathbb {C}}\). Such functional is necessarily compact,^{Footnote 8} i.e., completely continuous (see Definition 2.3).
Fréchet and functional derivatives
Next, we show that the Fréchet derivative of a continuous nonlinear functional defined on a compact subset of a real separable Hilbert space is a compact linear operator.
Lemma 3.1
(Compactness of the Fréchet derivative) Let K be a compact subset of a real separable Hilbert space H, and let \(F([\theta ])\) be a continuous real or complexvalued functional on H. If the Fréchet derivative \(F'([\theta ^*])\) exists at \(\theta ^*\in K\), then it is a compact linear operator.^{Footnote 9}
Proof
Continuous functionals on compact metric spaces are necessarily completely continuous (see Definition 2.3). To prove the Lemma, we proceed by contradiction. To this end, suppose that \(F'([\theta ^*])\) is not compact. Then, it is possible to find \(\epsilon > 0\) and a sequence \(\{\theta _k\}\in K\subseteq H\) such that \(\Vert \theta _k\Vert _H\le 1\) and
for all \(k\ne j\). By definition of Fréchet derivative at \(\theta ^*\), we have
for all \(\eta \in K\) with reasonably small norm, say \(\left\ \eta \right\ _H\le \delta \). In particular, we can choose \(\delta \) such that
Next, choose \(\tau \) small enough so that \((\theta ^*+\tau \theta _k)\in K\) and \(\left\ \tau \theta _k\right\ \le \delta \) for all \(k\in {\mathbb {N}}\). For such functions, we have
This means that the functional F is not completely continuous. In fact, the inequality (36) implies that it is not possible to extract a convergent subsequence from the sequence \(\{F([\theta ^*+\tau \theta _k])\}\), with \(\theta _k\) bounded. This proves the lemma. \(\square \)
Lemma 3.2
(Representation of the Fréchet derivative) Let K be a compact subset of a real separable Hilbert space H, and let \(F([\theta ])\) be a continuous real or complexvalued functional on K. If the Fréchet derivative of \(F([\theta ])\) exists at \(\theta ^*\in K\), then \(F'([\theta ^*])\) admits the unique integral representation
where \(\delta F([\theta ^*])/\delta \theta (x)\in H\) is the firstorder functional derivative (15).
Proof
We have seen in Lemma 3.1 that the Fréchet derivative of a continuous nonlinear functional defined on a compact subset of a real separable Hilbert space is a compact linear operator. Hence, \(\ell ([\eta ])=F'([\theta ^*])\eta \) is a bounded linear functional in H. By applying the Riesz representation theorem (37) to \(\ell \), we conclude that there exists a unique element \(\delta F([\theta ^*])/\delta \theta (x)\in H\) such that (37) holds. This proves the Lemma. \(\square \)
Example 1 Let \(H=L^2_p([0,2\pi ])\) and K be the Sobolev sphere (28), which includes its closure in \(L^2_p([0,2\pi ])\). Consider the nonlinear functional
The Fréchet derivative of (38) is
For fixed \(\theta \in K\), the linear functional \(F'([\theta ]) \eta \) is bounded in \(L^2_p([0,2\pi ])\). In fact, by the Cauchy–Schwarz inequality we have
for all \(\eta \in H\). Therefore, for each \(\theta \in K\) Lemma 3.1 holds, i.e., there exists a unique firstorder functional derivative
which is an element of \(L^2_p([0,2\pi ])\) (as a function of x).
Cylindrical approximation of nonlinear functionals in real separable Hilbert spaces
Let H be a real separable Hilbert space with inner product \((\cdot ,\cdot )_H\). Any element \(\theta \in H\) can be represented uniquely in terms of an orthonormal basis \(\{\varphi _1,\varphi _2,\ldots \}\) as
where the series converges in the norm \(\Vert \cdot \Vert _H\) induced by the inner product \((\cdot ,\cdot )_H\). We introduce the projection operator \(P_m\), which truncates the series expansion (42) to m terms
Clearly, \(P_m\) is an operator from H into the finitedimensional space
With this notation, we can represent any nonlinear functional \(F([\theta ])\) in H as a function depending on an infinite (countable) number of variables. To this end, we substitute (42) into \(F([\theta ])\) to obtain
A simple way to approximate this functional is to restrict its domain to the range of the projection \(P_m\), which is the finitedimensional space \(D_m\) in (44). This reduces \(F([\theta ])\) to a multivariate function \(f(a_1,\ldots ,a_m)\), which depends on as many variables as the number of basis elements of \(D_m\). Specifically, we have
In the theory of stochastic processes, the set
where B is a Borel set of \({\mathbb {R}}^m\), is known as cylindrical set (see [91, p. 55] or [88, p. 45]). Therefore, functionals of the form (46), i.e.,
where f is a multivariate function, are often referred to as cylindrical (or cylinder) functionals^{Footnote 10} [6, 39]. Such functionals play a fundamental role, e.g., in the approximation of functional integrals arising in quantum field theory [23, 105] (see also “Appendix A”).
Definition 4.1
(Cylindrical approximation of nonlinear functionals) Let H be a real separable Hilbert space, \(P_m\) the projection operator (43), and F a nonlinear functional on H. We will call \(F([P_m\theta ])\) cylindrical approximation of \(F([\theta ])\).
We will see in Sect. 5 that \(F([P_m \theta ])\) (Eq. (48)) converges uniformly to \(F([\theta ])\) (Eq. (45)) as m goes to infinity in any compact subset of a real separable Hilbert space H.
Next, we study the representation of the Fréchet and the firstorder functional derivatives. If \(F([\theta ])\) is Fréchet differentiable at \(\theta \in H\) with continuous Fréchet derivative \(F'([\theta ])\), then \(\ell (\eta )=F'([\theta ])\eta \) is a bounded linear functional. Hence, by Riesz’s representation theorem, there exists a unique element of H, which we denoted by \(\delta F([\theta ])/\delta \theta (x)\), such that
As we pointed out in Lemma 3.2, \(\delta F([\theta ])/\delta \theta (x)\) coincides with the firstorder functional derivative (15). Such derivative is an element of H, and therefore, it can be represented in terms of the orthonormal basis \(\{\varphi _1,\varphi _2,\ldots \}\) as
A differentiation of (45) with respect to \(a_k\) yields^{Footnote 11}
This means that the partial derivative of \(f(a_1,a_2,\ldots )\) with respect to \(a_k=(\theta ,\varphi _k)_H\) is the projection of the firstorder functional derivative of F onto the basis element \(\varphi _k\). By substituting (52) into (50) we obtain
This expression emphasizes that the firstorder functional derivative (53) is essentially a “dot product” between the (infinitedimensional) gradient of f and the (infinitedimensional) vector of basis elements. Evaluating (50) on the finitedimensional function space \(D_m\) yields the cylindrical approximation
Here, f depends solely on the variables \((a_1,\ldots , a_m)\). If the functional derivative \(\delta F([P_m \theta ])/\delta \theta (x)\) is an element of \( D_m\) (as a function of x), then the second term at the righthand side of (54) is clearly equal to zero.
Example 1 Let \(F([\theta ])\) be a continuously differentiable functional on a real separable Hilbert space H, and let \(P_m\) be the projection operator (43). Then, from Eqs. (49) and (52) it follows that
where f is the multivariate function defined in (46).
Convergence analysis of cylindrical approximations: continuous nonlinear functionals
In this section, we perform a convergence analysis for nonlinear functional approximations of the form (46) in compact subsets of real separable Hilbert spaces. We begin by recalling an approximation result first obtained by Prenter in [72].
Lemma 5.1
(Uniform convergence of cylindrical functional approximations [72, Lemma 5.3]) Let H be a real separable Hilbert space, K a compact subset of H, and \(P_m : H \rightarrow D_m\) the projection operator (43). If F is a continuous functional on H, then the sequence \(F([P_m\theta ])\) converges uniformly to \(F([\theta ])\) on K, i.e., for all \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that
for all \(\forall m\ge m_\epsilon \) and for all \(\theta \in K\)
The compactness hypothesis of the subset K in Lemma 5.1 can be replaced by the weaker assumption that \(K\subseteq H\) is bounded (e.g., a sphere), and F is uniformly continuous with respect to the socalled Stopology (see [9] for details).
Convergence rate
Lemma 5.1 establishes uniform convergence of the functional \(F([P_m\theta ])\) to \(F([\theta ])\) on compact subsets of real separable Hilbert spaces. We will now address how fast the approximation \(F([P_m\theta ])\) converges to \(F([\theta ])\). We will show that for continuously differentiable functionals (functionals with continuous Fréchet derivative) defined on compact, convex subset of real separable Hilbert spaces, \(F([P_m \theta ])\) converges to \(F([\theta ])\) at the same rate at which \(P_m \theta \) converges to \(\theta \) in H. This results follows from the wellknown mean value theorem.
Theorem 5.1
(Mean value theorem) Let F be a realvalued continuously differentiable functional on a compact convex subset K of a real separable Hilbert space H. Then, for all \(\theta _1, \theta _2\in K\) the following estimate holds:
where \(F'([\eta ])\) denotes the firstorder Fréchet derivative of F.
We omit the proof as this is a wellknown result. We simply recall that since \(F'([\theta ])\) is the Fréchet derivative of a continuously differentiable functional on a compact metric space, we have that \(F'([\theta ])\) is a compact linear operator (Theorem 3.1), and therefore, it is bounded on K. Hence,
We also emphasize that it is possible to relax the assumptions in Theorem 5.1. For instance, it is possible to drop the requirement that F is continuously differentiable and leverage the fact that for each \(\epsilon >0\) and any pair of points \(\theta _1,\theta _2\in K\) there exists a point \(\theta ^* \in K\) in which F is Fréchet differentiable, and
provided the line \(\theta _t= t\theta _1+(1t)\theta _2\) is in K for all \(t\in [0,1]\). However, for the purpose of the present paper we shall simply restrict the class of nonlinear functionals we study to continuously differentiable nonlinear functionals. This allows us to obtain the following convergence rate result using the mean value Theorem 5.1.
Lemma 5.2
(Convergence rate of cylindrical functional approximations) Let F be a realvalued, continuously differentiable functional on a compact and convex subset K of a real separable Hilbert space H. Then, for all \(\theta \in K\) and for any finitedimensional projection \(P_m\) of the form (43), we have
In particular, \(F([P_m\theta ])\) converges to \(F([\theta ])\) for all \(\theta \in K\) at the same rate as \(P_m \theta \) converges to \(\theta \) in H.
The proof follows directly from the mean value Theorem 5.1 by setting \(\theta _1=\theta \) and \(\theta _2=P_m\theta \).
Example 1 Consider the compact subset \(K\subseteq L^2_w([1,1])\) defined in equation (30). Then, for any continuously differentiable functional F on \(L^2_w([1,1])\), we have
where \(P_m\) is a projection onto ultraspherical polynomials in \([1,1]\), and C is the (finite) constant
Here, \(C_2\) and \(\rho ^2\) are defined in (32). If \(\theta \) is infinitely differentiable, then \(F([P_m \theta ])\) converges to \(F([\theta ])\) exponentially fast in m.
Convergence analysis of cylindrical approximations: Fréchet and functional derivatives
In this section, we study convergence of cylindrical approximations of \(F'([\theta ])\) and \(\delta F([\theta ])/\delta \theta (x)\) in a compact subset K of a separable real Hilbert space H. We begin with the following
Theorem 6.1
(Uniform approximation of firstorder Fréchet derivatives) Let H be a real separable Hilbert space, K a compact subset of H, and \(P_m:H \rightarrow D_m\) the projection operator (43). If F is continuously differentiable on K with Fréchet derivative \(F'([\theta ])\), then the sequence of operators \(F'([P_m \theta ])\) converges uniformly to \(F'([\theta ])\). In other words, for all \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that
for all \(m\ge m_\epsilon \), and for all \(\theta \in K\).
Proof
Let us define the functional \(G_\eta ([\theta ])=F'([\theta ])\eta \). For each fixed \(\eta \in H\), we have that \(G_\eta ([\theta ])\) is nonlinear and continuous in \(\theta \). Hence, we can apply Theorem 5.1 to conclude that \(G_\eta ([P_m \theta ])\) converges uniformly to \(G_\eta ([\theta ])\), i.e., that for each \(\epsilon _\eta >0\) and there exists \(m_\eta \in {\mathbb {N}}\) such that for all \(m \ge m_\eta \)
Since F is a continuously differentiable functional on a compact metric space, the Fréchet derivative \(F'([\theta ])\) is a compact linear operator (Theorem 3.1) on H for each \(\theta \in K\). This means that for each fixed \(\theta \in K\) the linear functional \(G_\eta ([\theta ])  G_\eta ([P_m \theta ])\) is bounded
By combining (63) and (64) we conclude that for each \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such for
for all \(m\ge m_\epsilon \) and for all \(\theta \in K\). This proves the theorem. \(\square \)
Next, we study convergence of the firstorder functional derivative (15). This is relatively straightforward given the convergence result we just obtained in Theorem 6.1. In fact, the linear functional \(F'([\theta ])\eta \) is bounded for each \(\theta \) in the compact set \(K\subseteq H\), and therefore, it admits the Riesz integral representation
where \((\cdot ,\cdot )_H\) is the inner product in H. Uniform convergence of \(F'([P_m\theta ])\) to \(F'([\theta ])\) for all \(\theta \) in the compact set \(K\subseteq H\) implies that for every \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that
and for all \(m\ge m_\epsilon \).
Lemma 6.1
(Uniform approximation of firstorder functional derivatives) Let H be a real separable Hilbert space, K a compact subset of H, \(\theta \in K\) and \(P_m: H \rightarrow D_m\) the projection (43). If F is continuously differentiable on K with Fréchet derivative \(F'([\theta ])\), then the sequence \(\delta F([P_m\theta ])/\delta \theta (x)\) converges uniformly to \(\delta F([\theta ])/\delta \theta (x)\). In other words, for all \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that for all \(m\ge m_\epsilon \)
Proof
Consider the linear functional of \(\eta \in H\)
It is well known that the norm of (69) is
By definition, \(M([\theta ]\)) is the smallest number such that
This observation together with (67) allows us to conclude that \(M([\theta ])<\epsilon \) for all \(\theta \in K\). This proves the theorem. \(\square \)
Convergence rate
Let us assume that \(G_\eta ([\theta ])=F'([\theta ])\eta \) is continuously Fréchet differentiable with respect to \(\theta \) in H. Denote by \(G'_\eta ([\theta ])\) the firstorder Fréchet derivative and let K be a compact convex subset of H. By applying the mean value Theorem 5.1, we obtain
The Fréchet derivative of \(G_\eta ([\theta ])\) can be written as^{Footnote 12}
If we divide (72) by \(\left\ \eta \right\ _H\) (\(\eta \ne 0\)) and take the supremum over \(\eta \in H\), we obtain
where
The symmetric bilinear form \(F'' ([\theta ])\) is continuous on \(H \times H\), and therefore, it is bounded. Moreover, \(\left\ F'' ([\theta ])\right\ \) is continuous in \(\theta \) and attains its minimum and maximum values in any compact set \(K\subseteq H\). By equation (74) this implies that the Fréchet derivative \(F'([P_m\theta ])\) converges to \(F'([\theta ])\) in K at the same rate as \(P_m\theta \) converges to \(\theta \) in H. We summarize these results in the following lemma.
Lemma 6.2
(Convergence rate of firstorder Fréchet derivatives) Let H be a real separable Hilbert space, and let \(F([\theta ])\) be a nonlinear functional with continuous first and secondorder Fréchet derivatives. Then, for all \(\theta \) in a compact convex subset K of H and for any finitedimensional projection \(P_m\) of the form (43), we have
In particular, \(F'([P_m\theta ])\) converges uniformly to \(F'([\theta ])\) in K at the same rate as \(P_m \theta \) converges to \(\theta \) in H.
Convergence rate results for higherorder Fréchet derivatives can be obtained in a similar way.
Approximation of linear functional differential equations
Let \({\mathcal {F}}(H)\) denote a Banach space of nonlinear functionals from a real separable Hilbert space H into \({\mathbb {R}}\) or \({\mathbb {C}}\). In this section, we develop necessary and sufficient conditions which guarantee that the solution to linear functional differential equations (FDEs) of the form
can be approximated by the solution of suitable finitedimensional linear partial differential equations. Equation (77) is a linear abstract evolution equation (Cauchy problem) in the Banach space \({\mathcal {F}}(H)\) [41]. The linear operator \({\mathcal {L}}([\theta ])\) is assumed to be in \({\mathcal {C}}({\mathcal {F}})\), which is the set of closed, densely defined and continuous linear operators on \({\mathcal {F}}(H)\). Note that \({\mathcal {L}}([\theta ])\) can be unbounded. To construct the approximation scheme for the FDE (77), we consider the following cylindrical approximation of the solution functional F
where \(P_m\) is the projection operator (43). We have seen in Sect. 5 that \(F_m([\theta ],t)= f(a_1,\ldots ,a_m,t)\) is a multivariate function in the m variables \(a_k=(\theta ,\varphi _k)_H\) (\(k=1,\ldots , m\)) which converges uniformly to \(F([\theta ],t)\) in every compact subset of H, for any fixed time t. From a functional analysis perspective, \(F_m\) is an element of a Banach space of functionals on H, which we denote by \({\mathcal {F}}_m(H)\). With this notation, we see that the functional approximation (78) is essentially induced by the application of a continuous linear operator \(B_m:{\mathcal {F}}(H)\rightarrow {\mathcal {F}}_m(H)\) defined as
Using the operator \(B_m\), we perform the following decomposition of the righthand side of (77)
where \({\mathcal {L}}_m([\theta ])\) is a linear operator acting on the mdimensional function \(F_m([\theta ],t)=f(a_1,\ldots ,a_m,t)\), and \(R_m\) is a functional residual. The operator \({\mathcal {L}}_m([\theta ])\) can be unbounded. As an example, let \(H=L_p^2([0,2\pi ])\) (space of square integrable periodic functions in \([0,2\pi ]\)) and consider
A substitution of (46) and (54) into (81) yields
where \(a_j=(\theta ,\varphi _j)_{H}\) (\(j=1,\ldots ,m\)). Note that \({\mathcal {L}}_m([\theta ])\) in (82) is a linear firstorder partial differential operator with nonconstant coefficients.
Definition 7.1
(Consistency) A sequence of linear operators \(\{{\mathcal {L}}_m\}\in {\mathcal {C}}({\mathcal {F}}_m)\), is said to be consistent (or compatible) with a linear operator \({\mathcal {L}}\in {\mathcal {C}}({\mathcal {F}})\) if for every \(F\in D({\mathcal {L}})\)^{Footnote 13} there exists a sequence \(F_m\in D({\mathcal {L}}_m)\) such that
as \(m\rightarrow \infty \). Moreover, if \(\left\ {\mathcal {L}}_m F_m  {\mathcal {L}} F\right\ ={\mathcal {O}}(m^{p})\), then we say that the sequence \(\{{\mathcal {L}}_m\}\) is consistent with \({\mathcal {L}}\) to order p.
Lemma 7.1
(Consistency of cylindrical approximations to FDEs) Let H be a real separable Hilbert space. Consider a functional \(F\in {\mathcal {F}}(H)\) and a densely defined closed linear operator \({\mathcal {L}}\in {\mathcal {C}}({\mathcal {F}})\). If \({\mathcal {L}}([\theta ])F([\theta ])\) is continuous in \(\theta \), then the sequence of operators \(\{{\mathcal {L}}_m\}\) defined in (80) is consistent with \({\mathcal {L}}\) on every compact subset K of H, provided \( \left\ R_m([\theta ])\right\ \rightarrow 0\) as \(m\rightarrow \infty \) for all \(\theta \in K\).
Proof
Equation (80) implies that
Since the functional \({\mathcal {L}}([\theta ])F([\theta ]) \) is continuous in \(\theta \), we can now use the uniform approximation Theorem 5.1 and claim that for any \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that
where K is a compact subset of H. A substitution of (85) into (84) yields
Hence, if \(\left R_m([\theta ]) \right \rightarrow 0\) for all \(\theta \in K\) as \(m\rightarrow \infty \), then \({\mathcal {L}}([\theta ])F([\theta ]) \rightarrow {\mathcal {L}}_m([\theta ])F_m([\theta ])\) for all \(\theta \) in K. By Theorem 5.1 we also have that \(F_m\rightarrow F\) on K. Hence the sequence \(\{{\mathcal {L}}_m\}\) is a consistent approximation of \({\mathcal {L}}\). \(\square \)
Corollary 7.1
Under the same assumptions of Lemma 7.1, if, in addition, K is convex, \({\mathcal {L}}([\theta ])F([\theta ])\) is continuously Fréchet differentiable in K, and \(\left\ R_m([\theta ])\right\ ={\mathcal {O}}( \left\ \theta P_m\theta \right\ _H)\), then \(\{{\mathcal {L}}_m\}\) is consistent with \({\mathcal {L}}\) to the same order as \(P_m\theta \) converges to \(\theta \) in H.
Proof
By using the mean value Theorem 5.1 and Eq. (84), we immediately conclude that
Hence, \({\mathcal {L}}_m\) is consistent with \({\mathcal {L}}\) to the same order as \(P_m\theta \) converges to \(\theta \) in K. \(\square \)
Example 1 Let \(H=L_p^2([0,2\pi ])\) be the space of square integrable periodic functions in \([0,2\pi ]\), \(\{\varphi _k\}\) an orthonormal Fourier basis in H, and \(K\subseteq H\) the Sobolev sphere (28) (together with its closure in H). We have seen in Sect. 3 that K is a compact subset of H. We now show that the operator \({\mathcal {L}}_m([\theta ])\) defined in (82) is a consistent approximation of the operator (81), in the compact set K. For all m larger than some fixed \(m_0\in {\mathbb {N}}\), we have
where \(\gamma \) is a constant independent of m. To obtain the last inequality, we used the fact that \(\varphi _k\) is orthonormal in H (\(\left\ \varphi _k\right\ _{H}=1\)), and that \(\left\ \partial (P_m \theta )/\partial x\right\ \) converges to \(\left\ \partial \theta /\partial x\right\ \) in H (uniformly in \(\theta \in K\)). The proof of this statement is based on the following inequalities [46, p. 38]
In the last inequality, we used the fact that \(\theta \) is in the Sobolev sphere (28). From (89) it follows that
where we repeatedly applied the Poincaré inequality \(\left\ f\right\ _{H}\le g \left\ \partial f/\partial x\right\ _{H}\) to obtain the constant \(\kappa \). Equation (90) defines the constant \(\gamma \) appearing in (88). At this point we recall that the functional derivative \(\delta F([P_m\theta ],t)/\delta \theta (x)\) converges strongly in H to \(\delta F([\theta ],t)/\delta \theta (x)\) as m goes to infinity for all \(\theta \in K\) (Theorem 6.1). This implies that (88) goes to zero for all \(\theta \in K\) as m goes to infinity,^{Footnote 14} i.e.,
The rate of convergence depends on the regularity of the firstorder functional derivative \(\delta F([\theta ],t)/\delta \theta (x)\) as a function of x. In particular, if \(\delta F([\theta ],t)/\delta \theta (x)\) is infinitely differentiable in x, then (92) goes to zero exponentially fast with m [46, p. 36].
Cylindrical approximation of FDEs: stability and convergence
Let us now consider the mdimensional linear PDE
where \(B_m\) and \(F_0\) are defined in (79) and (77), respectively. If the conditions of Lemma 7.1 are satisfied, then we say that the PDE (93) is a consistent approximation of the FDE (77). Moreover, if \(\{{\mathcal {L}}_m\}\) in (93) is consistent with \({\mathcal {L}}\) to order p, then we say that the PDE (93) is consistent with the FDE (77) with order p.
A fundamental question at this point is whether the solution of (93) converges to the solution of the FDE (77) as we send m to infinity. The Trotter–Kato approximation theorem for abstract evolution equations in Banach spaces [33, p. 209] states that this is indeed the case, provided the initial value problem (93) is “stable” in the following sense.
Definition 7.2
(Stability) Suppose that the linear operator \({\mathcal {L}}_m\) in (93) generates a strongly continuous semigroup \(e^{t {\mathcal {L}}_m}\). We say that the FDE approximation (93) is stable if there are two constants M and \(\omega \) independent of m such that \(\left\ e^{t {\mathcal {L}}_m}\right\ \le Me^{\omega t}\).
We now have all elements to state a version of the Trotter–Kato theorem [41, p. 8] that holds for cylindrical approximations of functional differential equations.
Theorem 7.1
(Convergence of cylindrical approximations to FDEs) Suppose that the initial value problem (77) is well posed in the time interval [0, T] (T finite), and that \({\mathcal {L}}([\theta ])\in {\mathcal {C}}({\mathcal {F}})\) generates a strongly continuous semigroup in [0, T]. Then, the FDE approximation (93) is stable and consistent in a compact subset K of a real separable Hilbert space H if and only if it is convergent, i.e.,
as \(m\rightarrow \infty \), provided \(F_m([\theta ],0)\rightarrow F_0([\theta ])\).
The proof of this theorem can be found in [33, p. 210]. In summary, to prove that a cylindrical approximations to FDEs is convergent we can proceed as follows:

(a)
Construct the multivariate PDE (93) and show that such PDE is a consistent approximation to the FDE (77) (Lemma 7.1)

(b)
Study stability of (93). This is a PDEspecific result stating that it is possible to control some norm of the solution of (93) by a constant multiple of a suitable norm of the initial condition, and all the norms involved (including the constant) do not depend on m. The simplest stability results arise from energy inequalities, e.g., for PDEs with continuous and coercive linear operators \({\mathcal {L}}_m\).

(c)
Apply Theorem 7.1 to claim that if a) and b) are satisfied, then the solution of the PDE (93) converges uniformly to the solution of the FDE (77) as the number of independent variables m goes to infinity.
Example 2 Consider the initial value problem
The FDE (95) is the Hopf equation corresponding to the linear PDE
where \(u_0(x)\) is random and periodic in \([0,2\pi ]\). To show this, let
be the Hopf functional associated with the solution to (96). The expectation operator \({\mathbb {E}}\left\{ \cdot \right\} \) in (97) is an integral over the probability measure of \(u_0(x)\). Differentiation of (97) with respect to time yields
We assume that \(\theta \) is in the compact set \(K\subseteq L^2_p([0,2\pi ])\) defined in (28). This is domain in which we solve the FDE (95). Let \(\{\varphi _1,\varphi _2,\ldots \}\) be an orthonormal basis of \(L^2_p([0,2\pi ])\). By equations (82) and (92), we have that the mdimensional PDE
where \(a_k=(\theta ,\varphi _k)_{L^2_p([0,2\pi ])}\), is a consistent cylindrical approximation to the FDE (95). Next, we show that such approximation is stable in the sense of Definition 7.2. By using the method of characteristics [75], it is straightforward to show that the solution of (99) can be bounded as
where \(f_0 = f(a_1,\ldots , a_m,0)\). Hence, if the \(L^\infty \) norm of \(f_0\) is bounded by a constant \(\kappa \) that is independent of m, then (99) is stable in the \(L^\infty ({\mathbb {R}}^m)\) norm.^{Footnote 15} Such strong bound implies that the solution (99) is also bounded in the \(L^2_\mu \) norm, where \(\mu \) is the measure defined in (A.2). In fact, we have
Note that this also implies that the functional integral defined in (A.1)–(A.3) converges, as it is bounded by the same constant \(\kappa \) independently of m. By using Theorem 7.1, we conclude that the solution of the PDE (99) converges uniformly to the solution of the FDE (95) in K, as the number of independent variables m goes to infinity.
We now have the main tools to study convergence of cylindrical approximations to FDEs. The main result is Theorem 7.1 which is based on the Trotter–Kato approximation theorem for abstract evolution equations in Banach spaces [53, 89]. The theorem states that stable consistent approximations of FDEs are convergent, but it does not provide an estimate on the rate of convergence of the approximation, i.e., how fast \(F_m\) converges to F. Estimates of such rate of convergence are available in rather general cases (e.g., [15]), but a thorough analysis for cylindrical approximations of FDEs is lacking. Nevertheless, in Sect. 9.3 we will show that the convergence rate of the cylindrical approximation to a prototype FDE can be exponential.
Approximation of nonlinear functionals and FDEs in real Banach spaces admitting a basis
In this section, we outline the extension of the functional approximation theory we developed in real separable Hilbert spaces to nonlinear functionals and FDEs defined on compact subsets of real Banach spaces admitting a basis.^{Footnote 16} Wellknown examples of such Banach spaces are:

\(C^{(0)}([0,1])\) (space of continuous functions in [0, 1]) [64, §5.2];

\(L^p(\Omega )\) for \(1<p<\infty \) (Lebesgue space defined on a compact domain \(\Omega \subseteq {\mathbb {R}}^n\)) [8, Theorem 2.1];

\(W^{k,p}(\Omega )\) for \(1<p<\infty \) (Sobolev space defined on a compact domain \(\Omega \subseteq {\mathbb {R}}^n\) with smooth or Lipschitz boundary [21, 60]).
Before we present the main results, let us briefly recall the definition and the basic properties of Schauder bases in Banach spaces.
Definition 8.1
(Schauder basis) A Schauder basis of a Banach space X is sequence of linearly independent elements \(\varphi _k\in X\) such that every \(\theta \in X\) can be uniquely represented as
where \(a_k: X \rightarrow {\mathbb {R}}\) is a sequence of bounded linear functionals^{Footnote 17} uniquely determined by the basis \(\{\varphi _k\}\).
As is well known, every basis in a Banach space is a Schauder basis (see, e.g., [86, p. 20] or [64, Proposition 5.3]). Hence, hereafter we will drop the adjective “Schauder” when referring to a basis in Banach space.
Compact subsets of real separable Banach spaces
Just like in the case of functional approximation in real separable Hilbert spaces, all approximation results we present hereafter hold in compact subsets of Banach spaces with a basis. Characterizing such compact subsets, is not as straightforward as in the case of Hilbert spaces (see the introduction of Sect. 3 and Theorem 3.1). Nevertheless, compactness results are available in rather general cases. For instance, the Arzelà–Ascoli theorem [78] provides necessary and sufficient conditions for a set \(K\subseteq C^{(0)}(\Omega )\) (\(\Omega \) compact subset of \({\mathbb {R}}^n\)) to be precompact. Specifically, the theorem states that K is precompact in the topology induced by the uniform norm if and only if K is equicontinuous and pointwise bounded. By using the Arzelà–Ascoli theorem, it is straightforward to prove, e.g., that the set of Lipschitzcontinuous (with the same Lipschitz constant) probability density functions on \(\Omega \) is precompact in \(C^{(0)}(\Omega )\). A similar compactness result, known as Kolmogorov–Riesz theorem [42, 43], can be obtained in \(L^p({\mathbb {R}}^n)\) and \(W^{1,p}({\mathbb {R}}^n)\) (\(1\le p<\infty \)). Such theorem can be stated as follows.
Theorem 8.1
(Compact subsets of \(L^p\) [43]) A subset K of \(L^p({\mathbb {R}}^n)\) (\(1\le p<\infty \)) is precompact if and only if
for all \(f\in K\).
The two conditions in (104) are known as equicontinuity and equitight conditions. Theorem 8.1 also holds in \(L^p(\Omega )\), where \(\Omega \) is a compact subset of \({\mathbb {R}}^n\). More generally, one can use wellknown compact embedding results such as the Rellich–Kondrachov theorem [1, §6]. Such theorem states that the Sobolev space \(W^{k,p}(\Omega )\) defined on a compact domain \(\Omega \subseteq {\mathbb {R}}^n\) with differentiable boundary is compactly embedded in \(W^{l,q}(\Omega )\), provided \(k>l\) and \(kp/n>lq/n\). This means, for example, that a closed sphere in \(W^{k,p}(\Omega )\) is precompact in \(L^q(\Omega )\) if \(k>(pq)/n\).
Approximation results for nonlinear functionals, functional derivatives and FDEs
Let X be a Banach with basis \(\{\varphi _k\}\) and let \(D_m=\text {span}\{\varphi _1,\ldots ,\varphi _m\}\). Define the linear projection operator \(P_m: X\rightarrow D_m\)
It is straightforward to show that \(P_m\) is bounded and that \(P_m \theta \) converges uniformly to \(\theta \) in every compact subset of X as m goes to infinity. In fact, we have the following
Lemma 8.1
Let X be a Banach space with basis \(\{\varphi _k\}\), and let K be compact subset of X. Then, for each \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that
The uniform convergence result (106) is known as “approximation property” in Banach space theory [61] (see [51, p. 638] for a proof). Hence, Lemma 8.1 shows that every Banach space with a basis has the approximation property. We remark that in a real separable Hilbert space the uniform approximation property follows immediately from the monotonicity of the sequence \(f_m([\theta ])=\left\ \theta  P_m\theta \right\ _H\) (Parseval’s identity implies \(f_{m+1}([\theta ])\le f_{m}([\theta ])\)), and Dini’s theorem.
Lemma 8.2
(Uniform convergence of functional approximations) Let X be a real Banach space with a basis, K a compact subset of H, and \(P_m\) the projection operator (105). If F is a continuous functional on X, then the sequence \(F([P_m\theta ])\) converges uniformly to \(F([\theta ])\) on K, i.e., for all \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that
The proof of this Lemma is essentially the same as the proof of Lemma 5.1 in [72]. We only need to replace the first equation at page 380 in [72] with (106). As before, we will refer to \(F([P_m \theta ])\) as “cylindrical approximation”^{Footnote 18} of \(F([\theta ])\).
Most of the approximation results we obtained for nonlinear functionals, Fréchet derivatives, functional derivatives and FDEs in compact subsets of in real separable Hilbert spaces hold also in compact subsets Banach spaces admitting a basis. Hereafter, we list the most important ones. The proofs are the same as in the case of real separable Hilbert spaces and therefore omitted.
Lemma 8.3
(Compactness of firstorder Fréchet derivatives) Let K be a compact subset of a real Banach space X admitting a basis, and let \(F([\theta ])\) be a continuous real or complexvalued functional on X. If the Fréchet derivative \(F'([\theta ^*])\) exists at \(\theta ^*\in K\), then it is a compact linear operator.
Lemma 8.4
(Convergence rate of functional approximations) Let F be a realvalued, continuously differentiable functional on a compact and convex subset K of a real Banach space X admitting a basis. Then, for all \(\theta \in K\) and for any finitedimensional projection \(P_m\) of the form (105), we have
In particular, \(F([P_m\theta ])\) converges uniformly to \(F([\theta ])\) in K at the same rate as \(P_m \theta \) converges to \(\theta \) in X.
Convergence rate estimates for \(\left\ \theta P_m\theta \right\ _X\) are available, e.g., in [21, 27, 60].
Theorem 8.2
(Uniform approximation of firstorder Fréchet derivatives) Let X be a real Banach space admitting a basis, K a compact subset of X, \(\theta \in K\) and \(P_m\) the projection operator (105). If F is continuously differentiable on K with Fréchet derivative \(F'([\theta ])\), then the sequence of operators \(F'([P_m \theta ])\) converges uniformly to \(F'([\theta ])\). In other words, for all \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that
for all \(m\ge m_\epsilon \), and for all \(\theta \in K\).
Lemma 8.5
(Convergence rate of firstorder Fréchet derivatives) Let X be a real Banach admitting a basis, and let \(F([\theta ])\) be a nonlinear functional with continuous first and secondorder Fréchet derivatives. Then, for all \(\theta \) in a convex compact subset K of X and for any projection \(P_m\) of the form (105), we have
In particular, \(F'([P_m\theta ])\) converges uniformly to \(F'([\theta ])\) in K at the same rate as \(P_m \theta \) converges to \(\theta \).
Regarding the extension of the approximation result for the firstorder functional derivative, i.e., Lemma 6.1, we can leverage various generalizations of the Riesz representation theorem (16) to specific Banach spaces. Hereafter, we consider the \(L^p\) generalization. More general versions may involve measure theory, e.g., in the case of spaces of continuous functions defined on compact subsets of \({\mathbb {R}}^n\) (see Eq. (17)).
Lemma 8.6
(Uniform approximation of firstorder functional derivatives) Let \(\Omega \) be a compact subset of \({\mathbb {R}}^n\), K a compact subset of \(L^p(\Omega )\) (\(1< p<\infty \)), and \(P_m\) the projection operator (105). If F is continuously differentiable on K with Fréchet derivative \(F'([\theta ])\), then the sequence \(\delta F([P_m\theta ])/\delta \theta (x)\) converges uniformly to \(\delta F([\theta ])/\delta \theta (x)\) in \(L^q(\Omega )\), where \(1/p +1/q=1\). In other words, for all \(\epsilon >0\) there exists \(m_\epsilon \in {\mathbb {N}}\) such that for all \(m\ge m_\epsilon \)
Proof
By Lemma 8.3 the Fréchet derivative \(F'(\theta )\) is a compact linear operator in \(L^p(\Omega )\) for each \(\theta \in K\). Hence, \(F'([\theta ])\eta \) is a bounded linear functional in \(L^{p}(\Omega )\) for each \(\theta \in K\). By using the Riesz representation theorem, we conclude that there exists a unique function \(\delta F([\theta ])/\delta \theta (x)\in L^q(\Omega )\) with \(1/q =1 1/p\) such that
By applying Theorem 8.2, we obtain that
As is well known, the norm of the linear functional \(\left( F([\theta ]) F([P_m \theta ])\right) \eta \) (linear functional of \(\eta \in L^p(\Omega )\)) is
By definition, \(M([\theta ]\)) is the smallest number such that
This fact, together with (114) allow us to conclude that \(M([\theta ])<\epsilon \) for all \(\theta \in K\). This proves the theorem. \(\square \)
A few comments on Lemma 8.6 are necessary at this point. First, compact subsets of \(L^p(\Omega )\) are identified by the equicontinuity and the equitight conditions in Theorem 8.1. Second, we excluded the case \(p=1\) as the Banach space \(L^1(\Omega )\) does not admit a basis. Regarding approximation of FDEs in real Banach spaces with a basis, we have the following results.
Lemma 8.7
(Consistency of cylindrical approximations to FDEs) Let X be a real Banach space with a basis. Consider a functional \(F\in {\mathcal {F}}(X)\) and a densely defined closed linear operator \({\mathcal {L}}\in {\mathcal {C}}({\mathcal {F}})\). If \({\mathcal {L}}([\theta ])F([\theta ])\) is continuous in \(\theta \), then the sequence of operators \(\{{\mathcal {L}}_m\}\) defined in (80) is consistent with \({\mathcal {L}}\) on every compact subset K of X, provided \( \left\ R_m([\theta ])\right\ \rightarrow 0\) as \(m\rightarrow \infty \) for all \(\theta \in K\).
Theorem 8.3
(Convergence of cylindrical approximations to FDEs) Suppose that the initial value problem (77) is well posed in the time interval [0, T] (T finite), and that \({\mathcal {L}}([\theta ])\in {\mathcal {C}}({\mathcal {F}})\) generates a strongly continuous semigroup in [0, T]. Then, the FDE approximation (93) is stable and consistent (in the sense of Definitions 7.1 and 7.2) in a compact subset K of a real Banach space X admitting a basis if and only if it is convergent, i.e.,
as \(m\rightarrow \infty \), provided \(F_m([\theta ],0)\rightarrow F_0([\theta ])\).
The proof of this theorem can be found in [33, p. 210]. We emphasize that the sequence of steps to prove convergence of functional approximations to FDEs in Banach spaces admitting a basis is the same (a)–(c) listed after Theorem 7.1.
Numerical examples
In this section, we provide numerical demonstrations of the approximation theorems we developed for nonlinear functionals and functional differential equations. To this end, we consider the function space defined by the following closure of a Sobolev sphere with radius \(\rho \):
We have seen in Sect. 3 that K is a convex compact subset of \(L_p^2([0,2\pi ])\). Hence, any realvalued continuous functional \(F([\theta ])\) defined on K can be represented as the limit of a uniformly convergent sequence of functionals of the form \(F([P_m \theta ])\), where \(P_m\) is the projection operator (43). We can sample elements from (118) by taking a truncated Fourier series of the form
and then choosing the modulus of the complex numbers \(\{c_0,\ldots ,c_N\}\) within an ellipsoid in \({\mathbb {R}}^{N+1}\). In fact, we have
Hence, the condition \(\left\ \theta \right\ ^2_{H_p^s}\le \rho ^2\) implies that
which defines the closure of an ellipsoid in the variables \(\{c_0,\ldots ,c_N\}\).
Generation of test functions with prescribed Fourier spectrum
The decay rate of the modulus of the Fourier coefficients \(\left c_k\right \) in the series expansion (119) is related to the degree of smoothness of \(\theta \), i.e., the value of s in (118) (see [46, §2]). Hence, by sampling \(\theta \) from a space of periodic functions with a prescribed spectral decay \(\left c_k\right \) we can study the effects of the regularity parameter s in (118) on the rate of convergence of the nonlinear functional approximations we developed in Sects. 4, 5 and 6. To sample test functions from (118), we represent \(c_k\) in (119) in polar form, prescribe the decay of the amplitudes \(c_k\) (\(k\ge 0\)) and introduce a uniformly distributed random shift \(\vartheta _k\in [0,2\pi ]\) subject to the constraint \(\vartheta _k=\vartheta _{k}\). This yields
We study two types of decay rates of the Fourier spectrum. The first is a powerlaw decay of the form
where \(\alpha \ge 1\) and \(k=1,\ldots , N\). In equation (123) a(0) is a uniformly distributed random variable in \([10,10]\) and \(\{a(1),\ldots , a(N)\}\) is a sequence of i.i.d. uniformly distributed random variables in [0, 10]. The algebraic decay (123) defines functions in a Sobolev sphere (118) with index \(s=\alpha \). The radius of such sphere can be computed by substituting (123) into (120), and then evaluating the supremum. The second power spectrum we consider has an exponential decay of the form
where \(\beta > 1\) and \(k=1,\ldots , N\). The random sequence \(\{b(0),b(1),\ldots , b(N)\}\) has the same properties as the sequence \(\{a(0),a(1),\ldots , a(N)\}\) in (123). The spectrum (124) defines functions in a Sobolev sphere (118) with index \(s\rightarrow \infty \).
In Fig. 1 we plot one sample of the random spectra (123) and (124), together with the corresponding sample functions (122) for \(N=500\), \(\alpha \in \{1.5, 2, 2.5, 3\}\) and \(\beta \in \{1.2, 1.5, 2, 3\}\). In the numerical examples presented, hereafter we choose N large enough so that the contribution of the tail of the spectrum is negligible in the series expansion (122). This allows us to generate highly accurate approximations of \(\theta \) in the space (118), which will then be projected onto a lowerdimensional subspace generated by a second trigonometric basis.
Specifically, we chose the following orthonormal basis consisting of discrete trigonometric polynomials [46, p. 29]
which yields the projection operator
As is well known, if the first \((s1)\) derivatives of \(\theta \) are all continuous, and if the sth derivative is in \(L_p^2([0,2\pi ])\), then the \(L_p^2([0,2\pi ])\) distance between \(\theta \) and \(P_m\theta \) as defined in (126) decays as \(1/m^s\). On the other hand, if \(\theta \) is of class \(C^{\infty }\), then \(P_m\theta \) converges to \(\theta \) exponentially fast in \(L_p^2([0,2\pi ])\) (see [46, §2.3]).
Approximation of nonlinear functionals
Consider the nonlinear functional
The Fréchet differential of \(F([\theta ])\) is given by
which is a linear operator in \(\eta \). We have shown in Sect. 3 that \(F'([\theta ])\) is compact in the function space (118), and therefore, it is bounded and continuous.^{Footnote 19} In fact, for all \(\theta \in K\) and \(\eta \in L^2_p([0,2\pi ])\) it follows from (128) that
Plugging this result into the mean value Theorem 5.1 yields the spectral convergence result
The last two inequalities follow from wellknown Fourier series approximation theory [46, p. 42], and from the fact that \(\theta \) is in the closure of a Sobolev sphere with radius bounded by \(\rho \).
Next, we determine the convergence rate of the firstorder Fréchet and functional derivative approximations. To this end, we notice that the secondorder Fréchet derivative of (128), i.e.,
is a continuous bilinear operator on the compact set \(K \times K\). Therefore, by equation (74), the firstorder Fréchet derivative must converge at the same rate as (130). The firstorder functional derivative of F, i.e., the kernel of the integral operator (128) is
As easily seen, if we evaluate (132) at \(P_m\theta \) we obtain the approximated functional derivative
which is an element of \(L_p^2([0,2\pi ])\) that converges to (132) uniformly in \(\theta \in K\) in the \(L_p^2([0,2\pi ])\) norm. This is because bounded and continuous functions such as \(\sin (x)\) preserve \(L^2\) convergence under composition (see [7, Theorem 7]). This result is also in agreement with Lemma 6.1. Hereafter, we provide a numerical verification of the convergence rate we just predicted. To this end, in Fig. 2 we plot
versus m. The error \(\epsilon _0(m)\) is computed numerically for each m by taking the maximum over \(10^3\) sample functions of the form (122), with \(N=1000\), and different spectra of the form (123) and (124) (see Fig. 1).
The error in the Fréchet derivative is defined as
and is computed as follows: for each given \(\theta \), we determine \(P_m\theta \) and then approximate the supremum over \(\eta \) using \(10^3\) sample functions \(\eta \). This is done for \(10^3\) functions \(\theta \) sampled from K as before. Notice that \(\eta \in K\) has the same form as \(\theta \), and therefore, it is taken from the same ensemble as \(\theta \) is taken from. The results of our calculations are shown in Fig. 3. As expected, the approximated functional derivative \(F'([P_m\theta ])\) converges to \(F'([\theta ])\) at the same rate as \(P_m\theta \) converges to \(\theta \) in \(L^2_p([0,2\pi ])\). The reason is that the Fréchet derivative (128) is continuously Fréchet differentiable,^{Footnote 20} and therefore the mean value formula (74) holds.
Approximation of functional differential equations
In this section, we provide a simple example of convergence analysis that shows how fast the solution of the multivariate PDE (99) converges to the solution of the FDE (95) as we send m to infinity. To this end, we first examine the analytical solution of the FDE (95).
Analytical solution
It was shown in [93, p. 76] that the analytical solution of the FDE (99) in the function space (118) is
Clearly, if \(F_0\) is invariant under translations, i.e., if \(F_0([\theta (xt)])=F_0([\theta (x)])\), then \(F([\theta ],t)= F_{0}([\theta ])\), i.e., the solution is constantly equal to the initial condition \(F_0([\theta ])\) at each time. Examples of such translationinvariant functionals are
On the other hand, the initial condition
is not translationinvariant. The solution to the initial value problem (95), with \(F_0\) given in (138), is
which is periodic in t with period \(2\pi \). It is easy to verify by direct calculation that (139) is indeed a solution to (95). To this end, let us define \(\partial _x=\partial /\partial x\). We begin by noting that
The firstorder functional derivative of (139) is obtained by analyzing its Fréchet differential
Here, we utilized the fact that the operator adjoint of the semigroup \(e^{t\partial _x}\) relative to standard \(L_p^2([0,2\pi ])\) inner product is \(e^{t\partial _x}\). Hence, the firstorder functional derivative of (139) is
Using again the fact that \(\partial _x\) is skewsymmetric relative to the \(L_p^2([0,2\pi ])\) inner product, we obtain
On the other hand, a temporal differentiation of (139) yields
By setting the equality between (143) and (144), we conclude that (139) is a solution to (95) if and only if
which is clearly an identity, given (140). This proof can be generalized to arbitrary Fréchet differentiable initial conditions \(F_0\).
FDE approximation and convergence analysis
We have seen in Sect. 7 that the cylindrical approximation of the FDE (95) yields the multivariate PDE (99). By using integration by parts, it can be shown that the matrix of coefficients
is skewsymmetric since the basis functions \(\varphi _j\) are periodic. The initial condition appearing in (99) is obtained by evaluating (138) on the range of \(P_m\). This yields the cylindrical functional
The solution to the initial value problem (99) with initial condition given in (147) is obtained as
We have seen in Sect. 9.2 that (147) converges uniformly to \(F_0([\theta ])\) as m goes to infinity at the same rate as \(\left\ \theta  P_m\theta \right\ _{L_p^2([0,2\pi ])}\) goes to zero. We also know that the residual of the finitedimensional PDE approximation (99) goes to zero as we send m to infinity (Example 1 in Sect. 7), and that (99)–(147) is stable in the \(L^{\infty }\) norm (Example 2 in Sect. 7.1). By Theorem 7.1 this is sufficient to guarantee that (148) converges uniformly in \(\theta \) to the FDE solution (139) as we increase m (Theorem 7.1). Hereafter, we calculate the convergence rate of such approximation and show that it can be exponential depending on degree of smoothness of \(\theta \in K\), which is measured by the index s in (118). To this end, we begin with
Recall that for any \(a,b\in {\mathbb {R}}\) we have
Hence, from equation (149) it follows that
At this point, we recall that \(\theta (x,t)=e^{t\partial _x}\theta (x)\) is the exact solution to the advection equation (145), while the function \(\theta _m(x,t)=\sum _{j=0}^m \left[ e^{tC}a\right] _j\varphi _j(x)\) is the solution to the Fourier–Galerkin discretization of (145)
It is well known that the Galerkin scheme (152) is stable in the \(L_p^2([0,2\pi ])\) norm (see, e.g., [16, §6.1.1]), and that the solution \(\theta _m(x,t)\) converges to \(\theta (x,t)\) at a rate that depends only on the smoothness of \(\theta (x,0)\) (initial condition). This implies that
where the parameter s measures the regularity of \(\theta \). If \(\theta \) is infinitely differentiable, then \(f(a_0,\ldots , a_m,t)\) converges to \(F([\theta ],t)\) exponentially fast in m. To validate (153) numerically, in Fig. 4 we plot the error
at time \(t=\pi \) in the case where \(\theta \) has power law or exponential decaying Fourier coefficients. It is seen that the cylindrical approximation \(f(a_0,\ldots ,a_m,t)\) indeed converges to \(F([\theta ],t)\) at the same rate at which \(P_m\theta \) converges to \(\theta \), which depends on the smoothness of \(\theta \in K\). It is worthwhile noticing that the convergence plots in Figs. 2, 3 and 4 are essentially a rescaled version of the same plot. The reason is that the FDE solution has continuous Fréchet derivatives up to any desired order. Hence, by the mean value Theorem 5.1, the convergence slopes are determined by the rate at which \(\left\ \theta P_m\theta \right\ _{L_p^2([0,2\pi ])}\) goes to zero.
Conclusions
We established rigorous convergence results for cylindrical approximations of nonlinear functionals, functional derivatives, and functional differential equations (FDEs) defined on a compact subset of a real Banach space X admitting a basis. Such approximations are constructed by restricting the domain of the functionals and the FDEs to the range of a finitedimensional projection acting on X. In this setting, we proved that continuous functionals and FDEs can be approximated by multivariate functions and multidimensional partial differential equations (PDEs), respectively. The convergence rate of such functional approximation can be exponential, depending on the regularity of the functional (in particular its Fréchet differentiability), and its domain. Rapidly converging approximations allow us to represent nonlinear functionals and FDEs in terms of multivariate functions and PDEs involving fewer independent variables. We also provided necessary and sufficient conditions for consistency, stability and convergence of functional approximations schemes to compute the solution of linear FDEs. The main results are Theorem 7.1 and Theorem 8.3, which are based on the Trotter–Kato approximation theorem for abstract evolution equations in Banach spaces. The results presented in this paper open the possibility to utilize techniques for highdimensional function representation such as deep neural networks [73, 74, 104] and numerical tensor methods [5, 10, 24,25,26, 55, 76, 76, 82] to approximate nonlinear functionals in terms of highdimensional functions, and to compute approximate solutions of functional differential equations by solving highdimensional PDEs.
Notes
 1.
 2.
The solution to a stochastic ordinary or partial differential equation is a nonlinear functional of the forcing terms, initial condition and boundary conditions. Effective methods to represent such functional dependence are based on polynomial chaos expansions [34, 97,98,99, 102], probabilistic collocation methods [29, 36, 101], and deep neural networks [74, 104]. Other techniques rely on a reformulation of the problem in terms of kinetic equations [11, 19, 96], or hierarchies of kinetic equations [13, 18, 94].
 3.
Recall that a closed sphere S in a Banach space X is not compact as not every sequence of elements in the sphere has a convergent subsequence with limit in S.
 4.
A Gaussnull set is a Borel set \(A\subseteq X\) such that \(\mu (A)=0\) for every nondegenerate Gaussian measure \(\mu \) on X.
 5.
 6.
A signed measure is a generalization of the concept of measure by allowing it to have negative values.
 7.
The condition (27) is known as equismall tail condition with respect to any orthonormal basis of H. It states that if we use enough basis elements, then we can bound the complementary energy (sum of the neglected Fourier amplitudes) associated with the series expansion of of any \(\theta \in E\).
 8.
We recall that a compact nonlinear functional is a continuous functional that maps bounded sets into precompact (relatively compact) sets.
 9.
 10.
In [39, p. 336] and [38, Ch. 1] Friedrichs and Shapiro defined cylinder functionals in a real separable Hilbert space H as those functionals which depend on their argument \(\theta \in H\) only in as much as they depend on \(P_m \theta \), where \(P_m\) is an orthogonal projection on H. In other words, a cylinder functional on H is a functional F such that \(F([\theta ])=F([P_m\theta ])\) for all \(\theta \in H\). Clearly, (48) is a cylinder functional.
 11.
Equation (52) can be equivalently written as
 12.
Note that \(F''\) is a compact symmetric bilinear form from \(H\times H\) into \({\mathbb {R}}\) or \({\mathbb {C}}\).
 13.
In Definition 7.1, \(D({\mathcal {L}})\) denotes the domain of the operator \({\mathcal {L}}\).
 14.
Recall that if \(f_n\rightarrow f\) is a strongly convergent sequence in a Hilbert space H, then, for any given \(\epsilon > 0\) and any orthonormal basis of H there exists \(m_\epsilon \in {\mathbb {N}}\) such that
 15.
 16.
A real Banach space with a basis \(\{\varphi _k\}\) is necessarily separable since the set of all finite linear combinations \(\sum _{k}a_k\varphi _k\) forms a countable dense subset of X [64]. The longstanding question of whether every separable Banach space possesses a basis was answered by Per Enflo [32] in 1973. He showed that there do exist separable Banach spaces that do not possess a basis.
 17.
It is shown in [86, p. 20] that \(1 \le a_k\left\ \varphi _k\right\ _X\le 2C\) for all \(k\in {\mathbb {N}}\), where \(C\ge 1\) is the socalled basis constant. In the case of real separable Hilbert spaces the linear functionals \(a_k([\theta ])\) are given by \(a_k([\theta ])=(\theta ,\varphi _k)_H\) (see Eq. (42)), and they are obviously bounded.
 18.
A cylinder functional on a real Banach space X admitting a basis is a functional f such that \(F([\theta ])=F([P_m\theta ])\) for all \(\theta \in X\). This definition relies on the fact that \(F([P_m \theta ])\) is a multivariate function of the coefficients \(a_k([\theta ])\), which define the cylinder set [91, p. 55]
where B is a Borel set of \({\mathbb {R}}^m\).
 19.
Recall that a linear functional in a Hilbert space is bounded if and only if it is continuous.
 20.
The functional (127) admits continuous Fréchet derivatives to any desired. In particular, we have
$$\begin{aligned} F''([\theta ])\eta _1\eta _2&= \,\, 2\int _{0}^{2\pi } \sin (x)\cos (2\theta (x)) \eta _1(x)\eta _2(x)\mathrm{d}x,\\ F'''([\theta ])\eta _1\eta _2\eta _3&= 4\int _{0}^{2\pi } \sin (x)\sin (2\theta (x)) \eta _1(x)\eta _2(x)\eta _3(x)\mathrm{d}x. \end{aligned}$$This implies that the mean value formula (74) can be applied to any of the Fréchet derivatives, by simply redefining the operator norm appearing at the righthand side of the inequality.
 21.
Recall that the coordinate system of \(D_m\) is \((a_1,\ldots ,a_m)\), and it depends on the choice of the orthonormal basis \(\{\varphi _1,\varphi _2,\ldots ,\varphi _m\}\).
References
 1.
Adams, R.A., Fournier, J.J.F.: Sobolev Spaces. Elsevier, Amsterdam (2003)
 2.
Alankus, T.: The generating functional for the probability density functions of Navier–Stokes turbulence. J. Stat. Phys. 53(5–6), 1261–1271 (1988)
 3.
Amit, D.J., MartínMayor, V.: Field Theory, the Renormalization Group and Critical Phenomena. World Scientific Publishing, New York (2005)
 4.
Aronszajn, N.: Differentiability of Lipschitz mapping between Banach spaces. Studia Math. 57, 147–190 (1976)
 5.
Bachmayr, M., Schneider, R., Uschmajew, A.: Tensor networks and hierarchical tensors for the solution of highdimensional partial differential equations. Found. Comput. Math. 16(6), 1423–1472 (2016)
 6.
Baez, J.C., Sawin, S.: Functional integration on spaces of connections. J. Funct. Anal. 150(1), 1–26 (1997)
 7.
Bartle, R.G., Joichi, J.T.: The preservation of convergence of measurable functions under composition. Proc. Am. Math. Soc. 12, 122–126 (1961)
 8.
Bellout, H.: On a special Schouder basis for the Sobolev spaces \(w_{0}^{1, p}(\omega )\). Ill. J. Math. 39(2), 187–195 (1995)
 9.
Bertuzzi, A., Gandolfi, A., Germani, A.: A Weierstrasslike theorem for real separable Hilbert spaces. J. Approx. Theory 32, 76–81 (1981)
 10.
Boelens, A.M.P., Venturi, D., Tartakovsky, D.M.: Parallel tensor methods for highdimensional linear PDEs. J. Comput. Phys. 375, 519–539 (2018)
 11.
Boelens, A.M.P., Venturi, D., Tartakovsky, D.M.: Tensor methods for the BoltzmannBGK equation. J. Comput. Phys. 421, 109744 (2020)
 12.
Bogoliubov, N.N.: On the theory of superfluidity. J. Phys. (USSR) 11, 23–32 (1947)
 13.
Brennan, C., Venturi, D.: Datadriven closures for stochastic dynamical systems. J. Comput. Phys. 372, 281–298 (2018)
 14.
Bukhvalov, A.V.: Integral representation of linear operators. J. Math. Sci. 9, 129–137 (1978)
 15.
Campiti, M., Tacelli, C.: Rate of convergence in Trotter’s approximation theorem. Constr. Approx. 28(2), 333–341 (2008)
 16.
Canuto, C., Hussaini, M.Y., Quarteroni, A., Zang, A.: Spectral Methods: Fundamentals in Single Domains. Springer, Berlin (2006)
 17.
Carmona, R., Delarue, F.: Probabilistic Theory of Mean Field Games with Applications III. Springer, Berlin (2018)
 18.
Cho, H., Venturi, D., Karniadakis, G.E.: Statistical analysis and simulation of random shocks in Burgers equation. Proc. R. Soc. A 2171(470), 1–21 (2014)
 19.
Cho, H., Venturi, D., Karniadakis, G.E.: Numerical methods for highdimensional kinetic equations. In: Jin, S., Pareschi, L. (eds.) Uncertainty Quantification for Kinetic and Hyperbolic Equations, pp. 93–125. Springer, Berlin (2017)
 20.
Chow, Y.T., Li, W., Osher, S., Yin, W.: Algorithm for Hamilton–Jacobi equations in density space via a generalized Hopf formula. J. Sci. Comput. 80, 1195–1239 (2019)
 21.
Ciesielski, Z., Figiel, T.: Spline bases in classical function spaces on compact \(c^{\infty }\) manifolds. Part I. Studia Mathematica 76, 1–58 (1983)
 22.
Cilia, R., Gutiérrez, J.M.: Operators with an integral representation. Proc. Am. Math. Soc. 144, 5275–5290 (2016)
 23.
Combe, P., Rodriguez, R., Rideau, G., SirugueCollin, M.: On the cylindrical approximation of the Feynman path integral. Rep. Math. Phys. 31, 279–294 (1978)
 24.
Dektor, A., Rodgers, B., Venturi, D.: Rankadaptive tensor methods for highdimensional nonlinear PDEs, pp. 1–24 (2020). arXiv:2012.05962
 25.
Dektor, A., Venturi, D.: Dynamic tensor approximation for highdimensional nonlinear PDEs pp. 1–23 (2020). arXiv: 2007.09538
 26.
Dektor, A., Venturi, D.: Dynamically orthogonal tensor methods for highdimensional nonlinear PDEs. J. Comput. Phys. 404, 109125 (2020)
 27.
DeVore, R., Petrova, G., Wojtaszczyk, P.: Greedy algorithms for reduced bases in Banach spaces. Construct. Approx. 37, 455–466 (2013)
 28.
Diestel, J., Uhl, J.J.: The Radon–Nikodym theorem for Banach space valued measures. Rocky Mt. J. Math. 6(1), 1–46 (1976)
 29.
Doostan, A., Owhadi, H.: A nonadapted sparse approximation of PDEs with stochastic inputs. J. Comput. Phys. 230(8), 3015–3034 (2011)
 30.
Dopazo, C., O’Brien, E.E.: Functional formulation of nonisothermal turbulent reactive flow. Phys. Fluids 17(11), 1968–1975 (1998)
 31.
Han, W.. E., J., Li, Q.: A meanfield optimal control formulation of deep learning. Res. Math. Sci 10, 6 (2019)
 32.
Enflo, P.: A counterexample to the approximation problem in Banach spaces. Acta Math. 130, 309–317 (1973)
 33.
Engel, K.J., Nagel, R.: OneParameter Semigroups for Linear Evolution Equations, vol. 194. Springer, Berlin (1999)
 34.
Ernst, O.G., Mugler, A., Starkloff, H.J., Ullmann, E.: On the convergence of generalized polynomial chaos expansions. ESAIM: Math. Model. Numer. Anal. 46(2), 317–339 (2012)
 35.
Folland, G.B.: Real Analysis: Modern Techniques and Their Applications. Wiley, Hoboken (2013)
 36.
Foo, J., Karniadakis, G.E.: Multielement probabilistic collocation method in high dimensions. J. Comput. Phys. 229, 1536–1557 (2010)
 37.
Fox, R.F.: Functionalcalculus approach to stochastic differential equations. Phys. Rev. A 33(1), 467–476 (1986)
 38.
Friedrichs, K.O., Shapiro, H.N.: Integration of functionals. New York University, Institute of Mathematical Sciences (1957)
 39.
Friedrichs, K.O., Shapiro, H.N.: Integration over a Hilbert space and outer extensions. Proc. Natl. Acad. Sci. 43(4), 336–338 (1957)
 40.
Gangbo, W., Li, W., Osher, S., Puthawala, M.: Unnormalized optimal transport. J. Comput. Phys. 399, 108940 (2019)
 41.
Guidetti, D., Karasozen, B., Piskarev, S.: Approximation of abstract differential equations. J. Math. Sci. 122, 3013–3054 (2004)
 42.
HancheOlsen, H., Holden, H.: The Kolmogorov–Riesz compactness theorem. Expo. Math. 28, 385–395 (2010)
 43.
HancheOlsen, H., Holden, H.: An improvement of the Kolmogorov–Riesz compactness theorem. Expo. Math. 37, 84–91 (2019)
 44.
Hänggi, P.: The functional derivative and its use in the description of noisy dynamical systems. In: Pesquera, L., Rodriguez, M. (eds.) Stochastic Processes Applied to Physics, pp. 69–95. World Scientific, New York (1985)
 45.
Hänggi, P.: Colored noise in continuous dynamical system. In: Moss, F., McClintock, P.V.E. (eds.) Noise in Nonlinear Dynamical Systems, vol. 1, pp. 307–347. Cambridge University Press, Cambridge (1989)
 46.
Hesthaven, J.S., Gottlieb, S., Gottlieb, D.: Spectral Methods for TimeDependent Problems. Cambridge University Press, Cambridge (2007)
 47.
Hohenberg, P., Kohn, W.: Inhomogeneous electron gas. Phys. Rev. 136, B864–B871 (1964)
 48.
Hopf, E.: Statistical hydromechanics and functional calculus. J. Rat. Mech. Anal. 1(1), 87–123 (1952)
 49.
Hunter, J.K., Nachtergaele, B.: Applied Analysis. World Scientific, New York (2001)
 50.
Jackson, D.: Fourier Series and Orthogonal Polynomials. Dover, Mineola (2004)
 51.
James, R.C.: Bases in Banach spaces. Am. Math. Monthly 89, 625–640 (1982)
 52.
Jensen, R.V.: Functional integral approach to classical statistical dynamics. J. Stat. Phys. 25(2), 183–210 (1981)
 53.
Kato, T.: Remarks on pseudoresolvents and infinitesimal generators of semigroups. Proc. Jpn. Acad. 35, 467–468 (1959)
 54.
Klyatskin, V.I.: Dynamics of Stochastic Systems. Elsevier Publishing Company, Amsterdam (2005)
 55.
Kolda, T., Bader, B.W.: Tensor decompositions and applications. SIREV 51, 455–500 (2009)
 56.
Lin, L., ZepedaNunez, L.: Projectionbased embedding theory for solving KohnSham density functional theory. SIAM Multiscale Model. Simul. 17(4), 1274–1300 (2019)
 57.
Lindenstrauss, J., Preiss, D.: On Fréchet differentiability of Lipschitz maps between Banach spaces. Ann. Math. 157, 257–288 (2003)
 58.
Mankiewicz, P.: On the differentiability of Lipschitz mappings in Fréchet spaces. Studia Math. 45, 15–29 (1973)
 59.
Martin, P.C., Siggia, E.D., Rose, H.A.: Statistical dynamics of classical systems. Phys. Rev. A 8, 423–437 (1973)
 60.
Matveev, O.V.: Bases in Sobolev spaces on bounded domains with Lipschitzian boundary. Math. Notes 72, 373–382 (2002)
 61.
McArthur, C.W.: Development in Schauder basis theory. Bull. Am. Math. Soc. 78, 877–908 (1972)
 62.
Melrose, R.: MIT Mathematics 18.102/18.102. Lecture Notes: Introduction to Functional Analysis, Springer, Berlin (2020)
 63.
Monin, A.S., Yaglom, A.M.: Statistical Fluid Mechanics. Volume II: Mechanics of Turbulence, Dover, Mineola (2007)
 64.
Morrison, T.J.: Functional Analysis: An Introduction to Banach Space Theory. Wiley, Hoboken (2001)
 65.
Nashed, M.Z.: Differentiability and related properties of nonlinear operators: some aspects of the role of differentials in nonlinear functional analysis. In: Rall, L.B. (ed.) Nonlinear Functional Analysis and Applications. Academic Press, Cambridge (1971)
 66.
Ohkitani, K.: Study of the Hopf functional equation for turbulence: Duhamel principle and dynamical scaling. Phys. Rev. E 101, 013104 (2020)
 67.
Parr, R.G., Weitao, Y.: DensityFunctional Theory of Atoms and Molecules. Oxford University Press, Oxford (1994)
 68.
Peskin, M.E., Schroede, D.V.: An Introduction to Quantum Field Theory. CRC Press, Boca Raton (2018)
 69.
Phythian, R.: The functional formalism of classical statistical dynamics. J. Phys. A: Math. Gen. 10(5), 777–788 (1977)
 70.
Pinkus, A.: \(N\)Widths in Approximation Theory. Springer, Berlin (1985)
 71.
Preiss, D.: Differentiability of Lipschitz functions. J. Funct. Anal. 91, 312–345 (1990)
 72.
Prenter, P.M.: A Weierstrass theorem for real, separable Hilbert spaces. J. Approx. Theory 3, 341–351 (1970)
 73.
Raissi, M., Karniadakis, G.E.: Hidden physics models: machine learning of nonlinear partial differential equations. J. Comput. Phys. 357, 125–141 (2018)
 74.
Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physicsinformed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 606–707 (2019)
 75.
Rhee, H.K., Aris, R., Amundson, N.R.: FirstOrder Partial Differential Equations. Volume 1: Theory and Applications of Single Equations, Dover, Mineola (2001)
 76.
Rodgers, A., Venturi, D.: Stability analysis of hierarchical tensors methods for timedependent PDEs. J. Comput. Phys. 409, 109341 (2020)
 77.
Rosen, G.: Functional calculus theory for incompressible fluid turbulence. J. Math. Phys. 12(5), 812–820 (1971)
 78.
Rudin, W.: Principles of Mathematical Analysis, 3rd edn. McGrawHill, New York (1976)
 79.
Ruthotto, L., Osher, S., Li, W., Nurbekyan, L., Fung, S.W.: A machine learning framework for solving highdimensional mean field game and mean field control problems. PNAS 117(17), 9183–9193 (2020)
 80.
Schachermayer, W.: Integral operators on \(l^p\) spaces. Indiana Uni. Math. J. 30(1), 123–140 (1981)
 81.
Schep, A.R.: Compactness properties of an operator which imply that it is an integral operator. Trans. Am. Math. Soc. 265(1), 111–119 (1981)
 82.
Schneider, R., Uschmajew, A.: Approximation rates for the hierarchical tensor format in periodic Sobolev spaces. J. Complex. 30(2), 56–71 (2014)
 83.
Schwartz, J.T.: Nonlinear Functional Analysis. Gordon and Breach Science Publishers, London (1969)
 84.
Seiringer, R.: he excitation spectrum for weakly interacting bosons. Commun. Math. Phys. 306, 565–578 (2011)
 85.
Semadeni, Z.: Spaces of continuous functions on compact sets. Adv. Math. 1, 319–382 (1965)
 86.
Singer, I.: Bases in Banach Spaces I. Springer, Berlin (1970)
 87.
Skorohod, A.V.: Integration in Hilbert Space. Springer. Ergebnisse der Mathematik und ihrer Grenzgebiete. 2. Folge (1974)
 88.
Gikhman, I.I., Skorokhod, A.V.: The Theory of Stochastic Processes I. Springer, Berlin (2004)
 89.
Trotter, H.F.: Approximation of semigroups of operators. Pacific J. Math. 8, 887–919 (1958)
 90.
Vainberg, M.M.: Variational Methods for the Study of Nonlinear Operators. HoldenDay, Toronto (1964)
 91.
van Neerven, J.: Stochastic evolution equations. ISEM Lecture Notes (2008)
 92.
Venturi, D.: Conjugate flow action functionals. J. Math. Phys. 54, 113502 (2013)
 93.
Venturi, D.: The numerical approximation of nonlinear functionals and functional differential equations. Phys. Rep. 732, 1–102 (2018)
 94.
Venturi, D., Karniadakis, G.E.: Convolutionless Nakajima–Zwanzig equations for stochastic analysis in nonlinear dynamical systems. Proc. R. Soc. A 470(2166), 1–20 (2014)
 95.
Venturi, D., Sapsis, T.P., Cho, H., Karniadakis, G.E.: A computable evolution equation for the joint responseexcitation probability density function of stochastic dynamical systems. Proc. R. Soc. A 468(2139), 759–783 (2012)
 96.
Venturi, D., Tartakovsky, D.M., Tartakovsky, A.M., Karniadakis, G.E.: Exact PDF equations and closure approximations for advective–reactive transport. J. Comput. Phys. 243, 323–343 (2013)
 97.
Venturi, D., Wan, X., Karniadakis, G.E.: Stochastic lowdimensional modelling of a random laminar wake past a circular cylinder. J. Fluid Mech. 606, 339–367 (2008)
 98.
Venturi, D., Wan, X., Karniadakis, G.E.: Stochastic bifurcation analysis of Rayleigh–Bénard convection. J. Fluid Mech. 650, 391–413 (2010)
 99.
Venturi, D., Wan, X., Mikulevicius, R., Rozovskii, B.L., Karniadakis, G.E.: Wick–Malliavin approximation to nonlinear stochastic partial differential equations: analysis and simulations. Proc. R. Soc. A 469(2158), 1–20 (2013)
 100.
Wiener, N.: Nonliner Problems in Random Theory. MIT Press, Cambridge (1966)
 101.
Xiu, D.: Numerical Methods for Stochastic Computations: A Spectral Approach. Princeton University Press, Princeton (2010)
 102.
Xiu, D., Karniadakis, G.E.: The Wiener–Askey polynomial chaos for stochastic differential equations. SIAM J. Sci. Comput. 24(2), 619–644 (2002)
 103.
Zakharov, V.K., Mikhalev, A.V., Rodionov, T.V.: Fundamentals of Functions and Measure Theory. De Gruyter, Berlin (2018)
 104.
Zhu, Y., Zabaras, N., Koutsourelakis, P.S., Perdikaris, P.: Physicsconstrained deep learning for highdimensional surrogate modeling and uncertainty quantification without labeled data. J. Comput. Phys. 394, 56–81 (2019)
 105.
ZinnJustin, J.: Quantum Field Theory and Critical Phenomena, 4th edn. Oxford University Press, Oxford (2002)
Acknowledgements
This research was supported by the U.S. Army Research Office (ARO) through the Grant W911NF1810309. Parts of this paper were completed while Daniele Venturi was in residence at the Institute for Computational and Experimental Research in Mathematics (ICERM) in Providence, RI, during the semester program “Model and dimension reduction in uncertain and dynamic systems”, supported by the NSFDMS Grant 1439786.
Author information
Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Cylindrical approximation of functional integrals in real separable Hilbert spaces
In this appendix, we study approximation of functional integrals defined on a real separable Hilbert space H, with particular emphasis on integrals involving cylindrical functionals, i.e., functionals of the form (48). This topic was first investigated by Friedrichs and Shapiro [38, 39], and it fits the framework of functional approximations we discussed in Sect. 4. To describe the method, we first recall that \(\{P_m\}\) is a hierarchical and complete sequence of orthogonal projections, i.e., \(P_m\subseteq P_{m+1}\) (meaning that the range of \(P_m\) is a subset of the range of \(P_{m+1}\)). Following Friedrichs, Shapiro and Sokorohod [38, 39, 87] (see also [93, Appendix B.1]), we define the functional integral over H relative to a measure \(\mu ([\theta ])\) as
where \(D_m\) is defined in (44). We pointed out in Sect. 4 that \(F([P_m\theta ])=f(a_1,\ldots , a_m)\) is a mdimensional function depending on the variables \(a_k=(\theta ,\varphi _k)_H\), which are the coordinates of \(\theta \) relative to the orthonormal basis \(\{\varphi _1,\varphi _2,\ldots ,\varphi _m\}\). The finitedimensional measure in each subspace \(D_m\) can be taken, e.g., as a Gaussian product measure
which is absolutely continuous [87, §17], and invariant [38, Ch. VI] under unitary coordinate transformations^{Footnote 21} in \(D_m\). Such transformations are induced by unitary transformations of the basis functions \(\{\varphi _1,\varphi _2,\ldots \}\) in H. There is a welldefined integration theory for
At this point we recall that cylindrical functionals \(F([P_m\theta ])\) converge to \(F([\theta ])\) uniformly in \(\theta \), if \(\theta \) is chosen a compact subset of H. Also, cylindrical functionals are completely invariant in the sense of [38, Ch V, §III]. This guarantees that the limit in (A.1) exists, and that the functional integral is well defined. This allows us to define the inner product between two cylindrical functionals F and G as (see [93, §5.1])
where \({\mathcal {F}}\) is the vector space of functionals defined on the Hilbert space H. The inner product (A.4) induces the norm
Example 1 Consider the nonlinear functional
in the space of squareintegrable periodic functions in \([0,2\pi ]\), i.e., \(L^2_p([0,2\pi ])\). We are interested in computing the functional integral
where the measure \(\mathrm{d}\mu ([\theta ])\) is the limit of the product measure (A.2) as \(m\rightarrow \infty \). To this end, we first project \(\theta \) onto the orthonormal Fourier basis
where \(a_k=\left( \theta ,\sin (k x)\right) _{L_p^2([0,2\pi ])}/\sqrt{\pi }\) and \(b_k=\left( \theta ,\cos (k x)\right) _{L_p^2([0,2\pi ])}/\sqrt{\pi }\), (\(k=1,\ldots ,m\)). A substitution of (A.8) into (A.6) yields
independently of m. Hence, the functional integral (A.7) is
Appendix B: Distance between function spaces and approximability of nonlinear functionals
A key concept when approximating a nonlinear functional \(F([\theta ])\) by restricting its domain D(F) to a finitedimensional space functions \(D_m\) is the distance between \(D_m\) and D(F). Such distance can be quantified in different ways (see, e.g., [70]). For example, we can define the deviation of \(D_m\) from D(F) as
The number E measures the extent to which the worst element of D(F) can be approximated from \(D_m\). One may also ask how well we can approximate \(D(F)\subseteq X\) with mdimensional subspaces of H which are allowed to vary within H. A measure of such approximation is given by the Kolmogorov mwidth
which quantifies the error of the best approximation to the elements of D(F) by elements in a vector subspace \(D_m\subseteq H\) of dimension at most m. The Kolmogorov mwidth can be rigorously defined, e.g., for nonlinear functionals in Hilbert spaces ( [70], Ch. 4). It should be emphasized that for a given domain of interest D(F), finding the optimal basis spanning \(D_m\) and minimizing the deviation \(E(D,D_m)\) is not an easy task. In some cases, however, asymptotic results are available, e.g., in the case of periodic Sobolev spaces [82]. It is important to emphasize that the approximation error and the computational complexity of approximating a nonlinear functional depends on the domain D(F) and the choice of basis \(\{\varphi _1,\varphi _2,\ldots \}\) spanning \(D_m\). In particular, an accurate functional approximation may be lowdimensional in one function space (i.e., for small \(\epsilon \), m is also small) and highdimensional in another (i.e., for small \(\epsilon \), m must be taken very large)—see §3.1.2 in [93] for examples.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Venturi, D., Dektor, A. Spectral methods for nonlinear functionals and functional differential equations. Res Math Sci 8, 27 (2021). https://doi.org/10.1007/s40687021002654
Received:
Accepted:
Published:
Mathematics Subject Classification
 46N40
 35R15
 47J05
 46G05
 65J15