Regularity of a Distribution and of the Boundary of Its Support

In two recent papers, Boman (J Geom Anal 31:2726–2741, 2020, https://doi.org/10.1007/s12220-020-00372-8, J Ill-posed Inverse Probl 2021, https://doi.org/10.1515/jiip-2020-0139), we proved that the Radon transform of a compactly supported distribution can be supported in the set of supporting planes to a bounded, convex domain D⊂Rn\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D\subset {\mathbb {R}}^n$$\end{document} only if the boundary of D is an ellipsoid. Using closely related methods we study here the relationship between the analytic wave front set for the characteristic function, χD\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi _D$$\end{document}, of a domain D⊂Rn\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D \subset {\mathbb {R}}^n$$\end{document} and singularities of the boundary ∂D\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\partial D$$\end{document} of the domain. For instance we prove that the boundary surface must be real analytic in a neighborhood of a point z∈∂D∈C1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$z \in \partial D \in C^1$$\end{document}, if the analytic wave front set of χD\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi _D$$\end{document} at z contains no other elements than the conormals to ∂D\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\partial D$$\end{document} at z.


Introduction
Hörmander's famous proof of Holmgren's uniqueness theorem showed that a socalled exterior conormal of the support of a distribution f (for instance the conormal of a supporting plane for the support) must belong to the analytic wave front set, WF A ( f ), of f , [6,Theorem 8.5.6]. The analogous fact for hyperfunctions was given independently by Kawai, and Kashiwara, [9,Theorem 4.4.1]. The question to be studied here is if singularities of the boundary of the support imply further singularities of the distribution in addition to the exterior conormal of the boundary. Later results by Hörmander [8] and Kashiwara showed that this is actually the case, if the boundary is sufficiently irregular in a certain sense. For instance, Corollary 2.7 in [8] implies that if z = (0, 0) ∈ supp f and supp f ⊂ {|x 2 | ≥ |x 1 | γ } for some γ < 2, then (z, ξ) ∈ WF A ( f ) for all ξ ∈ R 2 \{0}. On the other hand, simple examples show that an analytic singularity of the boundary surface in general does not lead to new singularities in WF A ( f ) (see Example 1 below). However, if f is a characteristic functions χ D for an open subset D ⊂ R n with C 1 boundary or a product of a characteristic function with a real analytic function, we show here that analytic singularities of the boundary do imply additional singularities of the function. For instance, if WF A (χ D ) at the boundary point z contains no other singularities than the conormals to the boundary at z, then the boundary must be real analytic in a neighborhood of z (Corollary 1).
It turns out that it is better to begin by studying distributions with support in a hypersurface. Let be a C 1 hypersurface y = (x) in an open set U ⊂ R n+1 , let q j be continuous functions on , and define a distribution f supported in by (1.1) If is C ∞ smooth and all q j ∈ C ∞ , then the wave front set of f is contained in the conormal to , in Hörmander's notation (

1.2)
Similarly, if the hypersurface is real analytic, and all q j are real analytic functions, then the same inclusion holds for the analytic wave front set of f We are interested in strong converses to those implications, stating that if f has some microlocal regularity, much weaker than (1.2) or (1.3), then and q j must be C ∞ or real analytic, respectively. In this direction we prove the following.

Theorem 1
Let be a C 1 hypersurface in a real analytic manifold M, let f ∈ D (M) be supported in , and let z ∈ supp f . Assume that v ∈ T z (M) is a tangent vector to M at z that is transversal to and that Then there exists a neighborhood U of z such that the surface is real analytic in U and the distribution f has the form (1.1) in suitable local coordinates in U with all q j real analytic.
satisfies the hypotheses of Theorem 1, since differentiation and multiplication by real analytic functions preserve the analytic wave front set.
Remark As we saw above, (1.3), the conclusion of Theorem 1 implies WF A ( f ) ⊂ N * ( ), which is a much stronger statement than the assumption. This is not surprising, because the fact that WF A ( f ) ⊂ N * ( ) follows directly from the assumptions by Kashiwara's Watermelon theorem [9,Theorem 4.4.3].
Theorem 1 can be used to obtain a considerably simplified proof of the main result, Theorem 1.1, of our recent article [3]. See Sect. 3 for details.
Here is an example of a function f in R 2 whose support is bounded by a C 2 curve with a discontinuity in the third derivative at z = (0, 0), yet WF A ( f ) z consists precisely of the conormal to the boundary at z.

Example 1
Let u(x, y) be the characteristic function for the region y > x 2 and v(x, y) the characteristic function for the region y > is equal to the union of the conormals of the two curves, and since the curves have the x-axis as common tangent at the the origin, the only wavefronts of f at the origin are (0, ±1). On the other hand, the boundary of the support of f is the curve y = ψ(x) = min(u(x), v(x)), which is C 2 but not C 3 , since ψ(x) = x 2 for x < 0 and ψ(x) = x 2 − x 3 for x > 0, and hence ψ (x) = −6H (x). This shows that a singularity of the boundary curve does not imply additional singularities of the distribution beyond the conormal of the boundary surface.
The assertion of Theorem 1 is no longer true if the analytic wave front set is replaced by the C ∞ wave front set.

Example 2
Let be the curve y = |x| 3 in R 2 and let q(x) be the function where ψ ∈ C ∞ c (R) and ψ = 1 in some neighborhood of the origin. Define a distribu- Then WF( f ) ⊂ N * ( ), although is not smooth.
Proof Since q(x) is smooth and the curve is smooth outside the origin we need only prove that the only wave fronts above the origin are (ξ, η) = (0, ±1). Let us compute the Fourier transform of f along an arbitrary ray through the origin, not parallel to the η axis, Replacing f by χ f , where χ is a smooth cut-off function, equal to 1 in a neighborhood of the origin and supported in |x| ≤ 1/ √ 2|a|, and making the change of variable x(1 ± ax 2 ) = t transforms the last two integrals into the form u a (λ) where u a ∈ C ∞ c and depends smoothly on a. This shows that χ f (ξ, η) tends rapidly to zero in the region |ξ | ≥ ε|η| for every ε > 0, hence WF( f ) ⊂ N * ( ) as claimed.

Proofs
We will first show that the assumptions of Theorem 1 imply that f can be represented as in (1.1) with ∈ C 1 and q j continuous.

Proposition 1 Under the assumptions of Theorem 1 there exists a neighborhood U of z and local coordinates in U such that can be written y
> 0, and the distribution f can be written as in (1.1) with q j continuous.
Proof Since the tangent vector v is transversal to we can choose coordinates (x, y) = (x 1 , . . . , x n , y) in a neighborhood of z such that v(ϕ) = ∂ y ϕ at z, in other words, v = (0, . . . , 0, 1). From now on we shall denote the coordinates of the point z by (x 0 , y 0 ). The hypersurface can then be represented y = (x) for some function ∈ C 1 in a neighborhood U of (x 0 , y 0 ). By a translation of the y-coordinate we can of course achieve that > 0. The assumption (1.4) can now be written and this implies of course the same for WF( f ), that is, for some δ > 0. For any f ∈ C ∞ c (R n+1 ) and ψ ∈ C ∞ c (R), denoting the partial Fourier transform with respect to x by F x , we can write ) and hence To see that we can choose a common m for all x ∈ U it is sufficient to observe that the distribution f must have finite order in U , after U has been replaced by a slightly smaller neighborhood of z, if necessary. To prove that all q k (x) must be continuous we choose a test function This completes the proof of the proposition.
We next prove that the assertions of Theorem 1 must hold under the simplifying assumption that the highest coefficient q m−1 in the expression (1.1) for f is different from zero at x = x 0 . As in [2] and [3] the arguments consist essentially of algebraic manipulations of sets of equations of the type (2.6).

Proposition 2
Let the distribution f be given by (1.1), where all q j are continuous and ∈ C 1 . Let (x 0 , (x 0 )) ∈ , assume that q m−1 (x 0 ) = 0 and that Then and q j , j = 0, 1, . . . , m − 1, are real analytic in some neighborhood of x 0 .
The proof of Proposition 2 will fill almost all of the rest of this section. Let π be the projection R n+1 (x, y) → x ∈ R n and π * the pullback of test The assumption (2.5) implies that π * f is real analytic near x 0 . If m = 1, the proof is now immediate: since π * ( f ) = q 0 (x) and π * (y f ) = q 0 (x) (x) are real analytic and q 0 (x 0 ) = 0 by assumption, it follows that (x) = π * (y f )/q 0 (x) must be real analytic in some neighborhood of x 0 .
To explain the idea of the proof we next consider the case m = 2. Choosing in turn ϕ(x, y) = ψ(x), ψ(x)y, ψ(x)y 2 , and ψ(x)y 3 in (1.1) and using the wave front assumption we see that the functions must be real analytic. In matrix form the system (2.6) can be written Our main goal is to prove that the function is real analytic. When this is done it will follow immediately from the Eq. (2.6) that all q j are real analytic. We can eliminate the quantities q 0 and q 1 from the system (2.6) in the following way. Multiply the first equation by 2 , the second by −2 , and the third by 1 and add all three equations. Then do the same with the second through fourth equations. This gives the system which in matrix notation can be written The determinant of the matrix and q 1 (x 0 ) = 0 by assumption. Hence we can solve for instance from the system (2.7) and obtain where F and G are real analytic and G( The corresponding fact for the general case is the main point of the proof of Proposition 2. We formulate it as a lemma.

Lemma 1 Under the hypotheses of Proposition 2 the function (x) must be real analytic in some neighborhood of x 0 .
Proof The proof for the general case is parallel to that of the special case just discussed: we will show (1) that we can eliminate all the functions q j to obtain a linear system in the "unknowns" , . . . , m , and (2) that the matrix of the resulting systemanalogous to A in (2.8)-is non-singular at x 0 .
Introduce the matrix M that consists of m columns and infinitely many rows and is defined as follows. The first column is 1, , 2 , . . ., the elements of the second column are the formal derivatives of those of the first, 0, 1, 2 , 3 2 , . . ., and the elements of the third column are the second derivatives of the same elements and so on. In other words, the entries m k, j of the matrix M can be written (D denotes formal differentiation with respect to ) We are interested in the dependence of the matrix M s on s. We will first show that and ( Observing that all derivatives of order < m with respect to t of the expression vanish at t = gives (2.14). This proves SM 0 = M 1 . The fact that M s T = M s+1 for all s ≥ 0, which implies the second identity in (2.10), follows immediately from the formula The identity SM s = M s+1 for arbitrary s now follows from This completes the proof of (2.10). Note that the identity SM 0 = M 0 T shows that det S = det T = m .
(2.15) Introduce a notation for column vectors as follows and Sets of equations π * ( f y k ) = h k of the type (2.6) can then be written It is now very easy to eliminate the functions q j from systems of Eq. The derivative of det B(t) with respect to t is equal to a sum of terms of the form But T (t) is equal to times the identity matrix. Therefore each of the matrices above contains two proportional columns, which implies that all the determinants are zero, so T (t) is independent of t as claimed. B(0) is a triangular matrix, since and det B(0) is therefore easily seen to be equal to b m q m m−1 with b m = 0. From (2.24) we see that for instance if m = 4 and similarly for arbitrary m we get (2.20). This completes the proof of Lemma 2.
The sign of b m is not important for us. However, it is easily seen that the sign of b m is equal to the parity of the permutation that reverses the order of m elements, and

End of proof of Lemma 1
Since the matrix H is non-singular at x 0 by Lemma 2 and we may assume that (x) > 0 it follows from (2.19) that (x) is real analytic in a neighborhood of x 0 .

End of proof of Proposition 2
By Lemma 1 we know that (x) must be real analytic in some neighborhood of x 0 . The triangular structure of the system of the first m of the equations (2.6) for arbitrary m shows that the argument given in the paragraph following (2.6) to prove that all q j must be real analytic in a neighborhood of x 0 is actually valid for arbitrary m. This completes the proof of Proposition 2.
Proof of Theorem 1 Choose coordinates according to Proposition 1. Let m be the largest number for which x 0 ∈ supp q m−1 . It remains only to consider the case when q m−1 (x 0 ) = 0. By definition a function h that is real analytic in a neighborhood of x 0 ∈ R n can be extended to a holomorphic function in a complex neighborhood in C n of x 0 . From now on we will therefore think of holomorphic germs at x 0 ∈ R n ⊂ C n . We will use the fact that unique factorization holds in the ring of germs of holomorphic functions of several variables, [7, Theorem 6. must be real analytic near x 0 for every s. For sufficiently large s this leads to a contradiction unless m = u/v is analytic near x 0 . And since we may assume that (x 0 ) > 0 we can conclude that is real analytic. And, as was mentioned earlier, the Eq. (2.6) now show that all q j must be real analytic. This completes the proof of Theorem 1.

Radon Transforms Supported in Hypersurfaces
For continuous compactly supported functions f we define the Radon transform R f as the integral of f over the hyperplane L(ω, p) = {x ∈ R n ; ω · x = p} with respect to surface measure ds R f is of course an even function, R f (ω, p) = R f (−ω, − p). The manifold H n of hyperplanes in R n is identified with S n−1 × R with (ω, p) and (−ω, − p) identified. It is well known that the Radon transform R f of a compactly supported distribution f can be defined by for even test functions ϕ(ω, p) on S n−1 × R. Here the action of the distribution g ∈ D (H n ) on the test function ϕ is denoted g, ϕ , and a locally integrable function g(ω, p) is identified with a distribution by the formula where dω is area measure on S n−1 , and R * ϕ is defined by For more details see [4] or [2,Sect. 2]. It was proved in [2] and [3] that the Radon transform of a compactly supported distribution can be supported in the set of tangent planes to a bounded convex domain D ⊂ R n only if the boundary of D is an ellipsoid. It has been asked if the following local version of that theorem is valid. Let D be a convex (not necessarily bounded) domain with C 1 boundary, and denote by D the set of tangent planes to ∂ D, which is a hypersurface in H n . Let L 0 be a tangent plane to ∂ D at the point z ∈ ∂ D. Assume that there exists a distribution f that is supported in D such that the restriction of R f to a neighborhood V ⊂ H n of L 0 is supported in V ∩ D . The question is if ∂ D must then be an ellipsoid in some neighborhood of z. A partial answer to this question was given by Mark Agranovsky [1]. If ∂ D is C 1 in a neighborhood of z, then Theorem 1 implies that under those conditions D must be a real analytic hypersurface in the manifold H n in a neighborhood of L 0 (Theorem 2). Moreover, if ∂ D is assumed to be C 1,1 , then ∂ D must itself be real analytic (Theorem 3). Here we have used the standard notation C 1,1 to denote the set of C 1 functions whose first derivatives are Lipschitz continuous, and a hypersurface is said to be C 1,1 if its defining function is C 1,1 . We wish to point out that in the proofs of those facts we do not need to use the assumption that the distribution g is a Radon transform, but only the fact that certain elements are not in the analytic wave front set of g.
Let D be a strictly convex (not necessarily bounded) domain in R n with C 1 boundary and let L 0 be a hyperplane that is tangent to the boundary ∂ D of D at the point z. Note that the intersection L ∩ D must be bounded for all hyperplanes L in some neighborhood of L 0 , since D is strictly convex. The set H n of hyperplanes in R n has a natural structure as a real analytic manifold. In a neighborhood of a hyperplane L(ω 0 , p 0 ) = {x ∈ R n ; x · ω 0 = p 0 } with ω 0 n = 0 we can take ω 1 , . . . , ω n−1 , p as coordinates on H n .

Theorem 2
Assume that there exists a distribution f , supported in D, and a neighborhood V of L 0 such that the restriction g = R f | V is supported in the set D ⊂ H n of tangent planes to ∂ D. Then g has the form (1.1) in suitable coordinates, and the surface = D and all q j are real analytic in a neighborhood of L 0 .
For the proof of Theorem 2 we shall need the following well known fact. g(ω, p) = R f (ω, p) is the Radon transform of a compactly supported distribution, then the conormal of every line R p → (ω, p) for ω ∈ S n−1 is disjoint from the analytic wave front set of g.

Lemma 3 If
Proof In two dimensions, the assertion of the lemma is closely related to the well known fact that the so-called sinogram, which is a density plot of the Radon transform R f (ω, p) in an α p plane where cos α = ω 1 , never contains discontinuities along curves with vertical tangents (the p-axis is vertical). The lemma is a special case of much stronger statements that are well known (see e.g. [6], Sects. 8.2 and 8.5), so we only sketch the proof. We first consider the case of the C ∞ wave front set, WF(g). Let L 0 be an arbitrary hyperplane in R n . Choose coordinates so that L 0 is defined by x n = 0. In a neighborhood V of L 0 , in fact in the open set of all hyperplanes L(ω, p) with ω n = 0, we can choose coordinates y 1 , . . . , y n on the manifold of hyperplanes by y j = ω j /ω n for 1 ≤ j ≤ n − 1 and y n = p/ω n . Writing x = (x , x n ) and ω = (ω , ω n ) we note that x · ω = x · ω + x n ω n = p is equivalent to The so-called parametric Radon transform R p f can be written in terms of the ycoordinates Since ds/dx = 1 + |y | 2 this gives the following expression for R f in terms of the y-coordinates R f (y) = 1 + |y | 2 R p f (y 1 , . . . , y n ) = 1 + |y | 2 The factor 1 + |y | 2 is a real analytic function of y, so we don't need to pay attention to it here. For each fixed x the integrand, as a function of y, has the form h(y n − x · y ), for some function h of one variable. The wave front set of such a function must be contained in the conormal of the level sets y n − x · y = c, which consists of elements of the form (y, η), where η = λ(−x , 1) (3.1) with λ ∈ R \ {0} and x = (x , x n ) ∈ supp f . If f is supported in the ball |x| ≤ b, it follows that, for any (y, η) ∈ WF(g), η must have the form (3.1) with |x | ≤ b, that is In particular, if (y, η) ∈ WF(g), then η n must be different from zero. This proves the assertion of the lemma with WF(g) instead of WF A (g). The argument is exactly the same for the case of WF A (g). This completes the proof.
Proof of Theorem 2 Lemma 3 shows that we can apply Theorem 1 to the distribution g and the hypersurface p = (ω) in the manifold of hyperplanes. The conclusion is that (ω) must be real analytic and that g(ω, p) must be expressible in the form (1.1) in suitable coordinates, where the surface and all densities q j are real analytic.
It is important to observe that the set of tangent planes to a hypersurface S ⊂ R n can be a real analytic hypersurface in the manifold of hyperplanes, even if S is not smooth. For example, if D is the set {x ∈ R 2 ; x 2 > (3/4)|x 1 | 4/3 }, then the line x 2 = ξ 1 x 1 + ξ 2 is tangent to ∂ D if and only if ξ 2 = −ξ 4 1 /4. However, if the boundary ∂ D in Theorem 2 is assumed to be in C 1,1 , then we can in fact conclude that the boundary is real analytic. This is an immediate consequence of the following property of the Legendre transform.

Proposition 3 Assume that the real-valued function h(x) = h(x 1 , . . . , x n ) is real analytic and strictly convex, and that its Legendre transform h(ξ ) is in C 1,1 . Then h (x) is non-degenerate (as a quadratic form) and h(ξ ) is real analytic.
The assertion of Proposition 3 is a special case of Theorem 4.2.2 in Chapter X of [5].

Theorem 3
Assume, in addition to the hypotheses of Theorem 2, that the boundary of D is in C 1,1 in a neighborhood of z ∈ L 0 ∩ ∂ D. Then the boundary of D is real analytic in a neighborhood of z.
Proof Choose coordinates so that L 0 is the hyperplane x n = 0 and D is given by Hence the hyperplane x n = x · ξ − q is tangent to ∂ D if and only if h(ξ ) = q. From Theorem 2 we know that the surface = D ⊂ H n of tangent planes to ∂ D is real analytic in a neighborhood V of L 0 . Using ξ 1 , . . . , ξ n−1 , q as coordinates on V ⊂ H n we see that the function h(ξ ) must be real analytic near ξ = (0, . . . , 0). Since h is convex, the Legendre transform of h is equal to h, [5, ch. X, vol. 2, Sect. 1.3], [10,Sect. 12]. And since h(x ) was assumed to be C 1,1 , Proposition 3 now implies that h(x ) is real analytic, which completes the proof.
Since the proof of Proposition 3 is much easier than that of the considerably sharper Theorem 4.2.2 in [5], we have included a proof of Proposition 3 here. Our arguments give without additional effort the following slightly stronger statement. h(x 1 , . . . , x n ) is strictly convex and in C p for some integer p ≥ 2, and that its Legendre transform h(ξ ) is in C 1,1 . Then h (x) is non-degenerate and h(ξ ) is in C p . Moreover, if h is real analytic, then h is real analytic.

Proposition 4 Assume that the real-valued function h(x) =
If h ∈ C p , p ≥ 2, and the second derivative h (x) (considered as a quadratic form) is non-degenerate, then it is very easy to see that h must belong to C p−1 , and hence where x is given by is non-degenerate, the inverse of that map must then also be C p−1 , that is, the map ξ → x = h (ξ ) must be C p−1 . This shows that h ∈ C p as claimed.
Therefore the assertions of Proposition 4 follow after we have proved the following. We shall use (3.4). Let x and ξ be connected by h (x) = ξ . Then h (ξ ) = x, so h (x)/x = ξ/ h (ξ ). Formula (3.5) now shows that ξ/ h (ξ ) tends to zero, and hence is unbounded, which proves that h is not C 1,1 .
As mentioned above Theorem 1 can be used to simplify the proof of Theorem 1.1 in [3]. A crucial step in the proof of Theorem 3.1, which implies Theorem 1.1, was to show that the determinant of 0 (ω) in (5.4) is not identically zero. The determinant of 0 (ω) is equal to a multiple of q m−1 (ω)q m−1 (−ω) as is seen from formula (5.5).
Here q m−1 (ω) is a function on S n−1 , and it is the highest coefficient in an expression for a distribution on H n with support in a hypersurface, similar to (1.1) above. Now, Theorem 1 shows that q m−1 (ω) must be real analytic, and therefore it is clear that q m−1 (ω)q m−1 (−ω) cannot vanish identically unless q m−1 vanishes identically.
Funding Open access funding provided by Stockholm University.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.