A hypersurface containing the support of a Radon transform must be an ellipsoid. I

If the Radon transform of a compactly supported distribution $f \ne 0$ in $\mathbb R^n$ is supported on the set of tangent planes to the boundary $\partial D$ of a bounded convex domain $D$, then $\partial D$ must be an ellipsoid. As a corollary of this result we get a new proof of a recent theorem of Koldobsky, Merkurjev, and Yaskin, which settled a special case of a conjecture of Arnold that was motivated by a famous lemma of Newton.

In other words, Rf is a distribution on the manifold of lines in the plane that vanishes in the open set of lines that intersect the open unit disk. Since Rf obviously vanishes on the open set of lines that are disjoint from the closed disk, it follows that the distribution Rf is supported on the set of lines that are tangent to the circle.
By means of an affine transformation it is easy to construct a similar example where the circle is replaced by an arbitrary ellipse. For an arbitrary ellipsoidal domain D ⊂ R n , n > 2, it is also easy to construct examples of distributions f supported in D such that the Radon transform Rf is supported on the set of tangent planes to the boundary of D. However, surprisingly, for other convex domains than ellipsoids such distributions do not exist.
Since we will consider arbitrary convex -not necessarily smooth -domains, we have to replace the notion of tangent plane by supporting plane. A supporting plane for D is a hyperplane L such that L ∩ D is non-empty and one of the components of R n \ L is disjoint from D.
Theorem 1. Let D be an open, convex, bounded and symmetric (that is D = −D) subset of R n , n ≥ 2, with boundary ∂D. If there exists a distribution f = 0 with support in D such that the Radon transform of f is supported on the set of supporting planes for D, then ∂D must be an ellipsoid.
The more general case when D is not assumed to be symmetric turned out to require different arguments from those given here. This case will therefore be treated in a forthcoming article.
Remark. Our arguments prove in fact a stronger statement, Theorem 3, which is local in ω but global in p; see Section 5.
Denote by V (ω, p) the volume of one of the components of D \ L(ω, p), where D ⊂ R n is a convex, bounded domain and L(ω, p) is the hyperplane x · ω = p that we assume intersects D. A famous conjecture of Arnold (Problem 1987-14 in Arnold's Problems, [A]) asserts that if V (ω, p) is an algebraic function, then n must be odd and D must be an ellipsoid. The case n even has been settled long ago by Vassiliev [V1], see also [V2]. For odd n the question is still open. The special case when n is odd and p → V (ω, p) is assumed to be a polynomial function of degree ≤ N for all ω and some N has also been studied and was settled recently by Koldobsky, Merkurjev, and Yaskin for domains D with smooth boundary, [KMY]; see also [AG1]. If the domain D is symmetric, then this question is answered by Theorem 1. In [AG2] the case when p → V (ω, p) is algebraic and satisfies a certain additional condition is reduced to the case p → V (ω, p) is polynomial.
Corollary 1. Let D ⊂ R n , n ≥ 2, be as in Theorem 1, and assume that there exists a number N such that p → V (ω, p) is a polynomial of degree ≤ N for all ω ∈ S n−1 . Then ∂D must be an ellipsoid.
Proof. Let χ D be the characteristic function of D and choose an integer k such that 2k > N . The assumption implies that ∂ 2k p (Rχ D )(ω, p) = 0 for all p in the interval inf{x · ω; x ∈ D} < p < sup{x · ω; x ∈ D}, and obviously (Rχ D )(ω, p) = 0 for all other p. This shows that the distribution ∂ 2k p Rχ D must be supported on the set of supporting planes to ∂D, a hypersurface in the manifold of hyperplanes in R n . Define the distribution f in R n by f = ∆ k χ D where ∆ denotes the Laplace operator. The formula p Rh(ω, p) with h = χ D now shows that the distribution Rf must be supported on the set of supporting hyperplanes. By Theorem 1 this implies that ∂D is an ellipsoid.
A somewhat related problem is treated in a recent article by Ilmavirta and Paternain, [IP]. It is proved that the existence of a function in L 1 (D), D ⊂ R n , whose X-ray transform (integral over lines) is constant, implies that the boundary of D is a ball.
In Section 2 we will write down an expression for an arbitrary distribution g on the manifold of hyperplanes that is equal to the Radon transform of some compactly supported distribution and is supported on the submanifold of supporting planes to ∂D. In Section 3 we will use the description of the range of the Radon transform to write down the conditions for g to be the Radon transform of a compactly supported distribution f . Those conditions will be an infinite number of polynomial identities in the supporting function ρ(ω) for D and the densities q j (ω) that define the distribution g. Thereby the problem is transformed to a purely algebraic question. In Section 4 we analyze the polynomial identities and prove (Theorem 2) that they imply that the supporting function ρ(ω) must be a quadratic polynomial, which together with the fact that ρ(ω) > 0 implies that ∂D is an ellipsoid. In Section 5 we finish the proof of Theorem 1 and prove the semi-local version Theorem 3. An outline of the proof of Theorem 2 is given in Section 4.
2. Distributions on the manifold of hyperplanes. As is well known the manifold P n of hyperplanes in R n can be identified with the manifold (S n−1 × R)/(±1), the set of pairs (ω, p) ∈ S n−1 × R, where (ω, p) is identified with (−ω, −p). Thus a function on P n can be represented as an even function g(ω, p) = g(−ω, −p) on S n−1 × R. In this article a distribution on P n will be a linear form on C ∞ e (S n−1 × R), the set of smooth even functions on S n−1 × R, and a locally integrable even function h(ω, p) on S n−1 × R will be identified with the distribution where dω is area measure on S n−1 . Using the standard definition of R * , we can then define the Radon transform of the compactly supported distribution f on R n as the linear form Let D be a bounded, convex subset of R n with boundary ∂D. Here we will also assume that D is symmetric with respect to some point, which we may assume to be the origin, so D = −D. We shall denote the supporting function of D by ρ(ω), that is Since D is symmetric, ρ will be an even function, because Clearly a hyperplane x · ω = p intersects D if and only if |p| < ρ(ω), and it is a supporting plane to ∂D if and only if p = ±ρ(ω). We shall consider the hypersurface in P n that consists of all the supporting planes to ∂D. Since the origin in R n is contained in (the interior of) D, none of the supporting planes can contain the origin, hence ρ(ω) > 0 for all ω.
We next show that we may assume that the distribution f is even, f (x) = f (−x), which implies that g = Rf is even in ω and p separately.
Lemma 1. Assume that there exists a compactly supported distribution f = 0 such that Rf is supported on p = ±ρ(ω). Then there exists an even distribution with the same property.
Proof. Let f = 0 be such that Rf is supported on p = ±ρ(ω). We have to construct an even distribution with the same property. It is clear that the distribution f (−x) has the same property. Hence the even part (f (x) + f (−x))/2 and the odd part (f (x) − f (−x))/2 of f both have the same property. If the even part is different from zero there is nothing more to prove, so we may assume that the odd part is different from zero. Set h(x) = (f (x) − f (−x))/2. Then h 1 = ∂h/∂x 1 is an even distribution. It remains to prove that Rh 1 is supported on p = ±ρ(ω). But this follows from the formula R(∂ x 1 h)(ω, p) = ω 1 ∂ p Rh(ω, p), which is easily seen by application of the formula Rϕ(ω, τ ) = ϕ(τ ω) to both members.
From now on we will therefore assume that the distribution f is even, which implies that g(ω, p) = Rf (ω, p) is even in ω and p separately. This implies that the measure q 0 in (1) must be even, so we may write for some q 0 (ω).
If the boundary ∂D is smooth and hence ρ(ω) is smooth, we can argue similarly, using the fact that δ (j) (·) is odd if j is odd and even if j is even, to see that an arbitrary distribution g(ω, p) that is even in ω and p separately and is supported on p = ±ρ(ω) can be written is not smooth, this is not always true. Note that δ (j) (p ± ρ(ω)) should be interpreted as the jth distribution derivative of δ(p ± ρ(ω)) with respect to p.
However, if g = Rf for some compactly supported distribution f , then we shall see that the representation (3) is valid and that the distributions q j (ω) must be continuous functions.
Lemma 2. Let f be a compactly supported even distribution in R n and let g = Rf . Assume that g is supported on the set of supporting planes to D.
Then there exists a number m and continuous functions q j (ω) such that the distribution g can be written in the form (3).
We note that the map ω → R ω f must be continuous in the sense that ω → R ω f, ψ is continuous for every test function ψ ∈ C ∞ (R). Rf can be expressed in terms of R ω f as follows. If ϕ(ω, p) = ϕ 0 (ω)ϕ 1 (p), then To prove the second last identity we replace the integrals by Riemann sums and observe that the function x → ϕ 1 (x · ω) together with all its derivatives depends continuously on ω. The formula (4) shows that if g = Rf is supported on the hypersurface p = ±ρ(ω), then R ω f must be supported on the union of the two points p = ±ρ(ω) for every ω. Hence R ω f can be represented as the right hand side of (3) for every ω. It remains only to prove that all q j (ω) are continuous. It is enough to prove that q j (ω) is continuous in some neighborhood of an arbitrary ω 0 ∈ S n−1 . If we choose ψ such that ψ(p) = 0 in some neighborhood of −ρ(ω 0 ) then We have seen that the expression on the right hand side must be a continuous function of ω for every ψ. Choosing ψ(p) such that ψ(p) = 1 in a neighborhood of ρ(ω 0 ) shows that q 0 (ω) is continuous at ω 0 . Next choosing ψ(p) such that ψ(p) = p in a neighborhood of ρ(ω 0 ) shows that q 1 (ω) is continuous. Continuing in this way completes the proof.
Our next goal will be to write down the conditions on q j (ω) and ρ(ω) for g(ω, p) to belong to the range of the Radon transform.
3. The range conditions. It is well known that a compactly supported (ω, p)-even function or distribution g(ω, p) belongs to the range of the Radon transform if and only if the function ω = (ω 1 , . . . , ω n ) → R g(ω, p)p k dp is the restriction to the unit sphere of a homogeneous polynomial of degree k in ω for every non-negative integer k.
We next compute the moments R g(ω, p)p k dp for the expression (3). By the definition of δ (j) , for any a ∈ R, R δ (j) (p − a)p k dp = 0 if j > k and Hence if j ≤ k and a = 0 we get R δ (j) (p − a) + (−1) j δ (j) (p + a) p k dp = 0 if k is odd and if k is even. For arbitrary non-negative integers k, j we define the constant c k,j by c 0,0 = 1 and Note that the second expression on the first line above makes sense also for j > k and is equal to zero then, although the first expression does not make sense if j > k. For instance, if j = 2, then c k,j = k(k − 1) for all k. Then we can now summarize our computations as follows: if g(ω, p) is defined by (3), then R g(ω, p)p k dp = 0 if k is odd and (6) R g(ω, p)p k dp = 2 k j=0 c k,j (−1) j q j (ω)ρ(ω) k−j , if k is even.
Thus, for g(ω, p) to be the Radon transform of a compactly supported distribution it is necessary and sufficient that is equal to the restriction to S n−1 of a homogeneous polynomial for every even k. In the next section we will show that those conditions imply that ρ(ω) 2 must be a quadratic polynomial.
The fact that ρ(ω) 2 is a quadratic polynomial, combined with the fact that ρ(ω) is everywhere positive on S n−1 , implies that ∂D, the boundary of the region D, is an ellipsoid. Indeed, if ρ(ω) is also rotationally invariant, ρ(ω) = c|ω| 2 , then it is obvious that D must be rotationally invariant, hence a ball. And since ρ(ω) is (strictly) positive we can find an affine transformation A such that ρ(Aω) 2 = c|ω| 2 for some c. This implies that D must be an affine image of a ball, hence an ellipsoid.
4. Analysis of the polynomial identities. The purpose of this section is to prove the following purely algebraic result. We shall denote the set of restrictions to the unit sphere of homogeneous polynomials of degree k by P k .
Theorem 2. Assume that the strictly positive and continuous function ρ(ω) on S n−1 and the continuous functions q 0 , q 1 , . . . , q m−1 , not all zero, satisfy the infinitely many identities where c k,j is defined by (5). Then ρ(ω) 2 is a (not identically vanishing) quadratic polynomial.
In (8) we have omitted the factor (−1) j that occurred in (7), because in the proof of Theorem 1 we can of course apply Theorem 2 to the functions (−1) j q j .
The first step of the proof of Theorem 2 will be to eliminate the m functions q j from sets of m + 1 of the equations (8). The result is a set of infinitely many polynomial identities in ρ(ω) 2 with the polynomials p k as coefficients, as will be explained in Lemma 4. Considering a set of m of those identities as a linear system of equations in the m quantities ρ 2 , ρ 4 , . . . , ρ 2m we can solve ρ 2 as a rational function in the coefficients p 2k and hence as a rational function of ω, provided the determinant of the corresponding coefficient matrix (17) is different from zero. Under the same assumption we can prove rather easily that ρ 2 must be a polynomial by considering sets of m such linear systems together. This entails considering the translation operator (p 2k , p 2k+2 , . . . , p 2k+2m−2 ) → (p 2k+2 , p 2k+4 , . . . , p 2k+2m ) on m-vectors of polynomials from the infinite sequence p 0 , p 2 , . . .. This operator is given by the matrix S introduced below (18). The matrix S has m identical eigenvalues ρ 2 , hence its determinant is ρ 2m . The crucial fact that the matrix (17) is non-singular (Proposition 1) is an easy consequence of the fact that that Jordan canonical form of S consists of just one Jordan block (Lemma 6).

Proof. Introduce the polynomials
Then c 2k,j = h j (2k). Since for any j for some constants a ν , it is clear that any matrix of the form (h j (t i )), j, i = 0, 1, . . . , m − 1, with all t i distinct can be transformed to a van der Monde matrix by elementary row operations, hence its determinant must be different from zero.
Lemma 3 shows that an arbitrary set of m consecutive rows C 2k = (c 2k,0 , . . . , c 2k,m−1 ), k = r, r + 1, . . . , r + m − 1, from the matrix c 2k,j forms a linearly independent set of m-vectors. Therefore it is clear that an arbitrary row C 2(r+m) with r ≥ 0 can be expressed as a linear combination of the m preceding rows. This is made precise in the next lemma. Proof. Observe first of all that the identity m k=0 (−1) k m k h(k) = 0 must be valid whenever h(t) is a polynomial of degree at most m − 1. This is obvious from the fact that the operator h → m k=0 (−1) k m k h(k) is the composition of m first order difference operators h → h(t + 1) − h(t). As we saw above, the function k → c 2k,j is a polynomial of degree j, hence This completes the proof of Lemma 4.
A natural start towards a proof of Theorem 2 would be to try to solve ρ 2 from some set of m of the equations (12) considering the equations as linear expressions in the m unknowns ρ 2 , . . . , ρ 2m . If the matrix is non-singular, we can solve ρ 2 from the linear system (12) with r = 0, 1, . . . , m − 1 and obtain ρ 2 as a rational function where F and G are polynomials and G = det A 0 . As we shall see below (Lemma 5) it is easy to strengthen this argument by considering m such systems together and thereby prove that ρ 2 is a polynomial. Therefore our main task in the rest of the proof of Theorem 2 will be to prove that the matrix A 0 is non-singular.
Proposition 1. Let the polynomials p 2k be defined as in Theorem 2 and assume that the function q m−1 (ω) is not identically zero. Then the matrix (17) is non-singular.
The proof will be given at the end of this section.
The recurrence relations (13) then show that the translation operator P 2k → P 2k+2 is given by the matrix so that SP 2k = P 2k+2 and S k P 0 = P 2k for all k.
For instance, if m = 3 and m = 4, then Defining r m = −1 and taking (15) into account we obtain so S has the m-fold eigenvalue ρ 2 , and the determinant of S is det S = (−1) m−1 r 0 = ρ 2m .
Proof. We have already seen that ρ(ω) 2 must be a rational function, if the matrix (17) is non-singular. The equations P 2k+2j = S k P 2j for j = k, k + 1, . . . , k + m − 1 can be combined to the matrix equation Denote the determinant of A 0 , which is a polynomial, by d(ω). Since det S = ρ 2m , equation (19) implies that is a polynomial for every k. Since ρ 2 is a rational function, this proves that ρ 2 must in fact be a polynomial as claimed.
We now turn to the proof of Proposition 1. To motivate the next lemma we make the following observations. Let B = (b k,j ) be the matrix of the system (8), with m columns and infinitely many rows, and let B 0 be the uppermost m × m minor of the matrix B obtained by restricting k to 0 ≤ k ≤ m − 1.
Since B 0 Q = P 0 , those vectors can be written B 0 Q, SB 0 Q, . . . , S m−1 B 0 Q. And since B 0 is non-singular, (21) is equivalent to Therefore we now study the matrix B −1 0 SB 0 . Lemma 6. The matrix B −1 0 SB 0 is an upper triangular matrix of the form where N = (n k,j ) is a nilpotent, upper triangular matrix whose elements next to the diagonal are given by For instance, if m = 5, The exact expression for the matrix N is inessential for us, apart from the fact that all the entries (22) are different from zero.
Proof of Lemma 6. Denote by u 0 , . . . , u m−1 the column vectors of B 0 . The assertion of the lemma is that the matrix of S with respect to the basis u 0 , . . . , u m−1 is ρ 2 I + N . In fact we shall prove that The components of u j = (u j 0 , . . . , u j m−1 ) are Denote by B 1 the second uppermost m × m minor of the matrix B, which is obtained by restricting k in (20) to 1 ≤ k ≤ m. The argument in the proof of Corollary 2 shows that SB 0 = B 1 , in other words if we define u j m as b m,j = c 2m,j ρ 2m−j . Denote by D the formal derivative with respect to ρ. Note that u 0 k = ρ 2k , u 1 k = Dρ 2k for 0 ≤ k ≤ m, and that more generally u j k = D j ρ 2k for 0 ≤ j ≤ m − 1, 0 ≤ k ≤ m. The identity (23) is obvious. To prove (24) we just note that the k:th component of Su 1 satisfies (Su 1 ) k = D(ρ 2 ρ 2k ) = ρ 2 Dρ 2k + 2ρ · ρ 2k = ρ 2 u 1 k + 2ρ u 0 k , 0 ≤ k ≤ m − 1.
For (25) we use Leibnitz' formula to get which completes the proof.
Lemma 7. Let A be an m × m matrix with entries in a field K. Assume that A has one m-fold eigenvalue λ. Let z ∈ K m . Then the vectors A k z, k = 0, 1, . . . , m − 1, span K m if and only if (A − λI) m−1 z = 0.
Proof. The condition is obviously necessary, because if (A − λI) m−1 z = 0, then the equation shows that the m vectors A k z, k = 0, 1, . . . , m − 1, are linearly dependent. The sufficiency follows from the fact that Jordan normal form (over the algebraic closure of K) must consist of just one Jordan block, but can also be seen more directly as follows. Introduce the subspace of K m defined by It is clear that E k ⊂ E k+1 , that E m = K m , and it is easily seen that so the vectors (A − λI) k z must span K m , which proves the lemma.
Proof of Proposition 1. By Lemma 7 it is enough to prove that (B −1 0 SB 0 − ρ 2 I) m−1 Q = 0. Using Lemma 6 we can write The matrix N m−1 has only one non-vanishing entry in the upper right corner. The value of this entry is equal to the product of of all the entries on the diagonal next to the main diagonal described in Lemma 6. And this product is equal to c = (2ρ) m−1 (m−1)!, hence different from zero. Recalling that Q = (q 0 , q 1 , . . . , q m−1 ) t we can now conclude that N m−1 Q = c q m−1 (ω)(1, 0, . . . , 0) t .
By assumption the continuous function q m−1 (ω) is different from zero on some open subset of S n−1 . Thus we have proved that the determinant of the matrix (17) must be different from zero on some open set. But the determinant is a polynomial function, hence equal to a non-zero polynomial. This completes the proof of Proposition 1.
End of proof of Theorem 1. Let m be the largest number for which the coefficient q m−1 in (3) is not identically zero. Then Proposition 1 shows that the matrix (17) must be non-singular. And then Lemma 5 shows that ρ(ω) 2 must be a positive quadratic polynomial. This completes the proof of Theorem 2 and hence of Theorem 1.

A semi-local result.
The arguments given here prove in fact a semilocal version of Theorem 1, where only an arbitrary open set of ω comes into play, but all p ∈ R. A set W of hyperplanes L ∈ P is called translation invariant, if L ∈ W implies that every translate x + L is contained in W .
Theorem 3. Let D be as in Theorem 1, let x 0 ∈ ∂D, and let ω 0 be one of the unit normals of a supporting plane L 0 to D at x 0 . If there exists a distribution f with support in D and a translation invariant open neighborhood W of L 0 , such that the restriction of the distribution Rf to W is supported on the set of supporting planes to D in W , then ∂D must be equal to the restriction of an ellipsoid in some neighborhood of ±x 0 .
This theorem implies Theorem 1, because if the assumptions of Theorem 1 are fulfilled, then the assumptions of Theorem 3 are valid for every x 0 ∈ ∂D, hence ∂D must be equal to an ellipsoid in some neighborhood of every point, hence be globally equal to an ellipsoid.
Proof of Theorem 3. Let f be as in the theorem and set Rf = g. By the range conditions the functions S n−1 ∋ ω → R g(ω, p)p k dp must belong to P k for all k. Set p 0 = x 0 · ω 0 . The assumptions imply that the restriction of g to some neighborhood of (±ω 0 , ±p 0 ) has the form (3). This shows that the assumption (8) of Theorem 2 must be valid for ω in some neighborhood E of ±ω 0 . Note that the functions q j are defined only in some neighborhood of ±ω 0 , whereas ρ(ω), the supporting function of D, is initially defined in all of S n−1 . But in this proof we are only concerned with the restriction of ρ to E. Taking restriction to E wherever relevant we see that the proofs of Corollary 2, Proposition 1, Lemma 5, and Lemma 6, work without change, so the restriction of ρ 2 to E must be a positive quadratic polynomial, and this implies the assertion.