Abstract
We consider steady state solutions of the massive, asymptotically flat, spherically symmetric Einstein–Vlasov system, i.e., relativistic models of galaxies or globular clusters, and steady state solutions of the Einstein–Euler system, i.e., relativistic models of stars. Such steady states are embedded into oneparameter families parameterized by their central redshift \(\kappa >0\). We prove their linear instability when \(\kappa \) is sufficiently large, i.e., when they are strongly relativistic, and prove that the instability is driven by a growing mode. Our work confirms the scenario of dynamic instability proposed in the 1960s by Zel’dovich & Podurets (for the Einstein–Vlasov system) and by Harrison, Thorne, Wakano, & Wheeler (for the Einstein–Euler system). Our results are in sharp contrast to the corresponding nonrelativistic, Newtonian setting. We carry out a careful analysis of the linearized dynamics around the above steady states and prove an exponential trichotomy result and the corresponding index theorems for the stable/unstable invariant spaces. Finally, in the case of the Einstein–Euler system we prove a rigorous version of the turning point principle which relates the stability of steady states along the oneparameter family to the winding points of the socalled massradius curve.
Introduction and Main Results
We consider a smooth spacetime manifold M equipped with a Lorentzian metric \(g_{\alpha \beta }\) with signature \(({}+{}+{}+)\). The Einstein equations read
where \(G_{\alpha \beta }\) is the Einstein tensor induced by the metric, and \(T_{\alpha \beta }\) is the energymomentum tensor given by the matter content of the spacetime; Greek indices run from 0 to 3, and we choose units in which the speed of light and the gravitational constant are equal to 1. To obtain a closed system, the field equations (1.1) have to be supplemented by evolution equations for the matter and by the definition of \(T_{\alpha \beta }\) in terms of the matter and the metric. We consider two matter models, namely a collisionless gas as described by the collisionless Boltzmann or Vlasov equation and an ideal fluid as described by the Euler equations. This results in the Einstein–Vlasov and Einstein–Euler systems, respectively. We study these systems under the assumption that the spacetime is spherically symmetric and asymptotically flat, but we first formulate them in general, together with our main results.
The Einstein–Vlasov System
In the case of a collisionless gas matter is described by the number density f of the particles on phase space. The world line of a test particle on M obeys the geodesic equation
where \(x^\alpha \) denote general coordinates on M, \(p^\alpha \) are the corresponding canonical momenta, \(\Gamma ^\alpha _{\beta \gamma }\) are the Christoffel symbols induced by the metric \(g_{\alpha \beta }\), the dot indicates differentiation with respect to proper time along the world line of the particle, and the Einstein summation convention is applied. We assume that all the particles have the same rest mass \(m_0\ge 0\) and move forward in time, i.e., their number density f is a nonnegative function supported on the mass shell
a submanifold of the tangent bundle TM of the spacetime manifold M which is invariant under the geodesic flow. Since we are interested in the massive case \(m_0>0\), we normalize \(m_0=1\), but at a crucial point in our analysis the massless case \(m_0=0\) will play an important role. Letting Latin indices range from 1 to 3 we use coordinates \((t,x^a)\) with zero shift which implies that \(g_{0a}=0\). On the mass shell PM the variable \(p^0\) then becomes a function of the remaining variables \((t,x^a,p^b)\):
Since the particles move like test particles in the given metric, their number density \(f=f(t,x^a,p^b)\) is constant along the geodesics and hence satisfies the Vlasov equation
The energymomentum tensor is given by
where g denotes the modulus of the determinant of the metric, and indices are raised and lowered using the metric, i.e., \(p_\alpha = g_{\alpha \beta }p^\beta \). The system (1.1), (1.2), (1.3) is the Einstein–Vlasov system in general coordinates. We want to describe isolated systems, and therefore we require that the spacetime is asymptotically flat.
In Sect. 3 we will see that steady states of this system can be obtained via an ansatz
where \(E= p_0\) is the local or particle energy and \(\phi \) is a prescribed ansatz function; we refer to (1.4) as the microscopic equation of state. We can now state our main result for the Einstein–Vlasov system in an informal way; the precise meaning of the parameter \(\kappa >0\) is explained in Sect. 3, in particular (3.10) and (3.11).
Theorem 1
Let \((f_\kappa )_{\kappa >0}\) be a oneparameter family of spherically symmetric steady states to the Einstein–Vlasov system with a fixed, decreasing microscopic equation of state \(\phi \) and \(\kappa \) the central redshift of the corresponding steady state. Then,

(a)
For \(\kappa \) sufficiently large, the associated steady state is dynamically unstable in the sense that the linearized Einstein–Vlasov system possesses an exponentially growing mode.

(b)
For any \(\kappa >0\) the phase space of the linearized system splits into three invariant subspaces: the stable, unstable, and center space. The dimension of the stable/unstable space is equal to the negative Morse index of a suitably defined, macroscopic Schrödingertype operator.
The Einstein–Vlasov system models large stellar systems such as galaxies or globular clusters, and the central redshift is a measure of how relativistic the corresponding state is. Part (a) of the theorem says that strongly relativistic galaxies or globular clusters are unstable. This is in sharp contrast to the corresponding Newtonian situation, i.e., to the VlasovPoisson system. For this system any steady state induced by a strictly decreasing microscopic equation of state is nonlinearly stable, cf. [28, 29, 46, 54]. The instability result in Theorem 1 is therefore a genuinely relativistic phenomenon.
A proof of instability which does not rely on the refined spectral information about the existence of growing modes can be found in Theorem 4.8, while the existence of a growing mode is given by Theorem 4.25. A rigorous statement of exponential trichotomy is provided in Theorem 4.28.
The Einstein–Euler System
In this case the matter is described by its massenergy density \(\rho \), its four velocity \(u^\alpha \) which is a futurepointing, timelike unit vector field, and its pressure p. These quantities are defined on the spacetime manifold M and induce the energymomentum tensor
The Bianchi identity applied to the field equations yields the evolution equations for the fluid
where \(\nabla ^\alpha \) denotes the covariant derivative associated with the metric. More explicitly,
In order to close the system we need to prescribe a (macroscopic) equation of state which relates p and \(\rho \),
with a prescribed function P. Notice that p will always denote the pressure as a function on spacetime, while P will denote its functional relation to the massenergy density \(\rho \). We state our main result for the Einstein–Euler system, which consists of (1.1), (1.5), (1.6), (1.7), (1.8), in an informal way.
Theorem 2
Let \((\rho _\kappa , p_\kappa , u^\alpha _\kappa \equiv 0)_{\kappa >0}\) be a oneparameter family of spherically symmetric steady states to the Einstein–Euler system with a fixed, strictly increasing equation of state (satisfying suitable assumptions) and \(\kappa \) the central redshift of the corresponding steady state. Then,

(a)
For \(\kappa \) sufficiently large, the associated steady state is dynamically unstable in the sense that the linearized Einstein–Euler system possesses an exponentially growing mode.

(b)
For any \(\kappa >0\), the phase space of the linearized system splits into three invariant subspaces: the stable, unstable, and center space. The dimension of the stable/unstable space is equal to the negative Morse index of a suitably defined Schrödingertype operator \(\Sigma _\kappa \).

(c)
A version of the turning point principle holds, i.e., as \(\kappa \in ]0,\infty [\) varies, the stability of the steady state can be inferred from the socalled massradius diagram and the knowledge of the negative Morse index of \(\Sigma _\kappa \).
The Einstein–Euler system models stars, and hence part (a) of the theorem says that strongly relativistic steady stars—those with very large central redshift—are unstable. The analogous comment as in the Vlasov case applies concerning the Newtonian situation. The rigorous statement and proof of the turning point principle is given in Theorem 5.17, the exponential trichotomy is shown in Theorem 5.20, and the instability for large \(\kappa \) can be found in Corollary 5.25.
Motivation and History of the Problem
The Newtonian analogue of the Einstein–Vlasov system is the gravitational VlasovPoisson system
Here \(f=f(t,x,v)\ge 0\), a function of time t, position \(x\in {{\mathbb {R}}^3}\), and velocity \(v\in {{\mathbb {R}}^3}\), is the phasespace density of the stars in a galaxy, and \(U =U(t,x)\) is the gravitational potential induced by the macroscopic, spatial mass density \(\rho =\rho (t,x)\); integrals without explicitly specified domain always extend over \({\mathbb {R}}^3\). A convenient way of constructing steady states of this system is to make an ansatz
where
is the local particle energy. Modulo some technical assumptions the profile \(\Phi \ge 0\), which we refer to as the microscopic equation of state, is arbitrary, but must vanish on \(]\infty ,0[\). Hence \(E^0<0\) represents the maximal particle energy allowed in the system. Any such choice of f satisfies (1.9) with the given potential U, and the problem of finding a steady state is reduced to solving (1.10), where the right hand side becomes a function of U obtained by substituting the ansatz into (1.11). Since such steady states necessarily are spherically symmetric, it seems convenient to parameterize them by U(0). However, in view of the boundary condition in (1.10) and the need of a cutoff energy \(E^0\) in the ansatz this seems one free parameter too many. Instead, one can use \(y=E^0U\) as the basic unknown function and use \(\kappa = y(0)>0\) as the free parameter. Under suitable assumptions on \(\Phi \) the function y has a unique zero at some radius \(R>0\) which marks the boundary of the support of the induced steady state which then also has finite mass, cf. [56] and the references there. With \(E^0=\lim _{r\rightarrow \infty }y(r)<0\) the corresponding potential U vanishes at infinity, and the parameter \(\kappa =y(0) = U(R)U(0)\) is the potential energy difference between the center and the boundary of the compactly supported steady state. Examples for ansatz functions \(\Phi \) which yield such oneparameter families of steady states are the polytropes where \(\Phi (\eta )=\eta ^k\) for \(\eta >0\) with \(1<k<7/2\), and the King model where \(\Phi (\eta )=e^\eta \), both of which are used in astrophysics, cf. [9] .
A central challenge in the qualitative description of the VlasovPoisson dynamics is to understand the behavior of solutions close to the above steady states. In the astrophysics literature formal arguments towards linearized stability were given for example by Antonov [7], Doremus, Feix, Baumann [18], and Kandrup, Sygnet [41]. The stability criterion proposed by these authors is that \(\Phi \) is strictly increasing on \([0,\infty [\). This is physically reasonable, as it states that the number of stars in the galaxy is a decreasing function of their energy. Much work has been invested in a mathematically rigorous, nonlinear proof of this stability criterion, cf. [28,29,30, 45, 46] and the references there. Remarkably, the size of \(\kappa \) is irrelevant for these stability results.
Theorem 1 shows that the above stability criterion for VlasovPoisson is false for the Einstein–Vlasov system: When the central redshift \(\kappa \) is sufficiently large, there exists a growing mode despite the monotonicity assumption on \(\Phi \). This fundamental difference between the VlasovPoisson and the Einstein–Vlasov system is driven by relativistic effects which become dominant at sufficiently large values of the central redshift \(\kappa \). Another difference between these two systems seems to be that in the nonrelativistic case nonlinearly stable steady states can be obtained as minimizers of suitable energyCasimir functionals, cf. [60] and the references there, but this does not work so well in the relativistic case, as the problem is supercritical; in [72] an attempt in this direction has been made, but as shown in [5] that paper is not correct. We also mention an interesting recent work of Andersson and Korzyński [1] on the variational derivation of the Einstein–Vlasov system.
The Newtonian analogue of the Einstein–Euler system is the EulerPoisson system, where the Euler equations
are coupled to the Poisson equation (1.10). Here \(\rho \) is the macroscopic fluid density, and \(\mathbf{u}\) is the fluid 3velocity. To close this system, one must prescribe an equation of state \(p = P (\rho )\). A wellinvestigated choice is again the polytropic one, i.e., \(P(\rho )=\rho ^\gamma \), \(\gamma >1\). When \(\gamma >\frac{6}{5}\) there exist spherically symmetric stationary distributions of compact support, cf. [9, 56]. A classical, linear stability analysis shows that polytropic steady stars are stable if \(\gamma >\frac{4}{3}\). By analogy to the VlasovPoisson case this result holds independently of the size of the central density \(\rho (0)\) of the steady state. Theorem 2 shows that this changes drastically in the relativistic context: strongly relativistic stars, i.e., stars with large central density or equivalently large central redshift, are unstable.
The subject of stability of isolated selfgravitating solutions of the Einstein–Euler system has a long history. The first linear stability result and a variational characterization of the stability question in spherical symmetry goes back to the pioneering work of Chandrasekhar [16] in 1964. The same stability criterion was then derived by Harrison, Thorne, Wakano, and Wheeler in 1965 [34], who were the first to show that Chandrasekhar’s result is equivalent to the positivedefiniteness of the second variation of the ADM mass, subject to the constant baryon number constraint, see also [8]. Moreover, a turning point principle is formulated in [34] and investigated numerically. It proposes that along the oneparameter family stability changes to instability or vice versa when the steady state passes through an extremal point of the socalled massradius curve. Recently, a proof of such a turning point principle was given in [31]. For a detailed survey of these and related results from the 1960s see [71]. A variational approach to stability and the turning point principle are elegantly formulated in [13,14,15]. In Theorem 5.17 we formulate and prove a rigorous version of the turning point principle for the Einstein–Euler system, i.e., of part (c) of Theorem 2. A general abstract discussion of turning point principle is astrophysics is given in [68]. For the different notion of thermodynamic instabilities and its relation to turning point principles, cf. the discussions in [43, 66]. We refer the reader to [27, 69] for a comprehensive overview of this topic from the physics point of view. By contrast to what is known in the vicinity of massive steady states, the vacuum solution of the Einstein–Vlasov system is asymptotically nonlinearly stable against small perturbations [23, 50, 61].
A burst of interest in the stability of highly relativistic, selfgravitating bodies occurred in the mid 1960s with the discovery of quasars [10, 75]. Since these objects are very powerful and very concentrated sources of energy, it was initially unclear whether their high redshifts were due to them being far away or due to them being very relativistic. Zel’dovich and Podurets gave an early contribution to the subject by showing numerically that certain spherically symmetric steady states turn unstable when their central redshift passes a threshold value [76]. This lead Ipser, Thorne, Fackerell and others to initiate a systematic study of the question of stability of steady relativistic galaxies [19,20,21,22, 36,37,38,39]. In particular, a turning point principle for the Einstein–Vlasov system was formulated. It states that the transition from a stable to an unstable configuration occurs at a critical point of the binding energy, plotted as a function of the central redshift. For some numerical results in this direction see [57, 67]. On the other hand, BisnovatyiKogan and Zel’dovich pursued the question whether there exist stable selfgravitating configurations with arbitrarily high central redshifts [10, 11]. The steady states studied in [10] are singular and have infinite mass and extent. However, in Sect. 3 we shall see that certain explicit steady states of the massless Einstein–Vlasov system have the same macroscopic properties as the solutions in [10]. These massless solutions, which we refer to as the BKZ solutions, capture the behavior of massive steady states close to the center of the galaxy when the central redshift is sufficiently large. An analogous assertion holds for the Einstein–Euler system, see Proposition 3.13. This observation appears to be new and of independent interest, but it also plays a fundamental role in our proof of instability in part (a) of Theorems 1 and 2 .
In the context of relativistic stars, Meltzer and Thorne [53] discuss the stability properties of very dense relativistic stars. They predict that perturbations localized to the core of a very dense star will result in gravitational collapse, which is consistent with our methodology, (see Sects. 4.2 and 5.6) .
Methodology
Under natural assumptions on a given equation of state \(f = \phi (E)\) or \(p=P(\rho )\), both the Einstein–Vlasov and the Einstein–Euler systems posses a corresponding oneparameter family of compactly supported steady states with finite mass, parameterized by the value of the central density \(\rho (0)\), or equivalently, by the central redshift \(\kappa >0\), cf. Sect. 3, in particular (3.10) and (3.11). Even though the equations of equilibrium are classical [55], a rigorous proof of the existence and finiteness of the total mass and support of a steady star/galaxy is nontrivial and depends crucially on the assumptions on the micro and macroscopic equations of state; see [56] and references therein. In what follows \(M_\kappa \) denotes the ADM mass and \(R_\kappa \) the radius of the support of the corresponding steady state.
In both cases, the formal linearization around the steady state leads to a Hamiltonian partial differential equation, which comes with a rich geometric structure. In the case of the Einstein–Vlasov system the steady state can be interpreted as a critical point of the socalled energyCasimir functional [32, 40]. To prove the instability for large values of \(\kappa >0\) it is therefore natural to investigate the sign of the second variation of this energyCasimir. In Theorem 4.3 we construct an explicit test function that turns the second variation negative for \(\kappa >0\) sufficiently large. The key to the construction is a precise understanding of the limiting behavior of the sequence of steady states \((f_\kappa )_{\kappa >0}\) in the singular limit \(\kappa \rightarrow \infty \). We show that in a suitably rescaled annulus around the center \(r=0\) the behavior of the steady states is at the leading order described by the BKZ solutions mentioned above, see Sects. 3.3 and 3.4. In the physics literature such a limit is referred to as the ultrarelativistic limit. Since the BKZ solutions are known explicitly, we obtain sharp a priori control over the second variation in the large \(\kappa \) regime, and we can make a judicious choice of a test function whose support is spatially localized to the aforementioned annulus, close to the center of the galaxy. In Theorem 4.3 we carry out careful estimates showing that such a test function makes the second variation functional negative. A similar procedure applies to the Einstein–Euler system, see Theorem 5.24.
The existence of a negative energy direction is sufficient for a proof of linear exponential instability as shown in Theorem 4.8 for the Einstein–Vlasov system; a similar proof can be given for the Einstein–Euler system. To this end we adapt a strategy developed in the study of plasma instabilities by Laval, Mercier, and Pellat [44]. However, with nonlinear applications in mind it is important to show that the instability is driven by a growing mode, a statement not afforded by Theorem 4.8. In the context of the Vlasov theory, this is a highly nontrivial question, as the presence of the continuous spectrum is generally unavoidable, and a more refined analysis is necessary. Thus to get a more precise spectral information, we must carefully analyze the linearized operator. For the EulerEinstein system Makino showed that the spectrum of the linearized operator is in fact discrete [51, 52] by formulating it as a singular SturmLiouville type operator on a bounded domain.
Both the linearized Einstein–Euler and the linearized Einstein–Vlasov system can be written in the second order form
where \(L_\kappa :X \supset D(L_\kappa ) \rightarrow X\) a selfadjoint linear operator, and X a Hilbert space. In both cases, it is shown that the second variation of the energy corresponds to to the quadratic form \((L_\kappa f,f)_X\). In the case of the Einstein–Vlasov system, a naive attempt to minimize the functional
leads to difficulties due to the possible loss of compactness along minimizing sequences. A related loss of compactness has been wellknown in the stability theory for various Vlasovmatter systems, most notably the VlasovPoisson and the VlasovMaxwell system [47]. We adopt a different approach.
The reduced operator. The key step in our analysis is to construct a suitable reduced operator which by definition is a selfdual, macroscopic, Schrödingertype operator \(S_\kappa :\dot{H}^1({\mathbb {R}}^3)\rightarrow \dot{H}^1({\mathbb {R}}^3)' \) such that for any \(f\in D(L_\kappa )\) there exists an element \(\psi \in \dot{H}^1({\mathbb {R}}^3)\) such that
and for any \(\psi \in \dot{H}^1({\mathbb {R}}^3)\) there exists an \(f\in D(L_\kappa )\) such that
Here \(S_\kappa = \Delta _\kappa + V_\kappa \), where \(\Delta _\kappa \) is an explicit, nondegenerate, linear, elliptic operator with variable coefficients, \(V_\kappa \) a compactly supported potential, and \(\dot{H}^1({\mathbb {R}}^3)\) is a homogeneous Sobolev space. Since \(S_\kappa \) has at most finitely many negative eigenvalues, we can use the bounds (1.12), (1.13) to conclude that the negative Morse index of \(L_\kappa \) is finite. The derivation of \(S_\kappa \) in both the kinetic and the fluid case is new.
An attempt to construct a reduced operator was made earlier by Ipser [37], but in that work the bound (1.12) with a different choice of \(S_\kappa \) is satisfied only under an additional hypothesis on the steady states, which appears to be hard to verify rigorously. We point out that a related construction of the reduced operators for the Newtonian analogues is nontrivial, yet considerably simpler; for the VlasovPoisson system see [30] and for the EulerPoisson system [49].
While the reduced operator can be constructed for any \(\kappa >0\), see Theorems 4.24 and 5.15 , only in the regime of large \(\kappa \) we know that there exists at least one negative direction. By proving that \(L_\kappa \) is selfadjoint and not merely symmetric, we then infer that the unstable space consists of eigenfunctions of finite multiplicity. The proof of selfadjointness of the operator \(L_\kappa \) in both the Einstein–Vlasov and the Einstein–Euler case is not obvious and is presented in Lemmas 4.21 and 5.23 respectively.
Exponential trichotomy. To obtain further information about the linearized dynamics, we are forced to work with the first order Hamiltonian formulation of the problem. This part of our analysis applies to all \(\kappa >0\). Abstractly, both problems can be recast in the form
where \({\mathscr {L}}_\kappa :{\mathcal {X}} \rightarrow {\mathcal {X}}'\) is a selfdual operator and \({\mathscr {J}}_\kappa :{\mathcal {X}}'\rightarrow {\mathcal {X}}\) is antiselfdual; \({\mathcal {X}}\) is an appropriate Hilbert space. The operators \({\mathscr {J}}_\kappa , {\mathscr {L}}_\kappa \) for the Einstein–Vlasov system are given in Lemma 4.20, for the Einstein–Euler system in Lemma 5.6. Using crucially the existence of the reduced operator and (1.14) we are able to apply the general framework developed by Lin and Zeng [48] to obtain a quantitative exponential trichotomy result, see Theorems 4.28 and 5.20 . Roughly speaking, we show that the phase space around the steady state naturally splits into a stable, unstable, and center invariant subspace. Under a nondegeneracy assumption on the reduced operator \(S_\kappa \), we can go a step further and prove a quantitative Lyapunov stability statement on the center space in the natural energy topology, see parts (v) of Theorems 4.28 and 5.20 .
The exponential trichotomy statement shall provide a foundation for the understanding of the nonlinear dynamics in the vicinity of steady states. For instance, one expects that in the presence of growing modes the phase space will split into perturbations leading to gravitational collapse, and those leading to global existence via dispersion, separated by an invariant (codimension 1 center stable) manifold. This is consistent with the dynamical picture proposed in the study of criticality phenomena [6, 25].
Turning point principle. In the context of selfgravitating relativistic bodies, it is desirable to have a simple criterion for stability that depends on certain bulk properties of the system under study. Precisely such an idea was put forward by Zel’dovich [74] and Harrison, Thorne, Wakano, and Wheeler in [34], wherein the socalled turning point principle was formulated. If one plots the mass \(M_\kappa \) and the steady star radius \(R_\kappa \) along a curve parameterized by the central redshift \(\kappa \), it is proposed in [34] that the stability can be lost to instability and vice versa only at the extremal points of the mass plotted as a function of \(\kappa \) , see also [66, 69].
The steady states of the Einstein–Euler system can be interpreted as critical points of the ADM mass among all densities of the same total baryon mass. This observation goes back to [34]. By computing the second variation of the ADM mass we can derive the wellknown Chandrasekhar stability criterion [16], which states that the static star is (spectrally) stable if the linearized operator is nonnegative on a codimension1 subspace of a certain Hilbert space, see Proposition 5.11. Using this characterization, we show in Theorem 5.17 that when \(\frac{\mathrm{d}}{\mathrm{d}\kappa } M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa }\left( \frac{M_\kappa }{R_\kappa }\right) \ne 0\) the number of growing modes associated with the linearized operator \({\mathscr {J}}_\kappa {\mathscr {L}}_\kappa \) equals \(n^(\Sigma _\kappa )i_\kappa \), where \(\Sigma _\kappa \) is the reduced operator associated with the Einstein–Euler system, \(n^(\Sigma _\kappa )\) is its negative Morse index, and the index \(i_\kappa \in \{0,1\}\) depends on the massradius curve through the formula
Finally, in line with Antonov’s First Law for Newtonian selfgravitating systems, we prove a relativistic “micromacro" stability principle in Theorem 5.26, which in a precise way states that a steady galaxy with a certain microscopic equation of state is spectrally stable if an individual star with the corresponding macroscopic equation of state is spectrally stable.
Some open questions A natural question for further study is the nonlinear instability of steady states when \(\kappa \gg 1\). For the Einstein–Vlasov case a local wellposedness result was established in [61] which can be used (or if necessary adapted) for initial data close to a steady state. A nontrivial problem which needs to be understood before attacking the nonlinear regime is the regularity of eigenfunctions corresponding to the growing modes of the linearized system. For the Einstein–Euler case we are naturally led to the vacuum free boundary problem wherein the location and the regularity of the boundary separating the star from the vacuum is an unknown. This question comes with a number of mathematical difficulties, and even the localintime wellposedness theory is an open problem in this context.
When \(\kappa \ll 1\) it is known [32, 33] that the steady states of the Einstein–Vlasov system are linearly stable. Since the Einstein equations are energy supercritical, this type of a priori control coming from the linearized problem is far from sufficient for proving any kind of nonlinear stability; note that the situation is different for the VlasovPoisson system. It is an open problem to show that nonlinear orbital stability is true for \(\kappa \ll 1\). A linear stability result for socalled hard stars is given in [24].
A further open question is to show, as conjectured in the physics literature [34, 69], that for the Einstein–Euler system the number of unstable directions grows to infinity as \(\kappa \rightarrow \infty \). As pointed out in Remark 5.19 there is strong evidence for this to be true.
The Einstein–Vlasov and Einstein–Euler System in Spherical Symmetry
For the above systems, questions like the stability or instability of steady states are at present out of reach of a rigorous mathematical treatment, unless simplifying symmetry assumptions are made. We assume spherical symmetry, use Schwarzschild coordinates \((t,r,\theta ,\varphi )\), and write the metric in the form
Here \(t\in {\mathbb {R}}\) is a time coordinate, and the polar angles \(\theta \in [0,\pi ]\) and \(\varphi \in [0,2\pi ]\) coordinatize the surfaces of constant t and \(r>0\). The latter are the orbits of \(\mathrm {SO}(3)\), which acts isometrically on this spacetime, and \(4 \pi r^2\) is the area of these surfaces. The boundary condition
guarantees asymptotic flatness, and in order to guarantee a regular center we impose the boundary condition
It is convenient to introduce the corresponding Cartesian coordinates
Instead of the corresponding canonical momenta \(p=(p^1,p^2,p^3)\) we use the noncanonical momentum variables
In these variables,
and f is spherically symmetric iff
The spherically symmetric, asymptotically flat Einstein–Vlasov system takes the following form:
where
Here \(\dot{}\) and \({}'\) denote the derivatives with respect to t and r respectively. For a detailed derivation of these equations we refer to [59]. It should be noted that in this formulation no raising and lowering of indices using the metric appears anywhere. It is a completely explicit system of partial differential equations where \(x,v \in {\mathbb {R}}^3\), \(x\cdot v\) denotes the Euclidean scalar product, \(v^2 = v\cdot v\), and \(\left\langle v\right\rangle \) is defined in (2.4); integrals without explicitly specified domain extend over \({\mathbb {R}}^3\).
We note that p and \(p_T\) are the pressure in the radial and tangential directions respectively, which for the Vlasov matter model in general are not equal. They are equal for the isotropic steady states which we consider in the next section, and of course also for the Euler matter model.
We now formulate the spherically symmetric Einstein–Euler system in Schwarzschild coordinates, i.e., for a metric of the form (2.1), where we keep the boundary conditions (2.2) and (2.3). The spherically symmetric matter quantities \(\rho =\rho (t,r)\), \(p=p(t,r)\), \(u=u(t,r)\) are scalar functions, and the four velocity is \(u^\alpha = (u^0,u,0,0)\) where
here \(u^2\) is a power and not to be confused with a component of the four velocity; no relativity index notation appears from this point on. The field equations become
The Euler equations become
One should keep in mind that (1.8) holds, which should be used to express p and its derivatives above.
Steady States
The Basic SetUp; OneParameter Families
We first consider the Vlasov case. The characteristic system of the stationary, spherically symmetric Vlasov equation reads as
and the particle energy
is constant along characteristics. Hence the static Vlasov equation is satisfied if f is taken to be a function of the particle energy, and the following form of this ansatz is convenient:
Here \(E^0>0\) is a prescribed cutoff energy—notice that the particle energy (3.1) is always positive. Because of spherical symmetry, the quantity
is conserved along characteristics as well so that we could include a dependence on L in the ansatz (3.2). This leads to anisotropic steady states which cannot be treated in parallel with the Euler case and are not pursued in this paper. We require that \(\Phi \) has the following properties; for the stability analysis in Sect. 4 this will be strengthened by (\(\Phi \)2):
Assumption (\(\Phi \)1). \(\Phi :{\mathbb {R}}\rightarrow [0,\infty [\), with \(\Phi (\eta )=0\) for \(\eta \le 0\), \(\Phi \in L^\infty _\mathrm {loc}(]0,\infty [)\), and there exists \(1/2<k<3/2\) and constants \(c_1, c_2 >0\) such that for all sufficiently small \(\eta >0\),
These assumptions are sufficient for the results of the present section, in particular for the analysis of the steady states in the limit of large central redshift, which we believe has some interest in itself. A typical guiding example which satisfies \((\Phi 1)\) is the function
with constants \(c>0\) and \(1/2<k<3/2\). These are the socalled polytropes, named so by analogy to the wellknown polytropic equations of state in compressible fluid dynamics.
Since only the metric quantity \(\mu \) enters into the definition of the particle energy E in (3.1), the field equations can be reduced to a single equation for \(\mu \). It is tempting to prescribe \(\mu (0)\), but since the ansatz (3.2) contains the cutoff energy \(E^0\) as another, in principle free parameter and since \(\mu \) must vanish at infinity due to (2.2) this approach is not feasible. Instead we define \(y:= \log E^0  \mu \) so that \(e^\mu = E^0 e^{y}\). For the ansatz (3.2) the spatial mass density and pressure become functions of y, i.e.,
where
and
The functions g and h are continuously differentiable on \({\mathbb {R}}\), cf. [62, Lemma 2.2], and they vanish for \(y<0\). The metric coefficient \(\lambda \) can be eliminated from the system, because the field equation (2.6) together with the boundary condition (2.3) at zero imply that
where the mass function m is defined by
Hence the static Einstein–Vlasov system is reduced to the equation
which is equivalent to (2.7); here m, \(\rho \), and p are given in terms of y by (3.4) and (3.8).
In [56] it is shown that for every central value
there exists a unique smooth solution \(y=y_\kappa \) to (3.9), which is defined on \([0,\infty [\) and which has a unique zero at some radius \(R>0\). In view of (3.4)–(3.6) this implies that the induced quantities \(\rho \) and p are supported on the interval [0, R], and a nontrivial steady state of the Einstein–Vlasov system with compact support and finite mass is obtained. We observe that the limit \(y(\infty ):=\lim _{r\rightarrow \infty } y(r) < 0\) exists, since \(r\mapsto y(r)\) is a decreasing function and has a unique zero as mentioned above. The metric quantity \(\mu \) is given by \(e^{\mu (r)}= E^0 e^{y(r)}\), and in order that \(\mu \) has the correct boundary value at infinity we must define \(E^0:=e^{y(\infty )}\). Since \(y(R)=0\) we also see that \(E^0=e^{\mu (R)}\). We want to relate the parameter \(\kappa \) to the redshift factor z of a photon which is emitted at the center \(r=0\) and received at the boundary R of the steady state; this is not the standard definition of the central redshift where the photon is received at infinity, but it is a more suitable parameter here:
Hence \(\kappa \) is in onetoone correspondence with the central redshift factor z with \(\kappa \rightarrow \infty \) iff \(z\rightarrow \infty \), and although this is not the standard terminology we refer to \(\kappa \) as the central redshift. For a fixed ansatz function \(\Phi \) we therefore obtain a family \((y_\kappa )_{\kappa >0}\) of solutions to (3.9), which induce steady states to the Einstein–Vlasov system parameterized by the central redshift \(\kappa \), and each member of this family represents a galaxy in equilibrium, which has finite mass and compact support.
Remark 3.1
The central redshift \(\kappa \) and the central density \(\rho _c=\rho (0)\) are related via \(\rho _c= g(\kappa )\), see (3.5). It can also be seen from (3.5) that g is monotonically increasing so that \(\kappa \) and \(\rho _c\) are in a 11 relationship. In the literature both \(\kappa \) and \(\rho _c\) are used to parameterize the steady state solutions, and the two parametrizations are equivalent.
We now consider the Euler case. For stationary, spherically symmetric solutions of the Einstein–Euler system the velocity field necessarily vanishes, \(u=0\), cf. (2.16). The remaining field equations (2.14), (2.15), (2.17) then become identical to the Vlasov case, the Euler equation (2.18) is satisfied identically, and (2.19) reduces to
We define
Then (3.12) can be written as
which means that on the support of the matter,
In analogy to the Vlasov case we introduce \(y=const \mu \) and find that \(\rho \) is given in terms of y,
Taking into account the equation of state (1.8) it follows that
Hence the stationary system takes exactly the same form as in the Vlasov case and can be reduced to the equation (3.9), the only difference being the definitions of the functions g and h in (3.5), (3.6) or (3.13), (3.14) respectively. The interpretation of \(\kappa =y(0)\) remains as explained for the Vlasov case above.
We now specify the assumptions on the function P which defines the equation of state.
Assumption (P1). \(P \in C^1([0,\infty [)\) with \(P'(\rho ) >0\) for \(\rho >0\), \(P(0)=0\), there exist constants \(c_1, c_2>0\) and \(0<n<3\) such that
and the inverse of P exists on \([0,\infty [\) and satisfies the estimate
Before we discuss examples for such equations of state in the next subsection, we briefly mention that the condition (3.15) forces the equation of state to take the approximate form \(P(\rho )\sim _{\rho \rightarrow 0}\rho ^{1+\frac{1}{n}}\) in the vicinity of the vacuum boundary, while (3.16) makes the equation of state linear at the leading order for large values of \(\rho \), i.e. \(P(\rho )\sim _{\rho \rightarrow \infty }\frac{1}{3}\rho \). The first assumption is the wellknown polytropic law from the classical gas dynamics, while the second assumption is necessary to ascertain that the speed of sound remains smaller than the speed of light at high densities.
Assumption (3.15) together with the required regularity guarantees that the function g and h defined in (3.13) and (3.14) are \(C^1\) on \({\mathbb {R}}\), and for each \(\kappa = y(0)>0\) the equation (3.9) has a unique solution with the same properties as stated for the Vlasov case, cf. [56]. For easier reference we collect the findings of this subsection.
Proposition 3.2

(a)
Let \(\Phi \) satisfy \((\Phi 1)\). Then there exists a oneparameter family of steady states \((f_{\kappa },\lambda _{\kappa },\mu _{\kappa })_{\kappa >0}\) of the spherically symmetric, asymptotically flat Einstein–Vlasov system.

(b)
Let P satisfy \(({\mathrm P}1)\). Then there exists a oneparameter family of steady states \((\rho _{\kappa },\lambda _{\kappa },\mu _{\kappa })_{\kappa >0}\) of the spherically symmetric, asymptotically flat Einstein–Euler system.
In both cases these steady states are compactly supported on some interval \([0,R_\kappa ]\), \(\rho _\kappa , p_\kappa \in C^1([0,\infty [)\), \(y_\kappa , \mu _\kappa , \lambda _\kappa \in C^2([0,\infty [)\), and \(\rho '_\kappa (0)=p'_\kappa (0)= y'_\kappa (0)= \mu '_\kappa (0)= \lambda '_\kappa (0)=0\). The following two identities hold on \([0,\infty [\), the second being known as the Tolman–Oppenheimer–Volkov equation:
Proof
Equation (3.17), which holds also in the timedependent case, follows by adding the field equations (2.6), (2.7) or (2.14), (2.15) respectively. For the Einstein–Euler case (3.18) was already stated above as (3.12). That this equation also holds in the Einstein–Vlasov case is due to the fact that a steady state of this system is macroscopically also one of the Einstein–Euler system, as will be seen in the next section. \(\quad \square \)
Microscopic and Macroscopic Equations of State
Let us consider a microscopic equation of state \(\Phi \) which satisfies the assumption (\(\Phi 1\)) (see (3.3)), and a corresponding steady state of the Einstein–Vlasov system. Given the relations (3.4) it is tempting to write
and interpret this steady state as a solution to the Einstein–Euler system with the macroscopic equation of state given by
we fix some \(\kappa >0\) and drop the corresponding dependence since it plays no role here. Indeed, this maneuver is perfectly rigorous and provides a class of examples \(P=P_\Phi \) which satisfy \(({\mathrm P} 1)\) (see (3.15)–(3.16)). To see this, we first observe that
with
The functions h and j and hence also g are continuously differentiable with
and
cf. [62, Lemma 2.2]. In particular, the functions g and h are strictly increasing on \([0,\infty [\) with \(\lim _{y\rightarrow \infty } g(y) = \lim _{y\rightarrow \infty } h(y) = \infty \) so that these functions are onetoone and onto on \([0,\infty [\).
Proposition 3.3
Assume that \(\Phi \) satisfies \((\Phi 1)\). Then \(P_\Phi \) defined in (3.19) satisfies \(({\mathrm P} 1)\). If in addition \(\Phi \in C^1([0,\infty [)\), then \(P_\Phi \in C^2([0,\infty [)\).
Proof
Let P be given by (3.19). Then for \(\rho >0\) and with \(y=g^{1}(\rho )>0\),
which implies that \(P\in C^1([0,\infty [)\) with \(P(0)=0\) and
We claim that
for some constant \(C>0\); such constants may depend only on \(\Phi \) and may change their value from line to line. For \(0\le \rho \le 1\) this is obvious, since by (3.25), \(P(\rho ) \le \rho /3\). With \(y=g^{1}(\rho )\) the estimate (3.26) is equivalent to
which needs to be shown for \(y\ge g^{1}(1)=:y_0\). Now
while
Combining both estimates yields (3.27) for \(y\ge y_0\). We rewrite (3.26) in terms of \(P^{1}\) to find that
By (3.26), \(P(\rho ) \ge \rho /6\) for \(\rho \) large which means that \(P^{1}(p) \le 6 p\) for p large so that (3.28) implies (3.16) in that case. We prove that there exist constants \(c_1, c_2 >0\) such that with \(\gamma = 1/(k+3/2)\),
This implies (3.15) with \(n=k+3/2 \in ]1,3[\), and \(P^{1}(p) \le C p^{1/(\gamma +1)}\) for p small. Thus
for p small which completes the proof of (3.16) so that (P1) holds; notice that \(\gamma < 1\) since \(k>1/2\). It therefore remains to prove (3.29), and it suffices to prove the estimate for \(P'\) which implies the one for P. In view of (3.24) and with \(\rho =g(y)\) the estimate for \(P'\) is equivalent to
Now
and \(j'\) is given by (3.23). Hence all the relevant terms are of the form
with \(a\in \{0,2,4\}\) and \(m\in \{1/2,1/2,3/2\}\), and these terms need to be estimated from above and from below for y small. We can drop the factor \(e^{a y}\). For \(\Phi (\eta )\) we use the assumption (3.3), and we observe that
Hence the relevant terms can be estimated from above and below by
This implies that \(h'(y)\) can be estimated from above and below by \((1e^{y})^{k+3/2}\), and \(g(y)^\gamma g'(y)\) can be estimated from above and below by \((1e^{y})^{\gamma (k+3/2)+(k+1/2)}\). But by the choice of \(\gamma \) the two exponents are equal and the estimate (3.30) holds.
It remains to show the regularity assertion. So let in addition \(\Phi \in C^1([0,\infty [)\). We observe that
We differentiate these expressions under the integral and change variables to find that
Arguing as in [62, Lemma 2.2] it follows that \(g,h \in C^2({\mathbb {R}})\), and hence by (3.24), \(P\in C^2([0,\infty [)\). \(\quad \square \)
Remark 3.4

(a)
The above proof shows that any equation of state derived from a microscopic ansatz satisfying \((\Phi 1)\) obeys the asymptotic relation
$$\begin{aligned} P(\rho ) \approx \frac{1}{3} \rho \ \text{ for }\ \rho \ \text{ large }, \end{aligned}$$(3.31)which for technical reasons is expressed in terms of \(P^{1}\) in \(({\mathrm P} 1)\). The limiting equation of state \(P(\rho )=\rho /3\) is known in astrophysics and cosmology as the equation of state for radiation. It is remarkable that this physically reasonable behavior is taken care of automatically if the equation of state derives from a microscopic one.

(b)
For the fluid model this asymptotic behavior must be put in by hand. In particular, it excludes equations of state of the form \(P(\rho )=c \rho ^\gamma \) with \(\gamma >1\). Such equations of state, which also violate the requirement that the speed of sound \(\sqrt{P'}\) must be less than the speed of light, are common for the EulerPoisson system.

(c)
It would be a minor technical modification to replace the factor 1/3 in (3.31) and \(({\mathrm P} 1)\) by some factor \(\alpha \in ]0,1[\).
The above discussion indicates that macroscopic quantities induced by an isotropic steady state of the Einstein–Vlasov system represent a steady state of the Einstein–Euler system with an equation of state given by (3.19). To complete this argument, it remains to check the stationary Euler equation (3.18). But using the relations (3.20) and (3.22),
As a genuinely fluid dynamical equation of state we consider the one used for neutron stars in [35]. Here for \(y\ge 0\),
and the relation between p and \(\rho \) is given by
or equivalently, by the equation of state
Proposition 3.5
The equation of state (3.32) for a neutron star is well defined, satisfies \(({\mathrm P}1)\), and in addition \(P_\mathrm {NS} \in C^2([0,\infty [)\) with \(P_\mathrm {NS}'' >0\) on \(]0,\infty [\).
Proof
Clearly, \({\tilde{g}}, {\tilde{h}} \in C^\infty ([0,\infty [)\) with
Also, both functions are onetoone and onto on the interval \([0,\infty [\). Hence \(P_\mathrm {NS} \in C^\infty (]0,\infty [)\cap C([0,\infty [)\) with \(P_\mathrm {NS}(0)=0\). Moreover,
This implies that for \(\rho >0\),
and
For \(y>0\) large it holds that \({\tilde{g}}(y)\le C y^4\) and hence \({\tilde{g}}^{1}(\rho )\ge C \rho ^{1/4}\) and
for \(\rho >0\) large, but due to (3.34) this estimate also holds for \(\rho >0\) small. Hence
Next we observe that for \(y>0\) small it holds that \(y^3 \le {\tilde{g}}(y) \le C y^3\) and hence for \(\rho >0\) small, \(c \rho ^{1/3} \le {\tilde{g}}^{1}(\rho ) \le \rho ^{1/3}\) which together with (3.33) implies that
for \(\rho \ge 0\) small; constants like \(0<c<C\) may change their value from line to line. In particular, (3.15) in \(({\mathrm P} 1)\) holds. The estimate (3.35) implies that
Since \(P_\mathrm {NS}(\rho )\ge \rho /6\) for \(\rho >0\) large and hence \(P_\mathrm {NS}^{1}(p) \le 6 p\) for \(p>0\) large, this implies (3.16) for such p. But if \(\rho \) and p are small, then (3.36) implies that \(P_\mathrm {NS}^{1}(p) \le C p^{3/5}\) and hence
which completes the proof of (3.16). From (3.33) it follows that
for \(\rho >0\), and the proof is complete. \(\quad \square \)
The BisnovatyiKogan–Zel’dovich (BKZ) Solution
A key ingredient in our instability analysis is rather precise information on the form of the steady states for large central redshift \(\kappa \). In order to obtain this information it is useful to understand that the steady states approach steady states of a certain version of the Einstein–Vlasov or Einstein–Euler system as \(\kappa \rightarrow \infty \). In this subsection we introduce this limiting system and discuss a special, explicit solution to it.
We first consider the Vlasov case. For \(\kappa \rightarrow \infty \) we expect that close to the center the corresponding solution of (3.9), (3.10) is large, so that formally,
and
where
and for the sake of notational simplicity we normalized
Since \(g^*\) and \(h^*\) are strictly positive, a corresponding steady state is never compactly supported. Let us for the moment assume that we have a solution y of (3.9), (3.10) where g and h are replaced by \(g^*\) and \(h^*\), and let \(\mu , \lambda , \rho , p\) be induced by y. Then these quantities satisfy the stationary Einstein equations together with the radiative equation of state
We want to understand what happens with the Vlasov equation in the limit \(\kappa \rightarrow \infty \). First, we note that the equation of state (3.38) can not come from an isotropic steady state particle distribution of the form (3.2) (and of course not from an anisotropic one either):
The physical meaning of this is simply that massive particles are not radiation. However, this observation carries its own cure: In the \(\kappa \rightarrow \infty \) limit the stationary Vlasov equation (2.5) must be replaced by its ultrarelativistic, or massless version
An isotropic solution of the latter equation is obtained by the ansatz
it is straightforward that this satisfies (3.39). Moreover, for such a particle distribution f,
as required by (3.38). Finally,
which is as expected from (3.37) so that the above f is a consistent solution to the massless Vlasov equation (3.39), and a stationary solution of the massless Einstein–Vlasov system, i.e., of the system (1.1), (1.2), (1.3) for particles with rest mass \(m_0=0\).
Lemma 3.6
For any \(\kappa >0\) there exists a unique solution \(y=y_\kappa \in C^1([0,\infty [)\) to the problem
where \(\rho ^*=g^*(y)\), \(p^*=h^*(y)\) with (3.37), and
Proof
The proof follows using the arguments in [58, Theorem 3.4]. \(\quad \square \)
We want to find a special, explicit solution of (3.40). To this end, we make the ansatz
Then
and
Substituting this into (3.40) it turns out that the latter equation holds iff
Thus
These macroscopic data of the solution are the same as for the massive solutions found by BisnovatyiKogan and Zel’dovich in [10], and we refer to it as the BKZ solution. We note that the corresponding y is singular at the origin, and the solution violates the condition (2.3) for a regular center. It has infinite mass and also violates the boundary condition (2.2) at infinity, i.e., it does not represent an isolated system.
Remark 3.7
(Geometric properties of the BKZ solution) To understand the BKZ solution better we compute its Ricci curvature R(r) and its Kretschmann scalar K(r). With the help of Maple it turns out that
so that this solution has a spacetime singularity at \(r=0\). A radially outgoing null geodesic satisfies the relation
hence
and
describe the radially outgoing null geodesics. They start at the singularity and escape to \(r=\infty \), i.e., the singularity is visible for observers away from the singularity, and hence it violates the strong cosmic censorship conjecture. The concept of weak cosmic censorship is not applicable to this solution, since it is not asymptotically flat. According to the cosmic censorship hypothesis “naked” singularities, i.e., those which are visible to distant observers, should be “nongeneric” and/or “unstable”. Our analysis turns out to be nicely consistent with this hypothesis: regular steady states, by being close to the BKZ solution for large central redshift, seem to inherit this instability and are indeed unstable themselves, as we will show below.
Remark 3.8
The massless Einstein–Vlasov system has been considered in the literature, for example in [4, 17, 70]. The results of the present section, more precisely of the next subsection, show that the massless system captures the behavior of solutions of the massive system in a strongly relativistic situation, at least in the context of spherically symmetric steady states. The massless system plays a similar role in [2].
The size of the central redshift is not the only possible measure of the strength of relativstic effects in a given steady state. Another possible choice is the compactness ratio \(\frac{2m(r)}{r}\), which by the Buchdahl inequality [3, 12] is always less than \(\frac{8}{9}\). We see from (3.43) that the compactness ration for the BKZ solution is \(\frac{3}{7}\) and it is therefore “far" from saturating the Buchdahl bound \(\frac{8}{9}\).
So far, the discussion of the present subsection was restricted to the Einstein–Vlasov system, but in view of the relation between the corresponding steady states explained in Sect. 3.2 it applies equally well to the Einstein–Euler system with the equation of state (3.31), which we would refer to as the massless or radiative Einstein–Euler system.
The Ultrarelativistic Limit \(\kappa \rightarrow \infty \)
In this subsection we prove that in a specified region close to the center the steady states provided by Proposition 3.2 are approximated by the BKZ solution when \(\kappa \) is sufficiently large. This is done in two steps. We first prove that the former steady states are approximated by solutions of the massless problem (3.40) with the same central value (3.41). In a second step we show that the behavior of the latter solutions is captured by the BKZ one.
For the first step we start with the Vlasov case. It is fairly simple to make the asymptotic relation between the functions g and h defined by (3.5) and (3.6) and their massless counterparts \(g^*\) and \(h^*\) precise:
It should be noted that this is indeed a good approximation for large y, since all the terms on the left are then of order \(e^{4y}\). This approximation would suffice to estimate the difference between solutions of (3.9), (3.10) and solutions of (3.40), (3.41). However, for the functions g and h defined by (3.13) and (3.14) in the Euler case the above estimate seems hard or impossible to obtain. For the latter case a different setup must be used to obtain the desired asymptotics, but that setup can be chosen such that it also works for the Vlasov case.
So from now on we treat both cases simultaneously and consider a steady state as provided by Proposition 3.2. For the moment we drop the subscript \(\kappa \) and note that by (3.18), which as we showed holds for the Vlasov case as well,
We want to read this as a differential equation for p, and so we observe that the equation of state \(p=P(\rho )\) can be inverted to define \(\rho \) in terms of p:
we notice that under our general assumptions the function P is onetoone and onto on the interval \([0,\infty [\) both in the Vlasov and in the Euler case and both for the massive and the massless equations. We also notice that p determines all the other components of the corresponding steady state. Hence we consider the equation
and its massless counterpart where according to (3.38) we set \(S(p)=3 p\):
In what follows, \(p_\kappa \) and \(p^*_\kappa \) denote the solutions to (3.45) and (3.46) respectively, satisfying the boundary condition
Remark 3.9
Strictly speaking we reparametrize our steady state family here and use the central pressure as the new parameter. However, the quantities y and p are in a strictly increasing, onetoone correspondence in such a way that \(y\rightarrow \infty \) iff \(p\rightarrow \infty \), i.e., \(p(0) \rightarrow \infty \) is equivalent to \(y(0) \rightarrow \infty \). In view of (3.37) the above way of writing p(0) guarantees that at least for \(\kappa \) large this quantity asymptotically coincides with its original definition, which may justify the misuse of notation.
We now show that close to the origin and for large \(\kappa \) the behavior of \(p_\kappa \) is captured by \(p^*_\kappa \).
Lemma 3.10
There exists a constant \(C>0\) which depends only on \(\Phi \) or P respectively such that for all \(\kappa > 0\) and \(r\ge 0\),
Proof
In order to keep the notation simple we drop the subscript \(\kappa \) and write p and \(p^*\) for the two solutions which we want to compare. We introduce rescaled variables as follows:
with \(\alpha >0\). A straightforward computation shows that
and
notice that (3.50) is actually the same equation as (3.46). We choose
In order to estimate the difference \(\sigma '  {\sigma ^*}'\) and apply a Gronwall argument we first collect a number of estimates. In these estimates C denotes a positive constant which depends only on the ansatz function \(\Phi \) or P and which may change its value from line to line. In particular, such constants are independent of \(\tau \) and \(\kappa \). First we note that \(\sigma \) and \(\sigma ^*\) are decreasing, and hence, for \(\tau \ge 0\),
By (3.16),
This implies that
and
A nontrivial issue is to get uniform control on the denominators in (3.49) and (3.50). But the spherically symmetric steady states of the Einstein–Vlasov or Einstein–Euler system which we consider here satisfy the estimate
which is known as Buchdahl’s inequality, cf. [12]; a proof for very general matter models which covers the present situation and also anisotropic steady states is given in [3]. Hence,
Let us abbreviate
Then these estimates together imply that
Integration of this estimate yields
Gronwall’s lemma implies that
by (3.48),
and the proof is complete. \(\quad \square \)
In order to understand the behavior of \(p^*_\kappa \) in the limit \(\kappa \rightarrow \infty \) the following observation is useful; we recall (3.47) and Remark 3.9 to avoid missinterpretation of the notation \(p^*_\kappa \).
Lemma 3.11
Let \(p^*_\kappa \) denote the solution of (3.46) with initial data \(p^*_\kappa (0)=e^{4\kappa }/3\). Then for all \(\kappa \ge 0\),
Proof
As we noted in the proof of Lemma 3.10 the rescaled function \(\sigma ^*(\tau ) = e^{4\kappa }p^*_\kappa (r)\) like \(p^*_\kappa \) solves (3.46), but to the initial data \(\sigma ^*(0)=\frac{1}{3}=p^*_0(0)\). Hence \(e^{4\kappa }p^*_\kappa (r)=p^*_0(\tau )=p^*_0(e^{2\kappa }r)\), which is the assertion. \(\quad \square \)
In view of Lemma 3.11 we now need to understand the behavior of \(p^*_0 (r)\) for large r.
Lemma 3.12
Let \(\rho ^*_0\) and \(m^*_0\) denote the quantities induced by \(p^*_0\). Then for any \(\gamma \in ]0,3/2[\),
The limiting constants above are the corresponding values of the BKZ solution (3.43).
Proof
The proof relies on turning the massless steady state equations into a planar, autonomous dynamical system. We recall the Tolman–Oppenheimer–Volkov equation (3.18), which as we showed above holds in both the Euler and the Vlasov case, and combine it with the massless equation of state \(p=\rho /3\) and (3.9). This leads to
We rewrite this in terms of \(u_1 (r) = r^2 \rho (r),\ u_2(r) = m(r)/r\) to obtain
Now we multiply both equations with r and introduce \(w_1(\tau ) = u_1(r),\ w_2(\tau )= u_2(r)\) with \(\tau =\ln r\). Then
We denote the right hand side of the system (3.54), (3.55) by F(w), which is defined and smooth for \(w_1\in {\mathbb {R}}\) and \(w_2\in ]\infty ,1/2[\). The system has two steady states:
We aim to show that the solution which corresponds to \(p^*_0\) converges to Z exponentially fast as \(\tau \rightarrow \infty \). To this end we first observe that
has eigenvalues
so that Z is an exponential sink. On the other hand,
with eigenvalues \(1\) and 2, stable direction (0, 1) and unstable direction \((3,4\pi )\). For the solution induced by \(p^*_0\) it holds that \(w(\tau ) \rightarrow (0,0)\) for \(\tau \rightarrow \infty \) which corresponds to \(r\rightarrow 0\). Since the corresponding trajectory lies in the first quadrant \(\{w_1>0,\ w_2>0\}\), it must coincide with the corresponding branch T of the unstable manifold of (0, 0). We want to show that the trajectory T approaches the point Z. To this end, let D denote the triangular region bounded by the lines
Clearly, \(Z\in D\subset ]0,\infty [\times ]0,1/2[\) so that D lies in the domain where F is defined and smooth. We want to show that \(T\subset D\). Since the unstable direction \((3,4\pi )\) points into D, this is at least true for the part of T close to the origin. The line \(\{w_1=0\}\) is invariant and can not be crossed by T; notice that this line is also the stable manifold of the point (0, 0). Due to Buchdahl’s inequality, T must lie below the line \(\{w_2 =4/9\}\). Finally, along the line \(\{w_1 = w_2\}\) the vector \((1,1)\) is normal to it and points into the domain D. Since along this line
so that the vector field F points into the domain D, the trajectory T cannot leave this domain. Hence according to PoincaréBendixson theory, the \(\omega \)limit set of T must either coincide with Z, or with a periodic orbit. However, according to Dulac’s negative criterion, the set \(]0,\infty [ \times ]0,1/2[\) does not contain a periodic orbit, since
In view of the real part of the eigenvalues \(\lambda _{1,2}\) it follows that, for any \(0<\gamma < 3/2\) and all \(\tau \) sufficiently large, i.e., all r sufficiently large,
When rewritten in terms of the original variables this is the assertion. \(\quad \square \)
We now combine the previous three lemmas to show that for \(\kappa \) large the steady state of the massive system which corresponds to \(p_\kappa \) is approximated well by the BKZ solution on some interval close to the origin; \(\rho _\kappa , m_\kappa , \mu _\kappa ,\lambda _\kappa \) denote quantities induced by \(p_\kappa \).
Proposition 3.13
There exist parameters \(0<\alpha _1< \alpha _2 < \frac{1}{4}\), \(\kappa _0>0\) sufficiently large, and constants \(\delta >0\) and \(C>0\) such that on the interval
and for any \(\kappa \ge \kappa _0\) the following estimates hold:
Proof
First we note that by Lemma 3.11, the radiative equation of state (3.38), and Lemma 3.12,
so that together with Lemma 3.10,
Below we will simplify the above right hand side on a suitably chosen r interval close to the origin, but first we estimate various other quantities in the same fashion. In order to estimate the difference in \(\rho \) we recall the rescaled variables introduced in (3.48) and the estimate (3.52). This implies that
If we combine this with the definition of \(\tau \) and (3.56) it follows that
where notice that \(E_\kappa \) also bounds the right hand side in (3.57). The estimate (3.58) implies that
and combining this with the second estimate in Lemma 3.12 and the scaling property in Lemma 3.11 it follows that
integrating the \(\rho \) estimate directly yields the same result, but \(E_\kappa \) is only integrable at the origin if we choose \(\gamma < 1\). Next we use (3.7) to find that
and, using Buchdahl’s inequality (3.53),
Using (2.15) for \(\mu _\kappa '\) and \(\mu _{\mathrm {BKZ}}'\) it follows that
where we have again used Buchdahl’s inequality (3.53); note that \(\mu _{\mathrm {BKZ}}'\) refers to the BKZ solution (3.43). Finally, with the established bounds on \(r^2\rho _\kappa \) and \(e^{2\lambda _\kappa }\) above, equation (2.6) immediately gives
We now analyze the error term \(E_\kappa (r)\) for \(r>0\) and \(\kappa >0\) such that
where \(0<r_1(\kappa ) < r_2(\kappa )\) and \(r_2(\kappa ) >1\). Then
We choose
and
Then for \(\kappa \) large, \(r_1(\kappa ) < r_2(\kappa )\) and \(1< r_2(\kappa )\) as required,
and with \(\delta := \gamma \alpha _1\) and any \(\alpha _2 \in ]\alpha _1,\frac{1}{4}[\) the proof is complete. \(\quad \square \)
In the above proposition no information is provided for \(\mu _\kappa \) itself. Such an estimate is derived next.
Proposition 3.14
Under the assumptions of Proposition 3.13 it follows that
on the r interval specified in Proposition 3.13. Here \(C_\kappa >0\) does depend on \(\kappa \), but \(C>0\) does not. The point in this estimate is that the exponential terms on both sides converge to 1 as \(\kappa \rightarrow \infty \).
Proof
Clearly,
Hence
Using Proposition 3.13 it follows that
as desired, and the lower estimate is completely analogous. \(\quad \square \)
Stability Analysis for the Einstein–Vlasov System
In this section we fix some ansatz function \(\Phi \), but we need to strengthen the assumptions on \(\Phi \) as follows.
Assumption (\(\Phi \)2). \(\Phi \) satisfies \((\Phi \,1)\), and \(\Phi \in C^1([0,\infty [)\cap C^2(]0,\infty [)\) with \(\Phi '(\eta )>0\) for \(\eta >0\).
A typical example is
with constants \(c>0\) and \(1\le k<3/2\); notice that we require the right hand side derivative at 0 to exist, but it need not vanish.
We consider a corresponding steady state \((f_\kappa ,\lambda _\kappa ,\mu _\kappa )\) as obtained in Proposition 3.2 with some \(\kappa >0\). We aim to prove that this steady state is linearly unstable when \(\kappa \) is sufficiently large. In order to keep the notation short, we write
where we recall the definition of the particle energy
and note the fact that the cutoff energy depends on \(\kappa \) as well; \(E^0_\kappa := e^{y_\kappa (\infty )} = e^{\mu _\kappa (R_\kappa )}\) where \(R_\kappa \) is the radius of the spatial support of the steady state, cf. Sect. 3.1. By (\(\Phi \)2), \(\phi _\kappa \in C^1([0,E^0_\kappa ])\) with
The basic strategy on which our instability result relies is the following. The Einstein–Vlasov system conserves the ADM mass. The second variation of the ADM mass at the given steady state is a conserved quantity for the linearized system, restricted to linearly dynamically accessible states. The key to linear instability then is to prove that there is a negative energy direction for the linearized system, i.e., a linearly dynamically accessible state on which the second variation of the ADM mass is negative. This is the content of Theorem 4.3.
This fact rather directly implies a linear, exponential instability result, cf. Theorem 4.8. However, much more precise instability information, including the existence of a simple growing mode, can be derived from Theorem 4.3. In order to do so the Hamiltonian character of the linearized system must be exploited which leads to the desired spectral properties of the generator of the \(C^0\) group induced by the linearized system, cf. Theorem 4.28. In order to emphasize the basic mechanism leading to our instability result we try to bring in the more functional analytic, abstract tools and terminology only when they are finally needed to derive Theorem 4.28.
Dynamic Accessibility and the Linearized Einstein–Vlasov System
Sufficiently regular solutions of the spherically symmetric Einstein–Vlasov system conserve the ADM mass
In addition, the Einstein–Vlasov flow conserves the Casimir functionals
Here \(\chi \in C^1({\mathbb {R}})\) is an arbitrary, prescribed function with \(\chi (0)=0\), and \(\lambda _f\) is defined by
which is the solution of (2.6) induced by f and satisfying (2.3), cf. (3.7). In a stability analysis it is natural to restrict the admissible perturbations of the given steady state to such as preserve all the Casimir invariants. These dynamically accessible perturbations form the symplectic leaf \({\mathcal {S}}_{f_\kappa }\) through the given steady state. Its formal tangent space at \(f_\kappa \) is the set of linearly dynamically accessible perturbations [40]. In order to proceed we need to recall the usual Poisson bracket
of two continuously differentiable functions f and g of \(x,v \in {\mathbb {R}}^3\). The product rule for the Poisson bracket reads
In addition we denote the radial component of a vector \(v\in {\mathbb {R}}^3\) by
An important subclass of linearly dynamically accessible states are of the form
the operator \({\mathcal {B}}_\kappa \) is defined in Definition 4.11, the concept of linearly dynamically accessible states which we adopt later is made precise in Definition 4.14. The generating function \(h=h(x,v)\) of the perturbation should be spherically symmetric and \(C^1\). In this case, we denote the perturbed metric field \(\lambda \) by \(\lambda _h\) and we have
On linearly dynamically accessible states of the form (4.5) the second variation of the ADM mass \({\mathcal {H}}_{\text {ADM}}\) takes the form
see [32].
Remark 4.1
We emphasize that the above variational structure and the discussion of dynamic accessibility is limited to radially symmetric perturbations; a detailed derivation is given in [32]. However, in our stability analysis of the steady states of the EVsystem variational methods do not play a role.
The relation of the quadratic form \({\mathcal {A}}_\kappa \) to the nonlinear Einstein–Vlasov system is not relevant for the present analysis, but it will become so in a possible extension of the linear instability result to the nonlinear system. For the present purposes it is important that \({\mathcal {A}}_\kappa \) is conserved along the linearized flow, which we need to recall next for the special class of perturbations of the form (4.5). The dynamics of the generating function h is determined by
where \(\lambda _h\) and \(\mu _h\) are determined by
together with the boundary conditions \(\lambda _h(t,0)=0=\mu _h(t,\infty )\), and
It can be checked from (4.9) that \(\lambda _h\) is indeed given by the formula (4.6), cf. [32].
Let us denote
so that \({\overline{D}}_\kappa \) is the support of the steady state under consideration. A simple iteration argument the details of which are indicated in [32] shows that for any \(\mathring{h} \in C^1({\overline{D}}_\kappa )\) there exists a unique solution \(h\in C^1([0,\infty [;C({\overline{D}}_\kappa )) \cap C ([0,\infty [;C^1({\overline{D}}_\kappa ))\) of the above system with \(h(0)=\mathring{h}\); in passing we note that Eq. (5.10) in [32], which corresponds to (4.8), contains a sign error. Moreover, we note that the characteristic flow of (4.8) is the one of the steady state solution and leaves the set \({\overline{D}}_\kappa \) invariant. Under the present regularity assumptions on \(\Phi \) this characteristic flow is \(C^3\), and the limiting factor concerning regularity of solutions to the above system is the factor \(\phi _\kappa '\) in (4.5). We recall from [32] that the energy \({\mathcal {A}}_\kappa (h,h)\) is conserved along solutions of the linearized system stated above.
Remark 4.2
It should be noted that by (4.1) and (4.5) the dynamically accessible perturbation \(f_h\) induced by some generator h vanishes outside \({\overline{D}}_\kappa \) and that only the values of h on that set matter for the definition of \(f_h\) and \({\mathcal {A}}_\kappa (h,h)\).
In [33] it was shown that for \(\kappa \) sufficiently small the quantity \({\mathcal {A}}_\kappa \) is positive definite, which leads to a linear stability result. Here we aim for the opposite.
A Negative Energy Direction for \(\kappa \) Sufficiently Large
Our next aim is to show the existence of a linearly dynamically accessible perturbation for which the bilinear form \({\mathcal {A}}_\kappa \) is negative.
Theorem 4.3
There exists \(\kappa _0>0\) such that for all \(\kappa >\kappa _0\) there exists a spherically symmetric function \(h\in C^2({\mathbb {R}}^6)\) which is odd in v, such that
where \(f_h\) is given by (4.5).
For the proof of this result we first establish two auxiliary results.
Lemma 4.4
For any \(a,b\in {\mathbb {R}}\) the following identity holds:
In particular,
Here w is defined in (4.4).
Proof
The proof of the first identity relies on the fact that for any \(a,b\in {\mathbb {R}}\),
We now integrate over \({\mathbb {R}}^3\) and integrate by parts. The choices \(a=3, b=1\), \(a=1, b=1\), and \(a=1, b=3\) together with the definitions of \(\rho _\kappa \) and \(p_\kappa \) complete the proof. \(\quad \square \)
Lemma 4.5
The following estimate holds:
Proof
By Lemma 4.4,
Now we note that
where the last inequality follows from the pointwise estimate \(w<\left\langle v\right\rangle \) for any \((x,v)\in {\mathbb {R}}^6\). Plugging the last inequality into (4.17) we obtain the claim of the lemma. \(\quad \square \)
Proof of Theorem 4.3
We shall look for a negative energy direction h of the form
where \(g\in C^2([0,\infty [)\) is to be chosen. We recall the definition of the coordinate w (4.4). As a function of v, h is clearly odd. With (4.18) and (4.6) in mind we can simplify the expression for \(\lambda _h\):
where we used (4.15). On the other hand,
Therefore
According to Proposition 3.13, on the spatial interval \([r_\kappa ^1, r_\kappa ^2]\) the steady state \((f_\kappa ,\lambda _\kappa ,\mu _\kappa )\) is well approximated by the BKZ solution of the massless Einstein–Vlasov system, provided \(\kappa \) is sufficiently large. Hence we localize the perturbation h given by (4.18) to this interval by setting
where \(0\le \chi _\kappa \le 1\) denotes a smooth cutoff function supported in the interval \([r_\kappa ^1, r_\kappa ^2]\) and identically equal to 1 on the interval \([2 r_\kappa ^1, r_\kappa ^2/2]\); note that the latter interval is nontrivial for \(\kappa \) sufficiently large. In addition, we require that
Then the perturbation \(f_{h}\) takes the form
We plug (4.20) into (4.7) and obtain the following identity:
The idea now is that if we replace the steady state \((f_\kappa ,\lambda _\kappa ,\mu _\kappa )\) in \(A_1\) by the corresponding limiting quantities according to Proposition 3.13, then a strictly negative term arises, together with various error terms, which like \(A_2\) and \(A_3\) do not destroy the overall negative sign of \({\mathcal {A}}_\kappa (h,h)\), provided that \(\kappa \) is sufficiently large.
Since by Proposition 3.13\(\frac{1}{r \mu _\kappa '} \approx 2\) on the support of \(\chi _\kappa \), we split \(A_1\) further:
A direct algebraic manipulation gives the estimate
Hence, by Lemma 4.4,
Using (3.17) it follows that
In order to estimate the various parts into which \({\mathcal {A}}_\kappa (h,h)\) has now been split we observe that by Proposition 3.13 the following estimates hold, provided \(\kappa \) is sufficiently large:
Hence using the fact that \(\chi _\kappa =1\) on \([2r_\kappa ^1,r_\kappa ^2/2]\) and Proposition 3.14,
Here and in what follows \(C_\kappa \) always denotes the constant introduced in Proposition 3.14, while C denotes a generic constant which does not depend on \(\kappa \), but which may change its value from line to line.
We now proceed to prove that all the remaining terms are smaller in modulus than the negative term just obtained, provided \(\kappa \) is sufficiently large. In order to estimate \(A_{112}\) we note that by Proposition 3.13,
Combining this with (4.24) and Proposition 3.14,
In order to estimate \(A_{12}\) we first observe that
Using Lemma 4.4 we find that
We insert the above two estimates into the definition of \(A_{12}\), and using (4.24) and Proposition 3.14 we conclude that
To estimate \(A_2\) we notice that
we use the bounds (4.19) for \(\chi _\kappa '\) and the estimate
cf. (4.15). Together with (4.24) this implies that
In order to estimate \(A_3\) we first estimate the vintegral contained in that term:
where we used (4.15). Hence
We add up (4.25), (4.26), (4.27), (4.28), and (4.29) to find that besides the constants \(C_\kappa \) and C introduced in Proposition 3.14 there exist positive constants \(C_1, C_2, C_3\) such that
provided \(\kappa \) is sufficiently large, and the proof is complete. \(\quad \square \)
Remark 4.6
The logarithmic gain in (4.25) is fundamental to the proof of Theorem 4.3. The perturbation \(f_{h}\) is carefully engineered to produce such a gain, taking advantage of the exact \(\kappa \rightarrow \infty \) asymptotics provided by Proposition 3.13.
Linear Exponential Instability
Theorem 4.3 is sufficient for showing a linear exponential instability result, without proving the existence of an exponentially growing mode. We present this simple argument first, before we turn to the growing mode in the following sections.
In order to exploit Theorem 4.3 we further analyze Eq. (4.8), which governs the dynamics of the linearly dynamically accessible perturbations. We split h into even and odd parts with respect to v, i.e., \(h_+ (x,v) = \frac{1}{2}\left( h(x,v) + h(x,v)\right) \) and \(h_ (x,v) = \frac{1}{2}\left( h(x,v)  h(x,v)\right) \). It is then easy to see that \(\lambda _h =\lambda _{h_}\), \(\mu _h =\mu _{h_}\), and (4.8) can be turned into the system
where we define
the operator \({\mathcal {T}}\) essentially represents the transport along the characteristic flow of the steady state under consideration.
Remark 4.7
It is tempting to turn the system (4.30), (4.31) into a second order equation for the odd part \(h_\) alone, but under the regularity assumption on \(\Phi \) the corresponding second order derivatives of \(h_\) need not exist. To do this rigorously we would have to interpret the equation in a distributional sense.
In what follows we use the weight \(W : = e^{\lambda _\kappa }\phi _\kappa '(E_\kappa )\) and denote by \(L^2_{W}=L^2_{W}(D_\kappa )\) the corresponding weighted \(L^2\) space on the set \(D_\kappa \), cf. (4.13); \((\cdot ,\cdot )_{L^2_{W}}\) denotes the corresponding scalar product. We obtain the following linear, exponential instability result:
Theorem 4.8
There exist initial data \(\mathring{h}_+, \mathring{h}_\) and constants \(c_1,c_2>0\) such that for the corresponding solution to the system (4.30), (4.31),
Proof
The proof relies on the fact that solutions to (4.30), (4.31) preserve energy, i.e.,
and on the virial identity
As to (4.33) we recall from [32] that the energy \({\mathcal {A}}_\kappa (h,h)\) is conserved, but if we substitute \(h=h_+ + h_\) and use the fact that \(\lambda _h = \lambda _{h_}\) it follows easily that
Now
and
here we used the antisymmetry of \({\mathcal {T}}\) stated in Lemma 4.10 below. The identity
implies that
If we substitute this into the above expression and use the linearized field equations (4.9) and (4.10) to replace the terms \(\rho _h\) and \(p_h\) it follows by a straightforward computation that
If we integrate the \(\mu '_h \lambda _h\) term by parts the r integral vanishes and (4.34) follows.
To prove the linear exponential instability we follow the approach used by Laval, Mercier, Pellat [44] in the context of stability of plasma flows. Since Jeans’ Theorem does not hold for the Einstein–Vlasov system, cf. [65], we must modify the argument of [44].
By Theorem 4.3 and Remark 4.2 there exists an oddinv function \(\mathring{h}_\in C^2 ({\overline{D}}_\kappa )\) such that
and to obtain initial data for the system (4.30), (4.31) we supplement this by
where \(\varepsilon >0\) is chosen so small that
Conservation of energy (4.33) and (4.36) imply that
By the CauchySchwarz inequality,
Combining this with the virial identity (4.34) and (4.37) it follows that
Setting
we see that \(y(0)=0\) and
by (4.38). Hence
and thus \(y(t) \ge c_\epsilon t\). By definition of y this is the assertion for \(\Vert h_(t)\Vert _{L^2_{W}}\). By CauchySchwarz,
which implies the estimate for \(\Vert {\mathcal {T}} h_+(t)\Vert _{L^2_{W}}\), and the proof is complete. \(\quad \square \)
Remark 4.9
It should be noted that \(\mathring{h}_+\) defined in this proof is indeed even so that \(\mathring{h}_\pm \) are the even and odd parts of some initial data \(\mathring{h}\) for the original linearized system (4.8)–(4.12).
Lemma 4.10
Let \(\chi \in C (]0,E^0_\kappa [)\), \(\chi >0\), and \(g,h \in C^1(D_\kappa )\). Then
provided both integrals exist.
Proof
Clearly,
For fixed \(\epsilon >0\) we now integrate by parts to get
all the other terms which appear actually drop out, because they contain either \(\partial _{x_i}\partial _{v_i} E_\kappa  \partial _{v_i}\partial _{x_i} E_\kappa \) or \(\nabla _v E_\kappa \cdot \nabla _x E_\kappa \nabla _x E_\kappa \cdot \nabla _v E_\kappa \), and we may temporarily replace \(\chi \) by a smooth approximation. But the outer unit normal \(\nu = (\nu _x,\nu _v)\) which appears in the boundary integral is parallel to \((\nabla _x E_\kappa ,\nabla _v E_\kappa )\) so that the boundary integral vanishes as well, and the assertion follows. \(\quad \square \)
It should be noted that the above integrationbyparts formula does not require any boundary conditions on either g, h or on the weight \(\chi \).
The Hamiltonian Structure of the Linearized Einstein–Vlasov System
We begin by defining a Hilbert space which is the suitable state space for the linearized dynamics, and a certain transport operator.
Definition 4.11
(The Hilbert space X and the operators \({\mathcal {T}}_\kappa \) and \({\mathcal {B}}_\kappa \).)

(a)
The Hilbert space X is the space of all spherically symmetric functions in the weighted \(L^2\) space on the set \(D_\kappa \) with weight \(e^{\lambda _\kappa }/\phi _{\kappa }^{\prime }\) and inner product
$$\begin{aligned} \left( f_{1},f_{2}\right) _X : =\iint _{D_\kappa } \frac{e^{\lambda _\kappa }}{\phi _\kappa '(E_\kappa )} f_{1} f_{2} \mathrm{d}v\,\mathrm{d}x. \end{aligned}$$(4.39)Functions \(f\in X\) are extended by 0 to all of \({\mathbb {R}}^6\).

(b)
For a function \(f\in X\) the transport term \({\mathcal {T}}_\kappa f :=  e^{\lambda _\kappa }\{f,E_\kappa \}\) exists weakly in X, iff there exists a function \(\eta \in X\) such that for all test functions \(\psi \in C^\infty _c (D_\kappa )\),
$$\begin{aligned} \left( \eta , \psi \right) _X =  \left( f, {\mathcal {T}}_\kappa \psi \right) _X. \end{aligned}$$Let
$$\begin{aligned} D({\mathcal {T}}_\kappa ) := \{f\in X \mid {\mathcal {T}}_\kappa f \ \text{ exists } \text{ weakly } \text{ in }\ X\}, \end{aligned}$$and \({\mathcal {T}}_\kappa f:=\eta \) for \(f\in D({\mathcal {T}}_\kappa )\).

(c)
The operator \({\mathcal {B}}_\kappa :D({\mathcal {B}}_\kappa )\subset X \rightarrow X\) is defined by \(D({\mathcal {B}}_\kappa ) = D({\mathcal {T}}_\kappa )\) and
$$\begin{aligned} {\mathcal {B}}_\kappa f&: = {\mathcal {T}}_\kappa f + 4\pi r \phi _\kappa '(E_\kappa ) e^{2 \mu _\kappa +\lambda _\kappa } \left( w \int f \frac{{\tilde{w}}^2}{\left\langle {\tilde{v}}\right\rangle }\,\mathrm{d}{\tilde{v}}  \frac{w^2}{\left\langle v\right\rangle }\int f {\tilde{w}}\,\mathrm{d}{\tilde{v}}\right) , \end{aligned}$$(4.40)where we recall the definition of w (4.4).
Remark 4.12

(a)
When it exists, the function \(\eta \) in Definition 4.11 (b) is uniquely determined by f.

(b)
The fact that the operator \({\mathcal {T}}_\kappa \) is antiselfadjoint (or skewadjoint) is shown in detail in [63].
In what follows we denote perturbations of \(f_\kappa ,\mu _\kappa ,\lambda _\kappa \) by \(f, \mu , \lambda \) respectively, i.e., \(f_\kappa +f,\mu _\kappa +\mu ,\lambda _\kappa +\lambda \) is the solution to the Einstein–Vlasov system (2.5)–(2.13), which is then linearized in \(f, \mu , \lambda \). In particular, \(\lambda =\lambda _f\) depends on f through the nonlocal relationship
the linearization of (4.3).
Remark 4.13
A simple application of the CauchySchwarz inequality implies that \(\lambda \) is welldefined for any \(f\in X\), i.e. there exists a \(C>0\) such that \( \left \int _0^r s^2 \int f \left\langle v\right\rangle \mathrm{d}v\,\mathrm{d}s\right \le C\), \(r\in [0,\infty [\), as f is supported on \(D_\kappa \). In particular \(\lambda (r)\le \frac{C}{r}\) and thus \(\int _0^\infty \lambda (s)^2\,\mathrm{d}s <\infty \).
As shown in [32] the formal variation of (4.2) yields the relation
Therefore, the formal tangent space of linearly dynamically accessible perturbations can be interpreted as the set of perturbations f satisfying the orthogonality condition
for all \(\chi \in C^1({\mathbb {R}})\), \(\chi (0)=0\). Here \(\lambda \) is defined through (4.41). By Proposition 3.2 in [32] for any \(h\in C^1(D_\kappa )\) the orthogonality condition (4.42) is satisfied if \(f={\mathcal {B}}_\kappa h\). A somewhat lengthy, but standard density argument shows that (4.42) is true for any \(f\in \overline{R({\mathcal {B}}_\kappa )}\). This motivates the following definition:
Definition 4.14
(Linearly dynamically accessible perturbations) A function \(f\in X\) is a linearly dynamically accessible perturbation if \(f\in \overline{R({\mathcal {B}}_\kappa )}\).
Remark 4.15

(a)
It is an open question whether \(R({\mathcal {B}}_\kappa )= \overline{R({\mathcal {B}}_\kappa )}\). In the context of the Vlasov–Poisson system, the analogous statement is true and can be inferred from [29]. The above question is intimately tied to the validity of Jeans’ theorem; if the former is true then one can indeed show that \(R({\mathcal {B}}_\kappa )= \overline{R({\mathcal {B}}_\kappa )}\). However, Jeans’ theorem is known to be false in general for the Einstein–Vlasov system [65].

(b)
We shall see in Lemma 4.20 that the operator \({\mathcal {B}}_\kappa \) and the Hilbert space X arise naturally in the study of the linearized Einstein–Vlasov system around the steady state \((f_\kappa ,\mu _\kappa ,\lambda _\kappa )\).
A key ingredient in the following analysis is a modified potential induced by a state \(f\in X\).
Definition 4.16
(The modified potential \({{\bar{\mu }}}\) and the operator \(\bar{{\mathcal {L}}}_\kappa \)).

(a)
For \(f\in X\) the induced modified potential \({{\bar{\mu }}}\) is defined as
$$\begin{aligned} {{\bar{\mu }}}(r) = {{\bar{\mu }}}_f(r) : = e^{\mu _\kappa \lambda _\kappa }\int _r^\infty \frac{1}{s}\, e^{\mu _\kappa (s)+\lambda _\kappa (s)}(2s\mu _\kappa '(s)+1)\,\lambda (s)\,\mathrm{d}s \end{aligned}$$(4.43)with \(\lambda \) defined by (4.41).

(b)
The operator \(\bar{{\mathcal {L}}}_\kappa :X \rightarrow X\) is defined by
$$\begin{aligned} \bar{{\mathcal {L}}}_\kappa f&: = f  \phi _\kappa '(E_\kappa )E_\kappa {{\bar{\mu }}}. \end{aligned}$$(4.44)
As it shall be often used in the remainder of this paper, we observe that
this follows since \(\mu _\kappa ' \ge 0\).
Lemma 4.17
For \(f\in X\), \({{\bar{\mu }}} = {{\bar{\mu }}}_f \in C([0,\infty [)\cap C^1(]0,\infty [)\), and \({{\bar{\mu }}} (r) \le C \Vert f\Vert _X\), \(r\ge 0\), with some \(C>0\) independent of f. Moreover,
\({{\bar{\mu }}} \in \dot{H}^1_r\), the subspace of spherically symmetric functions in the homogeneous Sobolev space \(\dot{H}^1({\mathbb {R}}^3)\), and
in the weak sense.
Proof
The properties of the steady state imply that the quantity \(e^{\mu _\kappa +\lambda _\kappa }(2 r \mu _\kappa '+1)\) is bounded; this is seen from the field equation (2.7) and the boundedness of \(p_\kappa ,\mu _\kappa ,\lambda _\kappa \) which is explained in Sect. 3.1. For \(f\in X\) the CauchySchwarz inequality implies that
which implies the estimate for \({{\bar{\mu }}}\). Its continuity is obvious, and since \(\lambda \) is continuous, \({{\bar{\mu }}}\) is continuously differentiable for \(r>0\) with
which follows after multiplying (4.43) by \(e^{\mu _\kappa +\lambda _\kappa }\) and then differentiating with respect to r. Equation (4.46) follows directly from (4.49). By Remark 4.13 and (4.45) we conclude from (4.46) that . Since \(\mu _\kappa '+\lambda _\kappa '=0\) outside \(D_\kappa \), it follows that \({{\bar{\mu }}}\in \dot{H}^1_r\). Multiplying both sides of (4.46) by \(e^{2\lambda _\kappa }r\) and taking one more radial derivative we obtain (4.47); note that . \(\quad \square \)
Lemma 4.18
The operator \(K:X\rightarrow X\), \(Kf: = \phi _\kappa '(E_\kappa )E_\kappa {{\bar{\mu }}}_f\) with \({{\bar{\mu }}}_f\) given by (4.43) is bounded, selfadjoint, and compact.
Proof
Boundedness follows from Lemma 4.17 and selfadjointness is obvious from an integrationbyparts argument and (4.47). To prove compactness, assume that \((f_n)_{n\in {\mathbb {N}}}\subset X\) converges weakly to some \(f\in X\). From (4.41),
If we denote \(C_n(s):=e^{2\lambda _\kappa }\left( f_nf, e^{\lambda _\kappa }\phi _\kappa '(E_\kappa ) \left\langle v\right\rangle \mathbf{1}_{B_s(0)}\right) _X\), it follows from the weak convergence of \((f_n)_{n\in {\mathbb {N}}}\) that \(\lim _{n\rightarrow \infty }C_n(s)=0\) for any \(s\ge 0\). Moreover for any \(s\ge R_\kappa \), \(C_n(s)=C_n(R_\kappa )\), where \([0,R_\kappa ]\) is the spatial support of the steady state. For any \(s,s'\in [0,R_\kappa ]\),
where the last bound follows from Hölder’s inequality. In particular, the sequence of functions \((C_n)_{n\in {\mathbb {N}}}\) is equicontinuous on \([0,R_\kappa ]\). By the ArzelaAscoli theorem it follows that along some subsequence \((f_{n_k})_{k\in {\mathbb {N}}}\), \(\lim _{k\rightarrow \infty } c_{n_k} = 0\), where \(c_n:=\sup _{s\ge 0}C_n(s)=\max _{s\in [0,R_\kappa ]}C_n(s)\). From this we conclude that
This shows the compactness of the map K. \(\quad \square \)
Lemma 4.19
The operator \({\mathcal {B}}_\kappa \) defined in Definition 4.11 is densely defined and anti selfadjoint on X. The operator \(\bar{{\mathcal {L}}}_\kappa \) is bounded and symmetric on X.
Proof
We write \({\mathcal {B}}_\kappa = {\mathcal {T}}_\kappa + {\mathcal {R}}_\kappa \) where for \(f\in X\),
notice that f is extended by 0 to all of \({\mathbb {R}}^6\). Since the functions \(r, \mu _\kappa , \lambda _\kappa , \phi '(E_\kappa ), v\) are all bounded on the set \(D_\kappa \) a straightforward computation shows that \({\mathcal {R}}_\kappa \) is bounded on X, and its antisymmetry is just as easy to check. Hence the assertion on \({\mathcal {B}}_\kappa \) follows from Remark 4.12. The estimate for \({{\bar{\mu }}}\) in Lemma 4.17 implies that \(\bar{{\mathcal {L}}}_\kappa \) is bounded on X, and it is easy to check that \(\bar{{\mathcal {L}}}_\kappa \) is symmetric. \(\quad \square \)
We now establish a first order Hamiltonian formulation of the linearized Einstein–Vlasov system.
Lemma 4.20
The linearized, spherically symmetric Einstein–Vlasov system takes the form
where the operators \({\mathcal {B}}_\kappa \) and \(\bar{{\mathcal {L}}}_\kappa \) are defined in (4.40) and (4.44) respectively. Furthermore \(D({\mathcal {B}}_\kappa \bar{{\mathcal {L}}}_\kappa )=D({\mathcal {B}}_\kappa )\). The operator \(\bar{{\mathcal {L}}}_\kappa \) induces a quadratic form on X which takes the form
where \(\lambda \) depends on f through (4.41). The flow of (4.50) preserves \({\mathcal {A}}_\kappa (f,f)\).
Proof
First we notice that for \(f\in X\) at least formally,
where recall that the coordinate w is given by (4.4). Since \(\lambda _f\) is given by (4.41) and the bound (4.48) holds, we conclude that \(\lambda _f(r) \le C r^{1/2}\). This bound together with (4.46) implies that \({{\bar{\mu }}}'(r) \le C r^{1/2}\), and the right hand side of (4.52) lies in X. Hence \(\bar{{\mathcal {L}}}_\kappa \) maps \(D({\mathcal {B}}_\kappa )=D({\mathcal {T}}_\kappa )\) into itself, and the operator product \({\mathcal {B}}_\kappa \bar{{\mathcal {L}}}_\kappa \) is defined on \(D({\mathcal {B}}_\kappa )\).
To linearize the equations we expand \(f = f_\kappa +\epsilon \delta f, \mu = \mu _\kappa +\epsilon \delta \mu , \lambda = \lambda _\kappa +\epsilon \delta \lambda \) into (2.5), use (2.8), and neglect terms of quadratic order and higher. By a slight abuse of notation we rename into \(f,\lambda ,\mu \) respectively, and obtain the linearized Vlasov equation
From the field equation (2.7) and the definition (2.11) of p it follows that
Plugging this back into (4.53) we obtain the equation
where \({\mathcal {B}}_\kappa \) is given by (4.40). Using (4.52), (4.15), and (4.49) it follows that
Plugging this into (4.54) we obtain
which proves (4.50).
The conservation of \((\bar{{\mathcal {L}}}_\kappa f,f)_X\) is a direct consequence of the antisymmetry of \({\mathcal {B}}_\kappa \) and (4.50). Finally, from the definition of \(\bar{{\mathcal {L}}}_\kappa \) we have
Replacing \(4\pi r^2 \rho \) by and using (4.46) we obtain (4.51). \(\quad \square \)
The above first order Hamiltonian formulation appears to be new and not exploited elsewhere in the literature. We now derive a second order formulation of the linearized Einstein–Vlasov system, which was first given in 1968 by Ipser and Thorne [36].
Lemma 4.21
A formal linearization of the spherically symmetric Einstein–Vlasov system takes the form
where \(f_\) denotes the odd part of f with respect to the variable v,
and \({\mathcal {B}}_\kappa \) is defined in (4.40). The operator \({\mathcal {L}}_\kappa \) is selfadjoint on X with
the latter being defined in analogy to Definition 4.11 (b). Therefore \(\Vert \partial _t f_\Vert _X^2 + ({\mathcal {L}}_\kappa f_,f_)_X\) is formally conserved along the flow of (4.55), and
Proof
We begin by examining the operator \({\mathcal {L}}_\kappa \). First we note that \({\mathcal {B}}_\kappa {\mathcal {B}}_\kappa f \in X\) is defined in a pointwise, classical sense, provided \(f\in C^\infty _c(D_\kappa )\); here the regularity assumption \(\Phi \in C^2(]0,\infty [)\), i. e., \(\phi _\kappa (E_\kappa )\in C^2(D_\kappa )\) enters. Thus \({\mathcal {B}}_\kappa {\mathcal {B}}_\kappa \) is densely defined and selfadjoint by arguments analogous to those in Remark 4.12. If we define
then it follows easily that \({\mathcal {R}}_\kappa \) is bounded on X and symmetric, which proves the assertion for \({\mathcal {L}}_\kappa \).
Since \(\phi _\kappa '(E_\kappa ) E_\kappa {{\bar{\mu }}}\) is even in v it is immediate from (4.50) that
We take the time derivative of the second equation and use the relation
which is the linearization of the field equation (2.8). Thus
Next,
here we recall (4.57) and observe that \(\partial _t \lambda = \lambda _{\partial _t f}\). Therefore
by (4.51) and the fact that \(f_+\) and \(f_\) are orthogonal to each other in the space X. \(\quad \square \)
Remark 4.22

(a)
The fact that \({\mathcal {A}}_\kappa (\partial _t f,\partial _t f)\) is conserved along the linearized flow can also be seen directly from the first order equation satisfied by \(\partial _{t}f\),
$$\begin{aligned} \partial _{t}\left( \partial _{t}f\right) ={\mathcal {B}}_{\kappa }{\mathcal {L}} _{\kappa }\partial _{t}f. \end{aligned}$$Note that by (4.50), \(\partial _{t}f\in R\left( {\mathcal {B}}_{\kappa }\right) \), the range of the operator \({\mathcal {B}}_{\kappa }\).

(b)
As noted at the beginning of Sect. 4.1 the quadratic form (4.51) arises as the second variation of a suitably chosen energyCasimir functional [32, 33, 40], which is conserved along the flow of the nonlinear Einstein–Vlasov system.
The Reduced Einstein–Vlasov Operator
The main result of this section is Theorem 4.24, which states in a precise manner that the operator \({\mathcal {L}}_\kappa \) is bounded from below and above by a simpler, Schrödingertype operator. Note that this operator acts only on functions that depend on the spatial variable r and we therefore refer to it as a macroscopic operator. In Sect. 4.6 we shall use this to obtain refined spectral information about the operator \({\mathcal {L}}_\kappa \).
An attempt to derive such a reduced operator was made by Ipser in [37], yet the operator derived there bounds the operator \({\mathcal {L}}_\kappa \) from below only under an additional assumption on the steady state, which appears to be hard to verify. Our approach is new and the key ingredient is the use of the modified potential \({{\bar{\mu }}}\) introduced in (4.43).
The operator \(\Delta _\kappa \) is given by
The reduced operator is given by
The nonlocal reduced operator \({\tilde{S}}_\kappa \) is given by
where \(\Pi \) denotes the projection onto the orthogonal complement of \(R({\mathcal {B}}_\kappa )\) in X, and \(\mathrm {id}\) is the identity. On a flat background, i.e., for \(\lambda _\kappa =\mu _\kappa =0\) the operator \( 4\pi \Delta _\kappa \) is the Laplacian applied to spherically symmetric functions.
To make sense classically of the above operators it is natural to require \(\psi \in H^2_r=H^2({\mathbb {R}}^3)\cap L^2_r\), where \(L^2_r\) denotes the space of all spherically symmetric \(L^2\) functions. However, this is too strong for our purposes, since the modified potentials \({{\bar{\mu }}}_f\) from Definition 4.16 must belong to the domain of definition of \(S_\kappa \) and \({\tilde{S}}_\kappa \), but they in general belong to \(\dot{H}^1_r\). Therefore we define \(S_\kappa \) and \({\tilde{S}}_\kappa \) using duality:
Definition 4.23
(The operators \(S_\kappa \) and \({\tilde{S}}_\kappa \) as symmetric quadratic forms) Let \((\dot{H}^1_r)'\) denote the dual space of \(\dot{H}^1_r\), the radial subspace of the homogeneous Sobolev space \(\dot{H}^1({\mathbb {R}}^3)\). The dual pairing is denoted by \(\langle \cdot ,\cdot \rangle \). Then \(S_\kappa :\dot{H}^1_r \rightarrow (\dot{H}^1_r)'\) is defined by
The operator \({\tilde{S}}_\kappa :\dot{H}^1_r \rightarrow (\dot{H}^1_r)'\) is defined analogously.
We use a standard definition of \(\dot{H}^1\) as given e.g. in [26], and we recall that \(D_\kappa = \mathrm {supp}\,f_\kappa \). As defined above the operators \(S_\kappa \) and \({\tilde{S}}_\kappa \) are clearly selfdual. Moreover, taking the r derivative of \(\rho _\kappa = \int \phi _\kappa (E_\kappa ) \left\langle v\right\rangle \,\mathrm{d}v\) it follows that
Thus the operator \(S_\kappa \) can formally be written in the form
The key result of this section is the following theorem.
Theorem 4.24

(a)
For every \(\mu \in \dot{H}^1_r\) and \(f = f_\mu := \phi _\kappa '(E_\kappa )E_\kappa \mu \) it holds that
$$\begin{aligned} \langle S_\kappa \mu ,\,\mu \rangle \ge {{\mathcal {A}}}_\kappa ( f_\mu , f_\mu ). \end{aligned}$$(4.63)For every \(f\in X\) and \({{\bar{\mu }}}_f\) as defined in (4.43),
$$\begin{aligned} {{\mathcal {A}}}_\kappa ( f, f) \ge \langle {\mathcal {S}}_\kappa {{\bar{\mu }}}_f,\,{{\bar{\mu }}}_f\rangle . \end{aligned}$$(4.64) 
(b)
For every \(\mu \in \dot{H}^1_r\) and \({\tilde{f}} = {\tilde{f}}_\mu := (\mathrm {id}\Pi )(\phi _\kappa '(E_\kappa )E_\kappa \mu )\in R({\mathcal {B}}_\kappa )\) it holds that
$$\begin{aligned} \langle {\tilde{S}}_\kappa \mu ,\,\mu \rangle \ge {{\mathcal {A}}}_\kappa ( {\tilde{f}}_\mu , {\tilde{f}}_\mu ). \end{aligned}$$(4.65)For every \(f\in X\),
$$\begin{aligned} {{\mathcal {A}}}_\kappa ( f, f) \ge \langle {\tilde{S}}_\kappa {{\bar{\mu }}}_f,\,{{\bar{\mu }}}_f\rangle . \end{aligned}$$(4.66)
Proof
We first observe that for \(\mu \in \dot{H}^1_r\),
in particular, using (4.43) and (4.46),
and on the other hand by (4.47),
The definitions of \({{\mathcal {A}}}_\kappa \) and \(S_\kappa \) together with (4.68)–(4.69) imply that
and the lower bound (4.64) is proven. To obtain the upper bound (4.63) we observe that by (4.67) for \(\mu \in H^2_r\) and \(\lambda =\lambda _f\) with \(f\in X\),
Now
Together with (4.70) this implies that
provided \(f= \phi _\kappa '(E_\kappa ) E_\kappa \mu \), and (4.63) is proven.
We now turn to part (b) and assume that \(f\in R({\mathcal {B}}_\kappa )\). By definition of the projection \(\Pi \) and (4.68)–(4.69),
and
Using (4.70) and (4.71) it follows that
which proves (4.66). To obtain the upper bound (4.65) we let \(f =(\mathrm {id}\Pi )\left( \phi _\kappa '(E_\kappa )E_\kappa \mu \right) \). Then in particular, \((f,\phi _\kappa '(E_\kappa )E_\kappa \mu )_X = \Vert f\Vert _X^2\). An argument analogous to the one used for (4.67) implies that
and the proof is complete. \(\quad \square \)
Main Results for the Einstein–Vlasov System
We first need to comment on some notation. Let L be either a selfadjoint, linear operator \(L:H\supset D(L) \rightarrow H\) on some Hilbert space H with scalar product \((\cdot ,\cdot )_H\) or a selfdual operator \(L:H\rightarrow H'\). In either case we we denote by \(n^{}\left( L\right) \) its negative Morse index, which by definition is the maximal dimension of subspaces of H on which \((L\cdot ,\cdot )_H <0\) or \(\langle L\cdot ,\cdot \rangle < 0\) in the selfdual case with \(\langle \cdot ,\cdot \rangle \) the dual pairing. For any decomposition \(H=H_{}\oplus \ker L\oplus H_{+}\) such that \((L\cdot ,\cdot )_H <0\) on \(H_{}\setminus \left\{ 0\right\} \) and \((L u,u)_H \ge \delta \left\ u\right\ _{H}^{2}\) on \(H_{+}\) it can be shown that \(\dim H_{}=n^{}\left( L\right) \), see Remark 2.2 and Lemma 12.1 in [48]. For any subspace \(S\subset H\), we also use \(n^{}\left( L_{S}\right) \) to denote the Morse index of L restricted to S.
The selfadjointness of the operator \({\mathcal {L}}_\kappa \) and Theorem 4.24 imply that the negative part of the spectrum has to be discrete and that part (a) of Theorem 1 holds, more precisely, that we have
Theorem 4.25
The negative part of the spectrum of \({\mathcal {L}}_\kappa :X \supset D({\mathcal {L}}_\kappa )\rightarrow X\) is either empty or consists of at most finitely many eigenvalues with finite multiplicities. For \(\kappa \) sufficiently large, there exists at least one negative eigenvalue and therefore a growing mode for the linearization of the Einstein–Vlasov system around \((f_\kappa , \lambda _\kappa ,\mu _\kappa )\).
Proof
By Lemma 4.21 the operator \({\mathcal {L}}_\kappa \) is selfadjoint. We note that for \(f\in D({\mathcal {L}}_\kappa )\),
By Theorem 4.24,
for the sake of completeness we recall the general argument behind the first two estimates in Lemma A.1 in the appendix. In order to show that \(n^{}\left( S_{\kappa }\right) <\infty \) we first observe that \(\langle {\mathcal {S}}_\kappa \phi ,\phi \rangle \ge C \langle {\mathcal {S}}'_\kappa \phi ,\phi \rangle \) for \(\phi \in \dot{H}^1_r\), where the selfdual operator \(S'_\kappa :\dot{H}^1_r\rightarrow (\dot{H}^1_r)'\) is formally given as \(S'_\kappa = \Delta V_\kappa \) with a nonnegative, continuous potential \(V_\kappa \) which has compact support. This follows from the corresponding bounds on \(\lambda _\kappa \) and \(\mu _\kappa \). Now we observe that the mapping \((\Delta )^{1/2} :\dot{H}^1({\mathbb {R}}^3) \rightarrow L^2({\mathbb {R}}^3)\), \(\phi \mapsto (2\pi \xi {{\hat{\phi }}})\check{}\) is an isomorphism which respects spherical symmetry. Passing to \(\psi = (\Delta )^{1/2}\phi \) we arrive at the relation that \(4 \pi \langle {\mathcal {S}}'_\kappa \phi ,\phi \rangle = ((\mathrm {id}K)\psi ,\psi )_{L^2}\), where \(K=(\Delta )^{1/2}V_\kappa (\Delta )^{1/2}:L^2({\mathbb {R}}^3) \rightarrow L^2({\mathbb {R}}^3)\) is compact, since \(V_\kappa \) is bounded and supported on the compact set \({\bar{B}}_{R_\kappa }(0)\), and the mapping \(\dot{H}^1({\mathbb {R}}^3) \ni f \mapsto \mathbf{1}_{{\bar{B}}_{R_\kappa }(0)} f\in L^2({\mathbb {R}}^3)\) is compact; notice that \(h = V_\kappa (\Delta )^{1/2}\psi \in L^1\cap L^2({\mathbb {R}}^3)\) so that \({\hat{h}}\in L^\infty \cap L^2({\mathbb {R}}^3)\), and hence \(\frac{1}{2 \pi \xi } {\hat{h}}\) and its inverse Fourier transform are in \(L^2({\mathbb {R}}^3)\). The spectral properties of compact operators imply that \(n^(\mathrm {id}K) <\infty \), and invoking Lemma A.1 again it follows that \(n^{}\left( S_{\kappa }\right) <\infty \) as claimed.
The assertion on the spectrum of \({\mathcal {L}}_\kappa \) follows from its spectral representation; for the sake of completeness we include the argument in Proposition A.2 in the appendix.
Each negative eigenvalue of \({\mathcal {L}}_{\kappa }\) gives rise to a pair of stable and unstable eigenvalues for (4.55). For \(\kappa \) sufficiently large, \(n^({\mathcal {L}}_\kappa )\ge 1\) by Theorem 4.3, and the claim follows. Note that the operator \({\mathcal {L}}_\kappa \) is nonnegative when restricted to the subspace of X of all eveninv functions in X, since for any \(f\in X\), \(\int _{{\mathbb {R}}^3} f_+ w \,\mathrm{d}v=0\), where \(f_+(x,v)=\frac{1}{2}(f(x,v)+f(x,v))\) is the even part of f. In particular, since the negative eigenvalues are of finite multiplicity, the associated eigenspace necessarily contains only oddinv functions. \(\quad \square \)
Remark 4.26
The finiteness of \(n^(\bar{{\mathcal {L}}}_\kappa )\) follows alternatively from the observation that the difference \(\text {id}  \bar{{\mathcal {L}}}_\kappa = K\), which is a compact operator by Lemma 4.18. Since \(\bar{{\mathcal {L}}}_\kappa \) is a compact perturbation of the identity, the only possible accumulation point is 1 and therefore \(n^(\bar{{\mathcal {L}}}_\kappa )<\infty \).
Remark 4.27
By the same arguments as in the proof above we can conclude that
which will be used below.
We now come to the existence of the linearized flow and the corresponding exponential trichotomy decomposition into a stable, unstable, and center space. Under a nondegeneracy assumption on the operator \({\tilde{S}}_\kappa \) we in fact obtain a Lyapunov stability of the flow on the center space for any \(\kappa >0\). The key in our analysis will be the first order formulation (4.50).
Theorem 4.28
The operator \({\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }\) generates a \(C^{0}\) group \(e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }}\) of bounded linear operators on X, and there exists a decomposition
with the following properties:

(i)
\(E^{u}\left( E^{s}\right) \) is the linear subspace spanned by the eigenvectors corresponding to positive (negative) eigenvalues of \({\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }\), and
$$\begin{aligned} \dim E^{u}=\dim E^{s}=n^{}\left( \tilde{S}_{\kappa }\right) . \end{aligned}$$(4.73) 
(ii)
The quadratic form \(\left( \bar{{\mathcal {L}}}_{\kappa }\cdot ,\cdot \right) _X\) vanishes on \(E^{u,s}\), but is nondegenerate on \(E^{u}\oplus E^{s}\), and
$$\begin{aligned} E^{c}=\{f\in X \mid {{\mathcal {A}}}_{\kappa }(f,g)= \left( \bar{{\mathcal {L}}}_{\kappa }f,\ g\right) _X =0\ \text{ for } \text{ all }\ g\in E^{s}\oplus E^{u} \}. \end{aligned}$$ 
(iii)
\(E^{c}\), \(E^{u}\), \(E^{s}\) are invariant under \(e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }}\).

(iv)
Let \(\lambda _{u}= \min \{\lambda \mid \lambda \in \sigma ({\mathcal {B}}_{\kappa } \bar{{\mathcal {L}}}_{\kappa }_{E^{u}})\}>0\). Then there exist \(M>0\) such that
$$\begin{aligned}&\left e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_\kappa }_{E^{s}}\right \le M e^{\lambda _{u}t},\ t\ge 0,\nonumber \\&\left e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_\kappa }_{E^{u}}\right \le M e^{\lambda _{u}t},\ t\le 0, \end{aligned}$$(4.74)$$\begin{aligned}&\left e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }}_{E^{c}}\right \le M (1+\left t\right )^{k_{0}},\ t\in {\mathbb {R}}, \end{aligned}$$(4.75)where
$$\begin{aligned} k_{0}\le 1+2\left( n^{}\left( S_{\kappa }\right)  n^{}\left( \tilde{S}_{\kappa }\right) \right) . \end{aligned}$$(4.76) 
(v)
When \(\ker \tilde{S}_{\kappa }=\left\{ 0\right\} \), then \(E^{c}\) can be further decomposed as
$$\begin{aligned} E^{c}=\ker \left( {\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }\right) \oplus \left( E^{c}\cap \overline{R({\mathcal {B}}_{\kappa })}\right) , \end{aligned}$$(4.77)and \(\bar{{\mathcal {L}}}_{\kappa }_{E^{c}\cap \overline{R({\mathcal {B}}_{\kappa })}}>\delta \) for some positive constant \(\delta \). As a consequence, \(e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }}_{E^{c}}\le M\) for some constant M. In particular, when \(\tilde{S}_{\kappa }>0\), then the steady state is stable in the sense that for all perturbations \(f\in X\),
$$\begin{aligned} \left\ e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }}f\right\ _{X} \le M\left\ f\right\ _{X}. \end{aligned}$$(4.78)
Remark 4.29

(a)
Equation (4.73) gives the sharp linear stability condition, namely \(\tilde{S}_{\kappa }\ge 0\).

(b)
We expect the exponential trichotomy estimates in (iii) to become useful for the possible construction of invariant (stable, unstable and center) manifolds, as well as for a possible proof of nonlinear instability in the future.

(c)
In the Newtonian limit the operator \(\tilde{S}_{\kappa }\) formally converges to its Newtonian counterpart, which was proved to be positive e.g. in [30]. This can be shown rigorously and for \(\kappa \ll 1\) it can be then shown that \(\tilde{S}_{\kappa }\ge 0\). By (4.78) we therefore obtain linear stability against general initial data in X. This improves the result in [32], where linear stability was proved for dynamically accessible initial data, i.e., when \(f \in R\left( {\mathcal {B}}_{\kappa }\right) \).
To prove Theorem 4.28 we first need to prove several preparatory statements.
Proposition 4.30
There exists a decomposition of X into the direct sum of three closed subspaces
with the following properties:

(H1)
\(\left( \bar{{\mathcal {L}}}_{\kappa }f,f\right) _X <0\) for all \(f\in X_{}\setminus \{0\}\), and
$$\begin{aligned} n^{}\left( \bar{{\mathcal {L}}}_{\kappa }\right) := \dim X_{}=n^{}\left( S_{\kappa }\right) <\infty ,\ \ \ \dim \ker \bar{{\mathcal {L}}}_{\kappa }=\dim \ker S_{\kappa }. \end{aligned}$$(4.79) 
(H2)
There exists \(\delta >0\) such that for all \(f\in X_{+}\),
$$\begin{aligned} \left( \bar{{\mathcal {L}}}_{\kappa }f,f\right) _X \ge \delta \left\ f\right\ _{X}^{2}. \end{aligned}$$
Proof
By Lemma 4.18 and the definition (4.44) of \(\bar{{\mathcal {L}}}_\kappa \) the operator \(\bar{{\mathcal {L}}}_\kappa \) is a compact perturbation of the identity, and therefore the only possible accumulation point of its spectrum is 1. Therefore the negative Morse index and the dimension of \(\ker \bar{{\mathcal {L}}}_\kappa \) are finite and the claimed decomposition follows, as well as the bound (H2).
To prove (4.79), first we note that
since it is easy to see that \(f\in \ker \bar{{\mathcal {L}}}_{\kappa }\) implies \(\mu \in \ker S_{\kappa }\) where \(\mu \) is defined from f via (4.47), and \(\mu \in \ker S_{\kappa }\) implies that \(\phi _{\kappa }'(E_{\kappa })E_{\kappa }\mu \in \ker \bar{{\mathcal {L}}}_{\kappa }\). Let \(n^{\le 0}\left( \bar{{\mathcal {L}}}_{\kappa }\right) \) and \(n^{\le 0}\left( S_{\kappa }\right) \) be the nonpositive dimensions of \(\left( \bar{{\mathcal {L}}}_{\kappa }\cdot ,\cdot \right) _X\) and \(\left( S_{\kappa }\cdot ,\cdot \right) _{L^2_r}\). By Theorem 4.24, \(n^{\le 0}\left( \bar{{\mathcal {L}}}_{\kappa }\right) =n^{\le 0}\left( S_{\kappa }\right) \) which combined with (4.80) implies that \(n^{}\left( \bar{{\mathcal {L}}}_{\kappa }\right) = n^{}\left( S_{\kappa }\right) \). This finishes the proof of the proposition. \(\quad \square \)
In the study of linear stability of (4.50), it is important to restrict to the dynamically accessible space \(R\left( {\mathcal {B}}_{\kappa }\right) \), for which we have the following result.
Lemma 4.31
The quadratic form \(\left( \bar{{\mathcal {L}}}_{\kappa }\cdot ,\cdot \right) _X \) is nondegenerate on the quotient space \(\overline{R({\mathcal {B}}_{\kappa })} / \left( \overline{R({\mathcal {B}}_{\kappa })}\cap \ker \bar{{\mathcal {L}}}_{\kappa }\right) \) if and only if \(\ker \tilde{S}_{\kappa }\subset \ker S_{\kappa }\). Moreover,
and
Proof
Since
it implies that \(f\in \ker \left( \left( \bar{{\mathcal {L}}}_{\kappa }\cdot ,\cdot \right) _X _{\overline{R({\mathcal {B}}_{\kappa })}}\right) \) if and only if \(f\in \overline{R({\mathcal {B}}_{\kappa })}\) and
here \(\Pi \) is the projection onto \(\ker {\mathcal {B}}_{\kappa }\) in X. In view of the the definition of \(\mu \) by (4.47), we deduce from (4.83) that \(\tilde{S}_{\kappa }\mu =0\). On the other hand, if \(\tilde{S}_{\kappa }\mu =0\), then it is easy to check that
Thus (4.81) is proved. By Theorem 4.24,
which together with (4.81) implies (4.82). \(\quad \square \)
Now we are ready to prove Theorem 4.28.
Proof of Theorem 4.28
To prove (i) we note that each pair of stable and unstable eigenvalues for the operator \({\mathcal {L}}_\kappa \) provided by Theorem 4.25 also gives a pair of stable and unstable eigenvalues for (4.50). Together with (4.82) we obtain the dimension formula (4.73).
Consider the Hamiltonian formulation (4.50). The operator \({\mathcal {B}}_{\kappa }\) can be defined as a selfdual operator \(X'\supset D\left( {\mathcal {B}}_{\kappa }\right) \rightarrow X\). Moreover, by Proposition 4.30, the quadratic form \(\left( \bar{{\mathcal {L}}}_{\kappa }\cdot ,\cdot \right) _X\) satisfies the assumptions (H1)–(H3) in [48]. Thus the conclusions (ii)–(iv) follow directly from [48, Thm. 2.2] for general Hamiltonian PDEs.
It remains to show (v). When \(\ker \tilde{S}_{\kappa }=\left\{ 0\right\} \), by Lemma 4.31, \(\left( \bar{{\mathcal {L}}}_{\kappa }\cdot ,\cdot \right) _X _{\overline{R({\mathcal {B}}_{\kappa })}}\) is nondegenerate and
By [48, Prop. 2.8] we have the direct sum decomposition
Since \(\ker {\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }\) is the steady solution space of (4.50) and is therefore contained in \(E^{c}\), we get the decomposition (4.77) for \(E^{c}\). Since
by [48, Cor. 6.1] and \(E^{c}\) is the orthogonal complement of \(E^{s}\oplus E^{u}\) in X, it follows that
Thus \(\mathcal {\bar{L}}_{\kappa }_{E^{c}\cap \overline{R({\mathcal {B}}_{\kappa })}} \ge \delta _{1}>0\), and by using \(\left( \mathcal {\bar{L}}_{\kappa }f,f\right) _X\) as the Lyapunov functional, we get the stability estimate \(\left\ e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }}f\right\ _{X} \le M\left\ f\right\ _{X}\) for \(f \in E^{c}\cap \overline{R({\mathcal {B}}_{\kappa })}\). Then the stability estimate \(e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }}_{E^{c}}\le M\) follows by the decomposition (4.77) and the fact that \(e^{t{\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }}f=f\) when \(f\in \ker {\mathcal {B}}_{\kappa }\bar{{\mathcal {L}}}_{\kappa }\). When \(\tilde{S}_{\kappa }>0\), (4.78) follows from the decomposition (4.84) by the same arguments as above. This finishes the proof of the theorem. \(\quad \square \)
Remark 4.32

(a)
The polynomial growth on \(E^{c}\) can be shown ( [49]) to be at most quadratic by using the second order formulation (4.55).

(b)
It is tempting to find the most unstable eigenvalue \(\lambda _{0}<0\ \) of \({\mathcal {L}}_\kappa \) by trying to minimize \(\left( {\mathcal {L}}_{\kappa }f,f\right) _X\) over the constraint set \(\{f\in D\left( {\mathcal {L}}_{\kappa }\right) \mid \left\ f\right\ _{X}=1\}\). However, in the current case it is difficult to study this variational problem directly due to the lack of compactness. It is the selfadjointness of \({\mathcal {L}}_{\kappa }\) and the finiteness of the negative Morse index that are crucial to the proof of the existence of unstable eigenvalues.
Stability Analysis for the Einstein–Euler System
The Number Density n
We need to introduce an additional quantity \(n=N(\rho )\), the number density. We define the function N as
Clearly, \(N\in C^2 (]0,\infty [)\) with
and
in particular, \(\frac{\mathrm{d}N}{\mathrm{d}\rho }\) is a positive and strictly decreasing function on \(]0,\infty [\). By \(({\mathrm P}1)\), \(P(s) \le c s^\gamma \) for \(s\in [0,1]\), where \(\gamma >1\). Hence for \(0<\rho \le 1\),
Together with (5.1), (5.2), and the monotonicity of \(\frac{\mathrm{d}N}{\mathrm{d}\rho }\) this implies that \(\lim _{\rho \searrow 0} N(\rho ) = 0\) and \(\lim _{\rho \searrow 0} \frac{\mathrm{d}N}{\mathrm{d}\rho }(\rho )\) exists in \(]0,\infty [\), in particular, \(N\in C^1([0,\infty [)\).
The asymptotic behavior (3.31) implies that \(\lim _{\rho \rightarrow \infty } N(\rho ) =\infty \) so that \(N:[0,\infty [ \rightarrow [0,\infty [\) is onetoone and onto. Given any solution of the Einstein–Euler system we define the corresponding number density \(n= N(\rho )\) so that \(\rho =N^{1}(n) = \rho (n)\), and by (5.2),
Since the functional relationship between the pressure and the energy density \(\rho \) is prescribed by (1.8), the expression p(n) simply means \(P(\rho (n))\).
Using (5.3), the Tolman–Oppenheimer–Volkov equation (3.18) and the fact that \(p'_\kappa = \frac{\mathrm{d}P}{\mathrm{d}\rho }(\rho _\kappa ) \rho '_\kappa \) it follows that
on the interval \([0,R_\kappa [\), where \([0,R_\kappa ]\) is the support of the steady state. We integrate this differential equation to find that for all \(0\le r, r_0 < R_\kappa \),
We take the limit \(r_0\nearrow R_\kappa \) and observe that \(\rho _\kappa (R_\kappa )=0\) so that
Now we observe that the function
with \(c_\kappa >0\) has the same properties as the function N stated above, and we can choose the normalization constant \(c_\kappa \) such that \(\frac{\mathrm{d}N_\kappa }{\mathrm{d}\rho }(0)= e^{\mu _\kappa (R_\kappa )}\). For the given steady state and with \(n_\kappa = N_\kappa (\rho _\kappa )\) the identity
holds on the interval \([0,R_\kappa ]\).
Remark 5.1
The normalization can not be achieved by taking \(r\rightarrow \infty \), since outside the support of the steady state the middle term is necessarily constant, but \(\mu _\kappa \) is not.
Before we linearize the Einstein–Euler system we collect a few properties of the steady state under consideration.
Lemma 5.2
Let \((\rho _{\kappa },\lambda _{\kappa },\mu _{\kappa })\) be a steady state given by Proposition 3.2 for some \(\kappa >0\). Then
Let
denote the specific enthalpy of the steady state; \([0,R_\kappa ]\) is the support of \(\rho _\kappa \). Then
and \(1/\Psi _\kappa \in C([0,R_\kappa ])\) with
Proof
The identity (5.7) is a simple consequence of (5.6). The proof of (5.9) relies on the Tolman–Oppenheimer–Volkov Eq. (3.18). Replacing \(\rho _\kappa +p_\kappa \) by \(n_\kappa \frac{\mathrm{d} \rho }{\mathrm{d} n}(n_\kappa )\) and \(p_\kappa '\) by \(\frac{\mathrm{d} P}{\mathrm{d} \rho }(\rho _\kappa )\frac{\mathrm{d} \rho }{\mathrm{d} n}(n_\kappa ) n_\kappa '\) we obtain the result. The function \(\Psi _\kappa \) is continuous on \(]0,R_\kappa [\), and \(\Psi _\kappa (0) >0\). Finally, (5.10) follows if we express \(1/\Psi _\kappa \) via (5.9) and observe that \(\mu _\kappa '(R_\kappa )>0\) and \(n_\kappa '(R_\kappa ) = \frac{\mathrm{d} N_\kappa }{\mathrm{d} \rho }(0)\, \rho _\kappa '(R_\kappa )=0\). \(\quad \square \)
Linearization
Let us now linearize the Einstein–Euler system (2.14)–(2.19) around a given steady state \((n_\kappa , u_\kappa \equiv 0, \lambda _\kappa ,\mu _\kappa )\). We write \(n, u, \rho , p, \lambda , \mu \) for the Eulerian perturbations, i.e., \(n_\kappa + n, u_\kappa + u, \rho _\kappa + \rho \) etc. correspond to the solution of the original, nonlinear system. Linearizing \(\rho _\kappa +\rho = N_\kappa ^{1}(n_\kappa + n)\) and \(p_\kappa + p = P(N_\kappa ^{1}(n_\kappa + n))\) we have the relations
where we have used (5.6) and (5.8).
Linearizing (2.18) we arrive at
From (2.16) it follows that \({\dot{\lambda }}= 4 \pi r e^{\mu _\kappa + 2\lambda _\kappa }\, u\, (\rho _\kappa + p_\kappa )\). Together with the relation (3.17) this yields
and (5.13) simplifies to
Linearizing (2.19) we obtain the equation
where we have used (5.9). Multiplying by \(e^{\lambda _\kappa }\) we find that
where we have used the field equation (2.15) in the third line to conclude that
In order to proceed we define a suitable phase space and a modified potential in complete analogy to Definition 4.16.
Definition 5.3

(a)
The Hilbert space \(X_1\) is the space of all spherically symmetric functions in the weighted \(L^2\) space on the set \(B_\kappa =B_{R_\kappa }\) (the ball with radius \(R_\kappa \) which is the support of \(\rho _{\kappa }\)) with weight \(e^{2\mu _\kappa +\lambda _\kappa }\Psi _{\kappa }\) and the corresponding inner product, \(X_{2}\) is the space of radial functions in \(L^2 \left( B_\kappa \right) \), and the phase space for the linearized Einstein–Euler system is
$$\begin{aligned} X:=X_{1}\times X_{2}. \end{aligned}$$ 
(b)
For \(\rho \in X_1\) the induced modified potential \({{\bar{\mu }}}\) is defined as
$$\begin{aligned} {{\bar{\mu }}}(r) = {{\bar{\mu }}}_\rho (r) : = e^{\mu _\kappa \lambda _\kappa }\int _r^\infty \frac{1}{s}\, e^{\mu _\kappa (s)+\lambda _\kappa (s)}(2s\mu _\kappa '(s)+1)\,\lambda (s)\,\mathrm{d}s, \end{aligned}$$(5.18)where
$$\begin{aligned} \lambda (r) = 4\pi \frac{e^{2\lambda _\kappa }}{r} \int _0^r s^2 \rho (s)\,\mathrm{d}s. \end{aligned}$$(5.19) 
(c)
The operators \(\tilde{{\mathcal {L}}}_\kappa :X_1 \rightarrow X_1'\) and \({\mathcal {L}}_\kappa :X_1 \rightarrow X_1\) are defined by
$$\begin{aligned} \tilde{{\mathcal {L}}}_\kappa \rho := & {} e^{2\mu _\kappa +\lambda _\kappa }\Psi _{\kappa }\rho +e^{\mu _\kappa +\lambda _\kappa }{\bar{\mu }}_\rho . \end{aligned}$$(5.20)$$\begin{aligned} {\mathcal {L}}_\kappa \rho := & {} e^{2\mu _\kappa \lambda _\kappa }\Psi ^{1}_{\kappa } \tilde{{\mathcal {L}}}_\kappa \rho = \rho + e^{\mu _\kappa }\Psi _\kappa ^{1}{\bar{\mu }}_\rho \end{aligned}$$(5.21)
Here the dual pairing is realized through the \(L^2\)inner product, so that
To check that the operators \(\tilde{{\mathcal {L}}}_\kappa \) and \({{\mathcal {L}}}_\kappa \) are welldefined and selfdual or selfadjoint respectively, it clearly suffices to show only the selfadjointness of \({\mathcal {L}}_\kappa \).
Lemma 5.4
The operator \({\mathcal {L}}_{\kappa }\) defined by (5.21) is welldefined, selfadjoint, and \({\mathcal {L}}_{\kappa }\mathrm {id}\) is compact.
Proof
The operator \({\mathcal {L}}_{\kappa }\) is clearly symmetric, i.e.,
We define the operator \(K :X_{1}\rightarrow X_{1}\) by \(K\rho =\frac{1}{e^{\mu _{\kappa }}\Psi _{\kappa }}{\bar{\mu }}\). It is straightforward to check that the modified potential \({{\bar{\mu }}}\) has the properties stated in Lemma 4.17, in particular
Evaluating the \(L^2({\mathbb {R}}^3)\)inner product of (5.22) with \(e^{\mu _{\kappa }+\lambda _{\kappa }}{\bar{\mu }}\), we get
Therefore
and thus \(\left\ \nabla \left( e^{\mu _\kappa +\lambda _\kappa }{\bar{\mu }}\right) \right\ _{L^{2}} \le C \left\ \rho \right\ _{X_{1}}\). Here we made use of the compact support of \(\frac{1}{\Psi _{\kappa }}\) and the embedding \({\dot{H}}^{1}\left( {\mathbb {R}}^{3}\right) \hookrightarrow L^{2}\left( B_\kappa \right) \). Since
and the mapping \({\dot{H}}_{r}^{1}\left( {\mathbb {R}}^{3}\right) \ni \rho \mapsto \rho _{S}\in L^{2}(S)\) is compact for any compact set \(S\subset {\mathbb {R}}^3\), the operator K is compact. Therefore, \({\mathcal {L}}_{\kappa }=\mathrm {id}+K\) is selfadjoint on \(X_{1}\). \(\quad \square \)
With the definition of \(\tilde{{\mathcal {L}}}_\kappa \) the linearized velocity equation (5.16) takes the form
It easy to check that formally the energy
is conserved along solutions of (5.15), (5.23). The functional I maps \(X_1 \times L_{e^{3\lambda _{\kappa }}n_{\kappa }}^{2}\left( B_{\kappa }\right) \) to \({\mathbb {R}}\). From the definition (5.20) one readily checks that for \(\rho \in X_1\),
where we used (4.46) and (5.19).
The quadratic form \(I(\rho ,u)\) motivates the definition
so that \(v\in X_{2}\) for any \(u\in L_{e^{3\lambda _{\kappa }}n_{\kappa }}^{2}\left( B_{\kappa }\right) \). Then the system (5.15), (5.23) can be equivalently rewritten in the form
In order to exhibit the Hamiltonian structure of this system it is convenient to define the modified divergence operator \(A_{\kappa }:D\left( A_{\kappa }\right) \subset X_{2}=X_2^*\rightarrow X_{1}\) by
and its dual operator (the modified gradient) \(A_{\kappa }^{\prime }:X_{1}' \supset D( A_{\kappa }') \rightarrow X_{2}\) by
Then for any \(v\in X_{2}\) and \(\rho \in X_{1}'\) it holds that
where \(\left\langle \cdot ,\cdot \right\rangle \) is the dual bracket. We also define the adjoint operator \(A_{\kappa }^{*}:X_{1} \supset D\left( A_{\kappa }^{*}\right) \rightarrow X_{2}\) by
Then for any \(v\in X_{2}\) and \(\rho \in X_{1}\)
We claim that both \(A_\kappa \) and \(A_\kappa ^\prime \) are densely defined and closed. The density is clear, and to show that they are closed we consider the isometries
Then to show that \(A_\kappa :X_2'\supset D(A_\kappa )\rightarrow X_1\) is closed it suffices to show that \({\mathscr {A}}_\kappa :=S_2A_\kappa S_1 :L^2(B_\kappa )\supset D({\mathscr {A}}_\kappa )\rightarrow L^2(B_\kappa )\) is closed, where
The adjoint of \({\mathscr {A}}_\kappa \) is given by \( {{\mathscr {A}}}_\kappa ^*\rho = e^{\frac{3}{2}\lambda _\kappa }n_\kappa ^{\frac{1}{2}} \frac{\hbox {d}}{{\hbox {d}}r}\left( e^{\mu _\kappa +\lambda _\kappa /2} \Psi _\kappa ^{\frac{1}{2}}\rho \right) \) with
Clearly \({{\mathscr {A}}}_\kappa ^*\) is also densely defined. To show that \({\mathscr {A}}_\kappa \) is closed it suffices to show . To justify the latter we need to show that the adjoint of \({{\mathscr {A}}}_\kappa ^*\) is precisely \({\mathscr {A}}_\kappa \), which is a simple consequence of integrationbyparts since by (5.8) and thus the boundary term vanishes.
Remark 5.5
(Notational conventions) We adopt the notation from Yosida’s book [73]: For an operator \(A:X\rightarrow Y\) between two Hilbert spaces X and Y, we use \(A':Y'\rightarrow X'\) to denote the dual operator and \(A^{*}:Y\rightarrow X\) to denote the adjoint operator of A. The operators \(A^{\prime }\) and \(A^{*}\) are related by
where \(I_{X}:X'\rightarrow X\) and \(I_{Y}:Y'\rightarrow Y\) are the isomorphisms defined by the Riesz representation theorem.
Lemma 5.6
The formal linearization of the spherically symmetric Einstein–Euler system takes the Hamiltonian form
where \(\left( \rho ,v\right) \in X\), and
Then \(J^{\kappa }:X'\rightarrow X\) and \(L^{\kappa }:X\rightarrow X'\) are antiselfdual and selfdual respectively. The conserved energy functional takes the form
Alternatively, one can rewrite (5.29) in the secondorder form
or equivalently
Proof
Equation (5.29) is just a rephrasing of (5.25), (5.26). The formal conservation of energy \(t\mapsto I(\rho ,v)(t)\) is a direct consequence of the Hamiltonian structure (5.29). That \(J^\kappa \) is antiselfdual follows easily from the definition of \(A_\kappa \) and \(A_\kappa ^\prime \). The selfduality of \(L^\kappa \) follows from the selfduality of \(\tilde{{\mathcal {L}}}_\kappa \), which in turn, is a trivial consequence of the selfadjointness of \({\mathcal {L}}_\kappa \). Finally, (5.31) follows formally by applying \(\frac{\mathrm{d}}{\mathrm{d}t}\) to the vcomponent of (5.29) and then plugging in the equation satisfied by the \(\rho \)component of (5.29). Equation (5.32) is just a restatement of (5.32). \(\quad \square \)
We will show below that the operator \(A_{\kappa }^{*}{\mathcal {L}}_{\kappa } A_{\kappa }\) is in fact selfadjoint, cf. Lemma 5.23.
Variational Approach to Stability
The variational picture explained in this section is naturally constrained to radially symmetric perturbations. However, by comparison to the corresponding variational picture for the EVsystem in Sect. 4.1, we need to provide more details, as it will play an important role in our formulation of the main results for the Einstein–Euler system in Sect. 5.5. The fundamental conserved quantities associated with the Einstein–Euler system are the total ADM mass
and the total particle (or baryon) number
Recalling (5.5) we write
By (3.7), \(e^{\lambda } = \left( 1\frac{2m(r)}{r}\right) ^{\frac{1}{2}}\) with \(m(r)=4\pi \int _0^r s^2\rho (s)\,\mathrm{d}s\), and therefore
We define the open set
Since by CauchySchwarz \(X_1 \subset L^1(B_\kappa )\), it follows that both \({\mathscr {M}}\) and \({\mathscr {N}}_\kappa \) are welldefined functionals on U.
Remark 5.7
For any \(\kappa >0\) the associated fluid density \(\rho _\kappa \) belongs to U. The estimate \(\max _{r\in [0,R_\kappa ]}\frac{2m(r)}{r} <1\) follows from the Buchdahl inequality, while the integral bound \(\int _{B_\kappa }e^{2\mu _\kappa +\lambda _\kappa }\rho _\kappa ^2 \Psi _\kappa <\infty \) follows from the boundary asymptotics \(\Psi _\kappa \sim \rho _\kappa ^{\gamma 1}\) in the neighborhood of the vacuum boundary \(r=R_\kappa \), for some \(\gamma >0\).
Since \({\mathscr {M}}\) is a linear map it is clearly twice Fréchet differentiable and by a standard argument so is \({\mathscr {N}}_\kappa \). In order to compute the first and the second Fréchet derivative of these conserved quantities we will as above replace \(\rho \) by \(\rho _\kappa +\rho \) etc., i.e., \(\rho \) now stands for the corresponding perturbation. Since by definition all perturbations \(\rho \) belong to \(X_1\) we have \(\mathrm {supp}\,\rho \subset B_\kappa \).
Lemma 5.8
Let \(\kappa >0\). The steady state \((\rho _\kappa ,\lambda _\kappa ,\mu _\kappa )\) of the Einstein–Euler system given by Proposition 3.2 is a critical point of the binding energy
and
cf. (5.24).
Proof
Since \({\mathscr {M}}\) is linear its expansion is trivial. For \({\mathscr {N}}_\kappa \) we recall (5.19) so that
By (5.6), \(\frac{\mathrm{d}N}{\mathrm{d}\rho }(\rho _\kappa ) = e^{\mu _\kappa }\). Hence
Using (5.7) and (3.17) it follows that
which we use to rewrite the last integral as follows:
in the last line we used (5.19) and the fact that due to (3.17), \(\mu _\kappa (R_\kappa )+\lambda _\kappa (R_\kappa )=\mu _\kappa (\infty )+\lambda _\kappa (\infty )=0\). It follows that \(D {\mathscr {E}}_\kappa [\rho _\kappa ] =0\), i.e., \(\rho _\kappa \) is a critical point of \({\mathscr {E}}_\kappa \).
Next we compute \(D^2{\mathscr {E}}_\kappa [\rho _\kappa ]\), using (5.36):
From (5.3), (5.6), and (5.8) it follows that
Therefore,
In order to compute \(I_2\) we recall (5.6) and note that \( \rho = \frac{1}{4\pi r^2}\left( e^{2\lambda _\kappa } r \lambda \right) '. \) Hence
since \(\lambda (r)\sim 1/r\) for large r there is no boundary term at infinity in the integration by parts. Finally, using (5.37) it follows that
Summing \(I_j\), \(j=1,2,3\), we obtain the asserted formula for \(D^2{\mathscr {E}}_\kappa \), cf. (5.24). \(\quad \square \)
In the context of stellar perturbations the natural notion of dynamically accessible perturbations parallels Definition 4.14 for the Einstein–Vlasov system.
Definition 5.9
(Linearly dynamically accessible perturbations) A function \(\rho \) is a linearly dynamically accessible perturbation if \(\rho \in \) \(\overline{R\left( A_{\kappa }\right) }\), the closure of \(R\left( A_{\kappa }\right) \) in \(X_1\).
This definition is motivated by (5.25), which states that for solutions of the linearized system, \({{\dot{\rho }}} \in \overline{R\left( A_{\kappa }\right) }\). In other words, the space of linearly dynamically accessible perturbations is identified with the tangent space at \(\rho _\kappa \) of the leaf of all density configurations of a fixed total mass \({\mathscr {M}}(\rho )={\mathscr {M}}(\rho _\kappa )\). Since \(\overline{R\left( A_{\kappa }\right) }=(\ker A_\kappa ')^\perp \) and \(A_\kappa ' =e^{\frac{3}{2}\lambda _{\kappa }} n_{\kappa }^{\frac{1}{2}} \partial _r\) we have \(\overline{R\left( A_{\kappa }\right) }=(\ker \partial _r)^\perp \) and therefore
A weaker notion of linear stability is spectral stability.
Definition 5.10
The steady state \((\rho _\kappa ,\lambda _\kappa ,\mu _\kappa )\) is spectrally stable if the spectrum of the linearized operator \(A_{\kappa }'\mathcal {\tilde{L}}_{\kappa }A_{\kappa }\) is nonnegative.
Combining Lemmas 5.6 and 5.8 , we can provide a variational criterion for spectral stability of steady states of the Einstein–Euler system, which in the physics literature goes back to Chandrasekhar [16].
Proposition 5.11
(Chandrasekhar stability criterion) A steady state \((\rho _\kappa ,\lambda _\kappa ,\mu _\kappa )\) of the Einstein–Euler system given by Proposition 3.2 is spectrally stable if and only if
where the constraint set \({\mathcal {C}}_\kappa =\overline{R(A_\kappa )}\) is the set of all linearly dynamically accessible perturbations.
Proof
We use the secondorder formulation (5.31) and note that the spectral stability is equivalent to the nonnegativity of the quadratic form
The claim now follows from (5.38). \(\quad \square \)
Remark 5.12
By Lemma 5.8 spectral stability is equivalent to the positive semidefiniteness of the second variation of the energy along the manifold of linearly dynamically accessible perturbations.
The Reduced Einstein–Euler Operator
In analogy to Sect. 4.5 our goal is to show that the operator \(\mathcal {\tilde{L}}_{\kappa }\) is bounded from above and below by a Schrödingertype operator. Our main estimate is contained in Theorem 5.15. It is of independent interest, but it is also a crucial ingredient in our proof of Theorem 2.
We first recall the definition (5.18) of the modified potential \({{\bar{\mu }}}\) and the fact that the relation (4.46) also holds in the present situation.
Definition 5.13
(Reduced operator for the Einstein–Euler system) By analogy to \(S_\kappa \) the reduced operator \(\Sigma _\kappa :\dot{H}^1_r\rightarrow (\dot{H}^1_r)'\) is defined by
with \(\dot{H}^1_r\) and \(\Delta _\kappa \) defined as in Definition 4.23.
Remark 5.14

(a)
We recall from Sect. 3.2 that the macroscopic quantities related to a steady state of the Einstein–Vlasov system constitute a steady state of the Einstein–Euler system. Indeed, in this situation \(S_\kappa =\Sigma _\kappa \), since by (5.6) and (5.9),
$$\begin{aligned} e^{\mu _\kappa +\lambda _\kappa } \frac{\rho _\kappa '}{\mu _\kappa '} =  e^{\mu _\kappa +\lambda _\kappa } \frac{\rho _\kappa '}{\Psi _\kappa n_\kappa '} =  \frac{e^{\lambda _\kappa }}{\Psi _\kappa }. \end{aligned}$$(5.40) 
(b)
Just as \(S_\kappa \) the operator \(\Sigma _\kappa \) is selfdual.
Theorem 5.15
Let \(\Sigma _\kappa \) be the reduced operator given in Definition 5.13. For every \(\mu \in \dot{H}^1_r\) and \(\rho = \rho _\mu :=  \frac{e^{\mu _\kappa }}{\Psi _\kappa }\mu \) it holds that
For every \(\rho \in X_{1}\) and \({{\bar{\mu }}}={{\bar{\mu }}}_\rho \) as defined in (5.18) we have
Proof
The proof is analogous to the corresponding result, Theorem 4.24, in the Einstein–Vlasov case. We first observe that the identities (4.68) and (4.69) remain valid in the present situation. Using these together with the definitions of \(\mathcal {\tilde{L}}_\kappa \) and \(\Sigma _\kappa \) implies that
and (5.42) is proven. To obtain the estimate (5.41) we observe that (4.70) remains valid for \(\mu \in \dot{H}^1_r\) and \(\lambda =\lambda _\rho \) with \(\rho \in X_1\), and hence
in the last equality we used (4.70), which again remains valid in the present situation. This implies that
provided \(\rho =  \frac{e^{\mu _\kappa }}{\Psi _\kappa }\mu \), and (5.41) is proven. \(\quad \square \)
Main Results for the Einstein–Euler System
Lemma 5.16
The identity
holds. In particular
Proof
Recall that for any \(r\ge 0\) \(e^{2\lambda _\kappa (r)} = 1  \frac{2m(r)}{r}\), where \(m(r)=\int _0^r 4\pi \rho _\kappa (s) s^2\,\mathrm{d}s\). In particular, \(M_\kappa =\int _0^\infty 4\pi s^2\rho _\kappa \,\mathrm{d}s = m(R_\kappa )\). Since \(e^{\mu _\kappa (r)+\lambda _\kappa (r)} = 1\) for all \(r\ge R_\kappa \), it finally follows that
\(\square \)
Theorem 5.17
Let \((\rho _\kappa ,\lambda _\kappa ,\mu _\kappa )\) be a steady state of the EulerEinstein system given by Proposition 3.2, and let
If \(\frac{\mathrm{d}}{\mathrm{d}\kappa } M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa }\left( \frac{M_\kappa }{R_\kappa }\right) \ne 0\), then the number of growing modes associated with the linearized operator \(J_\kappa L_\kappa \) equals \(n^(\Sigma _\kappa )i_\kappa \).
Proof
By Lemma 5.8 we have
where \({\mathscr {N}}_0(\rho )\) is defined in (5.35). Since \(c_\kappa >0\) the spectral stability is equivalent to checking the nonnegativity of \(\langle \left( \frac{\mathrm{d}^2{\mathscr {M}}[\rho _\kappa ]}{c_\kappa }  D^2{\mathscr {N}}_0[\rho _\kappa ]\right) \rho ,\,\rho \rangle \) for all \(\rho \in {\mathcal {C}}_\kappa \) (i.e. \(\rho \in X_1\), \(\int _{D_\kappa }\rho =0\)) by Proposition 5.11. Define the operator
By the above analysis it is clear that \(\rho _\kappa \) is a critical point of \(\tilde{{\mathscr {E}}}_\kappa \), i.e.,
It is not hard to see that the maps \(]0,\infty [ \, \ni \kappa \mapsto \frac{1}{c_\kappa }D{\mathscr {M}}[\rho _\kappa ]  D{\mathscr {N}}_0[\rho _\kappa ] \in {\mathcal {L}}(X_1, {\mathbb {R}})\) and \(]0,\infty [ \, \ni \kappa \mapsto \rho _\kappa \in X_1\) are \(C^1\)maps in the sense of Fréchet differentiability. The claim follows from the smooth dependence of the solutions of the steady state ODE (3.9) on the initial data. Differentiating (5.44) with respect to \(\kappa \) we arrive at
Therefore we have the relationship
for any \(\rho \in X_1\). In other words, \(\rho \) is orthogonal to \(\frac{\mathrm{d}\rho _\kappa }{\mathrm{d}\kappa }\) with respect to the (possibly sign indefinite) distance induced by the quadratic form \(\langle D^2\tilde{{\mathscr {E}}}_\kappa [\rho _\kappa ] \cdot , \cdot \rangle \). Furthermore
where \(M_\kappa = {\mathscr {M}}(\rho _\kappa )\) is the ADM mass of the steady star, and we recall from Sect. 5.1 that \(c_\kappa =c_0 e^{\mu _\kappa (R_\kappa )}\) for some \(\kappa \)independent constant \(c_0\). Recalling the reduced operator \(\Sigma _\kappa \) given by Definition 5.13, we then have
First assume \(\frac{\mathrm{d}}{\mathrm{d}\kappa } M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa } \left( \mu _\kappa (R_\kappa )\right) <0\). Then by the above calculation \(\frac{\mathrm{d}\rho _\kappa }{\mathrm{d}\kappa }\) is a negative direction for the quadratic form \(\langle D^2\tilde{{\mathscr {E}}}_\kappa [\rho _\kappa ]\cdot , \cdot \rangle \), i.e., for \(\langle \tilde{{\mathcal {L}}}_\kappa \cdot ,\cdot \rangle _{L^2}\). Since \({\mathcal {C}}_\kappa \) is orthogonal to \(\frac{\mathrm{d}\rho _\kappa }{\mathrm{d}\kappa }\) by (5.46) it follows that the number of growing dynamically accessible modes is \(n^(\Sigma _\kappa )1\). If \(\frac{\mathrm{d}}{\mathrm{d}\kappa } M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa } \left( \mu _\kappa (R_\kappa )\right) >0\), then by a similar argument
since \({\mathcal {C}}_\kappa \) is orthogonal to \(\frac{\mathrm{d}\rho _\kappa }{\mathrm{d}\kappa }\) by (5.46). Therefore, by (5.47) it follows that \(n^(\Sigma _\kappa ) = n^(L_\kappa )\) and the theorem is proved, since by Lemma 5.16, \(\text {sign}(\frac{\mathrm{d}}{\mathrm{d}\kappa }(\mu _\kappa (R_\kappa )) =\text {sign}(\frac{\mathrm{d}}{\mathrm{d}\kappa }\left( \frac{M_\kappa }{R_\kappa }\right) )\). \(\quad \square \)
A simple consequence of the above theorem is the following characterization of spectral stability.
Corollary 5.18
(A nearcharacterization of spectral stability) The steady state \(\rho _\kappa \) is spectrally stable if \(n^(\Sigma _\kappa )=1\) and \(\frac{\mathrm{d}}{\mathrm{d}\kappa } M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa }\left( \frac{M_\kappa }{R_\kappa }\right) <0\). The transition of stability to instability can only happen at the values \(\kappa \) satisfying \(n^{}\left( \Sigma _{\kappa }\right) =1\) and \(\frac{\mathrm{d}}{\mathrm{d}\kappa }M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa }\left( \frac{M_\kappa }{R_\kappa }\right) =0\).
Remark 5.19

(a)
By (5.45) it holds that at a critical point \(\kappa ^*\) of \(\kappa \mapsto \frac{M_\kappa }{R_{\kappa }}\),
$$\begin{aligned} {\mathcal {L}}_{\kappa ^*}\frac{\mathrm{d}\rho _{\kappa }}{\mathrm{d}\kappa }_{\kappa =\kappa ^*}=0. \end{aligned}$$Therefore, \({\mathcal {L}}_{\kappa ^*}\) (and equivalently the reduced operator \(\Sigma _{\kappa ^*}\)) has a zero eigenvalue at \(\kappa ^*\). This is a strong indication that the number of negative eigenvalues of \({\mathcal {L}}_{\kappa }\) increases or decreases by 1 as \(\kappa \) crosses \(\kappa ^*\). A detailed analysis of this subtle point will be given in [31].

(b)
Since \(M_\kappa \) is the ADM mass and \(R_\kappa \) the arearadius of the 2sphere \(t=0\) and \(r=R_\kappa \) it follows that \(i_\kappa \) has an invariant geometric meaning.
We now state the main result giving a detailed description of the linearized dynamics for the Einstein–Euler system.
Theorem 5.20
The operator \(J^{\kappa }L^{\kappa }\) generates a \( C^{0}\) group \(e^{tJ^{\kappa }L^{\kappa }}\) of bounded linear operators on X and there exists a decomposition
with the following properties:

(i)
\(E^{u}\left( E^{s}\right) \) consist only of eigenvectors corresponding to negative (positive) eigenvalues of \(J^{\kappa }L^{\kappa }\) and when \(\frac{\mathrm{d}}{\mathrm{d}\kappa } M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa }\left( \frac{M_\kappa }{R_\kappa }\right) \ne 0\),
$$\begin{aligned} \dim E^{u}=\dim E^{s}=n^{}\left( \Sigma _{\kappa }\right) i_{\kappa }, \end{aligned}$$(5.48)where \(i_{\kappa }\) is defined by (5.43).

(ii)
The quadratic form \(\left( L^{\kappa }\cdot ,\cdot \right) _X \) vanishes on \(E^{u,s}\), but is nondegenerate on \(E^{u}\oplus E^{s} \), and
$$\begin{aligned} E^{c}=\left\{ \begin{pmatrix} \rho \\ v \end{pmatrix} \in X \mid \left( L^{\kappa } \begin{pmatrix} \rho \\ v \end{pmatrix}, \begin{pmatrix} \rho _1 \\ v_1 \end{pmatrix} \right) _X =0\ \text{ for } \text{ all }\ \begin{pmatrix} \rho _1 \\ v_1 \end{pmatrix} \in E^{s}\oplus E^{u}\right\} . \end{aligned}$$ 
(iii)
\(E^{c}, E^{u}, E^{s}\) are invariant under \(e^{tJ^{\kappa }L^{\kappa }}\).

(iv)
Let \(\lambda _{u}=\min \{\lambda \mid \lambda \in \sigma (J^{\kappa }L^{\kappa }_{E^{u}})\}>0\). Then there exist \(M>0\) such that
$$\begin{aligned}&\left e^{tJ^{\kappa }L^{\kappa }}_{E^{s}}\right \le M e^{\lambda _{u}t},\ t\ge 0,\nonumber \\&\left e^{tJ^{\kappa }L^{\kappa }}_{E^{u}}\right \le M e^{\lambda _{u}t},\ t\le 0, \end{aligned}$$(5.49)$$\begin{aligned}&\left e^{tJ^{\kappa }L^{\kappa }}_{E^{c}}\right \le M (1+t)^{k_{0}}, \ t\in {\mathbb {R}}, \end{aligned}$$(5.50)where
$$\begin{aligned} k_{0}\le 1+2i_{\kappa }. \end{aligned}$$(5.51)
The proof of Theorem 5.20 is similar to the proof of Theorem 4.28. We need several auxiliary lemmas. By an argument analogous to the proof of Lemma 4.18, one can show that the linear operator \(X_1\ni \rho \mapsto e^{\mu _\kappa }\Psi _\kappa ^{1}{\bar{\mu }}_\rho \in X_1\) is compact and thus the only possible accumulation point of the spectrum of \({\mathcal {L}}_\kappa \) and therefore \(\tilde{{\mathcal {L}}}_\kappa \) is 1. Analogously to the proof of Proposition 4.30, we obtain the following result.
Lemma 5.21
There exists a decomposition of \(X_{1}\) into the direct sum of three closed subspaces
with the following properties:

(H1)
\(\left( \mathcal {\tilde{L}}_{\kappa }\rho ,\rho \right) _{L^2} <0\) for all \(\rho \in X_{1,}\backslash \{0\}\); and
$$\begin{aligned} n^{}\left( \tilde{{\mathcal {L}}}_{\kappa }\right) := \dim X_{1,}=n^{}\left( \Sigma _{\kappa }\right) ,\ \ \ \dim \ker \tilde{{\mathcal {L}}}_{\kappa }=\dim \ker \Sigma _{\kappa }. \end{aligned}$$ 
(H2)
There exists a \(\delta >0\) such that
$$\begin{aligned} \left( \mathcal {\tilde{L}}_{\kappa }\rho ,\rho \right) _{L^2} \ge \delta \left\ \rho \right\ _{X_{1}}^{2}\ ,\text { for any }\rho \in X_{1,+}. \end{aligned}$$
By the definition (5.30) of the operator \(L^{\kappa }\), the above lemma readily implies the following.
Proposition 5.22
There exists a decomposition of X into the direct sum of three closed subspaces
with the following properties:

(H1)
\(\left( L^{\kappa }\begin{pmatrix} \rho \\ v \end{pmatrix}, \begin{pmatrix} \rho \\ v \end{pmatrix} \right) _X <0\) for all \(\begin{pmatrix} \rho \\ v \end{pmatrix}\in X_{}\backslash \{0\}\); and
$$\begin{aligned} n^{}\left( L^{\kappa }\right) := \dim X_{}=n^{}\left( \Sigma _{\kappa }\right) ,\ \ \ \dim \ker L^{\kappa }=\dim \ker \Sigma _{\kappa }. \end{aligned}$$ 
(H2)
There exists \(\delta >0\) such that
$$\begin{aligned} \left( L^{\kappa }\begin{pmatrix} \rho \\ v \end{pmatrix}, \begin{pmatrix} \rho \\ v \end{pmatrix} \right) _X \ge \delta \left( \left\ \rho \right\ _{X_{1}}^{2} +\left\ v\right\ _{X_{2}}^{2}\right) \ \text { for all } \begin{pmatrix} \rho \\ v \end{pmatrix} \in X_{+}. \end{aligned}$$
Lemma 5.23
The operator \(A_{\kappa }^{*}{\mathcal {L}}_{\kappa }A_{\kappa }:X_{2}\rightarrow X_{2}\) is selfadjoint.
Proof
The operator \(A_{\kappa }:D\left( A_{\kappa }\right) \subset X_{2}\rightarrow X_{1}\) is clearly densely defined and closed. The operator \(A_{\kappa }^{*}:D\left( A_{\kappa }^{*}\right) \subset X_{1}\rightarrow X_{2}\) is also densely defined. By Lemma 5.4, the operator \({\mathcal {L}}_{\kappa }:X_{1}\rightarrow X_{1}\) is selfadjoint. Moreover, by Lemma 5.21, for the quadratic form \(\left( {\mathcal {L}}_{\kappa }\cdot ,\cdot \right) _{X_1} =\left\langle \mathcal {\tilde{L}}_{\kappa }\cdot ,\cdot \right\rangle \), there exists a decomposition \(X_{1}=X_{1,}\oplus \ker {\mathcal {L}}_{\kappa }\oplus X_{1,+}\) such that the properties (H1) and (H2) from Lemma 5.21 hold. Therefore, by Proposition A.3 in the Appendix the operator \(A_{\kappa }^{*}{\mathcal {L}}_{\kappa }A_{\kappa }\) is selfadjoint. \(\quad \square \)
Now we are ready to prove Theorem 5.20.
Proof of Theorem 5.20
To prove (i), we use the second order formulation (5.31). We note that
When \(\frac{\mathrm{d}}{\mathrm{d}\kappa }M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa } \left( \mu _{\kappa }(R_{\kappa })\right) \ne 0\), by (5.38) and the proof of Theorem 5.17, we have
Thus there exist exactly \(n^{}\left( \Sigma _{\kappa }\right) i_{\kappa }\) negative eigenvalues of \(A_{\kappa }'\mathcal {\tilde{L}}_{\kappa }A_{\kappa }\ \)in \(X_{2}\), each of which corresponds to a pair of stable and unstable eigenvalues for (5.31) and consequently for (5.29). This finishes the proof of part (i). The conclusions in (ii)–(iv) follow from [48, Thm. 2.2]. \(\quad \square \)
A Negative Energy Direction for \(\kappa \) Sufficiently Large
In order to prove that the negative Morse index of \(\tilde{{\mathcal {L}}}_\kappa \) is at least 1 we need to introduce an additional assumption on the equation of state.
Assumption (P2). There exists a constant \(C>0\) such that
We note that the examples \(P_\Phi \) and \(P_\mathrm {NS}\) which were discussed in Sect. 3.2 satisfy this assumption, cf. Propositions 3.3 and 3.5.
To analyze the sign of the quadratic form \(\langle \tilde{{\mathcal {L}}}_\kappa \rho ,\rho \rangle _{L^2}\) it will be convenient to rewrite in a slightly different form. Motivated by the Lagrangian coordinates point of view we consider perturbations of the form \(\rho = (r^2 n_\kappa \xi )'/r^2\) and the associated metric perturbation \(\lambda \). The second variation then can be written in the form
where \(\rho = A_\kappa \xi \), and therefore \(\rho \in {\mathcal {C}}_\kappa \) is linearly dynamically accessible in the sense of Definition 5.9.
Theorem 5.24
Let the equation of state \(p=P(\rho )\) satisfy the assumptions \(({\mathrm P} 1)\) and \(({\mathrm P} 2)\), and let \((\rho _{\kappa },\lambda _{\kappa },\mu _{\kappa })_{\kappa >0}\) be a family of steady states given by Proposition 3.2. There exists \(\kappa _0>0\) sufficiently large such that for any \(\kappa >\kappa _0\) there exists a nontrivial perturbation \(\rho \in {\mathcal {C}}_\kappa \) satisfying \((\tilde{{\mathcal {L}}}_\kappa \rho ,\rho )_{L^2}<0\). In particular, the negative Morse index \(n^(\tilde{{\mathcal {L}}}_\kappa )\) of the operator \(\tilde{{\mathcal {L}}}_\kappa \) is at least 1.
Proof
Some of the main ideas of this proof are analogous to the ones used in the proof of Theorem 4.3, but due to the local character of the operator \(L_\kappa \), the proof is actually simpler than the one of Theorem 4.3. Our starting point is the formula (5.53). We localize the perturbation \(\xi \) to the interval \([r_\kappa ^1, r_\kappa ^2]\) by setting \(\xi (r) = r^a\chi _\kappa (r)\) for an \(a\in {\mathbb {R}}\) to be specified later; the smooth cutoff function \(\chi _\kappa \) is as in (4.19). We split the quadratic form (5.53) into two parts: \(A_\kappa =A_{\kappa ,1}+A_{\kappa ,2}\), where
In order to understand the asymptotic behavior of these expressions as \(\kappa \rightarrow \infty \) we need to supplement the results from Propositions 3.13 and 3.14 with corresponding information on \(n_\kappa \), \(\Psi _\kappa \), and \(n'_\kappa \). But by (5.9) and (5.7) it holds that
We also recall (3.17) so that
By Proposition 3.13,
for \(\kappa \) sufficiently large, and hence by (P2),
on \([r_\kappa ^1, r_\kappa ^2]\) and for \(\kappa \) sufficiently large. Together with the estimates from Proposition 3.13 this implies that for any \(\epsilon >0\) the estimates
hold on \([r_\kappa ^1, r_\kappa ^2]\) for \(\kappa \) sufficiently large. If we substitute the exact asymptotic values of these expressions and the estimate from Proposition 3.14 into (5.54) the integrand becomes proportional to \(r^{1}\), provided we choose \(a=1/4\). With this choice and with the exact asymptotic values a short computation shows for the term \([\ldots ]\) in (5.54) that
Choosing \(\epsilon >0\) sufficiently small and \(\kappa \) sufficiently large it therefore follows that
where \(C>0\) is independent of \(\kappa \) and \(C_\kappa \) is the constant introduced in Proposition 3.14.
Turning now to \(A_{\kappa ,2}\) we must show that this term is smaller in modulus for \(\kappa \) large. We split this term further:
where
and
We recall that on the interval \([r_\kappa ^1,2r_\kappa ^1]\) the estimate \(\chi '_\kappa \le 2/r_\kappa ^1\) holds, and \(\chi '_\kappa \le 4/r_\kappa ^2\) holds \([r_\kappa ^2/2, r_\kappa ^2]\). Thus
and together with the estimate for \(A_{\kappa ,1}\) this implies that \(A_{\kappa } < 0\) for \(\kappa \) sufficiently large. \(\quad \square \)
Corollary 5.25
(Unstable star models) By Theorem 5.24 and Proposition 5.11 steady states of the Einstein–Euler system are dynamically unstable for sufficiently large central redshifts \(\kappa \gg 1\). In particular the stable and unstable spaces \(E^{u,s}\) are nonempty and the exponential trichotomy with precise estimates described in Theorem 5.20 holds.
The MicroMacro Stability Principle
The following theorem is a generalrelativistic analogue of Antonov’s First Law which can be expressed as follows: In the context of Newtonian physics a spherically symmetric stellar system with phase space density of the form \(f=\phi (E)\) with \(\phi \) strictly decreasing on its support is stable if the barotropic star with the corresponding equation of state and the same macroscopic density is stable, cf. [9, Sect. 5.2].
Theorem 5.26
(Micromacro stability comparison) Let \(\Phi \) be a microscopic equation of state satisfying the assumption \((\Phi 1)\) and let \(P=P_\Phi \) be the macroscopic equation of state generated by \(\Phi \) according to Proposition 3.3. Now fix some \(\kappa >0\). If the corresponding steady state \((f_\kappa , \lambda _\kappa ,\mu _\kappa )\) of the Einstein–Vlasov system is spectrally unstable, then the same is true for the corresponding steady state \((\rho _\kappa ,\lambda _\kappa ,\mu _\kappa )\) of the Einstein–Euler system.
Proof
If \(f_\kappa \) is spectrally unstable, then by the linear stability criterion for the Einstein–Vlasov system there exists a function \(f\in R\left( {\mathcal {B}}_{\kappa }\right) \) such that \({\mathcal {A}}_\kappa (f,f)<0\), where \({\mathcal {A}}_\kappa (f,f)\) is defined by (4.51), (4.41). We show that \(f\in R\left( {\mathcal {B}}_{\kappa }\right) \) implies that \(\int \rho _{f}\mathrm{d}x=0\), where \(\rho _{f}=\int f\, \left\langle v\right\rangle \, \mathrm{d}v\). Indeed, let \(f={\mathcal {B}}_{\kappa }h\), \(h\in D({\mathcal {B}}_\kappa )\). Then
Here we used the facts that \({\mathcal {B}}_{\kappa }\) is antiselfadjoint in X and \({\mathcal {B}}_{\kappa }\left( \phi _{\kappa }^{\prime }(E_{\kappa }) E_{\kappa }e^{\mu _{\kappa }\lambda _{\kappa }}\right) =0\); the latter follows from the definition of \({\mathcal {B}}_{\kappa }\) and (4.15). On the other hand by the CauchySchwarz inequality,
which simply says that in the present situation \(S_\kappa =\Sigma _\kappa \). This implies that
and
which in turn implies that
It follows that \(\left( \tilde{{\mathcal {L}}}_{\kappa } \rho _f ,\rho _f \right) _{L^2}<0\) with f chosen as above. By Proposition 5.11 this implies that the steady state \(\rho _\kappa \) of the Einstein–Euler system is unstable as well. \(\quad \square \)
A corollary of the above micromacro stability estimate is the stability part of the turning point principle for the Einstein–Vlasov system.
Theorem 5.27
(The stability part of the turning point principle for the Einstein–Vlasov system)
Under the assumptions of Theorem 5.26 the following holds: If \(n^(S_\kappa )=1\) and \(\frac{\mathrm{d}}{\mathrm{d}\kappa } M_\kappa \frac{\mathrm{d}}{\mathrm{d}\kappa } \left( \mu _\kappa (R_\kappa )\right) <0\), then the steady state \(f_\kappa \) is linearly stable. Here \(S_\kappa \) denotes the reduced operator defined in (4.59).
Proof
Let \((\rho _\kappa ,\lambda _\kappa ,\mu _\kappa )\) be the associated steady state of the Einstein–Euler system. Since \(S_\kappa =\Sigma _\kappa \) by (5.40), we have \(n^(\Sigma _\kappa )=1\). Thus by the part (i) of the turning point principle for the Einstein–Euler system, Theorem 5.17, we conclude that \((\rho _\kappa ,\lambda _\kappa ,\mu _\kappa )\) is spectrally stable. By (5.54) it follows that \({\mathcal {A}}_\kappa (f,f)\ge 0\) for all linearly dynamically accessible \(f\in \overline{R({\mathcal {B}}_\kappa )}\). This in turn implies the linear stability of \(f_\kappa \). \(\quad \square \)
Remark 5.28
The proof of the full turning point principle for the Einstein–Vlasov system is considerably harder than in the case of the Einstein–Euler system since the space of linearly dynamically accessible perturbations \(R({\mathcal {B}}_\kappa )\) is of infinite codimension, by contrast to \({\mathcal {C}}_\kappa \), see (5.38).
References
Andersson, L., Korzyński, M.: Variational principle for the EinsteinVlasov equations. Available online at arXiv:1910.12152
Andréasson, H.: On static shells and the Buchdahl inequality for the spherically symmetric EinsteinVlasov system. Commun. Math. Phys. 274, 409–425, 2007
Andréasson, H.: Sharp bounds on \(2m/r\) of general spherically symmetric static objects. J. Differ. Equ. 245, 2243–2266, 2008
Andréasson, H., Fajman, D., Thaller, M.: Models for selfgravitating photon shells and geons. Ann. Henri Poincaré 18, 681–705, 2017
Andréasson, H., Kunze, M.: Comments on the paper “Static solutions of the VlasovEinstein system” by G. Wolansky. Preprint. Available on arXiv as arXiv:1805.10683
Andréasson, H., Rein, G.: A numerical investigation of the stability of steady states and critical phenomena for the spherically symmetric EinsteinVlasov system. Class. Quantum Grav. 23, 3659–3677, 2006
Antonov, A.V.: Remarks on the problem of stability in stellar dynamics. Soviet Astr. A J. 4, 859–867, 1961
Bardeen, J.M., Thorne, K.S., Meltzer, D.W.: A Catalogue of Methods for Studying the Normal Modes of Radial Pulsation of GeneralRelativistic Stellar Models. Astrophys. J. 145, 505, 1966
Binney, J., Tremaine, S.: Galactic Dynamics, 2nd edn. Princeton Series in Astrophysics (2008)
BisnovatyiKogan, G.S., Zel’dovich, Ya B.: Models of clusters of point masses with great central red shift. Astrofizika 5, 223–234, 1969
BisnovatyiKogan, G.S., Thorne, K.S.: Relativistic gas spheres and clusters of point masses with arbitrarily large central redshifts: can they be stable? Astrophys. J. 160, 875, 1970
Buchdahl, H.A.: General relativistic fluid spheres. Phys. Rev. 116, 1027–1034, 1959
Calamai, G.: On a static criterion for the stability of the equilibrium. Astrophys. space Sci. 8, 53–58, 1970
Calamai, G.: The stability of the equilibrium of supermassive and superdense stars. Astrophys. Space Sci. 6, 240–257, 1970
Calamai, G.: A static theory for the stability of the equilibrium. Astrophys. Space Sci. 10, 165–177, 1971
Chandrasekhar, S.: A general variational principle governing the radial and the nonradial oscillations of Gaseous Masses. Astrophys. J. 139, 664, 1964
Dafermos, M.: A note on the collapse of small data selfgravitating massless collisionless matter. J. Hyperbol. Differ. Equ. 3, 589–598, 2006
Doremus, J.P., Feix, M.R., Baumann, G.: Stability of Encounterless Spherical Stellar Systems. Phys. Rev. Lett. 26, 725, 1971
Fackerell, E.D., Ipser, J.R., Thorne, K.S.: Relativistic star clusters. Comments Astrophys. Space Phys. 1, 134, 1969
Fackerell, E.D.: Relativistic, Spherically Symmetric Star Clusters IV  a Sufficient Condition for Instability of Isotropic Clusters against Radial Perturbations. Astrophys. J. 160, 859, 1970
Fackerell, E.D., Suffern, K.G.: The structure and dynamic instability of isothermal relativistic star clusters. Austr. J. Phys. 29, 311–328, 1976
Fackerell, E.D., Suffern, K.G.: The dynamic instability of isothermal relativistic star clusters. Astrophys. J. 203, 477–480, 1976
Fajman D., Joudioux, J., Smulevici, J.: The Stability of the Minkowski space for the EinsteinVlasov system. Preprint. arXiv:1707.06141
Fournodavlos, G., Schlue, V.: On ‘hard stars’ in general relativity. Annales Henri Poincaré 20(7), 2135–2172, 2019
MartínGarcía, J.M., Gundlach, C.: Critical phenomena in gravitational collapse. Living Rev. Relat. 10, 1–57, 2007
Gérard, P.: Description du défaut de compacité de l’injection de Sobolev. ESAIM COCV 3, 213–233, 1998
Glendenning, N.K.: Compact stars, 2nd edn. Springer, Berlin 2000
Guo, Y., Rein, G.: Isotropic steady states in galactic dynamics. Commun. Math. Phys. 219, 607–629, 2001
Guo, Y., Rein, G.: A nonvariational approach to nonlinear stability in stellar dynamics applied to the King model. Commun. Math. Phys. 271, 489–509, 2007
Guo, Y., Lin, Z.: Unstable and stable galaxy models. Commun. Math. Phys. 279, 789–813, 2008
Hadžić, M., Lin, Z.: Turning Point Principle for Relativistic Stars. arXiv:2006.09749.
Hadžić, M., Rein, G.: Stability for the spherically symmetric EinsteinVlasov system—a coercivity estimate. Math. Proc. Camb. Phil. Soc. 155, 529–556, 2013
Hadžić, M., Rein, G.: On the small redshift limit of steady states of the spherically symmetric EinsteinVlasov system and their stability. Math. Proc. Camb. Phil. Soc. 159, 529–546, 2015
Harrison, B.K., Thorne, K.S., Wakano, M., Wheeler, J.A.: Gravitation Theory and gravitational collapse. The University of Chicago press, Chicago 1965
Hsu, C.H., Lin, S.S., Makino, T.: On spherically symmetric solutions of the relativistic Euler equation. J. Differ. Equ. 201, 1–24, 2004
Ipser, J., Thorne, K .S.: Relativistic, spherically symmetric star clusters I. Stability theory for radial perturbations. Astrophys. J. 154, 251–270, 1968
Ipser, J.: Relativistic, spherically symmetric star clusters. II. Sufficient conditions for stability against radial perturbations. Astrophys. J. 156, 509–527, 1969
Ipser, J.: Relativistic, spherically symmetric star clusters. III. Stability of compact isotropic models. Astrophys. J. 158, 17–43, 1969
Ipser, J.: A bindingenergy criterion for the dynamical stability of spherical stellar systems in general relativity. Astrophys. J. 238, 1101–1110, 1980
Kandrup, H., Morrison, P.: Hamiltonian structure of the VlasovEinstein system and the problem of stability for spherical relativistic star clusters. Ann. Phys. 225, 114–166, 1993
Kandrup, H., Sygnet, J.F.: A simple proof of dynamical stability for a class of spherical clusters. Astrophys. J. 298, 27–33, 1985
Kato, T.: Perturbation Theory for Linear Operators. Springer, Berlin 1980
Katz, J., Horwitz, G., Klapisch, M.: Thermodynamic stability of relativistic stellar clusters. Astrophys. J. 199, 307–321, 1975
Laval, G., Mercier, P., Pellat, R.: Necessity of the energy principles for magnetostatic stability. Nuclear Fusion 5, 165, 1965
Lemou, M., Mehats, F., Raphaël, P.: A new variational approach to the stability of gravitational systems. Commun. Math. Phys. 302(1), 161–224, 2011
Lemou, M., Mehats, F., Raphaël, P.: Orbital stability of spherical systems. Inventiones Math. 187, 145–194, 2012
Lin, Z., Strauss, W.: A sharp stability criterion for the VlasovMaxwell system. Invent. Math. 173(3), 497–546, 2008
Lin, Z., Zeng, C.: Instability, index theorem, and exponential trichotomy for Linear Hamiltonian PDEs. To appear in Memoirs of AMS.
Lin, Z., Zeng, C.: Separable Hamiltonian PDEs and Turning point principle for stability of gaseous stars. To appear in Comm. Pure. Appl. Math.
Lindblad, H., Taylor, M.: Global stability of Minkowski space for the Einstein–Vlasov system in the harmonic gauge. To appear in Archive for Rational Mechanics and Analysis. arXiv:1707.06079
Makino, T.: On spherically symmetric stellar models in general relativity. J. Math. Kyoto Univ. 38(1), 55–69, 1998
Makino, T.: On spherically symmetric solutions of the EinsteinEuler Equations. Kyoto J. Math. 56(2), 243–282, 2016
Meltzer, D.W., Thorne, K.S.: Normal modes of radial pulsation of stars at the end point of thermonuclear evolution. Astrophys. J. 145, 514–543, 1966
Mouhot, C.: Stabilité orbitale pour le système de VlasovPoisson gravitationnel, d’après LemouMéhatsRaphaël, Guo, Lin, Rein et al. Séminaire Nicolas Bourbaki Nov. 2011, arXiv:1201.2275 2012
Oppenheimer, J.R., Volkoff, G.M.: On massive neutron cores. Phys. Rev. 55, 374–381, 1939
Ramming, T., Rein, G.: Spherically symmetric equilibria for selfgravitating kinetic or fluid models in the nonrelativistic and relativistic case—a simple proof for finite extension. SIAM J. Math. Anal. 45, 900–914, 2013
Rasio, F.A., Shapiro, S.L., Teukolsky, S.A.: Solving the Vlasov equation in general relativity. Astrophys. J. 344, 146–157, 1989
Rein, G.: Static solutions of the spherically symmetric VlasovEinstein system. Math. Proc. Camb. Phil. Soc. 115, 559–570, 1994
Rein, G.: The VlasovEinstein System with Surface Symmetry. Habilitationsschrift, München 1995
Rein, G.: Collisionless kinetic equations from astrophysics—The VlasovPoisson system. In Handbook of Differential Equations, Evolutionary Equations, vol. 3, edited by C. M. Dafermos and E. Feireisl, Elsevier 2007
Rein, G., Rendall, A.: Global existence of solutions of the spherically symmetric VlasovEinstein system with small initial data. Commun. Math. Phys. 150, 561–583 (1992). Erratum: Commun. Math. Phys. 176, 475–478 1996
Rein, G., Rendall, A.: Compact support of spherically symmetric equilibria in nonrelativistic and relativistic galactic dynamics. Math. Proc. Camb. Phil. Soc. 128, 363–380, 2000
Rein, G., Straub, C.: On the transport operators arising from linearizing the VlasovPoisson or EinsteinVlasov system about isotropic steady states. Kinetic & Related Models, to appear
Rudin, W.: Functional Analysis. International Series in Pure & Applied Mathematics, 2nd edition, 1990
Schaeffer, J.: A class of counterexamples to Jeans’ Theorem for the VlasovEinstein system. Commun. Math. Phys. 204, 313–327, 1999
Schiffrin, J.S., Wald, R.M.: Turning point instabilities for relativistic stars and black holes. Classical Quant. Grav. 31, 035024, 2014
Shapiro, S.L., Teukolsky, S.A.: Relativistic stellar dynamics on the computerpart twophysical applications. Astrophys. J. 298, 58–79, 1985
Sorkin, R.: A criterion for the onset of instability at a turning point. Astrophys. J. 249, 254, 1981
Straumann, N.: General Relativity. Graduate Texts in Physics, Springer (2013)
Taylor, M.: The global nonlinear stability of Minkowski space for the massless EinsteinVlasov system. Ann. PDE 3, 9, 2017
Thorne, K. S.: The GeneralRelativistic Theory of Stellar Structure and Dynamics. In: Proceedings of the International School of Physics “Enrico Fermi,” Course XXXV, at Varenna, Italy, July 12–24 , ed. L. Gratton, Academic Press, New York, 166–280 (1966)
Wolansky, G.: Static solutions of the VlasovEinstein system. Arch. Rational Mech. Anal. 156, 205–230, 2001
Yosida, K.: Functional Analysis. Springer, Grundlehren der mathematischen Wissenschaften 1974
Zel’dovich, Ya B.: Hydrodynamical stability of star. Voprosy Kosmogonii 9, 157–170, 1963
Zel’dovich, Ya B, Novikov, I .D.: Relativistic Astrophysics, vol. 1. Chicago University Press, Chicago 1971
Zel’dovich, Ya. B., Podurets, M. A.: The evolution of a system of gravitationally interacting point masses. Soviet Astronomy—AJ 9, 742–749 (1965), translated from Astronomicheskii Zhurnal, 42
Acknowledgements
MH acknowledges the support of EPSRC Grant EP/N016777/1 and the EPSRC Early Career Fellowship EP/S02218X/1. ZL is supported in part by NSF grants DMS1715201 and DMS2007457.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by C. Mouhot.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Auxiliary Results for Selfadjoint Operators
Auxiliary Results for Selfadjoint Operators
In this appendix we collect some auxiliary results on symmetric and selfadjoint operators; to make the present paper selfcontained, we include the proofs. First we show how estimates between two operators translate to corresponding information on their negative Morse index.
Lemma A.1
Let X and Y be Hilbert spaces with scalar products \((\cdot ,\cdot )_X\) and \((\cdot ,\cdot )_Y\), let \(A:X\supset D(A) \rightarrow X\) and \(B :Y\supset D(B) \rightarrow Y\) be linear, symmetric operators, let \(P :D(A) \rightarrow D(B)\) be linear, and assume that for all \(x\in D(A)\),
Then \(n^(A) \le n^(B)\). The analogous conclusion holds if \(B :Y \rightarrow Y'\) is linear and selfdual instead.
Proof
Let \(V\subset D(A)\) be a subspace such that \((Ax,x)_X <0\) for all \(x\in V\setminus \{0\}\), and define \(W:= P(V)\), which is a subspace of D(B). If \(x\in V\) such that \(Px=0\), then by (A.1) and the choice of V it follows that \(x=0\). Hence \(P:V \rightarrow W\) is onetoone and onto so that \(\dim V = \dim W\). Let \(y\in W\setminus \{0\}\) so that \(y=P x\) for some \(x\in V\setminus \{0\}\). Then by (A.1) and the choice of V it holds that \((By,y)_Y < 0\). Hence \(n^(B)\ge \dim V\), and the assertion follows. \(\quad \square \)
Next we consider the spectral properties of a selfadjoint operator with negative Morse index.
Proposition A.2
Let \(A :X\supset D(A)\rightarrow X\) be a selfadjoint operator on a Hilbert space X with scalar product \((\cdot ,\cdot )\) and assume that \(n^(A) < \infty \). Then the negative part of its spectrum, \(\sigma (A)\,\cap \, ]\infty ,0[\), consists of at most finitely many eigenvalues which have finite multiplicities. If \(n^(A) >0\), then A has a negative eigenvalue of finite multiplicity.
Proof
Let E denote the resolution of the identity corresponding to A; we follow the notation in [64]. Then for all \(x\in D(A)\),
cf. [64, Thm. 13.30, Def. 12.17]. Next we note that
as above, R(E(I)) denotes the range of the operator E(I). To see the latter we note that
Now let \(x=E(I) y\) with some \(y\in D(A)\). For \(\tau \) sufficiently large so that \(I\subset [\sqrt{\tau },\sqrt{\tau }]\),
Hence the integral in (A.4) converges and (A.3) follows from [64, Thm. 13.24].
Now we assume that the essential spectrum of A intersects the negative reals, i.e., there exists \(\lambda \in ]\infty ,0[\, \cap \, \sigma _\mathrm {ess}(A)\). We choose an \(\epsilon >0\) such that \(I:=]\lambda \epsilon ,\lambda +\epsilon [ \,\subset \, ]\infty ,0[\). Then \(\dim R(E(I)) = \infty \), see [42, X Remark 1.11]. We fix some \(k>n^(A)\) and choose linearly independent vectors \(x_1,\ldots ,x_k \in R(E(I))\). By (A.3), \(x_j \in D(A)\), and \(x_j=E(I) y_j\) with \(y_j\in X\), \(j=1,\ldots ,k\). We consider the integrals in (A.2) with \(x=x_j\). For \(\tau > \lambda +\epsilon \) it holds that
For \(\tau >0\) we have that \(]\tau ,\infty [ \cap I =\emptyset \) and therefore \((E(]\tau ,\infty [) x_j, x_j)=0\), and since \(E(]\infty ,\tau [)\) is a projection, \(E(]\infty ,\tau [) x_j, x_j)\ge 0\) for all \(\tau \). Therefore by (A.2), \((A x_j,x_j)<0\) for all \(j=1,\ldots k\), but since \((x_1,\ldots ,x_k)\) are linearly independent and \(k>n^(A)\) this is a contradiction. Hence \(\sigma (A) \cap ]\infty ,0[\) consists only of isolated eigenvalues with finite multiplicities, and there can be at most \(n^(A)\) of them.
If we now assume that in addition \(n^(A)>0\), then [64, Thm. 13.31] shows that \(\sigma (A)\) must intersect the negative reals, and the proof is complete. \(\quad \square \)
Finally we give a special version of an abstract result from [49], which is used in the proof of Lemma 5.23.
Proposition A.3
Let X, Y be Hilbert spaces and assume \(A :Y\supset D(A)\rightarrow X\) is densely defined and closed. In addition, assume the following:

(i)
The adjoint operator \(A^{*}:X\supset D(A^{*})\rightarrow Y\) is also densely defined.

(ii)
The operator \(L:X\rightarrow X\) is bounded and selfadjoint so that \(\left( Lu,v\right) \) is a bounded symmetric bilinear form on X, where \((\cdot ,\cdot )\) denotes the inner product in X.

(iii)
There exists a decomposition of X into the direct sum of three closed subspaces
$$\begin{aligned} X=X_{}\oplus \ker L\oplus X_{+},\quad n^{}(L) := \dim X_{}<\infty ,\ \ker L<\infty . \end{aligned}$$(A.5)such that the following holds:

(L1)
\(\left\langle Lu,u\right\rangle <0\) for all \(u\in X_{}\backslash \{0\}\),

(L2)
there exists \(\delta >0\) such that
$$\begin{aligned} \left( Lu,u\right) \ge \delta \left\ u\right\ _{X}^{2} \ \text {for all}\ u\in X_{+}. \end{aligned}$$

(L1)
Then the operator \(B=A^{*}L A :Y\rightarrow Y\) is selfadjoint with domain \(D\left( A^{*}LA\right) \subset D\left( A\right) \).
Proof
First, we show that there exists a new decomposition
such that (L1) and (L2) are satisfied and in addition
and
Indeed, Let \((f_{1},\ldots ,f_{k})\) be a basis of \(X_{+}^{\perp }\), where \(k=\dim \ker L+n^{}(L)\) is the codimension of \(X_{+}\). As \(D(A^{*})\) is dense in X, one may take \(g_{j}\in D(A^{*})\) sufficiently close to \(f_{j}\), \(j=1,\ldots ,k\). Let
Since \(X_{+}^{\prime }\) is close to \(X_{+}\), it is easy to show that (L2) is satisfied on \(X_{+}^{\prime }\). Define
Clearly, \(\ker L\subset X_{1}^{\prime }\) and \(\dim X_{1}^{\prime }=k\). Let \(X_{1}^{\prime }=\ker L\oplus X_{}^{\prime }\). Then (L1) is satisfied on \(X_{}^{\prime }\), since
and \(\dim X_{}^{\prime }=n^{}(L)\).
For convenience, we denote \(\left( X_{}^{\prime },X_{+}^{\prime }\right) \) by \(\left( X_{},X_{+}\right) \) below and let \(X_{1}=\ker L\oplus X_{}\). Let \(P_{+,1}\) be the projection of X to \(X_{+},_{1}\). By (A.7),
which implies that
with symmetric \(L_{+,1}\ge 0\). Since \(R(P_{1}^{*})=X_{+}^{\perp }\subset D(A^{*})\), by the Closed Graph Theorem \(A^{*}P_{1}^{*}\) is bounded. Therefore \(P_{1}A\) has a continuous extension \((A^{*}P_{1}^{*})^{*}=(P_{1}A)^{**}\), i.e., \(P_{1}A\) is bounded. Therefore \(P_{+}A\) is closed and densely defined. Let \(S_{+} :X\rightarrow X\) be a bounded symmetric linear operator such that
For any \(x\in X_{+}\),
which implies that for all \(x\in X_{+}\),
This lower bound of \(S_{+}\) implies that \(T_{+}:= S_{+}P_{+}A\) is also closed with the dense domain \(D(T_{+})=D(A)\) and thus \(T_{+}^{*}T_{+}\) is selfadjoint. Then
where \(B_{1}=A^{*}P_{1}^{*}L_{1}P_{1}A\) is bounded and symmetric. Therefore, \(A^{*}L A\) is selfadjoint by the Kato–Rellich Theorem. \(\quad \square \)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Hadžić, M., Lin, Z. & Rein, G. Stability and Instability of SelfGravitating Relativistic Matter Distributions. Arch Rational Mech Anal 241, 1–89 (2021). https://doi.org/10.1007/s00205021016472
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00205021016472