1 Introduction

This paper discusses recent progress in the area of collective linear dynamics of charged particle beams, when their space charge (SC) is important; both transverse and longitudinal planes are considered. Since such a task cannot be free from some subjectivity and arbitrariness, I beg a pardon for the inevitable omission of some valuable results.

More than 50 years ago, the first significant publication was presented on transverse collective instabilities of space-charge-dominated beams in circular machines; it was a preprint of  Merle and  Möhl “The stabilizing influence of nonlinear space charge on transverse coherent oscillations” [1]. A relatively simple equation of motion was suggested there as something obvious. Although it was, strictly speaking, neither obvious nor necessarily justified, it played and continues to play an extraordinarily important role. This equation implicitly relied on rigidity of the beam slices; the term slice refers to the group of beam particles whose Coulomb fields affect the given particle. Four decades later, it was shown that this approximation requires strong SC, i.e., the SC tune shift to exceed the lattice tune shifts and the synchrotron tune [2]. Soon after that, the concept of strong SC was used for bunched beams [3], including computation of Landau damping (LD) with space charge and octupoles taken into account. It was also demonstrated there that SC can help to avoid the transverse mode coupling instability (TMCI). Paradoxical features of this stabilization were realized few years ago, leading to discovery of the transverse convective instabilities [4], with exponential dependence of the head-to-tail amplification on the SC tune shift. These and related ideas are discussed in Sect. 2, devoted to the transverse collective oscillations.

Longitudinal modes are discussed in Sect. 3, for the bunched beams only, since the coasting beam theory seems to be well established long ago, see, e.g., Refs. [5, 6]. For bunched beams, the key intensity limitation is associated with the loss of LD, when a single-bunch impedance drives a collective mode outside the incoherent spectrum, making the beam defenseless against any coupled-bunch instability, no matter how weak it might be. First time this issue was noted and analyzed in this context by F. Sacherer almost half a century ago [7]; a certain threshold for SC impedance below transition was estimated. Later, this problem was addressed by other authors, who got the same result, up to a numerical factor; the smallest threshold was reported in Ref. [8]; it was about three times below the Sacherer’s one. These discrepancies were related to the model imperfections and to high sensitivity of the result on the distribution function. However, a recently published article of Karpov et al. [9] has shown that all those results are incorrect; for the pure SC impedance, the intensity threshold of the loss of Landau damping (LLD) is equal to zero. Soon after that, the entire collective spectrum for SC impedance below transition was described in Ref. [10], with generalization to any multipolarity and to other impedances. The single-bunch eigensystem problem was reduced to a parameter-less equation; also the coupled-bunch growth rates were analytically derived. These and related ideas are considered in Sect. 3.

2 Transverse oscillations

2.1 Coasting beams

To analyze the beam stability with SC, a linear equation of motion was suggested by  Merle and  Möhl in 1969 [1]:

$$\begin{aligned} \frac{d^2 x_i}{dt^2} + \omega _{x i}^2 x_i + 2 \omega _{x}\omega ^\mathrm {sc}_i\left( x_i -\bar{x}\right) + 2 \omega _{x}\omega ^\mathrm {c}\bar{x} =0 \,. \end{aligned}$$
(1)

Here \(x_i=x_i(t)\) is a transverse offset of a particle i, \(\omega _{x i}\) is the betatron frequency of the particle i, \(\omega _{x}\) is the average betatron frequency, \(\bar{x}=\bar{x}(t)\) is an average offset of that beam slice where the particle i is located at the given moment of time t, \(\omega ^\mathrm {sc}_i<0\) is the SC frequency shift of the particle, and \(\omega ^\mathrm {c}\) is the coherent frequency shift parameter, proportional to the ring impedance. The full time derivative d/dt is expressed through the partial ones, \(d/dt = \partial /\partial t + \omega _{i}\partial /\partial \theta \), where \(\omega _{i}\) is the revolution frequency of the particle, and \(\theta =s/R\) is the azimuthal angle, with s as the conventional longitudinal coordinate and R as the average ring radius. The term ‘slice’ refers to the group of beam particles which Coulomb fields affect the given particle number i, i.e., the particles with positions somewhere between \(s_i -a/\gamma \) and \(s_i +a/\gamma \), where a is the beam transverse size and \(\gamma \) is the Lorentz factor.

Equation (1) implies two important things.

First, this equation is a linearization over infinitesimally small collective perturbations \(\bar{x}\) and \(x_i\). In other words, it implies that \(x_i\) relates to the collective, driven part of the single-particle oscillations, excited by the collective motion of the slices’ centroids \(\bar{x}\), while amplitudes of free oscillations, the steady-state part, determine the space charge frequency shifts \(\omega ^\mathrm {sc}_i\). Thus, the offset \(x_i\) is of the order of centroid offsets, \(x_i-\bar{x}\), while the incoherent amplitudes are of the order of the beam transverse size.

Second, this equation assumes that each beam slice oscillates as a rigid body, allowing for a representation of the SC force in the simple way it is done there. Because of this assumption, Merle–Möhl approach is sometimes addressed as the rigid-slice or frozen-field model. Possible incorrectness of this assumption, as well as its very existence, was realized much later, when some strange features of Eq. (1) were discovered.

For a coasting beam, eigenfunctions of Eq. (1) have the form

$$\begin{aligned} x_i\,, \bar{x}\propto \exp \left[ -i\,\left( \omega _{x}+ n\omega _0 + \omega \right) t + in \theta \right] \, \end{aligned}$$
(2)

where n is an arbitrary integer, \(\omega _0\) is the average revolution frequency, and \(\omega \) is a frequency shift of the mode n. Substitution of this form into Eq. (1) yields for the complex amplitudes

$$\begin{aligned} x_i = \bar{x}\frac{\omega ^\mathrm {c}- \omega ^\mathrm {sc}_i}{\omega - \omega ^\mathrm {sc}_i-\delta \omega _{x i}-n\delta \omega _{i}}\,, \end{aligned}$$
(3)

with the lattice frequency shifts \(\delta \omega _{x i}=\omega _{x i}-\omega _{x}\) and \(\delta \omega _{i}=\omega _{i}-\omega _0\). Note that without lattice frequency spread, \(\delta \omega _{x i}=\delta \omega _{i}=0\), there is always the rigid-bunch solution, \(x_i=\bar{x}\), with \(\omega =\omega ^\mathrm {c}\), independently of the SC tune shifts \(\omega ^\mathrm {sc}_i\). This important physical property of Eq. (1) is a consequence of its SC representation by means of the term \(\propto x_i - \bar{x}\). In fact, the Merle–Möhl equation is the only possible linear dynamic equation consistent with the given incoherent spectrum, its lattice and SC parts, which represents the coherent SC term by means of \(\propto \bar{x}\) term only, preserving the rigid-bunch mode for zero lattice tune spread, as it must be from the first principles. Indeed, without lattice tune spread, the equations of motion for the full offsets \(X_i=a_i+x_i\) can be represented as

$$\begin{aligned} \frac{d^2 X_i}{dt^2} + \omega _{x}^2 X_i = \sum _j F\left( s_i-s_j, X_i - X_j\right) \,. \end{aligned}$$
(4)

Here F-terms in the right-hand side represent the SC forces. For the steady state, with \(x_i=0\), it yields

$$\begin{aligned} \frac{d^2 a_i}{dt^2} + \omega _{x}^2 a_i = \sum _j F\left( s_i-s_j, a_i - a_j\right) \,. \end{aligned}$$
(5)

Now, let us assume that the bunch oscillates as a whole, \(X_i = a_i + \bar{x}\). Then, it follows from the last two equations that

$$\begin{aligned} \frac{d^2 \bar{x}}{dt^2} + \omega _{x}^2 \bar{x}= 0\,, \end{aligned}$$
(6)

meaning that indeed there is no SC tune shift for the rigid-bunch mode.

By averaging over the particles, writing the sums as the phase space integrals with the distribution function, one gets the dispersion relation, i.e., the equation for the sought-after eigenfrequency \(\omega \),

$$\begin{aligned} 1 = - \int d\Gamma \, J_x \frac{\partial f}{\partial J_x} \frac{\omega ^\mathrm {c}- \omega ^\mathrm {sc}_i}{\omega - \omega ^\mathrm {sc}_i-\delta \omega _{x i}-n\delta \omega _{i}+i o} \end{aligned}$$
(7)

Here \(f=f(J_x, J_y, \delta p/p)\) is the unperturbed distribution density as a function of transverse actions \(J_{x,y}\) and the relative momentum offset \(\delta p/p\), normalized to unity, \(\int d\Gamma f =1\), where \(d\Gamma = dJ_x\,dJ_y\, d\delta p/p\); the single-particle subscript i has to be understood as indication to related functional dependences, i.e., \(\delta \omega _{x i}\rightarrow \delta \omega _x(J_x, J_y, \delta p/p)\), etc. To get Eq. (7) from Eq. (3), the Hereward rule [11] was used,

$$\begin{aligned} \sum _i \left( ...\right) \rightarrow - \int d\Gamma \, J_x\, \partial f/\partial J_x \,\left( ...\right) \,, \end{aligned}$$

and the Landau rule of going around the pole is explicitly marked, \(\omega \rightarrow \omega + i o\), where o is an infinitesimally small positive number.

It is straightforward to see from the dispersion relation (7) that without lattice frequency spreads, at \(\delta \omega _{x i}=\delta \omega _{i}=0\), the eigenfrequency \(\omega =\omega ^\mathrm {c}\), independently of the SC tune shift. Thus, even if the phase space density of the resonant particles were not zero, i.e., there were particles with the same tune as the coherent mode, still there would be no Landau damping (LD), irrespectively to nonlinearity of SC distribution.

The dispersion relation in the form (7) was first derived by  Möhl and  Schönauer in 1974 [12], not in the original preprint of Merle and Möhl. In the latter, some mathematical mistakes were adopted, so the dispersion relation was derived incorrectly. Due to this, it was mistakenly concluded there that SC nonlinearities may contribute to Landau damping of coasting beams even without the lattice frequency spread. This mistake was later repeated in Ref. [13] and corrected by Möhl and Schönauer [12]. That is why it seems fair to call Eq. (1) Merle–Möhl equation of motion and Eq. (7) Möhl–Schönauer dispersion relation.

After the simplest case of no lattice frequency spreads, the next by simplicity is a two-stream beam, \(\delta \omega _{i}=\pm \delta \omega _{0}\). For the Kapchinsky–Vladimirsky transverse distribution with a constant SC frequency shift, \(\omega ^\mathrm {sc}_i=\omega ^\mathrm {sc}\), Eq. (7) yields the following spectrum:

$$\begin{aligned} \omega =\frac{\omega ^\mathrm {c}+ \omega ^\mathrm {sc}}{2} \pm \sqrt{\frac{(\omega ^\mathrm {c}- \omega ^\mathrm {sc})^2}{4} + n^2 \delta \omega _{0}^2} \end{aligned}$$
(8)

The instability is driven by the coherent tune shift \(\omega ^\mathrm {c}\), so the most unstable modes are positive ones, i.e., those associated with the sign \(+\) in Eq. (8). A straightforward analysis shows that for them the two streams of the beam oscillate approximately in phase, so their wakes add to each other. For strong SC, \(|\omega ^\mathrm {sc}| \gg \max (|n|\delta \omega _{0},\, |\omega ^\mathrm {c}|)\), the spectrum of positive modes reduces to

$$\begin{aligned} \omega = \omega ^\mathrm {c}+ n^2 \delta \omega _{0}^2/\omega ^\mathrm {sc}\,, \end{aligned}$$
(9)

being far away from the incoherent spectrum localized around \(\omega ^\mathrm {sc}\). Thus, the negative modes not only are barely excited by the wake, but also stay close to the incoherent spectrum, so their stability would be provided automatically when the positive modes are stable.

A very general method of analysis of integral dispersion relations like Eq. (7) was presented in Ref. [14]. The idea was to reverse the problem: instead of finding the eigenfrequency \(\omega \) when the coherent tune shift \(\omega ^\mathrm {c}\) is given, let us do the opposite, find the coherent tune shift \(\omega ^\mathrm {c}\) for a given eigenfrequency \(\omega \) for the same dispersion relation. For the eigenfrequency moving along the real axis, the corresponding coherent tune shift follows a certain line in its complex plane, which is the conformal map of the real axis in the complex plane of \(\omega \) to the complex plane of \(\omega ^\mathrm {c}\). This line, \(\omega ^\mathrm {c}(\omega )\), is conventionally called the stability diagram; the beam is stable if and only if its actual coherent tune shift is below the diagram.

Certain historical investigations [15] convinced the author of this paper that it would be fair to call the stability diagram the Vaccaro diagram (VD), by name of Vittorio Vaccaro, who found how Nyquist’s stability plots can be modified to become an effective tool for the collective beam dynamics. With Möhl–Schönauer dispersion relation (7), VD is determined by the beam distribution function f, the lattice frequency shifts and SC tune shifts, being independent of wakes.

In 2001, M. Blaskiewicz suggested an original method of solving the Jeans–Vlasov equation with SC and lattice tune spread [16]; with respect to the equation naming, see Ref. [17]. The method was free from hidden assumptions of Merle–Möhl approach, being, arguably, more complicated and less transparent in the computations. The method allowed to make conclusions regarding SC effects for the stability diagram. In case of the chromatic lattice tune spread, the diagram of a Gaussian beam essentially shifted to the left by one half of the maximal SC tune shift. In case of the octupole nonlinearity, M. Blaskiewicz found that focusing octupoles are much more beneficial for LD than defocusing ones, confirming the same conclusion by  Möhl [18].

The first analytical attempt to build VD for Eq. (7) was presented in 2004 by  Metral and  Ruggiero [19]. Namely, they suggested a solution of the dispersion relation with SC and octupolar nonlinearity, where, instead of the coasting beam term \(n\delta \omega _{i}\), the bunched beam term \(k \omega _\mathrm {s}\) was put, with \(\omega _\mathrm {s}\) as the synchrotron frequency and k as the head-tail mode number. Such extension of the coasting beam theory to the bunched case was, however, left unexplained both in the paper itself and the references it suggested for that matter, including Ref. [18]. As it became more clear later, this extension works reasonably well only when the SC term could be safely omitted. However, the actual merit of the paper was not in its applicability to bunched beams with SC, but in its analytical building of VD for coasting beams with octupoles, nonlinear SC, and insignificant revolution frequency spread, \(n\delta \omega _{i}=0\). It was confirmed, in particular, that the octupole sign becomes crucial for strong SC; namely, the focusing octupoles are much preferable. The reason is that the octupoles affect mostly the tail particles, so the collective frequency is barely touched by them. Landau damping requires resonant particles, i.e., those whose individual tunes are the same as the collective one. Space charge moves the incoherent tunes down and does almost nothing for the collective tunes, thus killing LD. Hence, to restore the latter, the incoherent tunes have to be moved up to provide higher population of the resonant particles, so the octupoles have to be focusing.

Among multiple reasonable features, Vaccaro diagrams of Ref. [19] showed a strange one: for defocusing octupoles, there was a kink point of the curve at the real axis, which prevented the line from going to the lower half-plane, \(\Im \omega ^\mathrm {c}<0\). The kink point looked strange, since VD should be analytical by the definition.

In 2006, Pestrikov published an article [20] where a similar problem was solved, but instead of the kink point, the diagram smoothly continued to the lower half-plain, thus demonstrating Landau antidamping. Later that year, Landau antidamping was confirmed by K.Y. Ng for the same model as Metral and Ruggiero proposed [21]. On the ground of these findings, the kink point of Ref. [19] was dismissed as a mistake of the sign. Due to this, however, another problem appeared: at certain conditions, a Gaussian-like beam with time-independent Hamiltonian started looking unstable even when the coherent tune shift \(\omega ^\mathrm {c}\) suggested a decay of the mode. Next year, Pestrikov published another paper [22], presenting “a self-consistent model” which showed no antidamping, contrary to the Merle–Möhl model; he expressed doubt in the validity of the latter.

This doubt was enhanced to a stronger claim by V. Kornilov,  Boine-Frankenheim, and  Hofmann in their publication of 2008 [23]. First, they confirmed that VD of Eq. (7) indeed yields Landau antidamping for defocusing octupoles. Second, they supported this confirmation by macroparticle simulations within the frozen field model, equivalent to Merle–Möhl approach. Third, they ran self-consistent macroparticle simulations for the same conditions and saw no antidamping. From this, they concluded that “antidamping can be related to the non-self-consistent treatment of nonlinear space charge in the simulations and also in the dispersion relation.”

At that stage, several issues remained unresolved for coasting beams. First, it was not clear if Landau antidamping is ever possible for Gaussian-like beams with SC, octupoles and chromaticity. Second, with evidence of incorrectness of Merle–Möhl analytical approach at certain cases, it was not clear if their equation could be ever used at all, and under what conditions. Third, no analytical formulas for the instability thresholds were yet obtained. These issues were addressed in Ref. [2].

A possibility of Landau antidamping was denied there as contradicting to the Second Law of Thermodynamics. Indeed, a beam with real coherent tune shift \(\omega ^\mathrm {c}\), corresponding to imaginary transverse impedance, i.e., to zero energy losses, can be described by energy-preserving time-independent Hamiltonian, so the growing coherent oscillations might take energy from the incoherent degrees of freedom only. For a Gaussian beam, it would mean a perpetuum mobile of the second kind, forbidden by the Second Law. Landau antidamping, demonstrated for some parameters by Merle–Möhl dynamic system (1), is caused by the non-Hamiltonian character of its SC term. Specifically, the term \(\propto \omega ^\mathrm {sc}_i\bar{x}\) is non-Hamiltonian unless all the SC frequency shifts are identical within the beam slice.

When the SC tune shifts depend on the transverse actions, as they normally are, the defect of the model still should not play a role, if the slices were sufficiently rigid in their transverse oscillations. In this case only the tail particles would be responsible for LD, so the energy transfer to them could be reasonably approximated with the rigid slice model. To see when the core is really rigid, note that if the lattice tune shifts are small with respect to the tune separation,

$$\begin{aligned} |\delta \omega _{x i}+ n\delta \omega _{i}| \ll |\omega ^\mathrm {c}- \omega ^\mathrm {sc}_i|\,, \end{aligned}$$
(10)

the particles move together with the related centroids, \(x_i \approx \bar{x}\), as it follows from Eq. (3). Thus, if the SC is so strong that this condition is satisfied for the majority of particles, the slices oscillate almost without distortions, since almost all the particles oscillate almost identically to their centroids; so the rigid-slice approximation is justified. Luckily, for many low- and medium-energy machines, typical SC tune shifts are much larger than the imaginary part of the coherent tune shifts, \(|\omega ^\mathrm {sc}_i| \gg \Im \omega ^\mathrm {c}\), so stabilization is achieved at such a small lattice nonlinearity that Eq. (10) is satisfied, justifying Merle–Möhl equation. In this case of strong SC, instability thresholds were explicitly found in Ref. [2] for Gaussian beam, for both octupolar and chromatic frequency spreads. Recently, this method was extended to electron lenses; Landau damping rate introduced by a Gaussian e-lens for a coasting beam with SC was analytically estimated and presented in Ref. [24].

2.2 Bunched beams

The coherent spectrum of bunched beam with SC was presented for the first time by M. Blaskiewicz in 1998 [25] within a simple model of an air-bag bunch in a square potential well. For a delta-wake (the image charges), the eigenfrequencies were found to be the same as Eq. (8) for the two-stream coasting beam, with the substitution \(n \delta \omega _{0}\rightarrow k \omega _\mathrm {s}\), where \(k=0, 1, 2,\ldots \) is the mode counter and \(\omega _\mathrm {s}\) is the synchrotron frequency.

A new and rather surprising mathematical result of  Blaskiewicz [25] showed suppression of the transverse mode coupling instability (TMCI) by SC; the wake threshold was demonstrated to grow with SC tune shift, linearly at the strong SC limit, \(|\omega ^\mathrm {sc}| \gg \omega _\mathrm {s}\). This result was obtained for exponential wakes and the ABS model (Air-Bag, Square-well), so a question was raised about the sensitivity of this unexpected result to the details of the wake, potential well and bunch distribution. Also, it was not clear if there was any limit to this growth of the instability threshold. An explanation of this growth at moderate SC was suggested to the author by  Danilov [26] and reproduced in Ref. [27]. Without SC, TMCI typically results from crossing of the head-tail mode 0, shifted down by the wake, and the mode \(-1\), not shifted as much. Space charge, on the contrary, does not influence the mode 0 and shifts down the mode \(-1\), thus moving their coupling point to higher intensity. The effect of space charge and image currents on the transverse Schottky spectrum for the ABS model has been investigated in the year of 2009 by  Boine-Frankenheim and  Kornilov [28], for both simulations and the analytic model. They found a good agreement between the two approaches.

In the same year of 2009, when it was understood that Merle–Möhl approach of rigid slices is justified for sufficiently strong space charge, it was applied to bunched beams by the author [3]. Under the condition of SC tune shift being much stronger than all other tune shifts and spreads, as well as the synchrotron tune (strong space charge, SSC), an ordinary linear integro-differential equation was derived for the bunch modes for an arbitrary potential well, driving and detuning wakes, longitudinal and transverse bunch distribution functions. Later that same year,  Balbekov published a paper [29] with an alternative derivation of the SSC equation, whose result differed from mine. After checking his derivation and rechecking mine, I found an algebraic error in my calculations, and derived my ultimate form of the SSC mode equation, which agreed with Balbekov’s result, suggesting a slightly more compact form in the erratum [30],

$$\begin{aligned} i\frac{\partial \bar{x}}{\partial t} + \frac{1}{\omega ^\mathrm {sc}} \frac{\partial }{\partial s} \left( u^2 \frac{\partial \bar{x}}{\partial s} \right) = \mathbb {W} \bar{x}+ \mathbb {D} \bar{x}\,. \end{aligned}$$
(11)

Here \(\omega ^\mathrm {sc}= \omega ^\mathrm {sc}(s)\) is the SC frequency shift averaged over the transverse actions at every position s, \(u^2=u^2(s)\) is the local rms spread of the longitudinal velocities, \(u^2 \equiv < R^2 \delta \omega _{i}^2>\), while \(\mathbb {W}\) and \(\mathbb {D}\) are conventional driving and detuning wake linear integral operators [31]; for more details, see [30]. The equation is complemented by zero-derivative boundary conditions, \(\partial \bar{x}/\partial s =0\) at the bunch edges or at \(s=\pm \infty \). For the eigenfunctions, the time derivative has to be substituted by the sought-for eigenfrequency \(\nu \), i.e., \(i \partial /\partial t \rightarrow \nu \). Without wakes, this equation leads to the Blaskiewicz-type collective spectrum, \(\nu _k \simeq k^2 \omega _\mathrm {s}^2/\omega ^\mathrm {sc}\). The mathematical elegance of Eq. (11) has its price: missing is the Landau damping, which required additional ideas and computations.

Analytical estimations for LD at SSC were also suggested in Refs. [3, 30] for weak head-tail cases, when the wake does not influence the eigenfunction much. Contrary to coasting beams, it was found that there is an intrinsic LD, caused by the longitudinal variation of the SC tune shift only, even without any lattice tune spreads. The physical mechanism of the dissipation was associated with a break of the slice rigidity at the bunch edges, where the SC is not strong any more. The slice softening at the bunch edges opens a way for the energy transfer to the incoherent degrees of freedom. According to the related estimation, the intrinsic LD rate \(\Lambda _k\) at SSC was found to be a steep function of the SC parameter \(q \equiv \omega ^\mathrm {sc}/\omega _\mathrm {s}\) and the positive mode number k, \(\Lambda _k \simeq k^4 \omega _\mathrm {s}/q^3\); the SSC assumes \(q \gg 2k\). Since there is no LD in the opposite limit, \(q=0\), one may guess from here that the damping rate reaches its maximum at \(q \simeq 2k\). This has been demonstrated a year later by  Kornilov and  Boine-Frankenheim [32] in the numerical simulations for transversely homogeneous and longitudinally Gaussian bunch inside a parabolic potential well, i.e., for the so-called KV-Gaussian bunch. Five years later, the analytical results for SSC eigenfunctions and LD rates were fully confirmed in Synergia macroparticle simulations by  Macridin et al. [33], where the intrinsic LD rates were shown to have their maxima at \(q \simeq 2k\) both for the KV-Gaussian (as in the article of Kornilov and Boine-Frankenheim) and for 3D Gaussian distributions. A more subtle case of parametric Landau damping was treated by  Macridin et al. in Ref. [34] by means of analytical modeling and macroparticle simulations. Analytical estimations of octupoles-related LD suggested in Refs. [3, 30] are still waiting for at least numerical verifications; nothing yet has been published in that matter. Octupoles, however, are rather inefficient for LD, which requires significant nonlinearity inside the beam, not far outside, as octupoles provide. That is why a better instrument for LD is an electron lens, at least as thin as the beam. Such e-lenses are able to provide LD without deterioration of the dynamic aperture, as it was pointed out by  Shiltsev et al. [35]. Estimations of e-lens-caused LD rates for bunches with SC were suggested by  Alexahin,  Burov and  Shiltsev in 2017 [36].

With the wake taken into account, the Blaskiewicz’ result of linear growth of the TMCI wake threshold was confirmed in a series of publications, see Refs. [37, 38] and references therein. A hidden obstacle with this problem, sometimes caused misleading results, was realized by  Balbekov [39], who showed that convergence of the expansion of the sought-for eigenfunction over the zero-wake basis degrades with SC, requiring more and more terms for higher SC parameter q. The physical reason of the convergence worsening was recently found by the author of this paper; it is associated with the head-to-tail amplification, or the convective instabilities driven by wakes at SSC. When eigenfunctions are significantly amplified, their expansion over any even basis cannot be of a good convergence. Clearly, manifest subsiding of the instability with SC was presented in the two-particle model of Ref. [40].

With the theoretical proof of TMCI vanishing with SC, two problems became rather obvious, one experimental and the other theoretical. The former consisted in a reasonable agreement of the transverse instability at CERN SPS with no-SC theory, while SC tune shift was very strong there, especially with the old lattice [41, 42]. The latter problem was related to the linear growth of the wake threshold with SC. Due to this feature, the bunch should be stable up to infinite intensity, as soon as its emittance is low enough, which did not sound as a reasonable statement. The resolution of both problems was presented by the author few years ago [4]. The main idea, already mentioned above, was that, while moving out TMCI, SC moves another instability in its place, a convective one. Contrary to TMCI, which is an absolute instability, i.e., has nonzero growth rate, the convective instabilities grow not in time, but in space, from head to tail [43]. This head-to-tail amplification increases exponentially with bunch intensity, resulting in one or another physical limit, set by lattice nonlinearity, beam loss or feedback. When the amplification is large, even a tiny feedback from tail to head may be sufficient to close the loop and turn the convective instability into an absolute-convective one, like those with a microphone close to its loudspeaker. Such a feedback may be presented with a bunch-by-bunch damper, coupled-bunch wakes, or a halo of the same bunch. Here a question may be asked, why is the halo needed for the feedback? Why cannot core particles play this role, when they move to the bunch head with their high transverse amplitudes acquired at the tail? The answer is that due to strong SC, the bunch slices are rigid, as it was discussed in the previous section. Strong SC means that all tune shifts are small compared with the SC tune shift, so intra-slice degrees of freedom cannot be excited, and thus the tail particles do not preserve their large amplitudes while moving to the bunch head; instead, they just follow the existing spatial pattern of the rigid-slice oscillations.

However, what is impossible for the bunch’s core, might work for its halo, whose SC tune shift is smaller. That is why the halo particles traveling from the tail to the head may remember their tail offsets, providing a tail-to-head feedback. At strong SC, this feedback would be small due to the low population of that halo, but, if the convective amplification is large enough, even a small feedback could be sufficient to ignite the absolute-convective instability, as it was suggested and modeled in Ref. [44]. Apparently, the same effect is responsible for the non-monotonic behavior of the wake instability threshold on the SC parameter reported by Alexahin [45]. A good agreement of his analytically calculated highly convective mode with the pattern of oscillations seen by  Oeftiger in macroparticle simulations for the same conditions deserves to be mentioned. Alas, it was the last lifetime publication of Yuri, who suddenly passed away in a year after that [46].

In a recent Ref. [47], the joint action of wake and SC forces was examined by means of a numerical circulant matrix model (CMM) applied to a bunch represented by several thin rings, from one to fifty, in the longitudinal phase space with a parabolic potential well; no transverse variation of the SC tune shifts was assumed. For a single ring, i.e., for the conventional air-bag, all the conclusions of Ref. [4] with its ABS model were essentially confirmed. For many rings, a dense sequence of couplings-decouplings of the amplified radial modes were seen for the case of Gaussian longitudinal distribution, well below the air-bag TMCI threshold at the given strong SC. The authors note that ”the coupling of the radial modes seems in contradiction with the results in [38]. Nevertheless, the model used in the latter was obtained with the strong space charge approximation, which is not met at the edge of a longitudinal Gaussian distribution. Since the CMM is not based on such an approximation, the difference can likely be attributed to that.” In other words, this contradiction suggests that the longitudinal halo is an important part of the absolute-convective instability observed. The reason for this longitudinal core-halo instability could be in wider frequency range of the forces the halo particles see when they pass through the oscillating tail of the core, and this might be sufficient for them to bring the tail signal to the head. More detailed studies with this model are certainly needed for definite conclusions.

It is clear now, that convective instabilities constitute a common obstacle for high-intensity circular machines of low and medium energy, where SC is significant; they take place at CERN SPS [47, 48] and PS [49] rings, as well as at the Fermilab Booster [50]. That is why it is important to understand how they behave together with other factors of beam dynamics. Transverse instabilities of a bunch with SC, wake and damper were considered in Ref. [51]. In Ref. [52], measurements of a microwave instability at transition crossing in PS were reported; the instability was characterized as convective. Recently, an analytical model for it was proposed [49] by means of Eq. (11). A simple threshold formula derived there was found to be in good agreement with the data of Refs. [52, 53]. A statement made in Ref. [52] that “The bunch parameter measurements demonstrate... that the space-charge effect does not affect the instability thresholds” does not actually contradict to the rather weak dependence of the threshold bunch intensity \(N_\mathrm {th}\) on the transverse emittance, \(N_\mathrm {th} \propto \epsilon _\perp ^{1/4}\) of Ref. [49], since the limited range of the emittances examined in Ref. [52] and the measurement errors do not allow to resolve the rather weak dependence on the emittance on the ground of this set of measurements alone [54].

3 Longitudinal oscillations

In this review of the SC effects in the linear longitudinal collective oscillations, we limit ourselves by bunched beams, since the coasting beam theory seems to have been completed long ago [5, 6]. The only relatively new contribution to this theory, as far as I know, relates to the modification of the negative mass instability by the SC shift of the slippage factor, reported in Ref. [55].

Description of bunch dynamics generally requires solving of two consecutive problems: the bunch steady state has to be found, and, after that, dynamics of its small perturbations is to be analyzed. The phase space density F(I), the wake function W(z), and the RF potential \(U_\mathrm {rf}(z)\) have to be provided as input functions, where I and z are the action variable and the longitudinal position along the bunch. The steady-state Hamiltonian H(z, p), with p as the momentum variable associated with the coordinate z, full potential U(z), action I(H) and line density \(\lambda (z)\) have to be found then as solutions of self-consistent equations [56]. From that, the incoherent frequency spectrum \(\Omega (I)\) can be obtained as \(\Omega (I)=\mathrm {d}H/\mathrm {d}I\). For SC or inductive impedances, the steady state is determined by the distribution function F(I) and the dimensionless intensity parameter

$$\begin{aligned} k=-\frac{2 N r_0 \eta \, \omega _\mathrm {rf}^3}{\gamma \, c\, \Omega _0^2}\, \frac{\Im Z_n}{n\, Z_0}\,. \end{aligned}$$
(12)

Here N is the number of protons per bunch; \(r_0\) is the classical radius of the beam particles; c is the speed of light; \(\gamma \) is the Lorentz factor; \(\eta \) is the slippage factor, \(\omega _\mathrm {rf}\) is the RF frequency, \(\Omega _0\) is the bare RF synchrotron frequency, \(Z_n\) is the impedance at the azimuthal harmonic number n; for a pure SC case, \(Z_n/(n Z_0) = i \ln (b/a)/\gamma ^2\), where b and a are the aperture and the beam radii (corrections to the logarithmic factor are discussed in Ref. [57]). In particular, for a Gaussian bunch, the incoherent spectrum for the core particles at the first order of the intensity parameter k is found as

$$\begin{aligned} \frac{\Omega (I)}{\Omega _0}= 1-\frac{k}{2\sqrt{2\pi } \sigma ^3} - \frac{I}{8}\left( 1 - \frac{3 k}{\sqrt{2\pi } \sigma ^5} \right) \,, \end{aligned}$$
(13)

where the bunch length \(\sigma \) is measured in the RF radians, and the dimensionless units for the action I are such that its maximal value in the undistorted RF bucket is \(8/\pi \).

When the steady-state problem is solved, the collective dynamics of this state can be analyzed. Usually, it is done by means of linearized Jeans–Vlasov equation; the collective modes are found as its eigenfunctions. The main concern here relates to possible instabilities, when at least one of the eigenfrequencies has a positive imaginary part. However, one should keep in mind a possibility of convective instabilities with significant amplification along the bunch. Even if the growth rate is zero, large amplification may lead to beam loss or emittance degradation. A possibility of longitudinal convective instabilities for a bunched beam was demonstrated in Ref. [58] for a barrier bucket and when both SC and resonator impedances are large enough.

At sufficiently low beam intensity, all the collective frequencies normally supposed to lie within the incoherent spectrum, thus providing their damping by means of the Landau mechanism. As the intensity increases, some collective modes move outside the incoherent spectrum. Thus, at certain intensity, Landau damping becomes either insufficient or totally lost. The latter happens if the coherent tune is shifted by the collective interaction so far that it cannot meet resonant particles at all there; this case is termed loss of Landau damping, LLD. At the rest of this review, we concentrate on this issue, because of its high experimental importance and a recent breakthrough in this direction. Undamped collective longitudinal oscillations, interpreted as LLD-caused, were observed at RHIC [59], Tevatron [60], SPS [61] and LHC [62]. A special role in such phenomena is played by repulsive inductance above transition energy or space charge (SC) Coulomb forces below it. Being conservative, such forces cannot drive instabilities by themselves. However, they can move the coherent frequency outside the incoherent spectrum, above the maximal incoherent frequency, in which case even a tiny wake field of the preceding bunches causes an instability. If to liken Landau damping to the beam immune system, the SC force below transition would play a role of an immunodeficiency factor, while the wake fields of other bunches would be similar to all possible viruses. In this situation, the wall inductance plays the role of a guard of Landau damping, since it behave in same way as SC, but with the opposite sign; such a guard is not necessarily sufficient, of course. Above the transition, SC and the chamber inductance switch their roles: the SC becomes the guard of Landau damping, and the inductance the thief. Since SC normally dominates below transition, and sufficiently above it dominates the chamber inductance, the thief typically overcomes the guard. In this review, we limit ourselves by this situation of the repulsive inductance, where the force is proportional to the line density derivative, taken with the negative sign. For the vacuum chamber, this law is effective with wave numbers no higher than the inverse aperture; for hadron beams it is typically one or two orders of magnitude above the inverse bunch length. The SC impedance starts to roll off \(\gamma \) times further than that.

The possibility of LLD for a bunch in the inductive vacuum chamber above transition was first noted by  Sacherer [7]; he evaluated the threshold number of particles per bunch, \(N_\mathrm {th} \propto \sigma ^5\). Later this result was essentially confirmed, up to numerical factors, by many authors [8, 63,64,65,66]. Some discrepancies between these results were explained by different distributions and model imperfections. The problem of the LLD threshold calculation looked to be essentially solved until a recent article of  Karpov,  Argyropoulos and  Shaposhnikova [9] demonstrated that all the previous results were, in fact, incorrect: actually, there is no LLD threshold for such impedance, and the previous claims were all based in the insufficiency of the accepted limits on wave numbers q of the perturbations or insufficient number of the mesh points. This conclusion was demonstrated in several independent ways, leaving no doubt of its correctness. It was also shown that if the inductance iZ(q)/q rolls off at certain wave number \(q=q_c\), the LLD threshold would be inversely proportional to that value, \(N_\mathrm {th} \propto \sigma ^4/q_c\). When the intensity increased, emergence of a second mode of the discrete spectrum was demonstrated. The results of this breakthrough article were soon explained and generalized in Ref [10]. Below we reproduce this explanation.

Qualitatively, the leading collective mode can be approximated as oscillations of a central part of size a with a small amplitude \({\tilde{z}} \ll a\). Such oscillations result in the phase space density perturbation \(f \simeq F'\,a\, {\tilde{z}}\) and the line density perturbation \(\rho \simeq f\, a \simeq F' a^2\, {\tilde{z}}\), where \(F'=\mathrm {d}F/\mathrm {d}I\) at zero action, \(I=0\). For the inductive impedance, the related collective force is \(E \simeq k \rho /a \simeq k F' a {\tilde{z}}\). For the case under study, \(k>0\), this corresponds to an additional focusing seen by the collective mode, taking it above the incoherent spectrum. To avoid confusion, let us note that at the same time the incoherent frequencies are depressed by this wake. The related coherent tune shift is thus estimated as \(\Delta \omega \simeq -k F'\,a/2 \), in the units of \(\Omega _0\). For the same central cluster of particles, the incoherent tune spread is \(\delta \Omega \simeq \pm |\Omega '|a^2/4 \), where \(\Omega '=\mathrm {d}\Omega /\mathrm {d}I\) at \(I=0\), in the total potential well. The coherent motion dominates over the incoherent as soon as \(\Delta \omega \ge \delta \Omega \), or

$$\begin{aligned} a \le 2k |F'/\Omega '| \equiv \alpha \,. \end{aligned}$$
(14)

Thus, all the perturbations shorter than \(\alpha \) are of relatively small incoherent tune spread; in other words, there is no Landau damping for them. The maximal coherent tune shift for them is \(\Delta \omega \simeq -kF'\alpha /2 \simeq k^2 F'^2/|\Omega '|\). The suggested estimates assume a weak space charge, when the mode size is small, \(\alpha < \sigma \), leading to

$$\begin{aligned} k \le 0.2\, \sigma ^5 \,. \end{aligned}$$
(15)

Note that under this assumption the relative depression of the tune derivative \(|\mathrm {d}\Omega /\mathrm {d}I|\) (13) is not large.

The weak SC condition (15) can be compared with the condition of separated multipoles, or weak head–tail condition, requiring for the relative tune shift to be small. Thanks to Eq. (13), the latter can be presented as

$$\begin{aligned} k \ll 2 \sqrt{2\pi } \, \sigma ^3 \,. \end{aligned}$$
(16)

Thus, we come to a conclusion that there are four SC regimes:

  • Insignificant, \(\alpha \le q_c^{-1}\) or \(k<(\pi /8) \sigma ^4 q_c^{-1}\);

  • Weak, \((\pi /8) \sigma ^4 q_c^{-1} \le k \le 0.2\, \sigma ^5 \);

  • Medium, \(0.2\, \sigma ^5 < k \ll 2 \sqrt{2\pi } \, \sigma ^3\);

  • and strong, \(k \simeq 2 \sqrt{2\pi } \, \sigma ^3\).

For the first of them, the one with insignificant SC, all the modes are Landau damped due to the impedance roll-off or due to the intrabeam scattering at wave numbers \(q>q_c\). For the second, with weak SC, there is at least one discrete undamped mode, associated with oscillations of the relatively small central portion of the bunch, with the size \(a \simeq \alpha \). This size is determined by the equilibrium between SC tune shift and nonlinearity of the RF force. For the third regime, the medium one, the RF nonlinearity already does not play a role; all bunch particles are effectively involved into the undamped collective oscillations, but the synchrotron multipoles are still well-separated, the coherent and incoherent tune shifts are relatively small, and the bunch length is mostly determined by the given emittance and RF potential. For the fourth regime, the one with the strong SC, the potential well is significantly flattened by the SC forces, and the bunch length is determined by this condition, \(\sigma \simeq k^{1/3}\) [67, 68]. For all the regimes, except one of the insignificant SC, the Landau damping is lost, and even a tiny coupled-bunch (CB) wake may drive an instability.

The demonstrated back-of-the-envelope estimations essentially explain the findings of Ref. [9] that there is no LLD threshold for the pure inductive repulsive impedance, being in agreement with the leading mode character seen in that reference. It is also instructive to note that for the attractive wake, \(k<0\), the collective modes are shifted down with respect to the incoherent ones. Thus, LLD can happen only for the modes mostly associated with the high-amplitude particles, not the central ones. At high amplitudes, however, the derivative of the phase space density \(F'(I)\) normally tends to zero; thus, the nonzero LLD threshold has to be expected there, which also agrees with the analysis of Ref. [9].

It is worth noting that the method of estimations described in this section can be applied to any impedance. To show that, let us assume a repulsive impedance \(Z(q)= \zeta (-i q)^\kappa \), with some constant parameters \(\zeta \) and \(\kappa \). With that, the intensity parameter k is modified by a substitution \(-\Im Z/q \rightarrow \zeta \), yielding the coherent tune shift \(\Delta \omega \simeq -k F'\,a^{2-\kappa }/2\). Compared with the incoherent tune spread \(\delta \Omega = \pm |\Omega '|a^2/4\), it leads to a conclusion that at \(\kappa >0\) all the central perturbations with \(a \le |2 k F'/\Omega '|^{1/\kappa }\) are not damped. Thus, for all such impedances with \(\kappa >0\) the spectral properties should be qualitatively the same as for the inductive impedance. In particular, they all must correspond to zero LLD threshold above transition. Note that the resistive wall impedance belongs to this class; it is a case of \(\kappa =1/2\).

For the SC impedance below transition, a special interest is presented by the weak SC, in terms of the suggested classification, since it relates to lowest intensities where the stability problems may already appear. In this case, Jeans–Vlasov dynamic equation reduces, after a proper scaling transformation, to a parameter-less Hermitian integral equation for the eigenfunctions and eigenvalues. The eigenvalues which lie above the upper bound of the incoherent spectrum, constitute a discrete spectrum, relating to the modes without Landau damping. The tune of the leading mode sits at \(\Delta \omega = 0.22 |\Omega '| \alpha ^2\) above the maximal incoherent frequency \(\Omega (0)\). Formally speaking, the number of such modes is infinite; the eigenvalues have their limit point at \(\Omega (0)\). When the impedance roll-off or intra-beam scattering [59] is taken into account, it determines the highest-frequency discrete mode. In numerical computations, the number of discrete modes is also limited by the mesh size.

In case the inter-bunch wakes are smooth within a single bunch, the coupled-bunch growth rates can be either analytically expressed or found by means of the stability diagram [7], as it is shown in Ref. [10]. Also demonstrated there is how all these results are generalized for arbitrary multipolarity of the single-bunch modes.