1 Introduction

Gamma-gamma collisions have been studied since the 1970 s at \(e^+e^-\) storage rings in collisions of virtual photons (\(\gamma ^*\)). Two-photon physics is complementary to the \(e^+e^-\) physics program: in \(e^+e^-\) collisions, C-odd resonances with \(J^P=1^-\) are produced, while in \(\gamma \gamma \) collisions C-even resonances with various spins \(J\ne 1\) are produced. The first such resonance (\(\eta ^{\prime }\)) was observed in 1979 with detector Mark I at SPEAR [1], followed by many two-photon results from all \(e^+e^-\) facilities. Many results have been obtained at the high-luminosity KEKB and PEP-II [2], and studies continue at SuperKEKB. The number of virtual photons per electron is rather small, therefore \(\mathscr {L}_{\gamma \gamma }\) \(\ll \) \(\mathscr {L}_{e^+e^-}\) (however, it is complementary).

The future of the \(\gamma \gamma \) physics is closely connected with photon colliders based on high-energy linear colliders. At linear colliders, beams are used only once, which makes possible \(e \rightarrow \gamma \) conversion by Compton backscattering of laser light just before the interaction point, thus obtaining \(\gamma \gamma \), \(\gamma e\) beams with a luminosity comparable to that in \(e^+e^-\) collisions [3,4,5,6]. Since the late 1980 s, \(\gamma \gamma \) colliders have been considered a natural part of all linear collider projects; conceptual [7,8,9,10] and pre-technical designs [11, 12] have been published. The photon collider is being considered as one of the Higgs factory options [13,14,15]. However, no linear collider has yet been approved, and the future is rather unclear. Recently, V. Telnov proposed a photon collider [16] based on the electron linac of the existing linac of the European XFEL [17]. By pairing its 17.5 \(\mathrm {\,Ge V}\) electron beam with a 0.5 \({\,\upmu \mathrm m}\) laser, one can obtain a photon collider with a center-of-mass energy \(W_{\gamma \gamma } \le 12\) \({\mathrm {\,Ge V\!/}c^2}\). While the region \(W_{\gamma \gamma }<\) 4–5 \({\mathrm {\,Ge V\!/}c^2}\) can be studied at SuperKEKB, in the region \(W_{\gamma \gamma }=\) 5–12 \({\mathrm {\,Ge V\!/}c^2}\) the photon collider will have no competition in the study of a large number of \(b{\overline{b}}\) resonances, tetraquarks, or mesonic molecules.

In this paper, we investigate the question of the very possibility of observing and studying heavy C-even resonances in the presence of a large hadronic background. The effective cross section of resonance production is proportional to \(\varGamma _{\gamma \gamma }/M_R^2\); for bottomonium (\(b{\overline{b}}\)) states, this value is two orders of magnitude smaller than for charmonium (\(c{\overline{c}}\)) states. At the same time, the cross section of the background \(\gamma \gamma \ \rightarrow hadrons\) process in this energy region is nearly constant. At these “intermediate” energies, the angular distribution of hadronic backgrounds still differs not much from the isotropic distribution in resonance decays (for \(J=0\)), so the possibility of suppressing the background was not at all obvious.

In Sect. 2, we summarize theoretical predictions on \(\gamma \gamma \) widths \(\varGamma _{\gamma \gamma }\) of resonances in this energy region and give formulas for production cross sections in \(\gamma \gamma \) collisions. In Sect. 3, main parameters of the \(\gamma \gamma \) collider are presented and the differential luminosity \(d\mathscr {L}_{\gamma \gamma }/dW_{\gamma \gamma } \) is compared with that at SuperKEKB. In Sect. 4, we consider methods to suppress hadronic backgrounds (using realistic simulation) and determine detection efficiencies after background suppression. Finally, we find the values of \(\varGamma _{\gamma \gamma } (W)\) for which resonances can be observed at a \(5\sigma \) confidence level in 1 year of the \(\gamma \gamma \) collider operation.

2 Two-photon processes: general features

The spectrum of photons after Compton backscattering is broad, with a characteristic peak at maximum energies. Photons can have circular or linear polarizations depending on their energies and polarizations of initial electrons and laser photons. Due to the angle-energy correlation, in Compton scattering the \(\gamma \gamma \) luminosity cannot be described a by convolution of photon spectra. Due to complexity of processes in the conversion and interaction regions, the accuracy of prediction by simulation will be rather poor; therefore, one should measure all luminosity properties experimentally using well known QED processes [18].

Generally, the number of events in \(\gamma \gamma \) collision is given by [6, 18]

$$\begin{aligned} d{\dot{N}}_{\gamma \gamma \ \rightarrow X} = dL_{\gamma \gamma } \sum _{i,j=0}^3 \langle \xi _i \tilde{\xi _j} \rangle \sigma _{ij}, \end{aligned}$$
(1)

where \(\xi _i\) are Stokes parameters, \(\xi _2\equiv \lambda _{\gamma }\) is the circular polarization, \(\sqrt{\xi _1^2+\xi _3^2} \equiv l_{\gamma }\) the linear polarization, and \(\xi _0 \equiv 1\). Since photons have wide spectra and various polarizations, in general one has to measure 16 two-dimensional luminosity distributions \(d{\,^2}L_{ij}/d\omega _1 d\omega _2\), \(dL_{ij} = dL_{\gamma \gamma } \langle \xi _i \tilde{\xi _j} \rangle \), where the tilde sign marks the second colliding beam.

Among the 16 cross sections \(\sigma _{ij}\), there are three that are most important as they do not vanish after averaging over the spin states of the final particles and azimuthal angles, they are [6, 18]

$$\begin{aligned} \sigma ^{np}\equiv & {} \sigma _{00} = (\sigma _{\parallel }+\sigma _{\perp })/2= (\sigma _0+\sigma _2)/2, \nonumber \\ \tau ^c\equiv & {} \sigma _{22} = (\sigma _0-\sigma _2)/2, \nonumber \\ \tau ^l\equiv & {} (\sigma _{33} - \sigma _{11})/2 = (\sigma _{\parallel }-\sigma _{\perp })/2. \end{aligned}$$
(2)

Here the \(\sigma _{\parallel }\) and \(\sigma _{\perp }\) are the cross sections for collisions of linearly polarized photons with parallel and orthogonal relative polarizations and \(\sigma _0\) and \(\sigma _2\) are the cross sections for collisions of photons with \(J_z\) of two photons equal 0 and 2, respectively.

If only these three cross sections are of interest then formula (1) can be written as

$$\begin{aligned} d{\dot{N}}_{\gamma \gamma \ \rightarrow X} = dL_{\gamma \gamma }\; (\sigma ^{np} + \langle \xi _2 \tilde{\xi _2} \rangle \tau ^c + \langle \xi _3 \tilde{\xi _3} - \xi _1 \tilde{\xi _1} \rangle \tau ^l).\nonumber \\ \end{aligned}$$
(3)

Substituting \(\xi _2 \equiv \lambda _{\gamma }\),    \( \tilde{\xi _2} \equiv {\tilde{\lambda }}_{\gamma }\),    \(\xi _1 \equiv l_{\gamma } \sin 2\gamma \),    \(\tilde{\xi _1} \equiv -{\tilde{l}}_{\gamma } \sin 2{\tilde{\gamma }}\), \(\xi _3 \equiv l_{\gamma } \cos 2\gamma \), \(\tilde{\xi _3} \equiv {\tilde{l}}_{\gamma } \cos 2 {\tilde{\gamma }}\), and \(\varDelta \phi =\gamma -{\tilde{\gamma }}\) (azimuthal angles for linear polarizations are defined relative to one x axis), we get

$$\begin{aligned} d{\dot{N}}= & {} dL_{\gamma \gamma }( \sigma ^{np} + \lambda _{\gamma } {\tilde{\lambda }}_{\gamma }\; \tau ^c + l_{\gamma } {\tilde{l}}_{\gamma } \cos {2\varDelta \phi } \;\tau ^l) \nonumber \\\equiv & {} dL_{\gamma \gamma }\; \sigma ^{np} + (dL_0 - dL_2) \tau ^c + (dL_{\parallel } -dL_{\perp })\,\tau ^l \nonumber \\\equiv & {} dL_0 \sigma _0 + dL_2 \sigma _2 + (dL_{\parallel } -dL_{\perp })\, \tau ^l \nonumber \\\equiv & {} dL_{\parallel }\, \sigma _{\parallel } + dL_{\perp }\, \sigma _{\perp } + (dL_0 - dL_2)\, \tau ^c, \end{aligned}$$
(4)

where \(dL_0 = dL_{\gamma }(1+\lambda _{\gamma } {\tilde{\lambda }}_{\gamma })/2,\; dL_2 = dL_{\gamma }(1-\lambda _{\gamma } {\tilde{\lambda }}_{\gamma })/2,\) \(dL_{\parallel } = dL_{\gamma } (1+ l_{\gamma } {\tilde{l}}_{\gamma }\cos {2\varDelta \phi })/2,\; dL_{\perp } = dL_{\gamma }(1- l_{\gamma } {\tilde{l}}_{\gamma }\cos {2\varDelta \phi })/2\,.\)

So, one should measure \(dL_{\gamma \gamma }\), \(\langle \lambda _{\gamma } {\tilde{\lambda }}_{\gamma }\rangle \), \(\langle l_{\gamma } {\tilde{l}}_{\gamma }\rangle \) or, alternatively, \(dL_0,\, dL_2,\, dL_{\parallel },\, dL_{\perp }\). If both photon beams have no linear polarization or no circular polarization, the luminosity can be decomposed into two parts: \(L_0\) and \(L_2\), or \(L_{\parallel }\) and \(L_{\perp }\), respectively.

For example, for scalar/pseudoscalar resonances (\(J=0\)) \(\sigma _2=0\), while \(\sigma _{\parallel } = \sigma _0\),  \(\sigma _{\perp } = 0\) for \(CP=1\) (scalar) and \(\sigma _{\perp } = \sigma _0\),   \(\sigma _{\parallel } = 0\) for \(CP=-1\) (pseudoscalar). Then

$$\begin{aligned} d{\dot{N}} = dL_{\gamma \gamma } \,\sigma ^{np} (1+\lambda _{\gamma } {\tilde{\lambda }}_{\gamma } \pm l_{\gamma } {\tilde{l}}_{\gamma }\cos {2\varDelta \phi }). \end{aligned}$$
(5)

In the present work, we investigate the feasibility of studying two-photon production of C-even resonance states (charmoniums, bottomoniums, and various exotics in the energy range from 3 to 12 GeV. The cross section for production of narrow resonances in monochromatic non-polarized \(\gamma \gamma \) collisions (\(\hslash \equiv c \equiv 1\))

$$\begin{aligned} \sigma _{\gamma \gamma \rightarrow R}(W) = 8\pi ^2(2J+1) \frac{\varGamma _{\gamma \gamma }}{M} \delta (W^2-M^2). \end{aligned}$$
(6)

For broad luminosity spectra and polarized beams, the resonance production rate

$$\begin{aligned} {\dot{N}}= & {} \frac{d\mathscr {L}_{\gamma \gamma }}{dW_{\gamma \gamma }}\frac{4\pi ^2 (2J+1)\varGamma _{\gamma \gamma }}{M^2} \nonumber \\{} & {} \times \left( 1 + \frac{\tau ^c}{\sigma ^{np}} \lambda _{\gamma } {\tilde{\lambda }}_{\gamma }\; + CP \times \frac{\tau ^l}{\sigma ^{np}} l_{\gamma } {\tilde{l}}_{\gamma } \cos {2\varDelta \phi } \;\right) , \end{aligned}$$
(7)

where \(\sigma ^{np}, \tau ^c, \tau ^l\) are defined in (2).

At the photon collider under discussion, the degree of polarization in the high-energy part of the spectrum can be close to \(100\%\) for the circular and about \(85\%\) for linear polarization, which is controlled by the laser polarization.

For \(\lambda _{\gamma } {\tilde{\lambda }}_{\gamma }=1\), the number of scalars doubles (they are produced only in collisions of photons with the total helicity of zero, with the cross section \(\sigma _0\)). In the case of \(\lambda _{\gamma } {\tilde{\lambda }}_{\gamma }= - 1\), the total helicity is 2, scalar resonances are not produced, but the number of resonances with \(J=2\) almost doubles because it is known that they are produced mostly in the state with helicity 2 (\(\sigma _2 \gg \sigma _0\)). In the case of linearly polarized \(\gamma \)-beams, production of scalars doubles when the linear polarizations of the beams are parallel, while pseudoscalars, on the contrary, prefer perpendicular linear polarizations.

A nice feature of both \(e^+e^-\) and \(\gamma \gamma \) collisions is the single resonance production of hadrons. At \(e^+e^-\) colliders, resonances with the photon quantum numbers, \(J^{PC}=1^{--}\), can be single-produced, which includes the \({J/\psi }\) and \(\varUpsilon \) families. On the other hand, two real photons can single-produce C-even resonances with the following quantum numbers [19]: \(J^P=0^+\), \(0^-\), \(2^+\), \(2^-\), \(3^+\), \(4^+\), \(4^-\), \(5^+\), etc., the forbidden numbers being \(J^P=1^{\pm }\) and (odd \(J)^-\). Therefore, the \(\gamma \gamma \) collider presents a much richer opportunity for the study of hadronic resonances.

Resonance production cross sections in \(\gamma \gamma \) collisions depend on the total helicity of the two photons, \(J_z=0\) or 2. Assuming that the C and P parities are conserved, resonances are produced only in certain helicity states [19]: \(J_z=0\) for \(J^P=0^\pm \), (even \(J)^-\); \(J_z=2\) for (odd \(J \ne 1)^+\); \(J_z=0\) or 2 for \(J^P\) = (even \(J)^+\). In the experiment, the value of \(J_z\) is chosen by varying the laser photon helicities.

3 Expected C-even resonances

In photon-photon collisions, C-even resonances are produced with a wide range of spin and parity values. The first observation of the C-even resonance, \(\eta ^{\prime }\) meson at \(e^+e^-\) collider was done by Mark I collaboration in 1979 [1]. By now, many pseudoscalar (\(^1S_0\)), scalar (\(^3P_0\)) and tensor (\(^3P_2\)) resonances in a wide range of masses have been discovered at \(e^+e^-\) colliders in the two-photon fusion process \( e^+e^- \rightarrow \) \(e^+e^-\gamma ^\star \gamma ^\star \rightarrow e^+e^-X\) by BaBar and Belle [2], CLEO [20] and BESIII [21] collaborations. This process is dominated by events where both photons are nearly real and both \(e^+\) and \(e^-\) have very small scattering angles and are not detectable, therefore resonance X and its decay products have small total transverse momentum, which can be used as an experimental sign of the process. The cross section of narrow resonance production is proportional to the two-photon partial width \(\varGamma _{\gamma \gamma }\) of the resonance thus allowing the measurement of this quantity at photon colliders, which is the main experimental goal.

3.1 Heavy quarkonium pseudoscalar, scalar and tensor states

In Table 1 and Fig. 1, we list known \(c{\bar{c}}\) and \(b{\bar{b}}\) C-even resonances with experimental data from PDG [22] and a summary of theoretical predictions on their masses and two-photon widths [24,25,26,27,28].

Table 1 Masses and two-photon widths for various charmonium and bottomonium states from experiment (PDG) and various theoretical predictions [24,25,26,27,28]

Two-photon widths are successfully predicted with nonrelativistic quark models [28]. In the nonrelativistic limit, two-photon widths of mesons are proportional to the square of the wave function or its derivative at the origin. However, relativistic effects are important, especially for charmonium, and modify this relation [24,25,26,27]. The first-order correction is proportional to the QCD coupling \(\alpha _s\), which is estimated to be \(\alpha _s(m_b)\) = 0.18 for bottomonia and \(\alpha _s(m_c)\) = 0.26 for charmonia, respectively [23].

Another way to study non-perturbative QCD is an lattice QCD [29,30,31,32,33], which is quantum field theory defined on discrete Euclidean space-time.

Two-photon decay widths for scalar and pseudoscalar charmonia were recently estimated to be about 1 keV [32] which is smaller than the experimental values.

Besides quark-antiquark pairs mesons the quark model assumes the existance of exotic multiquark hadrons with more complex internal structures. Neutral mesons with exotic properties, namely, X- and Y-states in the mass range from 3.8 to 7.0 \({\mathrm {\,Ge V\!/}c^2}\), have been discovered experimentally. Different interpretations have been proposed for those resonances summarized in [35], such as tetraquarks, molecular states, quark-gluon hybrids, hadro-quarkonia, kinematic threshold effects, or mix states. Feasibility of observaion multiquark states in \(\gamma \gamma \) collisions is discussed below.

3.2 Tetraquarks

The simplest multiquark system is a tetraquark, which consists of two quarks and two antiquarks, is color- and charge-neutral, and has spin not equal to 1. One possible way to check for the existence of tetraquarks is to find a complete flavor-spin multiplet such as the standard quarkonium families. Scalar and tensor states are expected to be produced in two-photon collisions, although their two-photon widths are expected to be less than 1 keV [34]. A lot of tetraquarks that can be produced in \(\gamma \gamma \) collisions with masses from 3 to 12 \({\mathrm {\,Ge V\!/}c^2}\) are predicted in the relativistic quark model based on the quasipotential approach in the recent work [35]. In those calculations tetraquarks were assumed to have two or four heavy quarks and a diquark-antidiquark picture of heavy tetraquarks was used.

A narrow resonance around 6.9 \({\mathrm {\,Ge V\!/}c^2}\) in the invariant mass spectrum of \(J/\psi \) pairs was found by LHCb collaboration [36] and was called X(6900). Its mass and width were measured to be \(M_X\! =\! 6886\! \pm \! 2\) \({\mathrm {\,Me V\!/}c^2}\) and \(\varGamma _X\! =\! 168\! \pm \! 102\) MeV, while its quantum numbers can be \(0^{++}\) or \(2^{++}\). This resonance can be interpreted as a \(cc{\bar{c}}{\bar{c}}\) compact state. Using the vector meson dominance model in the assumption of its strong coupling to a di-\(J/\psi \) final state the X(6900) two-photon width has been estimated as 104 eV for \(J^{PC}=0^{++}\) and 86 eV for \(J^{PC}=2^{++}\) [37].

Fig. 1
figure 1

Values of the masses and two-photon widths for various charmonium and bottomonium states from PDG (circles) and various theoretical predictions (triangles), tetraquarks and molecular states

Scalar and tensor tetraquarks \(cc{\bar{q}}{\bar{q}}\) exist in the diquarkonium model but have not been observed yet in any experiment. Two states with quantum numbers \(J^{PC}=0^{++}\) and one with \(J^{PC}=2^{++}\) are predicted by the diquark-antidiquark model with the dominated cq interaction, and their masses are 3770 \({\mathrm {\,Me V\!/}c^2}\), 4000  \({\mathrm {\,Me V\!/}c^2}\) and 4000  \({\mathrm {\,Me V\!/}c^2}\), called \(X_0(3770)\)\(X_0^{\prime }(4000)\) and \(X_2(4000)\), respectively [38]. The partial \(\gamma \gamma \) widths of those tetraquarks are predicted to be 6.3 eV, 6.7 eV and 1.6 eV, respectively [37]. Experimental search for these states is an important test of the diquark–antidiquark picture of heavy tetraquarks.

3.3 Mesonic molecules

Hadronic molecules are bound states of two or more mesons. Particles with masses close to the sum of masses of two other mesons, on one hand, and away from the predictions of the quark model on the other are often considered to have a possible molecular structure. The most famous experimental candidate for a mesonic molecule is X(3872) resonance, which is considered to be \(D^0{\bar{D}}^{\star 0}\) molecule [39]. Other heavy meson candidates with mass greater than 3 \({\mathrm {\,Ge V\!/}c^2}\) considered to have a molecular structure are X(3915) [40, 41], Y(3940), Y(4140) and Y(4660) [42]. Identification of an observed resonance as a mesonic molecule is based not only on its mass and quantum numbers but also on the process in which the resonance was found. For predictions, the theory of electromagnetic interactions is usually used. So, properties of resonance produced in two-photon collisions provide information about its nature.

Partial two-photon widths calculated in the framework of a phenomenological Lagrangian approach of \(D{\bar{D}}\), \(D_s{\bar{D}}_s\), \(B{\bar{B}}\) molecules are expected in the range 0.1–2.8 keV [43]. Radiative widths of the molecules Y(3940) = \({D^{\star }{\bar{D}}^{\star }}\) and Y(4140) =\({D^{\star }_s {\bar{D}}^{\star }_s}\) are about 1 keV [44].

3.4 Glueballs

Glueballs predicted by QCD are color-neutral states that consist only of gluons. Gluons inside a glueball can self-interact but gluons remain stable, except the heaviest states that decay into lighter glueballs. Theory suggests a rich spectrum of glueballs. Their existence is compatible with recent experimental data, and several exotic meson candidates have been interpreted as glueballs: \(f_0(1370)\), \(f_0(1500)\), \(f_0(1710)\), \(f_J(2220)\) and others. The challenge is to identify observed particles as glueballs. The situation is complicated by the lack of knowledge on the glueball nature and possible mixing of glueballs with standard quark model states.

Glueball production in two-photon collisions is a unique process that can clearly distinguish a tensor glueball from a tensor meson [45]. Gluons do not participate in electromagnetic interactions. Two-photon widths of glueball states are significantly smaller in comparison with two photon widths of ordinary quarkonia [46]. The advantage is that two-photon width is model-independent in contrast with other glueball properties. The expected two-photon width is 1–10 eV.

Fig. 2
figure 2

\(\gamma \gamma \) luminosity distributions vs the invariant mass \(W_{\gamma \gamma }\): (left) unpolarized electrons; (right) longitudinal electron polarization \(2\lambda _e=0.8\) (80\(\%\)). In both cases the laser photons are circularly polarized, \(P_c=-1\). Solid lines are for the total helicity of the two colliding photons \(J_z=0\), dotted lines for \(J_z=2\). Red curves are luminosities with a cut on the longitudinal momentum

Glueballs are predicted in lattice QCD calculations. The mass of the first excited glueball in the tensor channel is estimated using lattices to be \(3320 \pm 20 \pm 160\) \({\mathrm {\,Me V\!/}c^2}\) [47]. States with quantum numbers and masses

$$\begin{aligned} J^{PC}= & {} 2^{-+},\; m_G=3040 \pm 40 \pm 150~{\mathrm {\,Me V\!/}c^2},\\ J^{PC}= & {} 3^{++}, \;m_G=3670 \pm 50 \pm 180~{\mathrm {\,Me V\!/}c^2} \end{aligned}$$

are predicted with the improved technique [48].

4 \(\gamma \gamma \) collider

The parameters of the \(\gamma \gamma \) collider based on the 17.5 GeV electron linac of the European XFEL is described in Ref. [16]. The maximum energy of scattered photons

$$\begin{aligned} \omega _m \approx \frac{x}{x+1}E_0, x = \frac{4E_0\omega _0}{m^2c^4}= 19\left[ \frac{E_0}{\mathrm {\,Te V}}\right] \left[ \frac{{\,\upmu \mathrm m}}{\lambda }\right] . \end{aligned}$$
(8)

For \(E_0=17.5\) \(\mathrm {\,Ge V}\) and the laser wavelength \(\lambda =0.5\) \({\,\upmu \mathrm m}\), \(x=0.65\), \(\omega _m/E_0=x/(x+1) \approx 0.394\), \(W_{\gamma \gamma , \mathrm max} \approx 13.3\) \({\mathrm {\,Ge V\!/}c^2}\), with a peak at 12 \({\mathrm {\,Ge V\!/}c^2}\), which covers the region of \(b{\overline{b}}\) resonances. The peak energy can be varied by adjusting the electron beam energy. The thickness of the laser target is taken to be equal to one scattering length for electrons with the initial energy. The required flash energy is about 3 J. We consider both unpolarized (as currently available at the European XFEL) and \(80\%\) longitudinally polarized electron beams. The laser beam should be circularly polarized, \(P_c= \pm 1\), when circularly polarized high-energy photons are needed. Collisions of linearly polarized photons would also be of interest for physics; for that, linearly polarized laser beams should be used. The degree of circular polarization in the high-energy part of the spectrum can be close to \(100\%\) (for any x) and about \(85\%\) for linear polarization (for \(x=0.65\)).

The \(\gamma \gamma \) luminosity spectra for non-polarized and longitudinally polarized electrons are shown in Fig. 2. The spectra are decomposed into states with the total helicity of the colliding photons \(J_z=0\) or 2; the total luminosity is the sum of the two spectra. Also shown are the luminosities with a cut on the relative longitudinal momentum of the produced system that suppresses boosted collisions of photons with very different energies. Luminosity distributions similar to those in Fig. 2 but for various distances b between the conversion and interaction points are given in Ref. [16]. As the distance increases, the luminosity spectra become more monochromatic at the cost of some reduction in luminosity.

For the study of resonances, when the invariant mass is determined by the detector, the maximum luminosity is needed; therefore a small distance is preferable, as in Fig. 2, where \(\rho =b/\gamma \sigma _y=1\) and corresponding \(b = 1.8\) mm.

Fig. 3
figure 3

Comparison of \(\gamma \gamma \) luminosities at the photon collider and Super-KEKB

The geometric electron–electron luminosity at the nominal energy of 17.5 GeV \(\mathscr {L}_{ee} = 1.45 \cdot 10^{33}\,\mathrm{\,cm}^{-2} \textrm{s}^{-1} \) (determined by the beam emittances and proportional to the energy), \(\mathscr {L}_{\gamma \gamma }(z>0.5z_m) \approx 2\cdot 10^{32}\,\mathrm{\,cm}^{-2} \textrm{s}^{-1} \) (\(\propto \mathscr {L}_{ee} \)). The resonance production rate is proportional to \(d\mathscr {L}_{\gamma \gamma }/dW_{\gamma \gamma } \) at the peak of the luminosity distribution. In Fig. 3 it is compared with that at SuperKEKB in \(\gamma ^*\gamma ^*\) collisions for \(2E_0=11\) GeV and \(\mathscr {L}_{ee} = 5\cdot 10^{34}\,\mathrm{\,cm}^{-2} \textrm{s}^{-1} \). At present, \(L_{\textrm{max}} \sim 4.5\cdot 10^{34}\,\mathrm{\,cm}^{-2} \textrm{s}^{-1} \) at SuperKEKB, the planned value (in the year 2028) is \(L_{\textrm{max}} \sim 8\cdot 10^{35}\, \mathrm{\,cm}^{-2} \textrm{s}^{-1} \). In any case, the photon collider is beyond competition in the \(b{\overline{b}}\) energy region.

5 Suppression of hadronic background

Below we study the possibility of detecting narrow C-even resonances in the process \(\gamma \gamma \rightarrow R\) at \(W=\)4–10 GeV. The selection of resonances requires registration of all final-state particles. The effective cross section of resonance production is proportional to \(\varGamma _{\gamma \gamma }/M_R^2\) (7). For bottomonium (\(b{\overline{b}}\)) states, this value is two orders of magnitude smaller than for charmonium (\(c{\overline{c}}\)) states. At the same time, the cross section of the background \(\gamma \gamma \ \rightarrow hadrons\) process in this region is almost constant, \(\sigma _{\gamma \gamma \rightarrow \textrm{hadr}}\sim 350\) nb. For example, the first candidate for studying is \(\eta _b(9400)\) with \(\varGamma _{\gamma \gamma } \sim 0.5\) keV (the largest in this mass region). The number of hadronic events in the resonance mass region \(\pm \sigma _M\) (\(\sigma _M \approx 50\) \({\mathrm {\,Me V\!/}c^2}\)) will be about 230 times greater than the number of resonances.

Fig. 4
figure 4

The layout of the detector (a quarter of the inner part)

In the present study, we carefully consider this problem, trying to suppress the background and to maximize the significance of the resonances, i.e. to increase the value \(S/\sqrt{B}\).

The procedure is the following. We simulate resonances and hadrons at several invariant masses, from 4 to 10 \({\mathrm {\,Ge V\!/}c^2}\), 100,000 events of resonances and hadrons at each point. Resonances and hadrons are generated by PYTHIA [49]. Resonances are modeled as \(\eta _b\), but with changed masses. Hadrons were modeled with a mass spread of 10% (similar to the width of the high-energy peak at the \(\gamma \gamma \) collider) at the same average invariant mass as the resonance under study. It is assumed that the peak in the luminosity distribution over invariant masses coincides with the resonance mass. If a hadronic event has passed all the selection conditions but its reconstructed mass is more than 20% lower than the average peak mass, then this is no longer a background for the studied resonance. There are quite a few such cases; the requirement of a small total transverse momentum cuts off all events with lost particles.

Simulated particles passed the detector (described by GEANT-4) shown in Fig. 4 (only the elements used for analysis are shown). It has a tracking system, a particle identification system (PID) and a CsI EM calorimeter. At this stage (it is just first look), we did not make a complete reconstruction of the event, but rather assigned a certain momentum and angular resolution to the particles that passed through the detector.

However, this is not just a geometric simulation: GEANT simulates interactions with a vacuum chamber and particle decays, which affects the reconstruction efficiency. A charged particle considered registered if it passes at least 55 cm through the track system. Reconstruction of resonances requires the registration of all (detectable) particles. The average particle multiplicity is about 17 at 10 GeV. Particles that are undetectable (e.g., neutrinos) or that could mess up the mass resolution (\(n,{\bar{n}},{\bar{p}},K_L\)) were simply removed from the events (after which the event did not pass the selection criteria or had a displaced mass). Removal of these particles has reduced the reconstruction efficiency by a factor of 1.7 and 3.4 for resonances with masses 4 and 10 \({\mathrm {\,Ge V\!/}c^2}\), respectively.

Thus, good events consist only of \(\pi ^{\pm }, K^{\pm }, e^{\pm }, \mu ^{\pm } \) and photons (events with protons are bad because they always contain \({\bar{p}}\) or \({\bar{n}}\)). Calculating the invariant mass, we assumed that electrons and positrons are completely identified (\(m=m_e\)). For muons, we assigned the pion mass. Kaons were considered identified (\(m=m_K\)) at \(pc<0.5\) GeV (by dE/dx in the tracking system) or when they crossed the PID system (it may be some kind of an aerogel system), otherwise they were assigned the the pion mass. If charged kaons are not identified at all, then the efficiency (the number of resonances in a narrow peak) will decrease by a factor of 1.3 times (at 10 \({\mathrm {\,Ge V\!/}c^2}\)).

Detector parameters are the following: the minimum angle is 0.15 rad (0.30 was checked as well), the magnetic field \(B= 1.2\) T, the tracking resolution

$$\begin{aligned} (\sigma _{p_{\perp }}/p_{\perp })^2=(2\cdot 10^{-3} p_{\perp }[\mathrm{GeV/c}])^2 + (3\cdot 10^{-3}/\beta )^2, \end{aligned}$$

the e.m. calorimeter resolution

$$\begin{aligned} \sigma _E/E = 0.025/\sqrt{E[\mathrm GeV]}. \end{aligned}$$

The invariant mass resolution is almost completely determined by the energy resolution of the calorimeter. The contribution of any typical photon angular resolution is negligible, since the system is created almost at rest (\(P_{\Vert }c<0.1\,W\)).

Below we choose the selection criteria to optimally suppress the hadron background and find the detection efficiency for resonances and the hadron background. Consideration is made for a minimum detector angle of 0.15 rad. At the end, we present the results for 0.3 rad. The difference in efficiencies turns out to be insignificant since events with a large sum of transverse particle momenta are selected to separate resonances from the background.

The sphericity angle distributions for the resonance and hadronic events are shown in Fig. 5. It can be seen that the hadronic background at \(W=10\) \({\mathrm {\,Ge V\!/}c^2}\) is pressed to the axis more strongly than at \(W=4\) \({\mathrm {\,Ge V\!/}c^2}\); these difference can be used to suppress hadrons. Below we consider the selection criteria based on the differences in the angular distribution of particles from the decay of resonances (with \(J=0\)) and the hadronic background.

Fig. 5
figure 5

The distributions of resonance and hadronic events on the sphericity angle for \(W=\) 4 and 10 GeV/\(c^2\)

Fig. 6
figure 6

The distributions of the parameter \(E(|\cos {\theta }|<0.7)/E\) for resonances with M = 4 and 10 GeV/\(c^2\) and corresponding hadronic background

One of the conditions is based on the ratio of the sum of the particles’ energies in the detector at an angle larger than some \(\theta _{\textrm{min}}\) to the total energy in the detector. For \(W=10\) GeV the optimum angle is about \(|\cos {\theta }|=0.7\). The distributions of the ratio \(E(|\cos {\theta }|<0.7)/E\) is shown in Fig. 6. We have found that the optimal value of this ratio for hadron suppression is about 0.7. This is the first constraint for the separation of resonances

$$\begin{aligned} (1) \;\;\;\;\; E(|\cos {\theta }|<0.7)/E <0.7. \end{aligned}$$
(9)

The distributions on \(\varSigma |p_t|\) are shown is Fig. 7. In terms of separating power, it is comparable to the previous cut. For the selection of a resonance with the mass M, we require \(\varSigma |p_t| > 0.75 Mc\); this is the second constraint:

$$\begin{aligned} (2) \;\;\;\;\;\varSigma |p_t| > 0.75\,Mc. \end{aligned}$$
(10)

The constraints (1) and (2) strongly correlate; nevertheless, together they give a slightly better result.

Fig. 7
figure 7

The distribution \(\varSigma |p_t|\) for resonances with M = 4 and 10 GeV/\(c^2\) and hadronic background

The distribution for all events (without any cuts) of the total transverse momentum \(|\varSigma \vec {p}_t|\) of detected particles is shown in Fig. 8. Only events with a small \(|\varSigma \vec {p}_t|\) (this indicates that all particles were registered by the detector) are suitable for observing narrow resonances. This defined our third cut:

$$\begin{aligned} (3) \;\;\;\;\;|\varSigma \vec {p}_t|<100\,\, \textrm{MeV}/c. \end{aligned}$$
(11)
Fig. 8
figure 8

Distributions on \(|\varSigma \vec {p}_t|\) in the detector (without any other cuts) for resonances with M = 4 and 10 GeV/\(c^2\) and hadronic background

Fig. 9
figure 9

Distributions on the invariant masses in the detector for resonances with M = 4 and 10 GeV/\(c^2\). Black curves – all events, blue – events with an even number of charged particles, red – with additional cut on the total transverse momentum. Hadronic background is not shown here

The distributions of the invariant masses in the detector are shown in Fig. 9. There are three distribution: all events, with an even number of charged particles, and with the cut \(|\varSigma p_t|<100\) MeV/c. The last condition leaves only events at the peak of the resonance. After adding constraints (1) and (2) to Fig. 9, we obtain final distributions of invariant masses for resonances shown in Fig. 10.

Fig. 10
figure 10

Distribution of resonances with M = 4 and 10 GeV/\(c^2\) on invariant masses in the detector after the cut on the sum transverse momentum (red curves in Fig. 9) plus cuts (1) and (2) (see the text) which suppress hadronic background. Hadronic background is not shown here

The efficiencies for resonances \(\epsilon _R\) and hadronic background \(\epsilon _h\) after applying all the selection criteria are presented in Fig. 11. The efficiency for resonances varies from 18 to 5% for \(M_R=\) 4–10 \({\mathrm {\,Ge V\!/}c^2}\). The efficiency for the hadron background is lower than for resonances by a factor of 2.5 times at \(W=4\) \({\mathrm {\,Ge V\!/}c^2}\), and a factor of 125 at 10 \({\mathrm {\,Ge V\!/}c^2}\). This is due to the fact that at higher energies the hadronic background is directed more forward and differs more from isotropic (at J = 0) decays of resonances. Such a behavior was expected but was not quantified. This result shows the possibility of studying C-even resonances in \(\gamma \gamma \) collisions in the energy region of 10 GeV, where the ratio of the non-resonant hadronic cross section to the bottomonium resonance cross sections is two orders of magnitude greater than in the charmonium energy region of \(W=\) 3–4 GeV\(/c^2\). The above analysis assumed a minimum detector angle of 0.15 rad. We repeated the same analysis for the angle 0.3 rad. For \(W=10\) \({\mathrm {\,Ge V\!/}c^2}\) the decrease in efficiency is about 15% and negligible for \(W=4\) \({\mathrm {\,Ge V\!/}c^2}\). The difference is so small because we selected events with a large sum of transverse momenta to suppress the hadronic background.

Fig. 11
figure 11

Efficiencies for resonances and hadrons after applying all selection criteria

Figure 12 shows how efficiency decreases when an additional cut is applied on the minimum \(p_t\) of charged particles in the detector. This information is useful when considering QED backgrounds with a small \(p_t\). It comes mainly from low-energy \(\gamma \gamma \rightarrow e^+e^- \) process. For the present analysis, the effective \(p_{t, \textrm{min}} \approx 50\) MeV/c, as it is seen in Fig. 12. The background \(e^+\) and \(e^-\) overlap with the events under study with a probability of about 2% for the collider parameters corresponding to Fig. 2. Such low \(p_t\) tracks, identified as \(e^+/e^-\), can simply be ignored in event analysis because the probability of such particles in the decay products of resonances is very small. The probability of imposing a hadron background on the effect is about 1.5%.

Fig. 12
figure 12

Efficiencies for resonances after all selection criteria (as in Fig. 11) with an additional cut on the minimum \(p_t\) of particles in the detector (if it will be needed for suppression of low \(p_t\) QED backgrounds, see the text)

Figure 13 shows the differential luminosity dL/dW of the \(\gamma \gamma \) collider under consideration at the high energy peak of luminosity spectra as a function of W (which is varied by the electron energy). The number of produced resonances (no cuts) with \(\varGamma _{\gamma \gamma } = 1\) keV for the running time at one energy point equal to 1/5 of the year is shown in Fig. 14.

Fig. 13
figure 13

The differential luminosity dL/dW at the high energy peak of luminosity spectra (at the photon collider under consideration) as a function of \(W_{\mathrm peak}\) which is varied by the electron energy

Fig. 14
figure 14

The number of produced resonances (no cuts) with \(\varGamma _{\gamma \gamma } = 1\) keV for the running time at one energy point equal to 1/5 of the year

The mass resolution of reconstructed resonances is given in Fig. 15. It is \(\sigma _{M_R}\approx \) 35–55 \({\mathrm {\,Me V\!/}c^2}\) for \(W=\) 4–10 \({\mathrm {\,Ge V\!/}c^2}\) for the chosen detector parameters. The minimum values of \(\varGamma _{\gamma \gamma }\) for detecting resonances at the 5\(\sigma \) confidence level in 1/5 year operation on the energy of the resonance is given Fig. 16. In this case, about 5 energy points (1 year) cover the entire region of invariant masses. It was assumed in calculations that \(\sigma _{\gamma \gamma \rightarrow \textrm{hadr}} \approx 350 \mathrm{\,nb} \) in mass region \(W=\) 4–10 \({\mathrm {\,Ge V\!/}c^2}\). The ratio of the resonance peak height to the non-resonant background

$$\begin{aligned} R=\frac{dN_R/dW}{dN_h/dW}= \frac{4\pi ^2(2J+1)\varGamma _{\gamma \gamma }(1+\lambda _{\gamma }\tilde{\lambda }_{\gamma })\epsilon _R}{\sqrt{2\pi }M_R^2\sigma _{M_R} \sigma _h \epsilon _h}. \end{aligned}$$
(12)
Fig. 15
figure 15

The mass resolution for resonances

For the lightest C-even charmonium \(\eta _c(2984)\) with \(\varGamma _{\gamma \gamma }\approx 5\) keV and the lightest bottonium \(\eta _b(9398)\) with \(\varGamma _{\gamma \gamma }\sim 0.5\) keV the values of R are approximately 1.4 and 0.4, respectively. The \(\eta _b(9398)\) meson has not yet been observed in the \(\gamma \gamma \) mode; at the photon collider it can be observed at the \(>5\sigma \) level in 1 day of operation.

Fig. 16
figure 16

The minimum values of \(\varGamma _{\gamma \gamma }\) for detecting resonances at the 5 sigma level for 1/5 year operation on the energy of the resonance

6 Conclusion

Our analysis showed that the hadron background in the \(b{\overline{b}}\) energy region (\(W \sim 10\) \({\mathrm {\,Ge V\!/}c^2}\)) can be suppressed by more than two orders of magnitude, which makes it possible to study C-even resonances at the \(\gamma \gamma \) collider with masses up to 12 \({\mathrm {\,Ge V\!/}c^2}\) by detecting all final particles (all hadronic decay modes together). As can be seen in Fig. 1, the region \(W=\) 3–12 \({\mathrm {\,Ge V\!/}c^2}\) is populated by many resonance states of various nature, which can be studied at the photon collider on the base of the European XFEL linac (or at any other photon collider at these energies).