1 Introduction

Quantum chromodynamics (QCD), a non-Abelian gauge field theory, predicts the existence of new types of hadrons with explicit gluonic degrees of freedom (e.g., glueballs, hybrids) [1,2,3]. The search for glueballs is an important field of research in hadron physics. It is, however, challenging since possible mixing of pure glueball states with nearby \(q{\bar{q}}\) nonet mesons makes the identification of glueballs difficult in both experiment and theory. Lattice QCD (LQCD) predicts the lowest-lying glueballs which are scalar (mass 1.5–1.7 \(\hbox {GeV}/c^2\)), tensor (mass 2.3–2.4 \(\hbox {GeV}/c^2\)), and pseudoscalar (mass 2.3–2.6 \(\hbox {GeV}/c^2\)) [4]. Radiative \(J/\psi \) decay is a gluon-rich process and it is therefore regarded as one of the most promising hunting grounds for glueballs [5, 6]. Recent LQCD calculations predict that the partial width of \(J/\psi \) radiatively decaying into the pure gauge pseudoscalar glueball is 0.0215(74) keV which corresponds to a branching ratio \(2.31(80)\times 10^{-4}\) [7]. Recently, three states, X(1835), X(2120) and X(2370), were observed in the BESIII experiment in the \(\pi ^{+}\pi ^{-}\eta '\) invariant-mass distribution through the decay of \(J/\psi \rightarrow \gamma \pi ^{+}\pi ^{-}\eta '\) with statistical significances larger than \(20\sigma \), \(7.2 \sigma \) and 6.4\(\sigma \), respectively [8]. The measured mass of X(2370) is consistent with the pseudoscalar glueball candidate predicted by LQCD calculations [4]. In the case of a pseudoscalar glueball, the branching fractions of X(2370) decaying into \(KK\eta '\) and \(\pi \pi \eta '\) are predicted to be 0.011 and 0.090 [9], respectively, in accordance with calculations that are based upon the chiral effective Lagrangian. Study on the decays to \(K{\bar{K}}\eta '\) of the glueball candidate X states is helpful to identify their natures.

In this paper, X(2370) and X(2120) are studied via the decays of \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta ' (K_{S}^{0}\rightarrow \pi ^{+}\pi ^{-})\) using \((1310.6 \pm 7.0) \times 10^6\) \(J/\psi \) decays [10] collected with the BESIII detector in 2009 and 2012. Two \(\eta '\) decay modes are used, namely \(\eta '\rightarrow \gamma \rho ^{0}(\rho ^{0}\rightarrow \pi ^{+}\pi ^{-})\) and \(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta (\eta \rightarrow \gamma \gamma )\).

2 Detector and Monte Carlo simulations

The BESIII detector is a magnetic spectrometer [11] located at the Beijing Electron Positron Collider II (BEPCII) [12]. The cylindrical core of the BESIII detector consists of a helium-based multilayer drift chamber (MDC), a plastic scintillator time-of-flight system (TOF), and a CsI(Tl) electromagnetic calorimeter (EMC), which are all enclosed in a superconducting solenoidal magnet providing a 1.0 T (0.9 T in 2012) magnetic field. The solenoid is supported by an octagonal flux-return yoke with resistive plate counter muon identifier modules interleaved with steel. The acceptance of charged particles and photons is 93% over \(4\pi \) solid angle. The charged-particle momentum resolution at \(1~\mathrm{GeV}/c\) is \(0.5\%\), and the dE/dx resolution is \(6\%\) for the electrons from Bhabha scattering. The EMC measures photon energies with a resolution of 2.5% (5%) at 1 GeV in the barrel (end cap) region. The time resolution of the TOF barrel part is 68 ps, while that of the end cap part is 110 ps.

Simulated samples produced with the geant4-based [13] Monte Carlo (MC) package which includes the geometric description of the BESIII detector and the detector response, are used to determine the detection efficiency and to estimate the backgrounds. The simulation includes the beam energy spread and initial-state radiation (ISR) in the \(e^+e^-\) annihilations modeled with the generator kkmc [14, 15]. The inclusive MC sample consists of the production of the \(J/\psi \) resonance, and the continuum processes incorporated in kkmc [14, 15]. The known decay modes are modeled with evtgen [16, 17] using branching fractions taken from the Particle Data Group [18], and the remaining unknown decays from the charmonium states are generated with lundcharm [19, 20]. The final-state radiations (FSR) from charged final-state particles are incorporated with the photos package [21]. Background is studied using a sample of \(1.2 \times 10^{9}\) simulated \(J/\psi \) events. Phase-space (PHSP) MC samples of \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) are generated to describe the nonresonant contribution. To estimate the selection efficiency and to optimize the selection criteria, signal MC events are generated for \(J/\psi \rightarrow \gamma X(2120)/X(2370) \rightarrow \gamma K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow \gamma X(2120)/X(2370)\rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) channel, respectively. The polar angle of the photon in the \(J/\psi \) center-of-mass system, \(\theta _{\gamma }\), follows a \(1+ \mathrm {cos}^{2}\theta _{\gamma }\) function. For the process of \(\eta '\rightarrow \gamma \rho ^{0}, \rho ^{0}\rightarrow \pi ^{+}\pi ^{-}\), a generator taking into account both the \(\rho \)\( \omega \) interference and the box anomaly is used [22]. The analysis is performed in the framework of the BESIII offline software system (BOSS) [23] incorporating the detector calibration, event reconstruction and data storage.

3 Event selection

Charged-particle tracks in the polar angle range \(|\cos \theta | < 0.93\) are reconstructed from hits in the MDC. Tracks (excluding those from \(K_{S}^{0}\) decays) are selected that extrapolated to be within 10 cm from the interaction point in the beam direction and 1 cm in the plane perpendicular to the beam. The combined information from energy-loss (\(\mathrm{d}E/\mathrm{d}x\)) measurements in the MDC and time in the TOF is used to obtain confidence levels for particle identification (PID) for \(\pi \), K and p hypotheses. For \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) decay, each track is assigned to the particle type corresponding to the highest confidence level; candidate events are required to have four charged tracks with zero net charge and with two opposite charged tracks identified as kaons and the other two identified as pions. For the \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) decay, each track is assumed to be a pion and no PID restrictions are applied; candidate events are required to have six charged tracks with zero net charge. \(K_{S}^{0}\) candidates are reconstructed from a secondary vertex fit to all \(\pi ^{+}\pi ^{-}\) pairs, and each \(K_{S}^{0}\) candidate is required to satisfy \(|M_{\pi ^{+}\pi ^{-}}-m_{K_{S}^{0}}|< 9~\hbox {MeV}/c^{2}\), where \(m_{K_{S}^{0}}\) is the nominal mass of \(K_{S}^{0}\) [18]. The reconstructed \(K_{S}^{0}\) candidates are used as input for the subsequent kinematic fit.

Photon candidates are required to have an energy deposition above 25 MeV in the barrel region (\(|\cos \theta |<0.80\)) and 50 MeV in the end cap (\(0.86<|\cos \theta |<0.92\)). To exclude showers from charged tracks, the angle between the shower position and the charged tracks extrapolated to the EMC must be greater than \(5^{\circ }\). A timing requirement in the EMC is used to suppress electronic noise and energy deposits unrelated to the event. At least two (three) photons are required for the \(\eta '\rightarrow \gamma \rho ^{0}\) (\(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta \)) mode.

For the \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta ' (\eta '\rightarrow \gamma \rho ^{0})\) channel, a four-constraint (4C) kinematic fit is performed to the hypothesis of \(J/\psi \rightarrow \gamma \gamma K^{+}K^{-}\pi ^{+}\pi ^{-}\) by requiring the total energy and each momentum component to be conserved. For events with more than two photon candidates, the combination with the minimum \(\chi _{4C}^{2}\) is selected, and \(\chi _{4C}^{2}< 25\) is required. Events with \(|M_{\gamma \gamma } - m_{\pi ^{0}}| < 30~\hbox {MeV}/c^{2}\) or \(|M_{\gamma \gamma } - m_{\eta }| <~30~\hbox {MeV}/c^{2}\) are rejected to suppress background containing \(\pi ^{0}\) or \(\eta \), where \(m_{\pi ^{0}}\) and \(m_{\eta }\) are the nominal masses of \(\pi ^{0}\) and \(\eta \) [18]. A clear \(\eta '\) signal is observed in the invariant-mass distribution of \(\gamma \pi ^{+}\pi ^{-}\) (\(M_{\gamma \pi ^{+}\pi ^{-}}\)), as shown in Fig. 1(a). Candidates of \(\rho \) and \(\eta '\) are reconstructed from the \(\pi ^{+}\pi ^{-}\) and \(\gamma \pi ^{+}\pi ^{-}\) combinations with \(0.55~\hbox {GeV}/c^{2}< M_{\pi ^{+}\pi ^{-}} < 0.85~\hbox {GeV}/c^{2}\) and \(|M_{\gamma \pi ^{+}\pi ^{-}} - m_{\eta '}| < 20~\hbox {MeV}/c^{2}\), where \(m_{\eta '}\) is the nominal mass of \(\eta '\) [18], respectively. If there is more than one combination satisfying the selection criteria, the combination with \(M_{\gamma \pi ^{+}\pi ^{-}}\) closest to \(m_{\eta '}\) is selected. After applying the above requirements, we obtain the invariant-mass distribution of \(K^{+}K^{-}\eta '\) (\(M_{K^{+}K^{-}\eta '}\)) as shown in Fig. 1(b). The peak around \(2.98 \, {\hbox {GeV/c}}^2\) is contributed from the decay of \(J/\psi \rightarrow \gamma \eta _c (\eta _c\rightarrow K^+K^-\eta ^\prime )\), while the peak around the right threshold is mainly from the background events of \(J/\psi \rightarrow K^+K^-\eta ^\prime \).

Fig. 1
figure 1

Invariant-mass distributions for the selected candidates of \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\). Plots a, b are invariant-mass distributions of \(\gamma \pi ^{+}\pi ^{-}\) and \(K^{+}K^{-}\eta '\) for \(\eta '\rightarrow \gamma \rho ^{0}\), \(\rho ^{0}\rightarrow \pi ^{+}\pi ^{-}\), respectively; plots c, d are the invariant-mass distributions of \(\pi ^{+}\pi ^{-}\eta \) and \(K^{+}K^{-}\eta '\) for \(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta \), \(\eta \rightarrow \gamma \gamma \), respectively. The dots with error bars correspond to data and the histograms are the results of PHSP MC simulations (arbitrary normalization)

To reduce background and to improve the mass resolution of the \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta ' (\eta '\rightarrow \pi ^{+}\pi ^{-}\eta )\) channel, a five-constraint (5C) kinematic fit is performed whereby the total four momenta of the final-state particles are constrained by the total initial four momentum of the colliding beams and the invariant mass of the two photons from the decay of \(\eta \) is constrained by its nominal mass. If there are more than three photon candidates, the combination with the minimum \(\chi _{5C}^{2}\) is retained, and \(\chi _{5C}^{2} < 45\) is required. To suppress background from \(\pi ^{0}\rightarrow \gamma \gamma \), \(|M_{\gamma \gamma } - m_{\pi ^{0}}| > 30~\hbox {MeV}/c^{2}\) is required for all photon pairs. The \(\eta '\) candidates are formed from the \(\pi ^{+}\pi ^{-}\eta \) combination satisfying \(|M_{\pi ^{+}\pi ^{-}\eta } - m_{\eta '}| < 15 \hbox {MeV}/c^{2}\), where \(M_{\pi ^{+}\pi ^{-}\eta }\) is the invariant mass of \(\pi ^{+}\pi ^{-}\eta \), as shown in Fig. 1c. After applying the mass restrictions, we obtain the invariant-mass distribution of \(K^{+}K^{-}\eta '(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta )\) as shown in Fig. 1d.

For the \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta ' (\eta '\rightarrow \gamma \rho ^{0})\) channel, the \(\gamma \gamma K_{S}^{0}K_{S}^{0}\pi ^+\pi ^-\) candidates are subjected to a 4C kinematic fit. For events with more than two photons or two \(K^{0}_{S}\) candidates, the combination with the smallest \(\chi ^{2}_{4C}\) is retained, and \(\chi ^{2}_{4C} < 45\) is required. To suppress background events containing a \(\pi ^{0}\) or \(\eta \), events with \(|M_{\gamma \gamma } - m_{\pi ^{0}}| < 30~\hbox {MeV}/c^{2}\) or \(|M_{\gamma \gamma } - m_{\eta }| < 30~\hbox {GeV}/c^{2}\) are rejected. The \(\pi ^{+}\pi ^{-}\) invariant mass is required to be in the \(\rho \) mass region, \(0.55~\hbox {GeV}/c^{2}< M_{\pi ^{+}\pi ^{-}} < 0.85~\hbox {GeV}/c^{2}\), and \(|M_{\gamma \pi ^{+}\pi ^{-}} - m_{\eta '}| < 20~\hbox {MeV}/c^{2}\) is applied to select the \(\eta '\) signal. If more than one combination of \(\gamma \pi ^{+}\pi ^{-}\) is obtained, the combination with \(M_{\gamma \pi ^{+}\pi ^{-}}\) closest to \(m_{\eta '}\) is selected as shown in Fig. 2a. After applying the above requirements, we obtain the \(K_{S}^{0}K_{S}^{0}\eta ' (\eta '\rightarrow \gamma \rho ^{0})\) invariant-mass spectrum as illustrated in Fig. 2b.

Fig. 2
figure 2

Invariant-mass distributions for the selected \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) candidate events. a, b Invariant-mass distributions of \(\gamma \pi ^{+}\pi ^{-}\) and \(K_{S}^{0}K_{S}^{0}\eta '\) for \(\eta '\rightarrow \gamma \rho ^{0}\), \(\rho ^{0}\rightarrow \pi ^{+}\pi ^{-}\), respectively; c, d Invariant-mass distribution of \(\pi ^{+}\pi ^{-}\eta \) and \(K_{S}^{0}K_{S}^{0}\eta '\) for \(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta \), \(\eta \rightarrow \gamma \gamma \), respectively. The dots with error bars represent the data and the histograms are the results of PHSP MC simulations (arbitrary normalization)

Candidate events of the \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) \((\eta '\rightarrow \pi ^{+}\pi ^{-}\eta )\) channel are subjected to a 5C kinematic fit, which is similar to that for the \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) \((\eta '\rightarrow \pi ^{+}\pi ^{-}\eta )\) mode. If there are more than three photons or more than two \(K^{0}_{S}\) candidates, only the combination with the minimum \(\chi _{5C}^{2}\) is selected and \(\chi _{5C}^{2} < \)50 is required. To reduce the combinatorial background from \(\pi ^{0}\rightarrow \gamma \gamma \) events, \(|M_{\gamma \gamma } - m_{\pi ^{0}}| > 30~\hbox {MeV}/c^{2}\) is required for all photon pairs. For selecting the \(\eta '\) signal, the \(\pi ^{+}\pi ^{-}\eta \) combination satisfying \(|M_{\pi ^{+}\pi ^{-}\eta } - m_{\eta '}| < 15~\hbox {MeV}/c^{2}\) is required, as shown in Fig. 2c. After applying the above selection criteria, we obtain the invariant-mass distribution of \(K_{S}^{0}K_{S}^{0}\eta ' (\eta '\rightarrow \pi ^{+}\pi ^{-}\eta )\) events as shown in Fig. 2d.

4 Signal extraction

Potential backgrounds are studied using an inclusive MC sample of \(1.2\times 10^{9} \, J/\psi \) decays. No significant peaking background is identified in the invariant-mass distributions of \(K^{+}K^{-}\eta '\) and \(K_{S}^{0}K_{S}^{0}\eta '\). Non-\(\eta '\) processes are studied using the \(\eta '\) mass sidebands. The major background in the decay \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) stem from \(J/\psi \rightarrow K^{*+} K^{-}\eta ' (K^{*+}\rightarrow K^{+}\pi ^{0}) + c.c.\). The contribution of \(J/\psi \rightarrow K^{*+} K^{-}\eta ' (K^{*+}\rightarrow K^{+}\pi ^{0}) + c.c.\) is estimated by the background-subtracted \(K^{+}K^{-}\eta '\) spectrum of \(J/\psi \rightarrow K^{*+} K^{-}\eta ' (K^{*+}\rightarrow K^{+}\pi ^{0}) + c.c.\) events selected from data. The spectrum is reweighted according to the ratio of efficiency of \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow K^{*+} K^{-}\eta ' (K^{*+}\rightarrow K^{+}\pi ^{0}) + c.c.\). For the \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) case, backgrounds from the process \(J/\psi \rightarrow \pi ^{0}K_{S}^{0}K_{S}^{0}\eta '\) are negligible, as it is forbidden due to charge conjugation invariance.

A structure near \(2.34 \, \hbox {GeV}/c^{2}\) is observed in the invariant-mass distribution of \(K^{+}K^{-}\eta '\) and \(K_{S}^{0}K_{S}^{0}\eta '\). We performed a simultaneous unbinned maximum-likelihood fit to the \(K^{+}K^{-}\eta '\) and \(K_{S}^{0}K_{S}^{0}\eta '\) invariant-mass distributions between 2.0 and \(2.7 \, \hbox {GeV}/c^{2}\), as shown in Fig 3. The signal is represented by an efficiency-weighted non-relativistic Breit–Wigner (BW) function convolved with a double Gaussian function to account for the mass resolution. The mass and width of the BW function are left free in the fit, while the parameters of the double Gaussian function are fixed on the results obtained from the fit of signal MC samples generated with zero width. The non-\(\eta '\) background events are described with \(\eta '\) sideband data and the yields from these sources are fixed; the \(J/\psi \rightarrow K^{*+}K^{-}\eta ' + c.c.\) contributions in the \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) decay channel are studied as discussed above and the shapes and the yields are fixed in the fit; the contribution from the nonresonant \(\gamma K{\bar{K}}\eta '\) production is described by the shape from the PHSP MC sample of \(J/\psi \rightarrow \gamma K{\bar{K}} \eta '\) and its absolute yield is set as a free parameter in the fit; the remaining background is described by a second order Chebychev polynomial function and its parameters are left to be free. In the simultaneous fit, the resonance parameters are free parameters and constrained to be the same for all four channels. The signal ratio for the two \(\eta '\) decay modes is fixed with a factor calculated by their branching fractions and efficiencies. The signal ratio between \(J/\psi \rightarrow \gamma X(2370)\rightarrow \gamma K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow \gamma X(2370)\rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) is a free parameter in the fit. The obtained mass, width and the number of signal events for X(2370) are listed in Table 1. A variety of fits with different fit ranges, \(\eta '\) sideband regions and background shapes are performed; after considering the systematic uncertainties like quantum number of X(2370) and the presence of X(2120)*, the smallest statistical significance among these fits is found to be \(8.3\sigma \). With the detection efficiencies listed in Table 2, the product branching fractions for \(J/\psi \rightarrow \gamma X(2370),X(2370)\rightarrow K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow \gamma X(2370),X(2370)\rightarrow K_{S}^{0}K_{S}^{0}\eta '\) are determined to be \((1.79\pm 0.23)\times 10^{-5}\) and \((1.18\pm 0.32)\times 10^{-5}\), respectively, where the uncertainties are statistical only.

Fig. 3
figure 3

The fit result for X(2370) in the invariant-mass distribution of \(K {{\bar{K}}} \eta '\) for the decays: a \(J/\psi \rightarrow \gamma X(2370),X(2370)\rightarrow \gamma K^{+}K^{-}\eta ', \eta '\rightarrow \pi ^{+}\pi ^{-}\eta \), \(\eta \rightarrow \gamma \gamma \), b \(J/\psi \rightarrow \gamma X(2370),X(2370)\rightarrow \gamma K^{+}K^{-}\eta ', \eta '\rightarrow \gamma \rho ^{0}, \rho ^{0}\rightarrow \pi ^{+}\pi ^{-}\), c \(J/\psi \rightarrow \gamma X(2370),X(2370)\rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta ', \eta '\rightarrow \pi ^{+}\pi ^{-}\eta \), \(\eta \rightarrow \gamma \gamma \), and d \(J/\psi \rightarrow \gamma X(2370),X(2370)\rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta ', \eta '\rightarrow \gamma \rho ^{0}, \rho ^{0}\rightarrow \pi ^{+}\pi ^{-}\). The dots with error bars represent the data; the solid curves show the fit results; the grid areas represent the signal of X(2370); the dotted lines are the background shapes from \(J/\psi \rightarrow K^{*+}K^{-}\eta '+c.c.\); the short dashed double dotted lines show the \(\eta '\) sidebands; the long dashed lines represent the Chebychev polynomial function; the gray short dashed lines are the contribution from PHSP MC and the dashed dotted lines show the sum of all backgrounds

No obvious signal of X(2120) is found in the \(K{\bar{K}}\eta '\) invariant-mass distribution. We performed a simultaneous unbinned maximum-likelihood fit to the \(K{\bar{K}}\eta '\) invariant-mass distribution in the range of \([2.0, 2.7] \, \hbox {GeV}/c^2\). The signal, X(2120), is described with an efficiency-weighted BW function convolved with a double Gaussian function. The mass and width of the BW function are fixed to previously published BESIII results [8]. The backgrounds are modeled with the same components as used in the fit of X(2370) as mentioned above. The contribution from X(2370) is included in the fit and its mass, width and the numbers of events are set free. The distribution of normalized likelihood values for a series of input signal event yields is taken as the probability density function (PDF) for the expected number of events. The number of events at 90% of the integral of the PDF from zero to the given number of events is defined as the upper limit, \(N^{UL}\), at the 90% confidence level (CL). We repeated this procedure with different signal shape parameters of X(2120) (by varying the values of mass and width with \(1\sigma \) of the uncertainties cited from [8]), fit ranges, \(\eta '\) sideband regions and background shapes, and the maximum upper limit among these cases is selected. The statistical significance of X(2120) is determined to be \(2.2\sigma \). To calculate \(N^{UL}\) for the \(J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K^{+}K^{-}\eta ' \, (J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta ')\) channel, the number of signal events for \(J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta ' \, (J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K^{+}K^{-}\eta ')\) channel is left free. The obtained upper limits of the signal yields are listed in Table 1, and the upper limit for the product branching fractions are calculated to be \({\mathcal {B}}(J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K^{+} K^{-} \eta ') < 1.41\times 10^{-5}\) and \({\mathcal {B}}(J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K_{S}^{0} K_{S}^{0} \eta ') < 6.15\times 10^{-6}\), respectively.

Table 1 Fit results for the structure around \(2.34 \, \hbox {GeV}/c^{2}\) and \(2.12 \, \hbox {GeV}/c^{2}\). The superscripts a and b represent the decay modes of \( X\rightarrow K^{+}K^{-}\eta '\) and \(X\rightarrow K_{S}^{0}K_{S}^{0}\eta '\), respectively. The uncertainties are statistical only
Table 2 Summary of the MC detection efficiencies of the signal yields for the two \(\eta '\) modes where the \(K{\bar{K}} \eta '\) invariant mass is constrained to the applied fitting range between 2.0 and \(2.7 \, \hbox {GeV}/c^{2}\). The superscripts a and b represent the decay modes of \( X\rightarrow K^{+}K^{-}\eta '\) and \(X\rightarrow K_{S}^{0}K_{S}^{0}\eta '\), respectively
Table 3 Absolute systematic uncertainties of resonance parameters of mass (M, in \(\hbox {MeV}/c^{2}\)) and width (\(\Gamma \), in MeV) for X(2370). The items with * are common uncertainties of both \(\eta '\) decay modes

5 Systematic uncertainties

Several sources of systematic uncertainties are considered for the determination of the mass and width of X(2370) and the product branching fractions. These include the efficiency differences between data and MC simulation in the MDC tracking, PID, the photon detection, \(K_{S}^{0}\) reconstruction, the kinematic fitting, and the mass-window requirements of \(\pi ^{0}\), \(\eta \), \(\rho \) and \(\eta '\). Furthermore, uncertainties associated with the fit ranges, the background shapes, the sideband regions, the signal shape parameters of X(2120), intermediate resonance decay branching fractions and the total number of \(J/\psi \) events are considered.

5.1 Efficiency estimation

The MDC tracking efficiencies of charged pions and kaons are investigated using nearly background-free (clean) control samples of \(J/\psi \rightarrow p{\bar{p}}\pi ^{+}\pi ^{-}\) and \(J/\psi \rightarrow K_{S}^{0}K^{\pm }\pi ^{\mp }\) [24, 25], respectively. The difference in tracking efficiencies between data and MC is 1.0% for each charged pion and kaon. The photon detection efficiency is studied with a clean sample of \(J/\psi \rightarrow \rho ^{0}\pi ^{0}\) [26], and the result shows that the difference of photon detection efficiencies between data and MC simulation is 1.0% for each photon. The systematic uncertainty from \(K_{S}^{0}\) reconstruction is determined from the control samples of \(J/\psi \rightarrow K^{*\pm }K^{\mp }\) and \(J/\psi \rightarrow \phi K_{S}^{0}K^{\pm }\pi ^{\mp }\), which indicates that the efficiency difference between data and MC is less than 1.5\(\%\) for each \(K_{S}^{0}\). Therefore, 3.0\(\%\) is taken as the systematic uncertainty for the two \(K_{S}^{0}\) in \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) channel.

For the decay channel of \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\), the PID has been used to identify the kaons and pions. Using a clean sample of \(J/\psi \rightarrow p{\bar{p}}\pi ^{+}\pi ^{-}\), the PID efficiency of \(\pi ^{+}/\pi ^{-}\) has been studied, which indicates that the \(\pi ^{+}/\pi ^{-}\) PID efficiency for data agrees with MC simulation within 1%. The PID efficiency for the kaon is measured with a clean sample of \(J/\psi \rightarrow K^+ K^-\eta \). The difference of the PID efficiency between data and MC is less than 1% for each kaon. Hence, in this analysis, four charged tracks are required to be identified as two pions and two kaons, and 4% is taken as the systematic uncertainty associated with the PID.

The systematic uncertainties associated with the kinematic fit are studied with the track helix parameter correction method, as described in Ref. [27]. The differences from those without corrections are taken as systematic uncertainties.

Due to the difference in the mass resolution between data and MC, uncertainties related to the \(\rho ^{0}\) and \(\eta '\) mass-window requirements are investigated by smearing the MC simulation to improve the consistency between data and MC simulation. The differences in the detection efficiency before and after smearing are assigned as systematic uncertainties for the \(\rho ^{0}\) and \(\eta '\) mass-window requirements. The uncertainties from the \(\pi ^{0}\) and \(\eta \) mass-window requirements are estimated by varying the mass windows of \(\pi ^{0}\) and \(\eta \), and differences in the resulting branching fractions are assigned as the systematic uncertainties of this item.

Furthermore, we considered the effects arising from different quantum numbers of X(2120) and X(2370). We generated \(J/\psi \rightarrow \gamma X(2120)\) and \(J/\psi \rightarrow \gamma X(2370)\) decays following a \(\mathrm {sin}^2\theta _{\gamma }\) angular distribution. The resulting differences in efficiency from the nominal value are taken as systematic uncertainties.

Table 4 Systematic uncertainties for determination of the branching fraction of \(J/\psi \rightarrow \gamma X(2370)\rightarrow \gamma K {\bar{K}} \eta '\) (in %). The items with * are common uncertainties of both \(\eta '\) decay modes. I and II represent the decay modes of \(\eta '\rightarrow \gamma \rho ^{0}\), \(\rho ^{0}\rightarrow \pi ^{+}\pi ^{-}\) and \(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta \), \(\eta \rightarrow \gamma \gamma \), respectively
Table 5 Systematic uncertainties for the determination of the upper limit of the branching fraction of \(J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K {\bar{K}} \eta '\)(in %). The items with * are common uncertainties of both \(\eta '\) decay modes. I and II represent the decay modes of \(\eta '\rightarrow \gamma \rho ^{0}\), \(\rho ^{0}\rightarrow \pi ^{+}\pi ^{-}\) and \(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta \), \(\eta \rightarrow \gamma \gamma \), respectively

5.2 Fit to the signal

To study the uncertainties from the fit range and \(\eta '\) sideband region, the fits are repeated with different fit ranges and sideband regions, the largest differences among these signal yields are taken as systematic uncertainties, respectively. To estimate the uncertainties in the description of various background contributions, we performed alternative fits with third-order Chebychev polynomials modeling the background of the \(K^{+}K^{-}\eta '\) and \(K_{S}^{0}K_{S}^{0}\eta '\) channels. The maximum differences in signal yield from the nominal fit are taken as systematic uncertainties. The uncertainties from the background of \(J/\psi \rightarrow K^{*+}K^{-}\eta ' + c.c.\) are estimated by absorbing this component into a Chebychev polynomial function, and the differences obtained by using the description with or without the background component of \(J/\psi \rightarrow K^{*+}K^{-}\eta ' + c.c.\) are taken as systematic uncertainties. The impact of X(2120) is also considered as a systematic uncertainty in the study of X(2370). The difference between a fit with and without a X(2120) contribution is taken as a systematic uncertainty associated to this item.

5.3 Others

Since no evident structures are observed in the invariant-mass distributions of \(M(K\eta ')\), \(M({\bar{K}}\eta ')\) and \(M(K{\bar{K}})\) for the events with a \(K{\bar{K}}\eta '\) invariant mass within the X(2370) mass region (\(2.2~\hbox {GeV}/c^{2}< M_{K{\bar{K}}\eta '} < 2.5~\hbox {GeV}/c^{2}\)), the systematic uncertainties of the reconstruction efficiency due to the possible intermediate states on the \(K\eta '\), \({\bar{K}}\eta '\) and \(K{\bar{K}}\) mass spectra are ignored. The uncertainties on the intermediate decay branching fractions of \(\eta '\rightarrow \gamma \rho ^{0}\rightarrow \gamma \pi ^{+}\pi ^{-}\), \(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta \), \(\eta \rightarrow \gamma \gamma \) and \(K_{S}^{0}\rightarrow \pi ^{+}\pi ^{-}\) are taken from the world average values [18], which are \(1.7\%\), \(1.6\%\), \(0.5\%\) and \(0.1\%\), respectively. The systematic uncertainty due to the number of \(J/\psi \) events is determined as 0.5% according to Ref. [10].

A summary of all the uncertainties is shown in Tables 34 and 5. The total systematic uncertainties are obtained by adding all individual uncertainties in quadrature, assuming all sources to be independent.

X(2120) and X(2370) are studied via \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) with two \(\eta '\) decay modes, respectively. The measurements from the two \(\eta '\) decay modes are, therefore, combined by considering the difference in uncertainties of these two measurements. The combined systematic uncertainties are calculated with the weighted least squares method [28] and the results are shown in Table 6.

Table 6 Combined results of the structure around \(2.34\, \hbox {GeV}/c^{2}\), the measured branching fractions and the upper limits

6 Results and summary

Using a sample of \(1.31\times 10^{9} ~J/\psi \) events collected with the BESIII detector, the decays of \(J/\psi \rightarrow \gamma K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow \gamma K_{S}^{0}K_{S}^{0}\eta '\) are investigated using the two \(\eta '\) decay modes, \(\eta '\rightarrow \gamma \rho ^{0}(\rho ^{0}\rightarrow \pi ^{+}\pi ^{-})\) and \(\eta '\rightarrow \pi ^{+}\pi ^{-}\eta ( \eta \rightarrow \gamma \gamma )\). X(2370) is observed in the \(K{\bar{K}}\eta '\) invariant-mass distribution with a statistical significance of \(8.3\sigma \). The mass and width are determined to be

$$\begin{aligned}&M_{X(2370)}=2341.6\pm 6.5\,\text {(stat.)}\pm 5.7\,\text {(syst.)} \, \hbox {MeV}/c^{2}, \\&\Gamma _{X(2370)}=117\pm 10\,\text {(stat.)}\pm 8\,\text {(syst.)} \, \hbox {MeV}, \end{aligned}$$

which are found to be consistent with those of X(2370) observed in the previous BESIII results [8]. The product branching fractions of \({\mathcal {B}}(J/\psi \rightarrow \gamma X(2370)\rightarrow \gamma K^{+} K^{-} \eta ')\) and \({\mathcal {B}}(J/\psi \rightarrow \gamma X(2370)\rightarrow \gamma K_{S}^{0} K_{S}^{0} \eta ')\) are measured to be \((1.79\pm 0.23~\text {(stat.)}\pm 0.65~\text {(syst.)})\times 10^{-5}\) and \((1.18\pm 0.32~\text {(stat.)}\pm 0.39~\text {(syst.)})\times 10^{-5}\), respectively. No evident signal for X(2120) is observed in the \(K{\bar{K}}\eta '\) invariant-mass distribution. For a conservative estimate of the upper limits of the product branching fractions of \(J/\psi \rightarrow \gamma X(2120)\rightarrow K^{+}K^{-}\eta '\) and \(J/\psi \rightarrow \gamma X(2120)\rightarrow K_{S}^{0}K_{S}^{0}\eta '\), the multiplicative uncertainties are considered by convolving the normalized likelihood function with a Gaussian function. The upper limits for product branching fractions at 90% C. L. are determined to be \({\mathcal {B}}(J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K^{+} K^{-} \eta ') < 1.49\times 10^{-5}\) and \({\mathcal {B}}(J/\psi \rightarrow \gamma X(2120)\rightarrow \gamma K_{S}^{0} K_{S}^{0} \eta ') < 6.38\times 10^{-6}\).

To understand the nature of X(2120) and X(2370), it is critical to measure their spin and parity and to search for them in more decay modes. A partial-wave analysis is needed to measure their masses and widths more precisely, and to determine their spin and parity. This might become possible in the future with the foreseen higher statistics of \(J/\psi \) data samples.