New-physics signatures via $C\!P$ violation in $\eta^{(\prime)}\to\pi^0\mu^+\mu^-$ and $\eta^\prime\to\eta\mu^+\mu^-$ decays

In this work we investigate the prospect of observing new-physics signatures via $C\!P$ violation in $\eta^{(\prime)}\to\pi^0\mu^+\mu^-$ and $\eta^\prime\to\eta\mu^+\mu^-$ decays at the REDTOP experiment. We make use of the SMEFT to parametrise the new-physics $C\!P$-violating effects and find that the projected REDTOP statistics are not competitive with respect to nEDM experiments. This reasserts the $\eta\to\mu^+\mu^-$ process as the most promising channel to find $C\!P$-violation at this experimental facility.


Introduction
Over the past few decades, high-energy particle colliders have not succeeded in the quest for finding evidence for physics beyond the Standard Model (BSM). The purpose of large experiments such as the Large Hadron Collider (LHC), apart from settling down the question around the source of electroweak symmetry breaking, was to provide experimental evidence for either supersymmetric particles or extra dimensions or both, as they enjoy from strong theoretical motivation based on naturalness arguments, but this has not happened.
The current lack of experimental evidence for new physics in direct searches, that would help guide theoretical effort, is forcing the community to increase their focus on low-energy, high luminosity precision measurements that attempt to find effects from BSM physics by looking for small discrepancies between SM predictions and measurements. To this end, one focuses on processes whose SM contribution is very precisely known or that have a very small SM background, hence, any positive experimental finding would be a confirmation for new physics. Accordingly, interest in BSM searches in meson factories has significantly increased in recent years, 1 as they can very precisely measure branching ratios of rare decays and test for violations of the basic symmetries. As an example, the observation of CP violation in processes mediated by the strong or electromagnetic interactions would be an unambiguous sign of new physics and the study of the η and η decays represents the perfect laboratory for this endeavour. This is because both mesons are eigenstates of the C, P , CP and G operators (i.e. I G J P C = 0 + 0 −+ ) and their additive quantum numbers are zero, which amounts to all their decays being flavour conserving. As a consequence, and unlike flavoured meson decays, they can be used to test C and CP symmetries, provided a large sample of η and η mesons is available. Furthermore, their strong and electromagnetic decays are forbidden at lowest order, increasing their sensitivity to rare decays.
In this context, a new experiment named REDTOP has been proposed [2,3], which aims at producing the largest sample of η and η mesons envisaged thus far, and is considering implementing dedicated detectors to perform muon polarimetry. In order to set their priorities, it is crucial to assess the most promising channels and the physics within reach. In Ref. [4], the possibility of observing new physics signatures via CP -violating effects at REDTOP was assessed using muon polarisation observables in η leptonic decays. In particular, the purely leptonic channels µ + µ − , µ + µ − γ, and µ + µ − + − were studied, finding that CP violation in the µ + µ − final state could be observed at REDTOP, while evading neutron electric dipole moment (nEDM) constraints.
In the present work, we investigate the suitability of some η and η semileptonic decays, which were not covered in the previous study as they require a dedicated analysis of hadronic matrix elements. In particular, we investigate the η ( ) → π 0 µ + µ − and η → ηµ + µ − decays using the SM effective field theory (SMEFT) as the general framework to capture new physics. 2 Using muon polarisation observables, we quantify the sensitivity that could be achieved at REDTOP for the relevant CP -violating Wilson coefficients. Our results show that these decays are not competitive when confronted against the stringent bounds derived from nEDM and D − s → µν µ decays. This contrasts with the η → µ + µ − decay that evades these bounds and ought to receive the highest priority.
The article is structured as follows. In Section 2, we discuss the general properties of the decay amplitudes and narrow down the range of SMEFT operators that are relevant to our study. In Section 3, we present the theoretical expressions for the required hadronic matrix elements obtained using large-N c chiral perturbation theory (LN c χPT). The polarised decay widths and the asymmetries that quantify the CP -violating effects in η ( ) → π 0 µ + µ − and η → ηµ + µ − decays are analysed in Section 4. The results from our investigation are presented in Section 5 and we briefly discuss their implications. Finally, in Section 6 we provide a summary of the work carried out and some final conclusions.
Turning to the BSM CP -violating contribution, which requires a careful study of the underlying hadron dynamics, one starts by assuming that the SMEFT provides a correct description of Nature. Accordingly, new physics degrees of freedom are expected to lie above the electroweak scale and, therefore, only the SM particle spectra are considered. In addition, the new-physics effects come from higher dimension operators, that are suppressed by increasing powers of a large energy scale, starting with D = 6 so long as B-L number conservation is assumed. In particular, the contribution from the different operators were outlined in Ref. [4] and we briefly recapitulate here. Quark and lepton EDM operators are highly constrained by nEDM bounds; likewise, CP violation in the hadronic sector requires CP -violating form factors with an additional electromagnetic α suppression, required in order to couple hadrons to leptons, which renders any such contribution not competitive. In addition, vector, axial and tensor operators have a vanishing coupling to the η ( ) π 0 and η η systems based on discrete symmetries. Finally, Fermi operators involving quarks and leptons provide the most significant contribution and, thus, the operators that are considered in this study are 3 O where prst are family indices (i.e. p, r, s, t = 1 or 2) [7]. These operators produce a non-vanishing CP -odd F 2 form factor 4 where v 2 = 1/( √ 2G F ) and the corresponding hadronic matrix elements need a careful treatment that we discuss in the following section within the framework of LN c χPT. To conclude this section, it is worth highlighting that at this order in the SMEFT there is no contribution to F 4 .
Before concluding this section, two remarks are in order: first, the matrix elements of the strange quark scalar current with η ( ) π 0 are suppressed by the isospin symmetry-breaking parameter 13 and, second, the contribution of theΛ parameter is in general significant for the matrix elements involving the η .

Polarised decays and asymmetries
Let us now compute the squared amplitude from Eq. (1), |M(λn,λn)| 2 , for the polarised decays that we are investigating. Using the conventions for the kinematics and the phase space given in Appendix A, and neglecting any contribution from the F 4 form factor as already mentioned at the end of Section 2, we find where n(n) is the µ + (µ − ) spin-polarisation axis defined in the µ ± rest frames and λ = ± denotes the two spin states. Note that terms in the first and second lines are CP conserving and violating, respectively, provided that F 1,2 ≡ F 1,2 (s, [q · k] 2n ) and F 3 ≡ F 3 (s, [q · k] 2n+1 ). The coefficients in Eq. (8) are given in Appendix B and are the necessary input for implementation in the Geant4 software [23]. The polarisation of the muons, however, cannot be directly measured and must be inferred from the velocities of the e ± associated to the corresponding µ ± decays. Using the expressions provided in Appendix B and making use of the spin-density formalism [24], one finds where the 3-body phase-space description from Appendix A has been employed for the initial η ( ) → π 0 µ + µ − and η → ηµ + µ − decays, and the first two brackets account for the phase space of the subsequent µ ± decays, cf. Appendix C. The coefficients in Eq. (9) are calculated from those in Appendix B and read c I 23 = 4λ where we have used the shorthand notation b(x) ≡ b and b(x) ≡b. As expected, integration over dΩdΩ results in the vanishing of all the terms involving spin correlations. Next, we make use of the identity dΩ/(4π)n(x)dx = 1, which allows one to write the total decay width as In order to quantify the CP -violating effects, one needs to construct the appropriate asymmetries that arise as a result of the interference of the SM CP -even and the SMEFT CP -odd amplitudes. Accordingly, we define the longitudinal and transverse asymmetries as follows 6 where the polar angles θ e ± refer to those of the e ± in the µ ± rest frames, φ(φ) correspond to the azimuthal e ± angles in the µ ± rest frames, and N refers to the number of η ( ) decays. It is important to highlight that only the terms associated toc R,I 12 andc R,I 23 contribute to the above asymmetries.

Results and discussion
In this section, we present quantitative results for the longitudinal and transverse asymmetries by plugging the theoretical expressions for F 1 and F 3 from Ref. [6] and the hadronic matrix elements from Section 3, required to compute F 2 , into Eqs. (20) and (21). The asymmetries for the three semileptonic processes read where the error quoted accounts for both the numerical integration and the model-dependence 7 uncertainties, with the latter strongly dominating over the former. Next, in order to assess the sensitivity to new physics, one starts by estimating the expected number of events at REDTOP, which can be obtained from the projected statistics 8 of 5 × 10 12 η/yr and 5 × 10 10 η /yr, and the SM branching ratios for the three muonic semileptonic processes from Ref. [6]. Accordingly, the estimated (statistical) SM backgrounds at the 1σ level, which can be assessed using σ = 1/ √ N , are found to be σ η→π 0 µ + µ − = 1.35 × 10 −2 , σ η →π 0 µ + µ − = 0.105 and σ η →ηµ + µ − = 0.354. It is now straightforward to estimate the REDTOP sensitivity to each of the SMEFT CP -violating Wilson coefficients from Eq. (2) by setting to zero two out of the three coefficients in Eqs. (22)(23)(24)(25)(26)(27). The corresponding results for the three decays studied in this work are summarised in Table 1. We also show in this table the REDTOP sensitivity to the same coefficients from η → µ + µ − [4], as well as the bounds set by the nEDM experiment using the most recent measurements from Ref. [27] (the bounds derived from D − s → µν µ decays are weaker and, thus, we do not quote them [26]). It must be highlighted that, strictly speaking, the nEDM experiment sets bounds on a particular linear combination of the three Wilson coefficients, which raises the question about possible cancellations that may weaken the nEDM bounds. From Eqs. (4.17) and (4.20) in Ref. [4], one can clearly see that partial cancellations are possible for c (1)2211 equ ∼ c 2211 edq , which would weaken the nEDM bounds by an order of magnitude. 9 Even in such scenario, REDTOP would still not be competitive.
Clearly, the most competitive observable amongst those studied in this work is the longitudinal asymmetry of the η → π 0 µ + µ − decay. As well as this, it can be seen that the constraints imposed by the η semileptonic decays are comparatively much weaker, which is down to the η REDTOP projected statistics being two orders of magnitude smaller than that of the η. If one compares the sensitivities obtained from the η ( ) → π 0 µ + µ − and η → ηµ + µ − decays to the CP -violating Wilson coefficients with the bounds extracted from nEDM experiments, one must conclude that the projected REDTOP statistics are not competitive enough for the above semileptonic processes, which can be attributed to the isospin-breaking suppression in the hadronic matrix elements, subject to the assumption that new physics can be parametrised by the SMEFT. Consequently, the leptonic η → µ + µ − decay studied in Ref. [4] remains the most promising channel to be studied at REDTOP.

Conclusions
In this work, we have analysed in detail possible effects of physics BSM via CP violation in η ( ) → π 0 µ + µ − and η → ηµ + µ − decays. This is particularly timely at present as the REDTOP experiment is studying the possibility of using polarisation techniques to study CP -violating newphysics effects. Assuming that BSM CP -violation appears in Nature via new heavy degrees of freedom, the use of the SMEFT is justified, which in turn provides a convenient connection to different observables, such as those from nEDM experiments and D − s → µν µ decays. The outcome of the present work is that the predicted statistics at REDTOP will fall short to detect any 7 In particular, we use the difference between the LNc χPT LO and NLO results as an estimation for the residual error associated to truncating the perturbative series, which in turn is used to, rather conservatively, quantify the error corresponding to the model. 8 A total production of 2.5×10 13 η/yr and 2.5×10 11 η /yr is expected [25], with assumed reconstruction efficiencies of approximately 20% [26]. 9 More drastic cancellations would require what it seems to us a large degree of fine-tuning. Furthermore, it seems unlikely that these cancellations would remain stable at higher order corrections. Table 1: Summary of REDTOP sensitivities to (the imaginary parts of) the Wilson coefficients associated to the SMEFT CP -violating operators in Eq. (2) for the processes studied in this work, as well as the η → µ + µ − decay analysed in Ref. [4]. In addition, the upper bounds from nEDM experiments are given in the last row for comparison purposes.

Process
Asymmetry Im c CP -violating effects in the semileptonic η ( ) → π 0 µ + µ − and η → ηµ + µ − decays, should one take into account the constraints set by nEDM and D − s → µν µ . This stands in stark contrast with the η → µ + µ − decay studied in Ref. [4] and can be understood by the fact that the less constrained strange quark contribution (cf. Table 1) is of isospin-breaking origin, which is very small in Nature. Accordingly, the leptonic η → µ + µ − decay is still the most promising channel to be investigated at REDTOP in search of new-physics signatures via CP -violating effects using muon polarimetry.

A Kinematics and phase space conventions
In this work, the phase space is described in terms of invariant masses and the µ + angle in the dilepton rest-frame, as shown in Fig. 1. This choice is convenient for the computation of the scalar products involving spin directions. The independent momenta for the η → π 0 µ + µ − decay can be written as The relevant scalar products can, in turn, be expressed as Figure 1: Left: the coordinates in the η rest frame. Right: the coordinates in the dilepton frame. Note that the longitudinal (ẑ) axis is chosen along the µ + direction and that θ =θ. Only p± sinθ = p * ± sin θ is preserved in the boost.
Across the entire manuscript, we use cθ ≡ cos θ and sθ ≡ sin θ for economy of notation. With these conventions, the differential decay width is It is also useful to quote all 4-momenta in the dilepton rest frame n * = (+γβ µ n L , n T , γn L ) ,n * = (−γβ µnL ,n T , γn L ) , where n T (n T ) and n L (n L ) are, respectively, the transverse and longitudinal µ ± spin components with respect to the µ + direction, and n kT is a unit vector representing the k momentum transverse to the µ + direction. Note that n T ,n T and n kT are 2-dimensional objects. The corresponding expressions for the other two processes are found by substituting η → η for η → π 0 µ + µ − , and η → η and π 0 → η for η → ηµ + µ − . The spin projectors, required when one does not sum over spins, are given here for completeness Note that the results obtained from the above equations are easily adapted to the Bouchiat-Michel formulae, Refs. [28,29], required in the spin-density formalism (see Ref. [24], Sect. 1.6).

B Polarised amplitudes
The c i coefficients in Eq. (8) are used as an intermediate step in our calculation but are relevant for implementing the decay processes in a Monte Carlo program, where the subsequent polarised µ ± decays are taken care of by Geant4. On these grounds, we provide them here for completeness c R 13 = 4β µ λ 1/2 K m µ 2m µ cθ(1 + n T ·n T − n LnL ) where the shorthand notation λn → n andλn →n has been employed.