Sensitivity study of the charged lepton flavor violating process τ→γμ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau \rightarrow \gamma \mu $$\end{document} at STCF

A sensitivity study for the search for the charged lepton flavor violating process τ→γμ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau \rightarrow \gamma \mu $$\end{document} at the Super τ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau $$\end{document}-Charm Facility is performed with a fast simulation. With the expected performance of the current detector design and an integrated luminosity of 1ab-1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1\,\textrm{ab}^{-1}$$\end{document} corresponding to one-year of data taking, the sensitivity on the branching fraction (BF) of τ→γμ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau \rightarrow \gamma \mu $$\end{document} is estimated to be at the level of 10-8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10^{-8}$$\end{document}. The sensitivity under different detector performances are also studied. With ideal performance, the BF could be probed to be 2.8×10-8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.8\,\times \,10^{-8}$$\end{document} at 90% confidence level. The sensitivity is expected to scale with the square root of the luminosity, therefore with a total luminosity of 10ab-1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10\,\hbox {ab}^{-1}$$\end{document} corresponding to ten-year of data taking, the sensitivity could reach 8.8×10-9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$8.8\,\times 10^{-9}$$\end{document}, which is about one order of magnitude improvement upon the current best upper limit.


Introduction
In the Standard Model (SM), the charged lepton flavor violating (cLFV) processes can occur through neutrino oscillation, but are highly suppressed due to the small mass of neutrino [1].For example, the branching fraction (BF) of where U is the PMNS matrix [2], i runs over the three neutrinos, m i and m W are the masses of neutrinos and W boson.
The BF in the SM is well beyond the sensitivity of current experiments, thus the observation of cLFV processes would be an unambiguous signature of new physics.On the other a E-mail: xiangteng@pku.edu.cnhand, the lepton flavor conservation, differing from other conservation laws in the SM, is not associated with an underlying conserved current, therefore many theoretical models beyond the SM naturally introduce cLFV processes, such as the Minimal Supersymmetric SM [3], the Grand Unified Theories [4,5], and seesaw mechanisms [6].Some of them predict BFs that are close to the current experimental sensitivity.
As the heaviest lepton, tau has many possible cLFV decay modes, amongst them τ → γ µ is regarded as one of the best probes, and is predicted in a wide variety of models with rates enhanced to observable level.For example, the BF is predicted to be up to 10 −9 in seesaw models [7], 10 −10 in Higgs-mediated SUSY models [8,9], 10 −8 in SO (10) SUSY models [10,11], and 10 −9 in non-universal Z ′ models [12].Experimentally, the most stringent upper limit (UL) on the BF of this channel is given by BABAR to be 4.4 × 10 −8 at 90 % confidence level (C.L.) [13] and Belle to be 4.2 × 10 −8 at 90 % C.L. [14].Next generation of experiments are aiming at pushing the sensitivity down for another one order of magnitude or even further [15].
The proposed Super τ-Charm Facility (STCF) [16] in China, which is an electron-positron collider with a centerof-mass energy in the region of the τ-charm threshold, is one of such next generation of experiments.In this paper, the sensitivity of searching for τ → γ µ at STCF is studied to explore the physics potential of STCF and guide the design of the experiment.STCF has several advantages on searching for τ → γ µ.As an electron-positron collider, the total four-momentum is known and the final state is fully reconstructed, leading to higher efficiency and lower background.The energy of STCF can be adjusted to be just above the threshold of tau pair production e + e − → τ + τ − , where the cross-section reaches the maximum [17], and the energy of radiative photons which is one of the main sources of fake signal photons in background is low thus can be well separated from signal.

Detector design and Monte Carlo simulation
The proposed STCF is a symmetric electron-positron collider operating at center-of-mass energy √ s from 2.0 GeV to 7.0 GeV with a designed peaking luminosity over 0.5 × 10 35 cm −2 s −1 [16].Such an environment will serve as an important high statistics and low background platform to test the SM and probe possible new physics beyond the SM, such as cLFV decays of tau.Assuming that it runs at √ s = 4.26 GeV, STCF will accumulate 3.5 × 10 9 tau pairs [18,19] per year for the corresponding expected integrated luminosity of 1 ab −1 .
As a general purpose detector designed for the electronpositron collider, the STCF detector consists a tracking system composed of the inner and outer trackers, a particle identification (PID) system with charged kaon/pion misidentification rate less than 2 % up to 2 GeV/c, an electromagnetic calorimeter (EMC) with an excellent energy resolution and a good time resolution, a super-conducting solenoid, and a muon detector (MUD) that provides good charged pion/muon separation.The design and the expected performances for each sub-detector are detailed in the conceptual design report [16].
At present, the STCF detector and the corresponding offline software system are under research and development.A fast simulation software package is therefore developed to access the physics study [20], which takes the most common event generators as input to perform a fast and realistic simulation.The simulation includes resolution and efficiency responses for tracking of final state charged particles and photons, PID system and kinematic fit related variables.Instead of performing the detailed simulation of interaction of the final objects with detector by GEANT4 [21], the responses of objects in each sub-system, including efficiency, resolution and other variables used in physics analyses, are modeled by sampling the shape and magnitude according to their performances.By default, all the parameters are based on empirical formulae or extracted from simulation of the BESIII detector [22,23], while the fast simulation also provides flexible interface for adjusting performance of each sub-system, which can be used to optimize the detector design according to the physics requirements.
This sensitivity study is performed based on Monte Carlo (MC) samples at 4.26 GeV, including samples for signal and expected possible background processes.Using the tag method with selected modes as detailed in Sec. 3, the major expected background processes are those that are discussed in the rest of this section.One of them is the dimu process e + e − → (γ)µ + µ − where the two muons and radiated photons can be misidentified as the tag pion and the signal muon and photon.Another background is the ditau process e + e − → τ + τ − where one tau decay is necessarily the same as tag side of signal event and the other tau decay can be misidentified as τ → γ µ.The hadronic process e + e − → q q(q = u, d, s, c) could also contribute to the background due to its high cross-section at this energy region.Samples for these processes are simulated with high luminosity, corresponding to 1 ab −1 , 6 ab −1 , and 1 ab −1 , respectively.The dimu samples are generated with BABAYAGA [24,25].For the ditau samples, the production of tau pair is generated with KKMC [18,19] and the decay of tau with TAUOLA [26].Hadronic process is generated with LUNDARLW [27].Other processes at this energy region are expected to be negligible.The background level of Bhabha process e + e − → e + e − and digamma process e + e − → γγ are much lower than dimu process due to the low probability of electron misidentified as muon or the lack of charged tracks.For the two-photon process e + e − → e + e − X where X is any system generated from the virtual photons radiated by the initial electron and positron, the electron and position tend to have small scattering angles, thus escape from the detector along the beam energy and cause large energy loss, and the remaining X can hardly pass the event selection.Dedicated MC samples for these processes are also generated, and it is demonstrated that they can be removed at early stage of event selection.The signal MC samples are simulated as process e + e − → τ + τ − with one tau goes to SM decay modes and the other decays to γ µ.The decay τ → γ µ is generated with the pure phase space model since the dynamics is unknown.

Analysis procedure
At STCF, τ + and τ − are produced in pairs, so we can tag τ + (denoted as tag side) by its SM decay modes and search for cLFV decay of its partner τ − (denoted as signal side).The charge-conjugated channels are always implied throughout the paper.For the tag side, amongst the five main 1-prong decay modes of τ + , e + ν e ντ , π + ντ and π + π 0 ντ are selected as tag modes, which account for 54 % of the total BF of tau decays [28].τ + → µ + ν µ ντ mode is not used due to high e + e − → (γ)µ + µ − background.As for τ + → π + π 0 π 0 ντ mode, due to the high photon multiplicity, the reconstruction efficiency is low and combinatorial background is high, and the tag photons can be easily misidentified as signal photon.The signal side consists of a signal photon and a signal muon and is featured by a peak around the beam energy on the total energy distribution and a peak around the tau mass on the invariant mass distribution of signal photon and muon, as shown in Fig. 1.
Charged tracks are selected after passing the criteria in fast simulation.For each charged track, the polar angle with respect to beam direction is required to satisfy |cos θ | <

E(γ µ) [GeV]
Fig. 1: The box plot is the two-dimensional distribution of E(γ µ) and M(γ µ) of signal samples where larger box indicates higher density, and the red line shows the signal region.0.93, and the closest approach to the interaction point must be within 10 cm in the beam direction and 1 cm in the plane perpendicular to the beam.Events with two charged tracks and zero net-charge are selected.PID is also performed for charged tracks by the fast simulation, where the identification of electron, pion and kaon is based on extracted likelihood of each particle type hypothesis calculated from the information of PID system and the one with the highest likelihood is assigned, while muon is identified by MUD based on preliminary research and development results.One of the tracks is required to be identified as electron or pion, which is tag charged particle, and the other is required to be identified as muon, which is signal muon.E/p information is then used for further identification, where E is the deposited energy in EMC and p is the momentum of the track.E/p > 0.8 is required for electron and E/p < 0.5 for muon and pion.Photons are selected with an energy threshold of 25 MeV in EMC barrel region (|cos θ | < 0.8) and 50 MeV in EMC end cap region (0.86 < |cos θ | < 0.92) to reject electronic noise and unrelated energy depositions.Neutral pions are reconstructed with two-photon combinations with invariant masses within a π 0 mass window of around 0.12 GeV/c 2 to 0.14 GeV/c 2 determined by fitting.The exact mass window depends on the detector performance design described later in Sec. 4. It is required that there is exactly one remaining photon after neutral pions reconstruction and it is denoted as the signal photon.To further suppress background, the momentum of the signal muon and energy of the signal photon are both required to be in [0.4,1.7] GeV, and the angle between them is required to satisfy cos θ γ µ < −0.35, all of which are constrained by the kinematics (shown in Fig. 2).Finally, a two-dimensional signal region is chosen on the total energy E(γ µ) and invariant mass M(γ µ) distributions of signal photon and signal muon.Since the two distributions are asymmetric and correlated, the signal region is an asymmetric oblique ellipse, as shown in Fig. 1.The tag mode for each event is then assigned based on the event selection result.The event is classified into e + ν e ντ mode if one electron is identified, and it is required that there are no neutral pions reconstructed.If one charged pion is identified, the event is further classified into π + ντ or π + π 0 ντ mode based on whether the number of neutral pions is 0 or 1. Events that do not fit into the classifications are discarded.
After above initial selections, there are mainly five kinds of background classified according to the tag modes and background processes.For e + ν e ντ tag mode, the main background is ditau process where tag tau radiative decays to electron, signal tau SM decays to muon, and signal photon is misidentified from radiative photon of tag side.For π + ντ tag mode, the main background is dimu and ditau processes.In the dimu process, one muon is misidentified as tag pion, the other muon is regarded as signal muon, and signal photon is misidentified from radiative photon.For backgrounds from the ditau process, signal tau SM decays to muon, tag tau decays to ππ 0 with π 0 not successfully reconstructed and the daughter photons are regarded as signal photon or not detected.For the π + π 0 ντ tag mode, one of the main background is the ditau process with tag tau decaying to ππ 0 or ππ 0 π 0 which is similar to the ditau background process in π + ντ tag mode.The other is hadronic process, mainly π + π − + (≥ 1)π 0 , where the signal muon is misidentified from charged pion and the signal photon is from π 0 (s) that are not successfully reconstructed.
Further event selection criteria are determined based on the background characteristics.For the ditau background in the e + ν e ντ tag mode, the signal photon is from radiative leptonic decay of tag tau, thus collinear with the tag charged track, as shown in Fig. 3(a).Moreover, the momentum of tag charged track is lowered due to the existence of radiative photon (Fig. 3(b)).Since neutrinos in the final states are not detected, there will be missing four-momentum defined as the total initial four-momentum subtracted by the fourmomenta of all detected final state particles.There are more neutrinos in background than in signal, so the missing energy E miss in background is higher (Fig. 3(c)), where E miss is defined as energy component of missing four-momentum.For the radiative dimu background in π + ντ tag mode, E miss is lower since there are no neutrinos (Fig. 4(a)), and the direction of missing momentum accumulates along the beam direction since missing momentum is mainly due to radiative photons which are collinear with beam and can escape in the beam direction which is beyond detector coverage (Fig. 4(b)).Furthermore, the energy of the signal photon is lower since it is from radiation (Fig. 4(c)).For ditau background in π + ντ tag mode, in contrast to the signal which has only one neutrino, there are more neutrinos in background, so the missing mass squared M 2 miss is around zero for signal while not for background (Fig. 4(d)).The M 2 miss is defined as the square of invariant mass of the missing fourmomentum.For ditau background in π + π 0 ντ tag mode, the M 2 miss distribution has similar characteristic with π + ντ tag mode (Fig. 5(a)).The distribution of helicity angle of signal muon is also different in signal and background due to different decay dynamics of signal tau (Fig. 5(b)), where the helicity angle is defined as the angle between the direction of signal muon in the signal tau rest frame and the direction of signal tau in center-of-mass frame.For the hadronic background in π + π 0 ντ tag mode, which is mainly π + π − + (≥ 1)π 0 , the missing momentum is due to photons escaping along the beam direction (Fig. 5(c)).
To determine the concrete selection criteria, Punzi significance ε/(1.5 + N bkg ) [29] is used as the figure of merit, where ε is signal efficiency and N bkg is the number of background events.A multidimensional optimization is performed for all the criteria simultaneously.The optimization result depends on the detector performance design, and the result with the best performance is shown here.Table 1 summarizes the further selection criteria and background levels and signal efficiencies before and after further selection.The final background level is suppressed to be only a few with signal efficiency of several percents.A Bayesian-based maximum likelihood estimator, extended from the profile likelihood approach [30], is used to determine the UL on BF of τ → γ µ with statistical fluctuations taking into account.The likelihood is constructed as where N obs is the observed number of events, N τ + τ − is the number of tau pairs, B is BF of τ → γ µ and is the parameter of interest, ε is signal efficiency, N bkg,i with i runs over all the background samples are the true values of background levels which are nuisance parameters, N obs bkg,i and f i are the observed number of events and scale factors for each background samples.The likelihood is then taken as probability distribution of parameters, and the posterior distribution of BF is obtained by integrating over nuisance parameters.Finally, the UL on BF is determined by integrating the posterior distribution, as shown in Fig. 6.The sensitivity of BF is defined as the UL that can be achieved assuming there are no signals.Pseudo-experiments are generated assuming background-only, and the median, the ±1σ and ±2σ quantiles on the distribution of UL are taken as the expectation and confidence interval of the sensitivity [31].With a total luminosity of 1 ab −1 , the sensitivity is estimated to be at the level of 10 −8 .A full systematic uncertainty evaluation which requires both experimental data and full MC simulation is not possible at this stage, so it will be qualitatively discussed.Referring to Eq. 2, the possible sources of systematic uncertainties include the number of tau pairs, the event selection efficiency, and the estimation of background.The uncertainty of number of tau pairs comes from the determination of luminosity and cross-section of e + e − → τ + τ − .For efficiency, the statistical uncertainty can be negligible with large MC samples; the uncertainty of tracking and PID of tracks and reconstruction of neutral pions, which is evaluated with difference between data and MC, can be studied with pure and high statistical control samples; uncertainties related with other selection criteria can be evaluated by control samples or varying the criteria and performing the Barlow test [32]; for the modeling of τ → γ µ decay, although the LFV in-teraction structure in unknown, the uncertainty can be estimated by assuming extreme cases such as pure V − A and V + A forms.The uncertainty of background estimation can be evaluated using control samples of data, such as from sidebands, and verifying that the estimated background is consistent with that in real data.The total systematic uncertainties at STCF are expected to be at the level of several percent or less, which only have a minor impact on the sensitivity.

Optimization of detector performance
The performance of STCF detector is tunable in the fast simulation, and the sensitivity of τ → γ µ under different performances are studied to guide the design of the detector.
The detector performance properties that are crucial to this analysis are determined based on the main background channels where photons and muons are misidentified as signals.One of the main origins of the signal muon is misidentification of pion, while the signal photon is misidentified from a photon with other origins.So, both pion/muon separation capability and photon detection resolution are relevant to this analysis.Better pion/muon separation capability can efficiently suppress background caused by pion/muon misidentification, and better photon detection resolution will improve the resolution of signal region thus exclude more background.With fast simulation, three kinds of detector responses can be studied: pion/muon separation, photon energy resolution and photon position resolution.Considering the feasibility of particular detector designs, a set of different values is assumed for each of the performance properties, from conservative to aggressive.The best detector performance is taken as benchmark, and the dependence of the sensitivity upon each performance property is checked by fixing the other properties and only varying the one under study.The benchmark result is sensitivity of 2.8 × 10 −8 with 1 ab −1 luminosity under the detector performance of 1 % in pion/muon misidentification rate and 3 mm and 2 % in photon position and energy resolutions.For each performance setting, the analysis is performed individually to reach the best sensitivity.pion/muon separation On the one hand, better pion/muon separation can suppress the background caused by pion misidentified as signal muon.On the other hand, better separation means tighter muon selection, which will cause lower signal efficiency.Three levels of pion/muon separation capability is assumed with overall pion to muon misidentification rates of 3 %, 1.7 % and 1 %, which corresponding to the muon identification efficiencies of 97 %, 92 % and 85 % at a momentum of 1 GeV/c, respectively.Table 2 summarizes the efficiency of muon identification in signal sample and the sensitivity of τ → γ µ with respect to pion/muon misidentification rate.The sensitivity together with its confidence intervals are also shown in Fig. 7.The result shows that the sensitivity improves with better pion/muon separation, but the improvement is small.This indicates that a trade-off between the muon efficiency and the pion/muon separation capability should be carefully considered.position resolution for photon Better photon position resolution will result in better signal region resolution, thus improve efficiency and suppress background.Besides, better photon resolution will result in higher tag accuracy and better separation of noise from real photons.The baseline for photon position resolution is 6 mm and an improvement of 30 % and 50 % is assumed for optimization.The signal resolution under different photon position resolutions is shown in Fig. 8, and the sensitivity is summarized in Table 3 and Fig. 9.It is shown that better photon position resolution will result in better sensitivity, but the influence is rather small, which is because the baseline resolution is already quite good.energy resolution for photon Similar to photon position resolution, better photon energy resolution will also result in better sensitivity.The baseline for photon energy resolution is 2.5 % at 1 GeV and an improvement of 10 % and 20 % is assumed for optimization.The signal resolution under different photon energy resolutions is shown in Fig. 10, and the sensitivity is summarized in Table 4 and Fig. 11.It is shown that better photon energy resolution will result in better sensitivity.2.9 % 6.9 10 % improved 3.0 % 5.1 20 % improved 3.1 % 2.8

Summary and discussion
The sensitivity on the cLFV process τ → γ µ at the Super τ-Charm Facility is studied based on 1 ab −1 MC samples, which corresponds to the one-year integrated luminosity of STCF.The sensitivity is expected to be at the level of 10 −8 .The optimization of detector performance is also studied in order to get the best sensitivity, and the result shows that the improvement of each of the three performance properties concerned in this analysis, namely pion/muon separation capability, position and energy resolution of photon, can all result in better sensitivity, though the importance of them is different.With ideal detector performance of 1 % in pion/muon misidentification rate and 3 mm and 2 % in photon position and energy resolutions, the best sensitivity of 2.8 × 10 −8 at 90 % confidence level is achieved.Since background-free can not be achieved, the sensitivity is expected to scale with the square root of the luminosity, and could reach 8.8 × 10 −9 with ten-year of data taking, which is about one order of magnitude improvement upon the current best result.

Fig. 2 :
Fig. 2: The kinematic distributions of (a) momentum of signal muon, (b) energy of signal photon, and (c) cosine of angle between signal photon and muon.The open histogram is signal, and the hatched histograms are background drawn in stacks.The arrows mark the event selection criteria.The lower end cutoff on momentum of muon is caused by MUD acceptance.

Fig. 3 :Fig. 4 :Fig. 5 :
Fig. 3: The comparison of signal (red solid histogram) and background (green dashed histogram) samples in e + ν e ντ tag mode.(a) the cosine of the angle between signal photon and tag charged track cos θ sig_γ,tag_charged , (b) the momentum of tag charged track p tag_charged , and (c) the missing energy E miss in ditau background.

Fig. 6 :
Fig. 6: Determination of UL of BF.The line shows the posterior probability distribution and the arrow marks the UL at 90 % C.L.

Fig. 7 :
Fig.7: Result for optimization of pion/muon separation capability.The dashed line is the expected sensitivity, the green and yellow bands are the 1σ and 2σ confidence intervals.

Fig. 8 :Fig. 9 :
Fig. 8: Signal resolution under different photon position resolution performance.The left and right figure shows the invariant mass and total energy of signal photon and muon, separately.The black solid line, the red dashed line and the blue dash-dotted line shows the result under baseline, 30 % and 50 % improved photon position resolution, separately.

Fig. 10 :
Fig. 10: Signal resolution under different photon energy resolution performance.The left and right figure shows the invariant mass and total energy of signal photon and muon, separately.The black solid line, the red dashed line and the blue dash-dotted line shows the result under baseline, 10 % and 20 % improved photon energy resolution, separately.

Table 1 :
The result of further event selection.The first column is the tag modes, the second column is the selection criteria, the third and fourth columns are the number of background N bkg and signal efficiency ε before and after further selection.

Table 2 :
Result of optimization for pion/muon separation.The first column shows the levels of pion/muon separation capability of the detector.The second and third columns show the muon identification efficiency and sensitivity on BF under different detector performance, separately.