Improved treatment of the T$_2$ molecular final-states uncertainties for the KATRIN neutrino-mass measurement

Abstract

The KArlsruhe TRItium Neutrino experiment (KATRIN) aims to determine the effective mass of the electron antineutrino via a high-precision measurement of the tritium $\upbeta$-decay spectrum in its end-point region. The target neutrino-mass sensitivity of ${0.2}~\text {eV}/\text {c}^2$ at 90% CL can only be achieved in the case of high statistics and good control of the systematic uncertainties. One key systematic effect originates from the calculation of the molecular final states of T$_2$ $\upbeta$ decay. In the first neutrino-mass analyses of KATRIN the contribution of the uncertainty of the molecular final-states distribution (FSD) was estimated via a conservative phenomenological approach to be $2 \times 10^{-2}~\text {eV}^{2}/\text {c}^{4}.$ In this work a new procedure is presented for estimating the FSD-related uncertainties by considering the details of the final-states calculation, i.e. the uncertainties of constants, parameters, and functions used in the calculation as well as its convergence itself as a function of the basis-set size used in expanding the molecular wave functions. The calculated uncertainties are directly propagated into the experimental observable, the squared neutrino mass $m_\nu ^2,$ and thus have to be determined individually for each experimental configuration. For the experimental conditions of the first KATRIN measurement campaign the new procedure is presented in detail, allowing for the application of this procedure to other experiments. This specific calculation leads to a constraint of the FSD-related uncertainty of $1.3 \times 10^{-3}~\text {eV}^{2}/\text {c}^{4},$ well below the design limit of $7.5 \times 10^{-3}~\text {eV}^{2}/\text {c}^{4}$ for any individual systematic contribution.

1 Introduction

The Karlsruhe Tritium Neutrino experiment (KATRIN) [1, 2] aims at determining the effective neutrino mass $m_\nu$¹ by precisely measuring the integrated electron-energy spectrum of the superallowed molecular tritium $\upbeta$ decay near the spectrum’s end point at about 18.57 keV. KATRIN combines an ultra-luminous Windowless Gaseous Tritium Source (WGTS) providing a $\upbeta$-decay rate of up to 1e11 Bq [3] with a large spectrometer of MAC-E-filter [4] type transmitting electrons above an adjustable energy threshold with ${{\mathcal {O}}}(1)~{\textrm{eV}}$ width. The target sensitivity is $m_\nu <{0.2\,\mathrm{\text {eV}/ c^{2}}}$ at $90\%$ CL after 5 years of data taking. The first four-week science campaign during spring 2019 (KNM1) yielded a limit of ${1.1\,\mathrm{\text {eV}/ c^2}}$ [5], and the first two science campaigns (KNM1 and KNM2) set an upper limit of $m_\nu < {0.8\,\mathrm{\text {eV}/ c^{2}}}$ (90% CL) [6].

Although the uncertainty of the neutrino mass extracted from these first measurement campaigns is dominated by the statistical error, a good control of the systematic effects and related uncertainties will be required in the future for reaching the target sensitivity of KATRIN. The main experimental uncertainties are connected to the distortions of the shape of the spectrum by various background effects, electrons’ starting potential, and scattering in the source, as well as the transmission properties of the spectrometer. Another source of uncertainty stems from the final states of molecular tritium (T$_2)$ $\upbeta$ decay. This distribution of electronic and ro-vibrational final-states (FSD) enters the computation of the differential $\upbeta$-decay spectrum that is used in the fit from which the squared neutrino mass is extracted. An FSD that is sufficiently accurate for the analysis of neutrino-mass experiments is so far only available from ab initio calculations [7]. As was shown in [8], a not-considered variance $\sigma ^2$ of the energy scale of the measured $\beta$ spectrum leads approximately to a shift in the squared neutrino mass extracted from the experiment according to $\updelta m_{\nu , \sigma }^2 = -2\cdot \sigma ^2$. From this relation a first naïve estimate of the FSD-related uncertainty on the neutrino mass can be derived: an uncertainty of the FSD used in the data analysis leads to an unknown bias of the $m^2_\nu$ observable and also to an additional contribution to the neutrino-mass uncertainty of $|\updelta m^2_{\nu , \sigma } |= 2 \cdot |\sigma ^2_{\textrm{true}}-\sigma ^2_{\textrm{fit}}|,$ if $\sigma ^2_{\textrm{true}}$ is the true variance of the FSD and $\sigma ^2_{\textrm{fit}}$ the FSD variance used in the data analysis.

In this work a new procedure for the assessment of the systematic uncertainties due to the molecular final states of T$_2$ (and ${\textrm{DT}},$ ${\textrm{HT}})$ $\upbeta$ decay is introduced. In contrast to the previous uncertainty estimation that was based on a fully phenomenological approach [9], the new procedure involves a detailed investigation of the various sources of uncertainties which enter the molecular FSD calculation. This includes the uncertainties from the use of a finite basis set in the ab initio calculation, from adopted approximations like the sudden approximation, and uncertainties on fundamental constants. Different FSDs are generated, e.g., by a systematic increase of the basis set or the inclusion (omission) of corrections to the adopted approximations. The comparison of the resulting squared neutrino masses $m^2_\nu$ – the KATRIN observable – that are obtained by a fit to a reference $\upbeta$ spectrum yields an effective shift $\varDelta m_\nu ^2.$ The absolute values of $\varDelta m_\nu ^2$ for different FSDs are then added to a total systematic uncertainty of $m_\nu ^2$ due to the FSD.

In order to allow for a direct comparison of the outcome of the uncertainty-analysis procedure introduced in this work compared to the already published outcome of the earlier FSD-uncertainty estimate, in this proof-of-principle demonstration the new procedure is applied adopting the experimental parameters (like the source temperature or isotopologue distribution) of KATRIN’s first science campaign KNM1. A direct re-evaluation of the uncertainty using this new approach is, however, not fully possible in the case of the FSD that was used for the analysis of the first two KATRIN science campaigns (named KNM1 FSD). Since the KNM1 FSD was constructed by adopting the best input data available in literature at that time, there is, e.g., no common basis set used for different states and the adopted corrections partially stem from calculations using again different basis sets [10]. Furthermore, a full reconstruction of the KNM1 FSD (in the sense of a re-evaluation of all values from scratch) is not possible, since some of the basis-set parameters are unavailable in literature (see Sect. 5.2), introducing the need for a pseudo-KNM1 FSD (described in detail in Sect. 5.3).

The procedure established in this work needs to be repeated for the analysis of future KATRIN measurement campaigns individually, i.e. depending on the experimental conditions of each campaign. Furthermore, an improved new FSD (not presented in this publication) will be adopted in the analysis of future campaigns that avoids the need for a pseudo FSD in the uncertainty estimate.

Since this publication is supposed to give a complete and thus a rather lengthy description of the new determination of the FSD systematics so that this procedure can also be applied to other experiments, first an executive summary of this work is given. The model of the tritium $\upbeta$-decay spectrum of KATRIN and the analysis procedure for the experimental conditions of the first measurement campaign are described in Sect. 2. The impact of the molecular final states on the $\upbeta$-decay spectrum and the general procedure of the FSD computation are given in Sect. 3. In Sect. 4 the following uncertainties of the FSD computation are introduced: Theoretical approximations adopted in the FSD calculation (e.g. the sudden approximation, the Born–Oppenheimer approximation), uncertainties on fundamental constants and conversion factors like the tritium end point, uncertainties due to the convergence of the chosen basis functions, and coding or input errors. Afterwards, the previous approach to the uncertainty estimation is summarised. This approach was based on a conservative comparison of different ab initio FSD calculations, the estimated uncertainties were propagated into a neutrino-mass uncertainty using a covariance matrix and a Monte-Carlo propagation method. Based on this approach, the neutrino-mass uncertainty originating from the FSD was constrained to ${0.02}~\text {eV}^{2}/\text {c}^{4}.$ The new procedure to systematically assess the FSD uncertainty is explained in Sect. 5. It is based on a convergence approach via a systematic basis-set increase. To investigate the impact of a specific FSD-related effect, a test FSD differing from the original FSD by the parameter of interest is used for a neutrino-mass fit onto a Monte-Carlo generated $\upbeta$-decay spectrum. The effect on $m_\nu ^2$ can be read from the deviation of the fitted $m_\nu ^2$ from the Monte-Carlo truth. As a proof-of-principle study, the resulting systematic contributions of the FSD calculation to the uncertainty of $m_\nu ^2$ obtained by the exemplary application of the new uncertainty approach to KATRIN’s first measurement campaign are presented in Sect. 6. The sum of all investigated contributions lies with ${1.3 \times 10^{-3}\,\text {eV}^{2}/\text {c}^{4}}$ far below the uncertainty contribution obtained from the old conservative approach. As will be shown, the dominant contribution stems from corrections to the Born–Oppenheimer approximation, which makes clear where future FSD calculations should focus. A brief summary and outlook is given in Sect. 7. For a list of all FSDs mentioned in this publication, see Appendix A. Details on the choice of parameters for the FSD computation in the present work can be found in Appendix B. Appendix C lists additional FSD uncertainties caused by experimental parameters. Finally, Appendix D compares the neutrino-mass shift induced from the use of the pseudo-KNM1 FSD instead of the KNM1 FSD.

2 Model of the experimental $\upbeta$-decay spectrum

In tritium neutrino-mass measurements the squared neutrino-mass parameter $m_\nu ^2$ is inferred by fitting a model spectrum to a measured kinetic energy spectrum of the electrons emitted during $\upbeta$ decay. In KATRIN all electrons with a kinetic energy above a specific threshold are detected. This threshold energy is scanned to obtain an integrated spectrum of $\upbeta$ electrons. The model of the integrated spectrum measured by KATRIN is described by a convolution of the theoretical differential $\upbeta$-decay spectrum $R_\upbeta (E),$ given by Fermi’s Golden Rule, with the experimental response function $f_{\text {calc}}(E, qU)$ [11],

$$\begin{aligned} R_{\text {calc}}(qU)= & {} A_{\text {s}}\cdot N_{\text {T}}\cdot \int _{qU}^{E_{0}}{R_\upbeta (E) \cdot f_{\text {calc}}(E, qU)\,dE} \,\nonumber \\{} & {} +\, R_{\text {bg}}. \end{aligned}$$

(1)

Here, E is the kinetic energy of the $\upbeta$ electron and qU is the retarding-voltage set point, which defines the energy threshold for the electrons transmitted by the spectrometer, $q=-e$ is the charge of the electron. $R_{\text {bg}}$ stands for a constant background rate which is a free parameter in the fit. The response function, $f_{\text {calc}}(E, qU),$ takes into account the energy losses due to scattering and synchrotron radiation, as well as the spectrometer transmission properties based on the magnetic fields along the beamline. $N_{\text {T}}$ is the number of tritium atoms $N_{\text {T, abs}}$ multiplied by the solid acceptance angle and the detector efficiency. $N_{\text {T, abs}}$ is defined via $N_{\text {T, abs}}=2\cdot \epsilon _{\text {T}}\cdot \rho d \cdot A,$ with A being the cross-section area of the flux tube within the windowless gaseous tritium source (WGTS), and $\epsilon _{\text {T}}=[N_{{\text{ T }}_2}+\frac{1}{2}(N_{\text {HT}}+N_{\text {DT}})]/\sum _i N_i$ the tritium purity. $N_i$ is the number of molecules of one of the isotopologues T$_{2},$ DT, D$_{2},$ HT, HD and H$_{2}.$ The amount of HT and DT in the source is described by the HT/DT ratio $\upkappa = N_{\text {HT}}/{N_{\text {DT}}}$ [12]. At the WGTS, a constant tritium flow is achieved by continuously injecting molecular tritium gas of high purity in the midpoint of the beam tube. It then diffuses to both sides, where it is pumped out. The column density $\rho d$ of the source is the integrated tritium density along the length $d={10\,\mathrm{\text {m}}}$ of the source cryostat. $N_{\text {T}}$ can vary between different measurement campaigns. Finally, $A_{\text {s}}$ in Eq. 1 is the relative signal amplitude, it is a free parameter in the fit. The other two fit parameters, the end point $E_{0}$ and the squared neutrino mass $m_\nu ^2,$ enter Eq. 1 via $R_\upbeta (E)$ (see Sect. 3, Eq. 2).

For the FSD-uncertainty studies presented in this work, the integrated spectrum $R_{\text {calc}}(qU)$ is evaluated at discrete retarding-voltage set points qU. In Sect. 2.1, the experimental parameters as well as the features of the spectrum model used in the analysis are described. The parameters correspond to the first KATRIN science campaign (KNM1). In Sect. 2.2, a description of the used reference Asimov Monte-Carlo data set is given. Finally, information on the fit methods is given in Sect. 2.3.

2.1 Experimental conditions of the first science campaign (KNM1)

Tritium source parameters The key source parameters of KNM1 which are used for the present FSD-uncertainty studies are listed in Table 1, more details can be found in [5].²

Table 1 Source parameters during KNM1 as used in the present analysis

Spectrometer and beamline conditions The $\upbeta$-decay electrons are emitted in a high magnetic field $B_{\text {src}}$ in the source. The magnetic field guides them adiabatically towards the spectrometers where they are filtered via the retarding potential energy qU. The filter width $\varDelta E /E$ is defined by the ratio of the minimum magnetic field $B_{\text {min}}$ in the spectrometer’s analysing plane³ and the maximal magnetic field in the beamline, $B_{\text {max}}.$ In the configuration of KNM1 [5] the fields have the values listed in Table 2, leading to $\varDelta E = {2.8\,\mathrm{\text {eV}}}$ at the $\upbeta$-decay end point $E_{0}.$ The values from Table 2 are used as input values for generating the Monte-Carlo data in the present studies.

Table 2 Magnetic fields along the beamline during KNM1, used as input for the Monte-Carlo data entering the FSD-convergence studies

Spectrum model and systematic corrections For the description of the tritium $\upbeta$-decay spectrum in this study a fully relativistic Fermi function is used. Radiative corrections due to virtual and real photons, described in more detail in [11, 13], are applied. Furthermore, synchrotron energy losses [14] of the electrons in the high magnetic fields are taken into account, while Doppler broadening is not applied.⁴

The experimental response function is influenced by energy losses of the electrons in the source due to scattering. An energy-independent inelastic cross-section of $3.64 \times 10^{-18}~\text {cm}^{2}$ [5] at $E_{0}$ is assumed. In the simulation, electrons with up to seven scatterings are taken into account.⁵ The scattering probability depends on the electron pitch angle with regard to the magnetic field lines due to the increase of the mean path length with increasing pitch angle [11]. This dependence is neglected in the study, the angle-averaged values are used instead.⁶

2.2 Generation of Monte-Carlo data

Table 3 Input parameters for the KNM1 Monte-Carlo data. $A_{\text {s}},$ $E_{0},$ $R_{\text {bg}}$ and $m_\nu ^2$ are the four free fit parameters in all fits performed for the FSD-uncertainty estimation

For the studies presented here a Monte-Carlo data set using the pseudo-KNM1 FSD introduced in detail in Sect. 5.3 was created. This Asimov Monte-Carlo data set does not contain any statistical fluctuations and it is based on the parameters of the first KATRIN campaign. The parameters are listed in Table 3. A time-dependent background component of the order of $10^{-6}~\text {cps}/\text {s}$ induced by Penning traps [5] is neglected in this study.⁷ The spectra of all 148 detector pixels are averaged to a single spectrum to facilitate the fits.⁸

During a measurement campaign, the integral $\upbeta$-decay spectrum is measured at discrete, non-equidistant retarding voltage set points which are applied repeatedly to scan the spectrum.⁹ The times spent at each retarding voltage set point in the Monte-Carlo-generated dataset mimic the times spent during KNM1. Each individual scan covers the energy interval $[E_{0}-{90\,\mathrm{\text {eV}}}, E_{0}+{50\,\mathrm{\text {eV}}}].$ The set points are assumed to be perfectly reached in the simulation. A total measurement time of 550 h is simulated, corresponding to 274 scans over the full energy range which corresponds to the total number of scans during KNM1. The generated Asimov Monte-Carlo spectrum is illustrated in Fig. 1.

Fig. 1

Illustration of the Asimov Monte-Carlo data set based on the first KATRIN campaign, with corresponding four-parameter fit, yielding as expected a $\chi ^2$-result close to zero as well as $m_\nu ^2 = 0 \,\text {eV}^{2}/\text {c}^{4}$. The same pseudo-KNM1 FSD is used for the Monte-Carlo data and the fit model. Since Asimov Monte-Carlo data is used, the statistical uncertainty on the $m_\nu ^2$ is not of relevance

2.3 Fit methods

The fit methods used in this work, as well as the model and fitting framework, are described in [9, 11]. The Monte-Carlo data are fitted using $A_{\text {s}},$ $E_{0},$ $R_{\text {bg}}$ and $m_\nu ^2$ as free fit parameters. In this study all other parameters in the model are fixed to the values used for generating the Asimov data set. Like in the KNM1 analysis, the lower limit of the fit interval in this work is $E_{0}-{40\,\mathrm{\text {eV}}}.$ The entire fit interval includes 27 individual discrete retarding-voltage set points. With 4 free fit parameters, this yields 23 degrees of freedom. As the Asimov Monte-Carlo data does not contain fluctuations, a fit to such data yields the exact input parameters from Table 3 as fit results, if the same FSD is applied for generating the data and in the fit model. The fit for $m_\nu ^2$ is shown in Fig. 1. The deviation of $m_\nu ^2=-1.9 \times 10^{-6}~\text {eV}^{2}/\text {c}^{4}$ from the input value of $m_\nu ^2={0}~\text {eV}^{2}/\text {c}^{4}$ is caused by numerical inaccuracies. The indicated uncertainties give an estimate of the 1-$\sigma$ statistical sensitivity of this data set.

The effect of a modified FSD on $m_\nu ^2$ can be investigated by fitting the Asimov data set generated using the KNM1-FSD with a model whose FSD has been modified. This mismatch between FSDs results in a shift away from the $m_\nu ^2$ Monte-Carlo truth of ${0}~\text {eV}^{2}/\text {c}^{4},$ which can be used as an estimate of the impact of the FSD modification on $m_\nu ^2.$ The numerical deviation of ${1.9 \times 10^{-6}~\text {eV}^{2}/\text {c}^{4}}$ mentioned above can be interpreted as the precision with which systematic uncertainties with regard to the FSDs can be determined in this study.

3 Molecular final-states distribution (FSD)

As a result of the $\upbeta$ decay the nuclear charge of the decaying nucleus in the parent molecule (here T$_2,$ HT, or DT) is increased and the formed molecular daughter ion (here $^3$HeT$^+,$ $^3$HeH$^+,$ or $^3$HeD$^+,$ respectively) may end up in any of its molecular final states. The FSD is the distribution that describes the probabilities $\zeta _j$ with which the energy $V_j$ is left within the daughter molecular ion, i.e. excluding the recoil energy to the translational degree of freedom.¹⁰ Thus the FSD over the molecular final states enters the model of KATRIN’s spectrum, Eq. (1), as it modifies the differential $\upbeta$-decay rate $R_\upbeta (E),$

$$\begin{aligned}{} & {} R_\upbeta (E)\propto (E+m_e c^2)\cdot \sqrt{(E+m_e c^2)^2-m_e^2 c^4} \nonumber \\{} & {} \quad \cdot \sum _j \zeta _j\cdot \epsilon _j \cdot \sqrt{\epsilon _j^2-m_\nu ^2 c^4} \cdot \varTheta (\epsilon _j -m_\nu c^2). \end{aligned}$$

(2)

Here, $m_e$ is the electron mass, $\epsilon _j=E_{0}-E-V_j$ is the total energy of the neutrino, and $E_{0}$ is the end point of the $\upbeta$-decay spectrum.¹¹ Due to the (energy conserving) Heaviside step function $\varTheta$ the differential decay rate $R_\beta$ is non-zero only if the kinetic energy of the neutrino $(\epsilon _j-m_\nu c^2)$ is larger than or equal to $E_0-E-V_j.$ At the end point the argument of $\varTheta$ is equal to zero, the neutrino has zero kinetic energy, the kinetic energy E of the $\upbeta$ electron has its maximum value which is equal to $E_0,$ and no excitation energy is given to the molecular system, i.e. $V_j=0.$ Note, in discussions of the $\upbeta$ spectrum, electron energies are generally defined relative to the end point $E_{0},$ thus one has decreasing $\upbeta$-electron energies when going away from the end point.¹² In contrast to this, the binned energies $V_j$ that are intrinsically defined to be positive¹³ start with zero value at the end point of the $\upbeta$ spectrum and increase until the end of the fit interval.

In the present case of a molecular system described in the Born–Oppenheimer approximation, the daughter molecular ion may be excited electronically, either to a bound state or into the ionization continuum. Within every molecular electronic state, the nuclear degrees of freedom allow for rotational and vibrational excitation. In the latter case, the system may be left in a bound or an unbound (i.e. dissociative) vibrational state. If the $^3$HeT$^+$ molecule is created in its absolute (electronic, vibrational, and rotational) ground state, the energy available to the $\upbeta$ electron and the neutrino is maximum. Until the onset of the first electronically excited state at about 19 eV below the end point only rotational and vibrational excitation, including dissociation, within the electronic ground state are possible (see Fig. 2). In the energy range between 19 and 40 eV below the end point only electronically bound excited states of $^3$HeT$^+$ occur. Since the potential curves of all these electronically excited states are purely repulsive in the relevant range of internuclear distances, the rotational and vibrational excitations lead to dissociation. Then, more than 40 eV below the end point, both an infinite series of (bound) Rydberg states and the ionisation continuum contribute to the spectrum. The theoretical treatment of the latter is much more challenging than the one of the electronically bound states. In order to avoid the complications arising from the theoretical treatment of the ionisation continuum, the first measurement campaigns of KATRIN limited the analysis to the 40 eV interval below the end point. For the future, an extension of the analysis interval e.g. up to 60 eV is under consideration and corresponding FSD calculations are in progress.

To reduce the amount of data entering the KATRIN analysis and thus also the computational times when performing the fits, the FSD is typically binned and provided as a single FSD (for a given experimental campaign). Since the initial population of the rotational states of the parent T$_2$ molecule depends on the temperature of the gas source, the FSD needs to be calculated for different initial rotational states. With the traces of HT and DT isotopologues in the gas mixture of the KATRIN source, the FSDs of HT and DT are also required for the model of the KATRIN spectrum. The single delivered FSD already considers the isotope mixture $({\textrm{HT}}, {\textrm{DT}}, {\textrm{T}_2})$ as well as the gas temperature in the evaluation of $\zeta _j$ and $V_j.$ To increase the information content, it was proposed in [7] to define the energy values $V_j$ as the mean transition energy for the bin j. The bin size should be adapted in accordance with the energy resolution of the experiment. The details on how the FSD is calculated will be discussed in the following subsections.

3.1 Molecular transition probabilities

The theoretical calculation of the FSD is based on the computation of the transition probabilities $P_{fi}^{\textrm{TS}}.$ For tritium $\upbeta$ decay this probability is given within the sudden approximation by

$$\begin{aligned} P_{fi}^{\textrm{TS}} = |\,M_{fi}^{\textrm{TS}}\,|^2 = \left| \, \langle \varPsi _{f}^{{\textrm{HeS}}^{+}} | \, e^{i{\tilde{\varvec{K}}} \cdot {\varvec{R}}} \, | \varPsi _{i}^{\textrm{TS}} \rangle \, \right| ^2. \end{aligned}$$

(3)

The details of the derivation as well as the validation of the sudden approximation can be found in [16, 17]; for a detailed description of the evaluation within the sudden approximation see [10]. In more detail, Eq. 3 describes the transition probability from a given molecular initial state i of the parent molecule TS to a specific final state f of the daughter molecular ion HeS$^+$ accompanying the nuclear $\upbeta$ decay. Here, the spectator S is any of the constituents in the molecule in addition to the decaying $\textrm{T}$ atom, thus S stands in this case for $\textrm{T},$ $\textrm{D},$ or $\textrm{H}$ in the case of T$_2,$ DT, and HT, respectively. In Eq. (3) ${\varvec{R}}$ is the vector connecting the two nuclei¹⁴ and ${\tilde{\varvec{K}}}$ is a fractional recoil momentum, i.e. the fraction of the recoil imparted onto the molecular ion HeS$^{+}$ by the emitted neutrino and the $\upbeta$-decay electron that is not transferred to the center-of-mass, but to the internal degrees of freedom.

If the final state lies within some continuum of states, e.g., dissociation or ionisation, it is in fact a transition probability per unit of energy in between $E_f$ and $E_f+{\textrm{d}}E.$ The FSD $(V_j,\zeta _j)$ is obtained from the transition probabilities by a sum (or integral) of all transition probabilities $P_{fi}^{\textrm{TS}}$ within a given energy interval. Different temperatures as they may occur in different experimental campaigns are considered by adding the transition-probability distributions of the different initial states i weighted by their statistical Boltzmann distribution probability. Similarly, the transition-probability contributions of the various isotopologues are added with a statistical weight representing the relative concentrations of ${\textrm{T}_2}, {\textrm{DT}},$ and ${\textrm{HT}}$ for a given experimental campaign. All these summations are performed incoherently, since the occurrence of temperature and isotope distribution is a purely classical statistical effect.

A key step is to evaluate the transition matrix element $M_{fi}^{\textrm{TS}}$ in Eq. (3) between the wavefunction $\varPsi _{i}^{\textrm{TS}}$ of the parent molecule in a given initial state i and the wavefunction $\varPsi _{f}^{{\textrm{HeS}}^{+}}$ of the daughter molecular ion in state f. The energy range of final states f that needs to be considered depends on the fit interval used in the analysis of a given experimental data set, $E_{\textrm{max}}=40$ eV in the case of KATRIN KNM1.

Adopting the Born–Oppenheimer approximation, the total wavefunction $\varPsi$ factorises into an electronic part $\varPhi$ and a nuclear-motion part $\xi ,$

$$\begin{aligned} \varPsi _{n,v_j,J_j,m_{J_j}}(\textbf{R}, \textbf{r}_1, \textbf{r}_2) = \varPhi _n (\textbf{r}_1, \textbf{r}_2; R) \cdot \xi _{n;v_j,J_j,m_{J_j}} (\textbf{R}). \nonumber \\ \end{aligned}$$

(4)

Here $R=|\textbf{R}|$ is the internuclear separation (distance between the two nuclei), $\textbf{r}_j$ $(j=1,2)$ are the coordinates of the two electrons, $J_j$ the total angular momentum, $m_{J_j}$ its projection along the z-axis, $v_j$ the vibrational quantum number and n the electronic state of the molecule. The electronic wavefunctions $\varPhi$ have only a parametric dependency on R,  i.e. R is kept constant when solving the eigenvalue equation

$$\begin{aligned} \hat{\textrm{H}}_{\textrm{el}} \, \varPhi _n (\textbf{r}_1, \textbf{r}_2; R) = V_n^{\textrm{BO}}(R) \, \varPhi _n (\textbf{r}_1, \textbf{r}_2; R) \end{aligned}$$

(5)

with the electronic Hamiltonian

$$\begin{aligned} \hat{\textrm{H}}_{\textrm{el}}= & {} -\,\frac{\hbar ^2}{2 m^2_e} \, \left( \nabla _1^2 + \nabla _2^2\right) - \, \frac{e^2}{4\,\pi \epsilon _0} \nonumber \\{} & {} \times \left( \frac{Z_\text {A}}{r_{1\text {A}}} - \frac{Z_\text {A}}{r_{2\text {A}}} - \frac{Z_\text {B}}{r_{1\text {B}}} - \frac{Z_\text {B}}{r_{2\text {B}}} + \frac{1}{r_{12}} + \frac{Z_\text {A} Z_\text {B}}{R}\right) . \end{aligned}$$

(6)

Here the $V_n^{\textrm{BO}}$ are the internuclear-separation dependent electronic energies for electronic state n,  the so-called Born–Oppenheimer potential curves. A and B denote the nuclei, 1 and 2 the electrons, $r_{ij} = |\textbf{r}_i - \textbf{r}_j|,$ and $Z_A$ $(Z_B)$ is the charge number of nucleus A (B).

The substitution of Eq. (4) into Eq. (3), considering only the decay of molecules that were initially in the electronic ground state $(n_i=0)$, and adopting the notation $n_f\equiv n$ yields the volume integral with respect to the nuclear-separation coordinate ${\varvec{R}}$

$$\begin{aligned} M_{n,f;i}^{\textrm{TS}}(\tilde{K}) \,= & {} \, \iiint \limits _{-\infty }^{+\infty } \left\{ \xi _{n;v_f,J_f,m_f}^{{\textrm{HeS}}^{+}}({\varvec{R}}) \right\} ^* \; S_n(R)\nonumber \\{} & {} \cdot e^{i {\varvec{\tilde{K}} \cdot {\varvec{R}}}} \; \xi _{0;v_i,J_i,m_i}^{\textrm{TS}}({\varvec{R}}) \; {\textrm{d}}V_R \end{aligned}$$

(7)

with the electronic transition overlap matrix element $S_n(R)$ defined as the six-dimensional integral

$$\begin{aligned}{} & {} S_n (R) =\nonumber \\{} & {} \iiint \limits _{-\infty }^{+\infty }\iiint \limits _{-\infty }^{+\infty } \left\{ \varPhi _n^{{\textrm{HeS}}^+}(\textbf{r}_1,\! \textbf{r}_2; R)\right\} ^*\,\varPhi _0^{\textrm{TS}}(\textbf{r}_1, \textbf{r}_2; R)\;\text {d}{V}_1 \text {d}{V}_2. \nonumber \\ \end{aligned}$$

(8)

The nuclear-motion wavefunctions $\xi$ describing the rotational $J_j\, , m_{J_j}$ and vibrational $v_j$ degrees of freedom of the molecule, i.e. all but the translational degree of motion, are the solutions of the eigenvalue equation

$$\begin{aligned} \hat{\textrm{H}}_\textrm{nuc}^{(n)} \; \xi _{n;v_j,J_j,m_{J_j}} (\textbf{R}) = E_{n;v_j,J_j,m_{J_j}} \;\xi _{n;v_j,J_j,m_{J_j}} (\textbf{R}) \end{aligned}$$

(9)

with the nuclear-motion Hamiltonian

$$\begin{aligned} \hat{\textrm{H}}_\textrm{nuc}^{(n)} = -\frac{\hbar ^2}{2m_{\mu }} \,\nabla _{R}^2 + V_{n}^{\textrm{BO}}(R). \end{aligned}$$

(10)

Here $m_{\mu }$ is the reduced mass of the two atoms T and S, which will be discussed in more detail at the end of Sect. 4.1.1 and in Sect. 6.1.3. Since $V_n^{\textrm{BO}}$ is only a function of the absolute value of the internuclear distance, the angular solutions of Eq. (10) are simply the spherical harmonics Y, 

$$\begin{aligned} \xi _{n;v_j,J_j,m_{J_j}} (\textbf{R}) = \frac{\chi _{n;v_j,J_j} (R)}{R} \; Y_{J_j,m_{J_j}} (\theta _R,\varphi _R), \end{aligned}$$

(11)

while the radial wavefunctions $\tilde{\chi }=\chi \, R^{-1}$ are solutions of the eigenvalue equation

$$\begin{aligned} \hat{\textrm{H}}_{\textrm{nuc},J}^{(n)} \, \tilde{\chi }_{n;v_j,J_j} (R) = E_{n;v_j,J_j} \, \tilde{\chi }_{n;v_j,J_j} (R) \end{aligned}$$

(12)

with the radial nuclear-motion Hamiltonian

$$\begin{aligned} \hat{\textrm{H}}_{\textrm{nuc},J}^{(n)} = -\, \frac{\hbar ^2}{2m_{\mu }} \, \frac{\textrm{d}^2}{\textrm{d}R^2} + \frac{\hbar ^2}{2m_{\mu }} \, \frac{J(J+1)}{R^2} +V_{n}^{\textrm{BO}}(R). \end{aligned}$$

(13)

As a consequence of the spherical symmetry, the energy $E_{n;v_j,J_j}$ does not depend on the magnetic quantum number $m_{J_j}.$

The separation of the wavefunctions $\xi$ into an angular and a radial part together with an expansion of the $\exp (i{\tilde{\varvec{K}}}{\varvec{R}})$ term in a series of spherical harmonics times spherical Bessel functions $j_\ell (\tilde{K} R)$ allows for a straightforward integration over the angles by using the properties of the spherical harmonics (leading to terms that can be expressed using Wigner 3j symbols or Clebsch–Gordan coefficients), for details see [10]. The remaining task is the calculation of one-dimensional integrals over the radial coordinate R, 

$$\begin{aligned} I_{0;v_i.J_i}^{n;v_f,J_f} (\tilde{K})= & {} \int \limits _{0}^{\infty } \left\{ \tilde{\chi }_{n;v_f,J_f}^{{\textrm{HeS}}^{+}}(R) \right\} ^* \nonumber \\{} & {} \times S_n(R)\; j_\ell (\tilde{K} R) \; \tilde{\chi }_{0;v_i,J_i}^{\textrm{TS}}(R) \, {\textrm{d}}R. \end{aligned}$$

(14)

The evaluation of the transition matrix elements $M_{fi}$ thus splits basically into four steps.

• 1. The solution of the electronic eigenvalue problem Eq. (5) is required in order to obtain the electronic wavefunctions $\varPhi _n$ and the Born–Oppenheimer potential curves $V_n^{\textrm{BO}}(R).$ The former are needed for the calculation of the electronic transition overlaps $S_n$ according to Eq. (8) and the latter as an input for the radial nuclear-motion eigenvalue problem Eq. (13).

• 2. The electronic overlaps $S_n(R)$ are calculated according to Eq. (8).

• 3. The wavefunctions of nuclear motion $\chi$ and the complete molecular (rotational, vibrational, and electronic) energies $E_{n;v_j,J_j}$ are obtained by solving Eq. (12).

• 4. The matrix elements $M_{fi}$ in Eq. (7) are evaluated adopting the corresponding Clebsch–Gordan algebra and calculating the integrals I in Eq. (14) by means of numerical quadrature.

At the temperature of 30 K in the first KATRIN science campaign mostly rotational and some vibrational excitation of the parent molecule is possible, while the electronic excitation is negligible and thus $n_i=0$ is used. In the case of a homonuclear diatomic parent molecule like ${\textrm{T}_2}$ also the spin statistics (ortho, para) needs to be taken into account, since the symmetry of the nuclear spins allows only for either even or odd values of the rotational quantum number J. When adding the binned FSDs for different initial states and isotopologues of the parent molecules one needs to adjust the energy scales (energy offsets).¹⁵

The calculation is performed for all isotopologues, i.e. steps 3 and 4 as well as the binning (including the temperature effects) are repeated for ${\textrm{T}_2},$ ${\textrm{DT}},$ and ${\textrm{HT}}.$ The three binned spectra are combined by first adjusting the different energy scales and then adding the probabilities with the corresponding weights determined by relative isotopologue concentration. Consequently, also new mean transition energies per bin are obtained. This whole procedure results finally in the FSD (a list of $(V_j,$ $\zeta _j)$ values) used in the fit of $m_\nu ^2.$ Once the transition matrix elements $M_{f,i}^{\textrm{TS}}(\tilde{K})$ (see Eq. (3)) for the three isotopologues and a sufficient number of initial rotational states are evaluated and stored, the FSDs for different bin sizes, temperatures, or isotopologue mixtures can be obtained straightforwardly. The molecular calculations are performed adopting atomic units in version Hartree (setting $\hbar =m_e=4\pi \epsilon _0=e=1)$ with distances given in units of Bohr, $a_0\approx 5.29\cdot 10^{-11}$ m, energies in Hartree, $E_H = \hbar ^2/(m_e a_0^2) \approx 27.2114$ eV, and masses in units of the mass of the electron, $m_e\approx 5.11\cdot 10^5$ eV/c$^2.$

These units are used in the following if not stated otherwise.

3.2 Electronic potential curves and overlaps

There are various ways to find the solutions of the electronic eigenvalue problem Eq. (5) for diatomic two-electron molecules. In order to efficiently obtain highly accurate solutions, the wavefunctions $\varPhi$ may be expressed as a linear combination of N explicitly correlated two-electron basis functions $\phi ,$ so called geminals,

$$\begin{aligned} \varPhi _n({\varvec{r}}_1,{\varvec{r}}_2; R)=\sum _{j=1}^N \, c_{n,j} (R) \; \phi _j({\varvec{r}}_1,{\varvec{r}}_2; R). \end{aligned}$$

(15)

These geminal expansions converge much faster than the most accurate standard quantum-chemistry approach known as full configuration interaction (CI) (also known as exact diagonalisation, see [18]) and thus accurate results can be obtained very efficiently. In the CI approach, the basis functions $\phi$ are uncorrelated tensor products (more accurately properly anti-symmetrised Slater determinants) of two one-electron basis functions, usually Hartree-Fock orbitals. Since accuracy is essential, the geminal expansion is used in the following.

The (non-relativistic) one-electron diatomic problem is separable in prolate spheroidal coordinates. For an electron i these coordinates are defined as $\xi _i=(r_{i,A}+r_{i,B})/R,$ $\eta _i=(r_{i,A}-r_{i,B})/R,$ and the angle $\phi _i$ around the internuclear axis, where A and B are the two foci of the ellipse, naturally to be chosen as the positions of the two nuclei. The so-called Kołos–Wolniewicz (or James-Coolidge) basis functions (see, e.g., [19]) are defined as

$$\begin{aligned} \phi _{j,\text {KW}} ({\varvec{r}}_1,{\varvec{r}}_2; R)= & {} \mathcal{N}_j \, \rho _{12}^{\mu _{j}} \, \xi _1^{\lambda _{j}} \, \xi _2^{\bar{\lambda }_{j}} \, \eta _1^{\nu _{j}} \, \eta _2^{\bar{\nu }_{j}} \;\nonumber \\ {}{} & {} \quad e^{- \alpha \xi _1 - \bar{\alpha } \xi _2 - \beta \eta _1 - \bar{\beta } \eta _2 } . \end{aligned}$$

(16)

Here, $\rho _{12}=|{\varvec{r}}_1-{\varvec{r}}_2|$ and $\mathcal{N}_j$ is a prefactor that depends on R,  but is constant for a specific value of R.¹⁶ Furthermore, $\alpha ,$ $\bar{\alpha },$ $\beta ,$ and $\bar{\beta }$ are (in general) non-integer parameters that form a base. This base is identical for all basis functions of a given basis set. A specific basis function j for a given base is characterised uniquely by a set of five integers, the quintuple $\{\mu _j, \lambda _{j}, \bar{\lambda }_j, \nu _j, \bar{\nu }_j\}.$ More accurately, linear combinations of the basis functions in Eq. (16) are adopted depending on the symmetry. A basis set is thus defined by the specification of a base and a set of quintuples, one quintuple per basis function. A summary of the namings related to the base used in this work is given in Appendix A.

Since the basis functions defined in Eq. (16) are not orthogonal to each other, their use in the ansatz Eq. (15) for solving the electronic eigenvalue Eq. (5) yields the generalised eigenvalue problem

$$\begin{aligned} \textbf{H}_{\textrm{el}} \, \textbf{c}_n(R) \;=\; V_n^{\textrm{BO}}(R) \, {\tilde{\textbf{S}}} \, \textbf{c}_n \end{aligned}$$

(17)

with the electronic Hamiltonian matrix $\textbf{H}_{\textrm{el}}$ with the matrix elements

$$\begin{aligned} h_{i,j}(R) =\langle \,\phi _i(R)\,|\,\hat{\textrm{H}}_{\textrm{el}}\,|\,\phi _j(R) \,\rangle \end{aligned}$$

(18)

and the overlap matrix ${\tilde{\textbf{S}}}$ with the matrix elements

$$\begin{aligned} \tilde{s}_{i,j}(R) =\langle \,\phi _i(R)\,|\,\phi _j(R) \,\rangle . \end{aligned}$$

(19)

In Eqs. (18) and (19) the bra-ket notation implies an integration over the coordinates of the two electrons, only. The solution of the generalised eigenvalue problem Eq. (17) for different values of the internuclear separation R provides the Born–Oppenheimer potential curves $V_n^{\textrm{BO}}(R)$ that enter the corresponding nuclear-motion Hamiltonian $\hat{\textrm{H}}_{\textrm{nuc},J}^{(n)}$ Eq. (13) and the eigenvector coefficients $\textbf{c}_n(R)$ that define the electronic wavefunctions according to Eq. (15). Solving this equation for both, the parent molecule TS and the daughter molecular ion HeS$^+,$ allows finally for the evaluation of the electronic transition matrix elements (asymmetric overlaps) $S_n(R)$ according to Eq. (8) as

$$\begin{aligned} S_n(R) =\sum \limits _{i=1}^{N^{{\textrm{HeS}}^+}} \, \sum \limits _{j=1}^{N^{\textrm{TS}}} \; c_{n,i}^{{{\textrm{HeS}}}^+} (R) \, s_{i,j}(R) \, c_{0,j}^{\textrm{TS}}(R) \end{aligned}$$

(20)

with the matrix elements

$$\begin{aligned} s_{i,j}(R) =\langle \,\phi _i^{{{\textrm{HeS}}}^+}(R)\,|\,\phi _j^{\textrm{TS}}(R) \,\rangle . \end{aligned}$$

(21)

The indices denoting the molecular system emphasise that usually the number of basis functions N and the base are chosen differently for parent and daughter molecules as to optimally describe these different systems in a more efficient way. The potential curves and electronic overlaps used in the KNM1-FSD calculation were obtained with a variant of a code originally written by Kołos, Wolniewicz and co-workers (for brevity abbreviated as Kołos code in the following).¹⁷

3.3 Basis-set convergence parameter $\varOmega$ and number of included final states

According to the variational principle, the energies (and eigenvectors) obtained in a basis-set calculation can be systematically improved by increasing the basis set, i.e. by the addition of more and more basis functions. However, an increase of the size of a basis set typically causes the contained set of basis functions to become linearly dependent due to the finite numerical precision. Instead of canonical orthogonalisation that is of limited use, another solution is provided by using variable precision in the calculation, i.e. the number of digits used in the representation of the floating point numbers. Based on a library (MPFUN) that allows for the use of flexible precision with FORTRAN programs, Pachucki, Zientkiewicz, and Yerokhin [21] implemented a new code (H2SOLV), which uses Kołos–Wolniewicz basis functions but allows for the systematic increase of the basis set. More accurately, Pachucki et al. use the slightly different notation and form,

$$\begin{aligned} \phi _{j,{\textrm{Pa}}} = e^{- u \xi _1 R - w \xi _2 R - y \eta _1 R - x \eta _2 R} \rho _{12}^{n_{0,j}} \eta _1^{n_{1,j}} \eta _2^{n_{2,j}} \xi _1^{n_{3,j}} \xi _2^{n_{4,j}},\nonumber \\ \end{aligned}$$

(22)

compared to the basis function in Eq. (16). The difference is the explicit inclusion of the internuclear separation R in the non-integer base parameters u, w, y,  and x. For all calculations in this work a modified version of the H2SOLV code, in the following called H2SOLVm, was used.

With the linear-dependency problem under control (though paying the price of requiring much larger computational resources), the basis set can be systematically increased by simply adding more and more basis functions j that differ only by the quintuples, which are denoted by $\{n_{0,j}, n_{1,j}, n_{2,j}, n_{3,j}, n_{4,j}\}$ in the H2SOLV code. In order to systematically increase the basis set, the H2SOLV code introduces a single parameter $\varOmega$ that, together with the base, i.e. the values of u, w, y,  and x,  defines the complete basis set. For a given value of $\varOmega$ all basis functions j,  i.e. all quintuples $\{n_{0,j}, n_{1,j}, n_{2,j}, n_{3,j}, n_{4,j}\}$ with integer values $n_{i,j},$ are included that fulfill

$$\begin{aligned} \sum _{i = 0}^{4} n_{i,j} \le \varOmega . \end{aligned}$$

(23)

In more detail, symmetries (especially spin symmetry as well as D$_{\infty h}$ and C$_{\infty v}$ symmetries¹⁸ for a homonuclear or heteronuclear diatomic molecule, respectively) are explicitly considered. Therefore, corresponding linear combinations of the basis functions Eq. (22) are used that transform like irreducible representations of the corresponding symmetry group. As a consequence, the total number of basis functions $N(\varOmega )$ is smaller than the value obtained from the restriction Eq. (23). Due to the variational principle, the wavefunction is improved (or remains of equal quality¹⁹), if more basis functions are added. Therefore, $\varOmega$ is the central convergence parameter. An exact solution will be obtained, if $\varOmega$ approaches infinity. As shown in Sect. 6.1.1, if $\varOmega$ is increased in a set of test FSDs, a well-defined convergence behaviour of the extracted $m_{\nu }^2$ is observed.

It should be noted that the basis-set optimisation based only on the $\varOmega$ variation is less efficient than the careful selection of basis functions (quintuples), as it was done for the previous FSD calculations [7, 22, 23], including the KNM1 FSD. However, especially for the present purpose of an uncertainty investigation it is advantageous to have a single (or only a few) convergence parameter(s).

As already mentioned, all final states need to be considered in the FSD that have energies lying in the energy interval used in the KATRIN analysis. Even within the 40 eV wide energy interval below the end point considered in this work, we must consider the infinite number of electronically bound states: the Rydberg states. On the other hand, if a finite basis set comprising N basis functions is used, only a finite number of N states can be obtained. Furthermore, depending on the chosen base, the fraction of these states representing bound or discretised continuum states differs, even for the same value of $\varOmega .$ More importantly, since the convergence study is based on a systematic enlargement of $\varOmega ,$ the size of the basis-set and thus the number of basis functions N increases. As a consequence, the number of electronic states obtained within the 40 eV energy interval increases with $\varOmega ,$ and this in a rather unpredictable way. While this would be already an issue for atoms, in the case of molecules the nuclear motion leads to the potential curves (see Eqs. (5) and (10)). In principle, any electronic state n may contribute to the 40 eV interval, if its energy $V_n^{\textrm{BO}}(R)$ lies for some value of the internuclear separation R below 40 eV. In summary, the number of electronic states that is included in the generation of the test FSDs needs to be adapted, if $\varOmega$ is varied, in order to obtain consistent FSDs covering the same energy interval. Consequently, and keeping in mind that all the electronically excited states of ${^3\textrm{HeH}^+}$ are purely repulsive in the Franck–Condon window, see [10, 24], the Born–Oppenheimer potential curves for all states obtained with a given basis set are computed. Then the number of states, $N_{\text {states}},$ are determined that lie below the ${40\,\mathrm{\text {eV}}}$ threshold, i.e. the energy difference to the absolute (electronic, vibrational, and rotational) ground state is less than 40 eV. In this selection of states the maximum distance considered for the calculation of the excited states is set to $R_{\text {max, ex}} = 4\,\text {a}_0.$ The resulting value of $N_{\text {states}}$ for each $\varOmega$ and thus included in the corresponding FSD calculation is listed in Table 4. (Note, in the calculation of the KNM1 FSD 13 electronically bound states were considered, as obtained with the basis set used in that calculation.) Since the adiabatic corrections (see Sect. 6.1.9) are not considered for the electronically excited states, this leads to a slight overestimation of $N_{\text {states}}.$ These additional states will result in additional probability in the FSD, but it will only affect the FSD more than 40 eV below the end point, so they do not need consideration here.

Table 4 Number of electronic states $N_{\text {states}}$ below ${40\,\mathrm{\text {eV}}}$ used for the computation of the FSD in dependence of $\varOmega$

3.4 Nuclear motion and transition matrix elements

For every electronic state $(n_i=0$ for the electronic ground state of TS; $n_f$ for the electronic ground and excited states of $^3$HeS$^+,$ S = H, D, or T) the radial nuclear Schrödinger Eq. (12) has to be solved separately using the corresponding Born–Oppenheimer potential curve $V_n^{\textrm{BO}}$ of that electronic state. This yields the radial part $\tilde{\chi }_{n;v,J}$ of the nuclear-motion wavefunction and the complete molecular energy $E_{n;v,J}$ of the electronic, vibrational, and rotational state within the Born–Oppenheimer approximation. While the electronic ground states of parent and daughter molecules support a number of bound solutions besides states within the dissociation continuum, the potential curves of the electronically excited states are almost purely repulsive, i.e. they do not support bound molecular states, but only dissociative ones; at least within the considered R range. Clearly, due to the dissociation continua there is an infinite number of nuclear-motion states for each potential curve, and thus a selection is needed in practice.

There are two basic types of approaches for solving the eigenvalue equation (12) and thus for finding the nuclear-motion wavefunctions and total molecular energies within the Born–Oppenheimer approximation: direct numerical integration or the variational (basis-set) approach. Direct numerical integration was adopted in the older FSD calculations in [7, 22]. However, the treatment of the dissociation continuum turns out to be very difficult to be automatised, if many predissociative resonances occur, as is the case for the electronic ground state of $^3$HeT$^+.$ Thus the calculation involved a lot of human interaction and is prone to errors, since different boundary conditions need to be applied which can lead to double counting of the same state (for details see, e.g., [25]). As the uncertainty analysis proposed in this work requires the generation of a very large number of FSDs, such a procedure is prohibitive and a variational approach based on B-spline basis functions was adopted.

In the case of the KNM1 FSD as well as all FSDs generated in this work the radial nuclear-motion eigenvalue problem was solved by adopting a basis-set expansion similar to the one that was described in Sect. 3.2 for the solution of the two-electron problem. Much simpler basis functions are needed in this one-dimensional case and the convergence is much faster. One popular and flexible choice of basis functions are B splines, see, e.g. [26], that are a generalization of cubic splines to arbitrary polynomial order k, 

$$\begin{aligned} \tilde{\chi }_{n;v,J} =\sum \limits _{n_B=1}^{N_B} \, d_{n;v,J;n_B} \, B_{n_B}^{(k)}(R). \end{aligned}$$

(24)

This ansatz inserted into the eigenvalue equation leads to a generalised, but sparse eigenvalue problem, since the B splines are non-orthogonal and only overlap with a small number of neighbor B splines. The bandwidth of the sparse matrix depends directly on the adopted order k of the B splines. Besides their order, the B-spline basis is defined by the so-called knot sequence, i.e. the sequence of radial grid points defining their support. The radial grid is finite and thus the wavefunctions are confined within some finite (spherical) box with radius $R_{\textrm{max}}.$ Fixed boundary conditions are applied at the upper grid boundary (setting the wavefunction and its derivatives to zero at the last grid point) yielding discretised states in the dissociative continuum with Kronecker $\delta$ normalization. For convenience, the wavefunction and its derivative at $R=0$ are set to zero. These boundary conditions are easily implemented by removing the corresponding B splines. Therefore, if $N_B$ is the number of adopted B splines the number of states that can be obtained is $N_B-2.$ While the density of the knot points is decisive for the state with maximum energy that is yielded, the box size and thus $R_{\textrm{max}}$ determines the density of states in the resulting discretised dissociation continuum. Since it is a single-channel scattering problem, a renormalisation to energy-normalised states is, in contrast to the case of the ionization continuum of two electrons, straightforward and thus transition-probability densities (per unit of energy) can easily be obtained from the discretised transition probabilities (using a renormalisation procedure based on the energy density of states).

Once the radial nuclear-motion wavefunctions $\tilde{\chi }$ are obtained in either way, they are then used (together with the electronic transition matrix elements $S_n$ and the spherical Bessel functions) to solve the corresponding radial integrals I,  see Eq. (14). This integration is done numerically using quadrature. This quadrature is exact in the case of B splines, if Gaussian quadrature is adopted and the product of the spherical Bessel function and the electronic transition matrix elements can be expressed as a finite polynomial. The integrals finally yield the state-to-state transition probabilities P according to Eq. (3).

4 Quantifying the FSD uncertainty

The calculation steps for the FSD as described in Sect. 3 introduce uncertainties on the resulting shape of the FSD and therefore on the neutrino mass extracted from a measured $\upbeta$-decay spectrum. In the following, the different kinds of uncertainties which contribute to the FSD uncertainty are introduced. Afterwards, it is described how the total FSD uncertainty had previously been estimated for the first two measurement campaigns of KATRIN.

4.1 Types of uncertainty contributions from the FSD

There are different types of uncertainties of the FSD obtained from theoretical calculations, four with origin in the theory and calculation itself, and one with experimental origin. First of all, there are uncertainties due to the adopted approximations (Sect. 4.1.1), e.g. the sudden approximation. Assessing the validity of the approximations ensures that the uncertainty induced by them is sufficiently small. Second, there are possible uncertainties in constants entering the calculation (Sect. 4.1.2), for example the nuclear masses. Third, there are uncertainties due to the use of finite basis-sets and finite numerical precision in the ab initio calculations of energies, wavefunctions, transition matrix elements, and probabilities (Sect. 4.1.3). Since some of the corrections to the adopted approximations are calculated using the calculated energies and wavefunctions the evaluated corrections themselves have uncertainties due to the adopted finite basis sets. Finally, there is also a possibility of errors made in the computer codes or the input files (Sect. 4.1.4).

If the FSD is binned and isotopologue composition as well as temperature (including possible spin-statistical effects) are considered, i.e. if the FSD is a sum of FSDs with corresponding statistical weighting factors, an additional uncertainty is introduced. However, in this case the uncertainty has a purely experimental origin, since it is caused by the uncertainty in temperature and isotopologue distribution in a given experiment. Estimations on the effect of these uncertainties with experimental origin on $m_\nu ^2$ can be found in Sect. 6.1.5 and Appendix C.

4.1.1 Approximations adopted in the FSD calculation

The FSDs used so far in the analysis of tritium $\upbeta$-decay experiments rely on the sudden approximation. Near the end point of the $\upbeta$-decay spectrum, the $\upbeta$-decay electron is emitted with large kinetic energies of more than 18 keV and thus with very high speed. The remaining atomic or molecular system experiences in first order the decay process as a sudden change of the charge of the decaying nucleus. The validity of the sudden approximation including an explicit calculation of the leading corrections was investigated in [16, 17, 27]. It was found that the leading correction stems from the Coulomb distortion of the wavefunction of the emitted $\upbeta$-decay electron by the decaying nucleus and this effect is described by a simple (and well-known) factor, the so-called Fermi function. The function depends on the kinetic energy of the $\upbeta$-decay electron and the nuclear charge of the decaying nucleus. This factor is included in the spectrum model of KATRIN (as was also done in the analysis of previous experiments). The next largest correction to the sudden approximation due to the spectator nucleus and the two molecular electrons is explicitly given in [17] where it was found that it is mostly proportional to the probability in the sudden approximation. Since the $m_\nu ^2$ measurement by KATRIN does not depend on the total probability, but the relative distribution of the probabilities as a function of energy, the effective impact of the correction to the sudden approximation is practically reduced by about one order of magnitude [17]. Based on these results it was concluded that at least for the first science campaigns of KATRIN the sudden approximation is applicable, see Sect. 6.1.8 for more details.

In the calculation of the KNM1 FSD (as in the ones before) the Born–Oppenheimer approximation was applied. The electronic problem in Eq. 5 was solved as a function of a fixed internuclear separation yielding potential curves as described in Sect. 3.2. The electronic wavefunctions were then used in the calculation of the electronic transition matrix elements $S_n.$ The potential curves enter the Schrödinger equation describing the vibrational and rotational degrees of freedom of the nuclei. In the derivation of the Born–Oppenheimer approximation, the matrix elements $\langle \,\varPhi _n\,|\,\varDelta _R\,|\,\varPhi _m\,\rangle$ stemming from the action of the kinetic-energy operator of the nuclei $\varDelta _R$ on the electronic wavefunctions $\varPhi$ were neglected. Considering the diagonal matrix elements (the ones between the same electronic wavefunction, $n=m)$ leads to the adiabatic approximation, while the off-diagonal matrix elements $(n\ne m)$ that couple different electronic Born–Oppenheimer states are known as non-adiabatic corrections. For the ground states of both the parent and the daughter molecules the adiabatic corrections were included in the KNM1 FSD (as they were in the FSDs in [7, 22]).

The non-adiabatic (off-diagonal) corrections have been considered in the calculation of the transition probabilities to a variety of rovibrational states of ${^3\textrm{HeT}^+}$ in [28, 29]. It was found that although the transition probabilities to some individual states changed by the order of ${\mathcal {O}}(10^{-3})$, these transition probabilities are effectively only shifted within a very small energy window, i.e. within two states that lie energetically very close to each other. The total change to the electronic ground state was found to be of order ${\mathcal {O}}(10^{-4})$ which is accidentally very similar to the change due to beyond-sudden-approximation effects (see Sect. 6.1.8). The corresponding uncertainty analysis following the procedure introduced in this work yields an uncertainty of the order of ${\mathcal {O}}(10^{-4}$ eV$^2/$c$^4)$. Thus the non-adiabatic corrections were concluded to be neglected for the first science campaigns of KATRIN, but will be re-investigated in the future.

The calculation of the FSDs created in this work was performed using basically non-relativistic quantum mechanics and neglecting effects stemming from quantum electrodynamics. Only for the electronic ground state of the parent molecule first-order relativistic and radiative corrections were included, since they are available in literature [30]. All of the corrections (adiabatic $V_n^{\textrm{ad}},$ relativistic $V_n^{\textrm{rel}}$ and radiative $V_n^{\textrm{rad}})$ were included by adding them to the corresponding Born–Oppenheimer potential curve $V_n^{\textrm{BO}},$ resulting in the corrected potential curve

$$\begin{aligned} V_n^{\textrm{cor}} = V_n^{\textrm{BO}} + V_n^{\textrm{ad}} + V_n^{\textrm{rel}} + V_n^{\textrm{qed}}. \end{aligned}$$

(25)

The corrected potential curve $V_n^{\textrm{cor}}$ was used, instead of the Born–Oppenheimer potential curve $V_n^{\textrm{BO}},$ in the nuclear-motion Hamiltonian from Eq. (13). Since the Born–Oppenheimer potential curves and corrections are not given for the same internuclear separations R,  the corrections were spline interpolated and then evaluated at the internuclear separations of the Born–Oppenheimer potential curves. The effect of the corrections to the Born–Oppenheimer approximation on $m_\nu ^2$ is given in Sect. 6.1.9.

Another approximation has been adopted in order to obtain the operator $e^{{i{\tilde{\varvec{K}}}\cdot {\varvec{R}}}}$ in Eq. (3), see [16]. The transition operator resulting from the transformation from the lab to the molecular frame is only approximately correct in the form given in Eq. (3). The physical origin of the transition operator in the lab frame is that it describes transitions induced in the daughter molecule by the recoil of the departing $\upbeta$ electron and the neutrino. In view of the large mass difference and the fact that close to the end point of the $\upbeta$ spectrum basically all decay energy is given to the electron, the influence of the recoil induced by the neutrino is, however, negligible, see [16]. The transformation to the transition operator in the molecular frame splits the effect of the recoil of the emitted electron into two parts. One is responsible for excitation of the center-of-mass motion of the molecular ion, i.e. for excitation of the translational degree of freedom, the other one for excitation of the internal molecular degrees of freedom (electronic, vibrational, and rotational). In the approximated form given in Eq. (3) only rotational and vibrational excitations can be induced, while the correction describing electronic excitation due to the recoil is ignored. In [16] the size of this effect, i.e. the probability for recoil-induced electronic transitions, was estimated based on a sum rule and found to be of the order of $5\cdot 10^{-5}$ and was therefore neglected.

Clearly, also the value of this fractional recoil $\tilde{K}=|\tilde{\textbf{K}}|$ changes with the energy of the $\upbeta$-decay electron. This energy is reduced by the amount of energy left in the daughter molecular ion, so $\tilde{K}$ varies as a function of the energy at which the transition probability (and thus FSD) is evaluated. This effect was, however, ignored in the KNM1 FSD, assuming it is negligibly small. It was neglected in the pseudo-KNM1 FSD as well. An analysis of the uncertainty induced by this approximation is given in Sect. 6.1.7.

Finally, an approximation is adopted with respect to the reduced mass used when solving for the rovibrational wavefunctions $\chi$ according to Eq. (12) both in the KNM1 FSD and in the FSDs of this work. In principle, within the Born–Oppenheimer description, the Schrödinger equation describing nuclear motion should include the masses of the electrons, but is dependent on the electron density. To give a simplified example, consider if both electrons are in the case of ${^3\textrm{HeT}^+}$ mostly at the He nucleus (ionic bond). In this case, both electron masses should be added to this nucleus. If the electrons are equally distributed (covalent bond), then the mass of a single electron should be added to each nucleus. Since the electron density depends evidently on the electronic state and even within one state it changes with internuclear separation, this effect is non-trivial to be included exactly into the calculation. Clearly, an R dependent reduced mass (that needs to be obtained for every electronic state separately) should be included. In Sect. 6.1.3 the most extreme case of inclusion and exclusion of the electron masses in the calculation of the reduced mass, but independent of the internuclear separation, is investigated.

4.1.2 Uncertainties on fundamental constants and conversion factors

The Hamiltonians used in the calculation contain fundamental constants like the electron mass, the Planck constant, etc. The Hamiltonian describing nuclear motion depends on the nuclear masses. As already discussed, the fractional recoil $\tilde{\textbf{K}}$ depends on the energy of the $\upbeta$-decay electron, where the maximal electron energy (end-point energy $E_0)$ is not exactly known. The KNM1 FSD adopted a more precise value of $E_0$ than the FSDs in [7, 22]. Using different $E_0$ values in the FSDs results in an effect on the uncertainty similar to the neglect of the variation of $\tilde{K}$ within the 40 eV interval discussed in Sect. 4.1.1. These effects on $m_\nu ^2$ are given in Sects. 6.1.6 and 6.1.7.

In practice, all FSD calculations are performed using atomic units (Hartree), but for the analysis of KATRIN data the energies need to be converted into eV. This conversion factor depends on fundamental constants like the Planck constant or the elementary charge. The conversion factor changes over time as a result of more precise determinations of the fundamental constants. In fact, the corresponding recommended CODATA value changed in between the publication of the FSD by Fackler et al. [22] and the calculation of the KNM1 FSD more than once. The relative error between the recommended CODATA values from 1986 and 2018 is about $3.6 \times 10^{-7}.$

4.1.3 Uncertainties due to convergence and precision

Although the choice of molecular tritium in neutrino-mass experiments like KATRIN is also motivated by the fact that it is amenable to very accurate theoretical calculations, these calculations are always limited by the finite basis sets and numerical precision adopted in the calculation. In this section a compact overview is given indicating in which steps of the FSD calculation convergence and precision issues enter. The most critical part is the treatment of the electronic problem, despite the use of very efficient basis functions. A further complication arises from the fact that there is no universal basis set for treating all the electronic states at the same time within the same precision. This complicates the uncertainty analysis, since not only different uncertainties need to be assigned to different electronic states, but these uncertainties enter different, but partially overlapping energy ranges of the FSD. Consequently, the impact of a specific state depends on both the fit range and the relative statistical weights given to the fit intervals.

In the case of the one-electron tritium atom and its decay to $^3$He$^+,$ the transition probabilities can be calculated analytically if a non-relativistic description within the sudden approximation is adopted and possible electronic excitation due to the recoil is also neglected. The case of a two-electron atom requires on the other hand numerical solutions of the Schrödinger equation. In the case of a two-electron molecule like ${\textrm{T}_2},$ even within the Born–Oppenheimer approximation a numerical solution of the Schrödinger equation describing both electronic motion and nuclear motion is required, as was discussed in Sect. 3.4. An additional challenge is the need for a calculation of the energies and transition probabilities of all molecular final-states that lie within the energy interval used in the analysis of the $\upbeta$-decay spectrum, including possibly dissociation and ionisation continua.

The use of explicitly correlated exponential basis functions in prolate-spheroidal coordinates known as Kołos–Wolniewicz functions, see Eq. (16), for obtaining the solutions of the electronic Schrödinger equation of a diatomic two-electron molecule (and adopted in the Fackler FSD [22], the Saenz FSD [7], and the KNM1 FSD [10]) has proven to be extremely efficient and accurate. It is, however, limited by the fact that one basis set is defined by one base, i.e. a single set of the non-integer parameters $\{\alpha , \bar{\alpha }, \beta , \bar{\beta }\}$ (Kołos–Wolniewicz) or $\{u,w,y,x\}$ (Pachucki et al.), see Sect. 3.3. While a proper, judicious choice of the base allows for obtaining very accurate results for a given electronic state with a very small number of basis functions (where each basis function j being defined by the integer quintuple $\{\mu _j,\lambda _j,\bar{\lambda }_j,\nu ,\bar{\nu }_j\})$, formally an infinite number of basis functions would be required to describe an electronic state exactly. In the case of a finite basis set, the base should, in principle, be optimised individually both for every electronic state and internuclear separation R. However, the choice of the optimal base (non-integer parameters) depends also on the selection of basis functions (integer quintuples) and vice versa.

In order to obtain the best possible results for all considered final states, inputs from different calculations and basis sets have been used in the generation of the KNM1 FSD. For the electronic ground states of parent and daughter molecules highly optimised potential curves (including adiabatic corrections for all molecules, for the parent molecules TS also relativistic and radiative ones), see Appendix B.1, and electronic overlaps $S_n(R)$ were adopted from literature [19]. Similarly, the potential curves and overlaps for the lowest-lying five excited electronic states were taken from literature (see [29]) where they had been optimised individually for the various states. The remaining (Rydberg) states (corresponding for the adopted basis to the electronic states 7 to 13) were taken from literature where they had been obtained with a single basis set that was found to provide the best compromise for representing all excited states, as well as the electronic continuum. Clearly, while the accuracy of the electronic input data is supposed to be very high (and for the electronic ground-state potential curves this was confirmed by a comparison to various experimentally determined transition energies, see, e.g., [29, 31, 32]), the use of finite, rather than infinite, basis sets leads to an FSD uncertainty. Note, it is easier to achieve the basis-set completeness and thus higher accuracy in the one-dimensional equation of nuclear motion, Eq. (12), compared to the six-dimensional electronic part. Of course, in all cases the question of finite precision arises, for example also in the quadrature used when calculating the transition probabilities, i.e. the integrals in Eq. (14). The effect of using a finite basis set is evaluated in Sect. 6.1.1, the effect of the choice of specific base parameters on $m_\nu ^2$ in Sect. 6.1.2. The choice of ground-state energies, which is related to the convergence of the basis set, has an impact on $m_\nu ^2$ as described in Sect. 6.1.4.

4.1.4 Coding or input errors

Besides uncertainties there is, despite careful testing and cross-checking of the used codes and inputs, evidently also the possibility of errors in the various computer codes used in order to evaluate the FSD and the quantities entering this calculation. Furthermore, input parameters may not be chosen properly, or typos may have occurred. Clearly, these are errors and not uncertainties, but in those cases where such errors were detected after an FSD had already been used in the analysis of experimental data, it is evidently of interest to evaluate the effect that a given error has on the extracted $m_\nu ^2.$ Based on the analysis of the impact of a given error on the fit result it can be decided whether the experimental data need to be re-analysed with a properly corrected FSD, or whether the impact of the error is negligible with respect to the other uncertainties. As in the case of the uncertainties the impact of a given error on the extracted neutrino mass depends on the experimental conditions and the fit range. The new uncertainty analysis proposed in this work in Sect. 5 takes these experimental conditions into account and provides thus also a helpful tool for the analysis of the impact of errors. For example, the reduced masses of the different isotopologues used in the generation of the KNM1 FSD turned out to be not fully consistent and the present work confirms that for the first KATRIN measurement campaign the impact is negligible (the largest effect is of order ${\mathcal {O}} (10^{-4}$ eV$^2/$c$^4)$), see Sect. 6.1.3 for details.

4.2 FSD-uncertainty estimate adopted in KNM1

In the analysis of the first neutrino-mass measurement campaign of KATRIN the systematic contribution of the uncertainties of the FSD was estimated based on more general properties of the FSDs, namely, the moments of the distribution. A conservative estimate from a comparison of the ab initio calculations by Saenz et al. [7] and the earlier ones by Fackler et al. [22] yielded for the electronic ground-state contribution to the FSD (energy interval below ${10\,\mathrm{\text {eV}}})$ an uncertainty of 1% on the variance. However, in the generation of the Saenz and Fackler FSDs, the same Born–Oppenheimer potential curves and corrections to them as well as the same electronic transition moments were used for the electronic ground states. This is legitimate because the ones introduced by Fackler were found to be very accurate. Only for the electronically excited bound states slightly improved potential curves and matrix elements were used in the Saenz FSD in [7]. The main difference between the two FSDs consisted in the treatment of the electronic continuum that, however, only contributes to the FSD at excitation energies above 40 eV, i.e. to the $\beta$-spectrum below $E_0-40$ eV. It is thus not relevant for the first two science campaigns of KATRIN in which the fit range used in the analysis did only extend to 40 eV below the end point. On the other hand, in [15] it was shown that the non-physical $m_\nu ^2$ in the results of the LANL [33] and LLNL [34] experiments that were analysed with the Fackler FSD could be resolved, if the Saenz FSD with a better description of the continuum was adopted. Below 40 eV the dominant difference between the Fackler FSD and the Saenz FSD is due to the use of the non-relativistic and the relativistic value for the fractional recoil $\tilde{K},$ respectively, as is explained in [29]. It leads to a shift of the mean excitation energy by approximately 0.03 eV and a change in variance of the order of 1%.

As a rather conservative uncertainty estimate for the FSD, and considering a good agreement of the theoretical predictions and the measurement of the probability of dissociated to non-dissociated molecular fragments by the TRIMS experiment [35], a 1% uncertainty on the normalisation of the ground to excited states populations was assumed. Based on these findings, the uncertainty of the total variance of the FSD was constrained to 2%. The corresponding uncertainty of the variance of the electronically excited states and ionisation continuum was set to 4%.

The FSD uncertainty estimated this way was propagated into the $m_\nu ^2$ uncertainty using two different techniques: the covariance matrix and the Monte-Carlo propagation. In both approaches the binned FSD was modified randomly in a bin-to-bin uncorrelated way to provide the corresponding variations of the electronic ground-state variance (1%), its probability (1%), and the variance of the electronically excited states (4%). Repeating this modification many times and calculating the corresponding integrated spectra a covariance matrix was built for the FSD-related uncertainty. The constructed covariance matrix in the first approach and randomised FSDs in the Monte-Carlo propagation method were used in the spectral fit of the data. The corresponding variation of the $m_\nu ^2$ parameter yielded the estimate of the additional uncertainty. In a typical narrow analysis interval of 40 eV below the end point $E_0$ the electronic ground state contributes the largest fraction to the measured spectrum and the FSD-related $m{_\nu }^2$ uncertainty was given as ${0.02}~\text {eV}^{2}/\text {c}^{4}$ [5, 9].

For the first two science campaigns of KATRIN, which both used the KNM1 FSD as input, this uncertainty estimate for the FSD was sufficient, since even the very conservative estimate indicated that the FSD contribution to the uncertainty budget was sufficiently small compared to the statistical uncertainty. However, with the increase of the statistics and other systematic uncertainties being reduced, it is crucial to provide a more stringent way to determine the FSD uncertainty; also because an unjustified too conservative estimate would unnecessarily limit the capability of KATRIN. This new way of uncertainty determination should also discriminate the contribution of each of the types of FSD uncertainties discussed in Sect. 4.1.

5 New FSD uncertainty analysis

In this chapter, the new procedure for determining the FSD uncertainty via a basis-set convergence approach is introduced. The general requirements for the uncertainty analysis are motivated and described in Sect. 5.1. As will be shown in Sect. 5.2, the new way of uncertainty determination cannot be applied to the KNM1 FSD as is. Finally, the concrete procedure of the uncertainty determination which fulfills the requirements listed in Sect. 5.1 is presented in Sect. 5.3.

5.1 General strategy for the uncertainty estimation

As discussed in Sect. 4.2, an estimate of the FSD uncertainty based on the agreement of the Fackler and the Saenz FSDs as was done previously for the uncertainty estimate of the KNM1 FSD may be not reliable enough: there is a substantial overlap both of the inputs and the computational approaches. This includes the potential curves and electronic overlaps for the electronic ground states of parent and daughter molecules. Although improved data were used for the electronically excited states, they were still calculated with the same type of basis functions, Kołos–Wolniewicz geminals. The Saenz FSD contains a substantial improvement in the electronic continuum, but this part of the FSD is only relevant, if the fit range of an experiment extends beyond 40 eV below the end point.

While there exists no direct experimental measurement of the molecular FSD besides the already mentioned recent re-determination of the dissociation branching ratio by the TRIMS experiment [35], there are numerous reasons and data that provide confidence in the theoretically obtained FSDs (Fackler FSD, Saenz FSD, and KNM1 FSD), which will be discussed in the following.

Electronic ground states: The potential curves of the electronic ground states of both the parent and the daughter molecules have been evaluated multiple times with various approaches, see e.g. [36,37,38,39,40]. Furthermore, there exists a wealth of spectroscopic data in which the transition energies between many rovibrational states have been very accurately measured for a number of isotopologues [32]. In [29] the energies obtained when adopting the potential curves that were used in generating the Fackler FSD were compared with numerous spectroscopic data and extremely good agreement was found. This finding together with the fact that basis-set variations, though also using Kołos–Wolniewicz geminals, did not lead to lower energies, and thus not to better results (as can be uniquely concluded on the basis of the variational principle), motivated their use in the evaluation of the Saenz FSD. In view of the requirements of KATRIN (higher temperature and the presence of the additional isotopologue DT), a new FSD was calculated in [23, 25], but using the same input and codes as used in the evaluation of the Saenz FSD. Furthermore, a comparison to new spectroscopic data was performed, again finding very good agreement [25]. Also more recent improved calculations of the electronic ground states of the parent and daughter molecules [40, 41] only increased the number of digits, but did not modify the potential curves adopted in the Fackler, Saenz, and KNM1 FSDs within the precision (number of digits) given therein.

Electronically excited states: There is much less confirmation of the correctness of the potential curves of the electronically excited states. In the case of the electronic transition matrix elements (always involving the electronic ground state of the parent molecule) there could even have been implementation errors, cf. Sect. 4.1.4. Therefore, before the KNM1 FSD was generated, numerous checks were performed that will be reported elsewhere [42]. For example, the electronic problem (potential curves, wavefunctions, and transition matrix elements) was additionally solved adopting a different method (configuration interaction adopting a B-spline basis in prolate-spheroidal coordinates [43]) and thus also using completely independently developed computer codes. The convergence behaviour was much slower compared to the geminal approach. Nevertheless, within the convergence that could be achieved, the input data (potential curves and matrix elements) used for the Fackler and Saenz FSDs were validated. Adopting Kołos–Wolniewicz geminals, but using now the new H2SOLV code [21] that allows for systematic basis-set improvements, again the high accuracy of the previously used FSD input data was confirmed.

Nuclear problem: Also the nuclear problem that was solved before by adopting numerical integration based on the Numerov method was re-evaluated with a different approach (expansion of the radial part in B-splines) and thus again a completely independently written computer code. Also in this case very good agreement was found. All these findings motivated the way the KNM1 FSD was generated and used in the analysis of the first two science campaigns of KATRIN.

Though the numerous convergence studies provided validation of the generated FSD, they do not provide an explicit uncertainty, i.e. a single value specifying the contribution of the uncertainty of the FSD to the overall systematic uncertainty of $m_{\nu }^2$ extracted from the KATRIN data. For example, the accuracy of the adopted potential curves as estimated from the convergence studies depends on the electronic state, and for a given state on the internuclear separation. An uncertainty in the potential curve leads to an uncertainty in the energies of the rovibronic (rotational, vibrational and electronic) states, simultaneously the electronic wavefunction (the eigenvector belonging to a given potential curve and thus the eigenvalue) and its possible uncertainty influences the electronic transition matrix elements. Clearly, the uncertainties are highly correlated in a non-trivial way.

In contrast to the situation for the electronic ground state, the accuracy of the electronically excited states and their uncertainties are much less precisely known due to the lack of alternative calculations, but especially due to the lack of experimentally determined spectroscopic data. However, the influence on the extracted $m_\nu ^2$ is much larger for the ground state, but its relative importance depends on the fit range included in the analysis.

There are thus different aspects that should be considered in the new uncertainty analysis. First of all, it has to depend on the experimental conditions: the energy interval included in the fit determining the molecular final states that contribute to the FSD, the temperature and the isotopologue distributions which affect the statistical weights with which different initial states contribute, the energy resolution requiring some minimal bin sizes, etc. Second, the impact on the extracted $m_\nu ^2$ by the individual sources of uncertainties should become transparent, also allowing for the identification of those ingredients or approximations entering the FSD that need to be improved most urgently, if the FSD uncertainty needs to be reduced. In this way, artificial reduction of uncertainties due to counteracting or canceling effects may be identified. Third, the accuracy of the potential curves, transition matrix elements, and corrections that are calculated with finite-basis-set approaches should be assessed based on systematic basis-set convergence studies. The uncertainty analysis should thus incorporate correlations between, e.g., the quality of the potential curves and the one of the corresponding wavefunctions, in a consistent way. Most importantly, the variable accuracy of the input data, e.g. the higher accuracy for the electronic ground states compared to the electronically excited ones, as well as their impact depending on the fit interval should be contained in the uncertainty analysis.

It is important to note, that the systematic basis-set convergence study, however, cannot be performed with the KNM1 FSD itself due to the mixed bases and partly missing input information, which motivates the need for a pseudo-KNM1 FSD for the studies presented here. In the following Sect. 5.2 the generation of this pseudo-KNM1 FSD is described, while Sect. 5.3 introduces the new procedure and discusses how these requirements are fulfilled.

5.2 Need of a pseudo-KNM1 FSD

Applying the proposed new uncertainty analysis to the KNM1 FSD results in some difficulties. First of all, different basis-sets (different values of both the base $\{\alpha ,\bar{\alpha },\beta ,\bar{\beta }\}$ and basis-set expansions, i.e. the number and the selection of the quintuples $\{\mu _j,\lambda _j,\bar{\lambda }_j,\nu _j,\bar{\nu }_j\},$ see Eq. (16) in Sect. 3.2) had been adopted in the evaluation of the electronic input data. Even worse, it was not possible to fully recover the complete information of all involved basis sets at all internuclear separations entering the previous calculations. For example, there is some cross-referencing between publications that do not contain sufficiently detailed information. Sometimes, simple typos are evident (the same basis function appearing twice) but nevertheless not (simply) recoverable (as it is unclear which basis function was used instead) or there were corrections given in some other publication, but it was not found to be certain whether this correction is correct itself. In fact, in some cases the canonical orthogonalisation had been adopted to handle numerically induced linear dependencies. Even if the cut-off threshold used for reducing the basis set is given, the results depend on the order of the basis functions (and thus the quintuples), but this information is not always available. Even the numerical precision of the adopted hardware and compiler, but especially the adopted expansion length in the von Neumann expansion used for solving the integrals may influence the result of the orthogonalisation and this information is often missing. It should be emphasised that these problems apply only to some of the input values. For all internuclear separations that contribute substantially to the FSD the achieved agreement to the literature values is $10^{-6}$ to $10^{-9}.$ Most of the input values could be reproduced to a sufficient degree (after a corresponding very laborious trial-and-error procedure based on inverse engineering), see Appendix B for details. Also, the input values itself (potential-curve values including the corrections to them or electronic overlaps) that entered the KNM1 FSD are all reproducible in the sense that every value used and taken from literature can be found in the correspondingly cited publications.

It was thus concluded that the demands in computational resources and the time that would be needed to recover every single basis-set parameter for every input value are too large while the impact on the here performed uncertainty analysis would be completely negligible in order to justify the attempt of a full input-data recovery. It should be again emphasised that these data are solely required for a fully consistent uncertainty analysis as it is proposed here, but not for the reproduction of the KNM1 FSD itself.

Due to the difficulties described above, it was decided to create a pseudo-KNM1 FSD, which is as close as possible to the KNM1 FSD, but for which all input parameters are known and can thus be systematically varied. The generation of the pseudo-KNM1 FSD is based on the input data and basically the same procedure and numerical apparatus used for obtaining the KNM1 FSD [10]. Most importantly, the pseudo-KNM1 FSD allows for the systematic basis-set enlargement required for the uncertainty analysis. Clearly, the need for a pseudo FSD stems from the way the KNM1 FSD was constructed that is not suitable for a systematic and consistent uncertainty analysis. This motivated the use of a differently obtained FSD for the analysis of more recent KATRIN measurement campaigns [44].

5.3 Procedure to estimate the KNM1 uncertainty

In view of the requirements the uncertainty analysis should fulfill, the following procedure is proposed and is illustrated by its application to the KNM1 FSD adopting the experimental conditions from the KNM1 measurement campaign. With the aid of a theoretically obtained pseudo-KNM1 FSD a $\upbeta$-decay spectrum is generated on the basis of the experimental parameters, setting the neutrino mass to zero. The pseudo-KNM1 FSD is close to the KNM1 FSD, but differs from the latter in the following ways:

Fig. 2

FSD for ${\textrm{T}_2}$ up to 40 eV (corresponding to 40 eV below the end point) as used in the analysis of the first KATRIN science campaign (KNM1 FSD), in comparison to the FSD for ${\textrm{T}_2}$ used for the uncertainty investigations in this work (pseudo-KNM1 FSD). On the left hand side the ground state is shown, while on the right hand side the excited states can be seen. (Note, the energy range in between is not shown, since the transition probability therein is negligibly small)

• The pseudo-KNM1 FSD uses nuclear reduced masses $\mu _{\text {n}}$ for the parent nuclei of all isotopologues and effective reduced masses $\mu _{\text {eff}}$ for the daughter nuclei, while the KNM1 FSD adopts simultaneously different types of masses, i.e. not all daughter isotopologues use $\mu _{\text {eff}}$ (see Sect. 6.1.3)

• The pseudo-KNM1 FSD uses a density approach [10] with probability densities for the description of the dissociation continua of the electronically excited states, while the KNM1 FSD uses a discretised approach. The consequences of using either of the approaches are discussed in Appendix B.2.

• The pseudo-KNM1 FSD is constructed with a basis-set convergence parameter $\varOmega =10,$ while the basis set of the KNM1 FSD corresponds effectively to values between $\varOmega =5$ and $\varOmega =7$ (see Sect. 3.3)

While the Born–Oppenheimer potential curves are independent of the nuclear masses, this is not the case in the adiabatic approximation. In the adiabatic case individual potential curves for the different isotopologues ${\textrm{TS}}$ as well as the corresponding daughter isotopologues are used in the FSD calculation. A comparison of the KNM1 FSD and the pseudo-KNM1 FSD is shown in Fig. 2. The impact of the difference between adopting either the KNM1 FSD or the pseudo-KNM1 FSD on $m_\nu ^2$ is discussed in Appendix D.

In order to investigate a specific uncertainty, the $\upbeta$-decay spectrum generated using the pseudo-KNM1 FSD is fitted by a $\upbeta$-decay spectrum using a test FSD (or set of test FSDs) describing this specific uncertainty (see Eq. (2)) and the $m_\nu ^2$ value is extracted. The resulting shift in $m_{\nu }^2$ provides a direct measure of the impact of the specific uncertainty. This specific uncertainty could be a specific limitation of the calculation (like the finite basis set), an adopted approximation (like the sudden approximation), or some input error.

Clearly, the result depends on the basis set adopted in the evaluation of the Born–Oppenheimer potential curves and electronic transition matrix elements. Therefore the investigation is repeated with a set of test FSDs with systematically improved basis sets. The values obtained for $m_\nu ^2$ by a fit on the Monte-Carlo data set described in Sect. 2.2 in which always the same pseudo-KNM1 FSD is used should then systematically converge to $m_\nu ^2=0$ eV$^2$/$c^4.$ Its deviation from this value provides a quantitative estimate of the uncertainty due to the considered limitation, approximation, or input error independently of the basis-set limitation. The results on the individual FSD uncertainties obtained by using the procedure described in this section will be discussed in Sect. 6.

6 Results and discussion

In this chapter, the results for the individual uncertainty contributions from the FSD calculation onto $m_\nu ^2$ are presented (see Sect. 6.1). To achieve consistency in the estimation of the uncertainty, all of the following plots use the same Asimov Monte-Carlo data set generated with the pseudo-KNM1 FSD specified in Sect. 5 and $m^2_\nu =0$ as input. The other parameters of the data set are as described in Sects. 2.1 and 2.2. The KNM1 FSD and the pseudo-KNM1 FSD are similar though their differences do introduce a tiny and constant shift in all fit results for $m_\nu ^2,$ see Appendix D. Besides the constant shift, the FSD used for the Monte-Carlo data does not impact the results from the convergence studies.

Unless otherwise stated, the graphs in this section all comprise the same y axis. They show the fit result for $m_\nu ^2$ when fitting the same Asimov Monte-Carlo data set described above while using an FSD other than the pseudo-KNM1 FSD for the fit model which differs from the pseudo-KNM1 FSD by one parameter of interest.²⁰ With two exceptions, these fit values are plotted against $\varOmega$ of the basis set (see Sect. 3.3) on the x axis. The FSDs generated with the convergence parameter set to $\varOmega =10$ appear to be a good compromise between convergence that is achieved and computational efforts. Therefore, the deviation of the $m_\nu ^2$ obtained for $\varOmega =10$ gives the uncertainty associated with the effect under investigation.²¹ In the following Sect. 6.1, all individual uncertainty contributions which are obtained following the described procedure are listed. The final new total uncertainty for the KNM1 FSD which is obtained is presented in Sect. 6.2.

6.1 Individual uncertainty contributions

In the following, the individual effects leading to an FSD uncertainty are discussed separately. They are

• the impact of the convergence parameter $\varOmega$ (see Sect. 6.1.1),

• the choice of the base for describing the excited states (see Sect. 6.1.2),

• the choice of the reduced masses (nuclear or effective, see Sect. 6.1.3),

• the influence of the molecular ground-state energies (see Sect. 6.1.4),

• the influence of binning (see Sect. 6.1.5),

• the impact of the uncertainty on the tritium Q value (see Sect. 6.1.6),

• the neglect of the fractional-recoil variation with energy (see Sect. 6.1.7) and

• the use of the sudden approximation (see Sect. 6.1.8),

• the impact of the inclusion of theoretical corrections (relativistic, radiative, and adiabatic corrections, see Sect. 6.1.9).

All of them are summarised as well in Table 6.

6.1.1 $\varOmega$ convergence

The electronic part of the original KNM1 FSD was computed using a variant of a code originally written by Kołos and collaborators (see Sect. 3.2). The basis sets used in the KNM1-FSD calculation entering Eq. (16) had been taken from literature, as is described in Appendix B.1. Therein, they had been optimised individually as a function of the internuclear separation for the ground electronic states of ${\textrm{H}_2}$ and HeH$^+$ with respect to both the base (non-linear parameters) and the basis functions (quintuples) [10]. For the electronically excited states of HeH$^+$, 400 basis functions (integer quintuples) and three different sets of the base parameters $\alpha ,\; \bar{\alpha },\; \beta \; {\textrm{and}}\; \bar{\beta }$ were chosen. Out of the results obtained with the different bases, for every one of the five lowest-lying electronically excited states and every value of the internuclear separation the lowest energies (and corresponding overlaps) were chosen. For some internuclear separations and excited states the potential curves and transition matrix elements already used in the Fackler FSD [22] were adopted, if the corresponding energies were lower. According to the variational principle both energies and wavefunctions are then more accurate. Evidently, such a combination of individually selected data obtained with different basis sets cannot be mapped onto a simple $\varOmega$ variation. However, such a mapping would be required for a clear systematic uncertainty analysis as is proposed in this work. Therefore for the KNM1 FSD the result is approximate. In order to validate the approximation, some consistency checks are performed.

A first assessment of the quality of the KNM1 FSD is obtained by a comparison of the Born–Oppenheimer potential curves $V_{n}^{\textrm{BO}}$ from KNM1 with those obtained in the present work for the different values of $\varOmega ,$ both for the ground states of ${\textrm{H}_2}$ and of HeH$^+.$ Since the ground-state population of ${^3\textrm{HeH}^+}$ after $\upbeta$ decay of ${\textrm{T}_2}$ is about 57% of the total probability [35], it is most relevant and thus chosen. From such a comparison (not shown) it is concluded that the potential curves used in generating the KNM1 FSD are of a quality similar to the ones obtained with $\varOmega = 5 - 6$ for HeH$^+$ and $\varOmega = 6 - 7$ for ${\textrm{H}_2}.$ Therefore, it is to be expected that the results of the KNM1 FSD should be comparable to test FSDs obtained with values of $\varOmega = 5 - 7.$ Thus, an effective $\varOmega _{\text {eff, KNM1}}$ describing the convergence level of the KNM1 potential curves should be between 5 and 7.

In order to assign the KNM1 potential curves an $\varOmega _{\text {eff, KNM1}}$ the KNM1 FSD potential curves are associated an $m_{\nu , \text {as}}^2$ that is obtained from using a test FSD with the KNM1 potential curves in the fit model for the pseudo-KNM1 Monte-Carlo data set. This way it can be determined where the single-valued KNM1-FSD uncertainty is placed on the $\varOmega$ convergence curve. This is illustrated in Fig. 3 where the horizontal line shown is the aforementioned value $m_{\nu , \text {as}}^2$ corresponding to the extracted $m_\nu ^2$ when KNM1-FSD potential curves are used, while the crosses (connected by a dotted line to guide the eye) show the convergence curve which shows the results obtained for an increasing value of the convergence parameter $\varOmega .$ The intersection of the two curves delivers the estimate $\varOmega _{\text {eff, KNM1}}\approx 6.$ The difference between $m_{\nu , \text {as}}^2$ and the $m_\nu ^2$ of the convergence curve at $\varOmega = 10$ delivers the error of the KNM1 FSD compared to an FSD that is obtained for potential curves that are supposed to be converged. The error found this way is ${-1.7 \times 10^{-4}~\text {eV}^{2}/\text {c}^{4}}.$

Fig. 3

Convergence curve using $\varOmega$-dependent potential curves in comparison to the horizontal $m_{\nu , \text {as}}^2$ line obtained when the original potential curves from the KNM1 FSD are used

6.1.2 Base parameters

As discussed earlier, each of the potential curves produced with the H2SOLVm code depends only on the two inputs specifying the basis set: the base, i.e. $\{\alpha ,$ $\bar{\alpha },$ $\beta ,$ $\bar{\beta }\},$ and the convergence parameter $\varOmega$ (see Sect. 3.3). As the KNM1 FSD adopted the potential and overlap curves from literature, the base $\{\alpha ,$ $\bar{\alpha },$ $\beta ,$ $\bar{\beta }\}$ used to produce these curves is needed in order to perform a convergence study. However, complete information on a base (that usually varies with the nuclear separation, and for different electronic states) is not always available in the references. In these cases, an attempt was made to reconstruct the bases for the pseudo-KNM1 FSD. Additionally the choice of base is limited by the fact that the recurrence relation used for solving the integrals in H2SOLV imposes certain restrictions on the bases that can be used. This excludes some of the bases used in generating the KNM1 FSD (where the Kołos code was adopted). All bases that were used, but could not be found in literature, are given explicitly in Appendix B.1. The bases adopted for the ground states, even the ones for ${^3\textrm{HeH}^+}$ that were obtained by a reconstruction, yield very accurate results (low energies) as they had been very carefully optimised. Thus only the much larger influence of the choice of different bases on the excited states is discussed in this work.

The potential curves of the excited states in the KNM1 FSD are composed of four different bases, as described in Appendix B.1.3. For each of the excited states n the lowest energy for a given internuclear separation R was chosen for constructing the corresponding potential curve $V_n^{\textrm{BO}}(R)$ and electronic overlap matrix element $S_n(R).$ In order to estimate the uncertainty due to the choice of the base, for each of the three different (known) bases that were used in generating the pseudo-KNM1 FSD a separate FSD was calculated as a function of the convergence parameter $\varOmega$ (based on the corresponding base-dependent potential curves and electronic overlaps for the electronically excited states of ${^3\textrm{HeH}^+})$. The convergence of the squared neutrino mass $m_{\nu }^2$ obtained from using these three different sets of FSDs is shown in Fig. 4 and compared with the results for the pseudo-KNM1.

Despite the fact that the different bases show quite a different convergence behaviour, all curves converge for larger values of $\varOmega$ to $m_{\nu }^2$ values which are in agreement with each other to within ${3 \times 10^{-5}~\text {eV}^{2}/\text {c}^{4}}$ for $\varOmega =10.$ The results in Fig. 4 confirm the expectation that a poor choice of the base (large deviation from an accurate calculation) for small values of $\varOmega$ can be compensated by using a large value of $\varOmega ,$ since for sufficiently large values of $\varOmega$ the results for different bases converge to the same value.

Fig. 4

Comparison of the fitted squared neutrino mass $m_\nu ^2$ for different bases used for the computation of the electronically excited ${^3\textrm{HeH}^+}$ states. The results obtained for the three different Bases 1, 2, and 3 are compared with the results of the pseudo-KNM1 FSD. Those three bases (see text and Appendix B.1.3 for details) are constituents of the combined base in the pseudo-KNM1 FSD calculation

6.1.3 Reduced mass

When solving for the wavefunction of nuclear motion one needs to use the reduced mass $\mu$ of the parent and of the daughter molecules. This raises the question how to choose a proper value for the reduced mass $\mu$ (see end of Sect. 4.1.1). In [25] four different ways are discussed for choosing the reduced mass (in the case of the electronic ground state) and the results are compared to spectroscopic data. In the present work, two oppositional cases of these options were considered: the nuclear reduced mass $\mu _{\text {n}}$ which only takes into account the masses of the nuclei without the electrons and the effective reduced mass $\mu _{\text {eff}}$ that distributes the masses of the two electrons according to the electron density. In the latter case it is assumed that one electron is bound to the $\text {He}^{2+}$ nucleus and the other one is distributed between both nuclei. The effective reduced mass was also chosen, because in [25] it is demonstrated that using $\mu _{\text {eff}}$ gives better agreement with spectroscopy data, if used for the daughter molecules ${^3\textrm{HeH}^+}$ and ${^3\textrm{HeD}^+}.$ For both the effective reduced mass and the nuclear reduced mass, the same relation

$$\begin{aligned} \mu = \frac{m_1 m_2}{m_1 + m_2} \end{aligned}$$

(26)

applies, but different values are used for the masses $m_1,m_2$ of the molecular constituents, either ignoring or including the electron mass.

Table 5 The values of the nuclear and effective reduced masses (given in terms of the electron mass $m_{\text {e}})$ used in this work for the uncertainty estimation of the FSD are given together with the values used in the calculation of the KNM1 FSD

Fig. 5

Comparison of the fitted squared neutrino mass $m_\nu ^2$ obtained for different choices of the reduced mass. Shown are the results for different combinations of adopting the nuclear or the effective reduced mass for either the parent or the daughter molecule, or for both. Furthermore, the results obtained with the reduced masses adopted in the KNM1 FSD calculation are shown

The corresponding values for the reduced masses in the two approximations that were used in the present work are listed in Table 5. Furthermore, the values used in the KNM1-FSD calculation are also given. As it turns out, there appear to be some minute inconsistencies with respect to the adopted reduced masses in the KNM1-FSD calculation.²² However, the effect of the likely erroneous values for ${\textrm{DT}}$ and ${\textrm{HT}}$ entering the KNM1 FSD should be negligible for two reasons. First, the differences between the values used and the ones that should be used when using the effective reduced masses are very small. Second, the FSDs of ${\textrm{DT}}$ and ${\textrm{HT}}$ are of much smaller importance compared to the one of ${\textrm{T}_2}$ because of the high ${\textrm{T}_2}$ purity of 97.6 % in the first KATRIN science campaign [5]. Evidently, for the same reason (high ${\textrm{T}_2}$ purity) the use of the nuclear instead of the effective reduced mass for ${^3\textrm{HeT}^+}$ appears more relevant. The difference between $\mu _{\text {n}}$ and $\mu _{\text {eff}}$ is, however, relatively small.

In order to estimate the uncertainty on $m_{\nu }^2$ due to the choice of the reduced mass, convergence curves for five different mass combinations were considered: the original masses as used in the KNM1 FSD and all four combinations of $\mu _{\text {n}}$ or $\mu _{\text {eff}}$ for the parent and daughter isotopologues. This should provide a conservative upper limit on the effect that an inaccurate choice of $\mu$ has on the FSD. In Fig. 5 these five convergence studies are shown.

The choice of the reduced mass for the parent molecule has a negligible effect onto $m_\nu ^2$ that is of the order of less than $1 \times 10^{-5}\,\text {eV}^{2}/\text {c}^{4}$ and thus not resolved on the scale used in Fig. 5. This is to be expected, since the electrons are distributed equally between both nuclei in the electronic ground state of ${\textrm{T}_2}.$ Within the Born–Oppenheimer approximation this applies also to ${\textrm{DT}}$ and ${\textrm{HT}}.$ Nevertheless, adopting the (smaller) nuclear reduced mass compared to the effective reduced mass lowers in first order all energy levels by a constant value. The choice of the reduced mass for the daughter isotopologues has a larger effect since it accounts for the unequal distribution of the electron density around the two nuclei. The shift between the $m_\nu ^2$ obtained using the nuclear reduced mass and the one obtained using the effective reduced mass for the daughter molecule yields (independent of the choice of the reduced mass of the parent molecule) between $\varOmega =6$ and $\varOmega = 10$ an uncertainty estimate of $1.5 \times 10^{-4}\,\text {eV}^{2}/\text {c}^{4}$ due to the choice of reduced masses. The same uncertainty is found when using the reduced masses adopted in the calculation of the KNM1 FSD.

6.1.4 Molecular ground-state energies

Since the molecular tritium source in KATRIN contains impurities like the tritium-containing isotopologues ${\textrm{DT}}$ and ${\textrm{HT}},$ the FSD used in the fit needs to be an incoherent superposition of the FSDs for the different isotopologues, weighted with the composition probability (see Sect. 3). When combining these different FSDs it is important to properly adjust the three energy scales, i.e. to use a consistent energy zero. If the parent molecule is initially in its absolute ground state, the energy available to the $\upbeta$ electron is maximised when the daughter molecular ion is also left in its absolute ground state and the neutrino is at rest. The corresponding molecular ground-state energies are different for the different isotopologues even within the Born–Oppenheimer approximation, since the zero-point energy depends on the reduced mass. As a consequence, the maximum energy available to the $\upbeta$ electron is isotopologue dependent. The corresponding relative energy difference needs to be accounted for when creating the incoherent superposition of the FSDs for the three isotopologues.

Fig. 6

Comparison of the fitted squared neutrino mass $m_\nu ^2$ for different choices of the ground-state energies of the parent and the daughter molecules in the adjustment of the energy scales for the different isotopologues. The results obtained with the potential curves used on the KNM1-FSD calculation are independent of $\varOmega$ and denoted by “KNM1”. If the $\varOmega$-dependent ground-state energies from the calculation of the test FSDs are used in the scale adjustment, the results denoted “consistent” are used

Since the value obtained for the absolute ground-state energy depends on the adopted potential curve, uncertainties in the calculated potential curves need to be considered when adjusting the energy scales of the different isotopologues relative to each other. In order to determine the impact of uncertainties in the energy-scale adjustment on $m_{\nu }^2,$ either the ground-state potential curves used in the KNM1 FSD or the ones obtained within the $\varOmega$ variation were used when adding the FSDs of the different isotopologues in the generation of the test FSDs. The effect was investigated for daughter and parent ground-state energies separately. Thus four combinations of ground-state energies were obtained, for which the convergence curves are shown in Fig. 6. Since the choice of the ground-state energies in the energy-scale adjustment does not cause a constant shift in $m_\nu ^2$ as a function of $\varOmega ,$ the uncertainty is conservatively estimated with the maximal difference between the different options considered here which is found for $\varOmega = 8.$ The uncertainty estimated this way is smaller than $1 \times 10^{-5}~\text {eV}^{2}/\text {c}^{4}$ for both parent and daughter molecules.

6.1.5 Binning

The molecular transition probability $P_{fi}^{\textrm{TS}}$ in Eq. 3 comprises two parts. First, a set of discrete values describing the transitions to the bound states of ${^3\textrm{HeT}^+}$ (or its isotopologues). Second, a continuous transition probability (per energy) describing the transitions into dissociative and (or) ionisation continua. As was explained in Sect. 3 such a spectrum is impractical for the analysis of a neutrino-mass experiment like KATRIN. Instead, a binned probability spectrum is used in the fit model when extracting the squared neutrino mass. The influence of equidistant bin sizes on $m_\nu ^2$ is investigated in Fig. 7. The Monte-Carlo data set which served as reference spectrum used the pseudo-KNM1 FSD with non-equidistant bins. The KNM1 FSD was binned with ${0.02\,\mathrm{\text {eV}}}$ wide bins between ${-0.5\,\mathrm{\text {eV}}}$ and ${5.0\,\mathrm{\text {eV}}}$ (covering the transitions to the electronic ground state of the daughter molecular ion) and ${0.2\,\mathrm{\text {eV}}}$ wide bins between ${19\,\mathrm{\text {eV}}}$ and ${40\,\mathrm{\text {eV}}}$ (electronically excited states). Negative energies can occur due to the non-zero temperature and the combination of different isotopologues since the zero energy is defined via the ground state of T$_2.$ For test bin sizes up to 0.025 eV, a small constant positive shift in $m_\nu ^2$ around $1 \times 10^{-4}\,\text {eV}^{2}/\text {c}^{4}$ occurs. With increasing bin size, the $m_\nu ^2$ shift becomes more negative when the test FSD bin size becomes larger than the pseudo FSD bin size. This (non-equidistant) binning of the KNM1 FSD is found to yield a shift in $m_\nu ^2$ of ${1.5 \times 10^{-4}\,\text {eV}^{2}/\text {c}^{4}}$ at $\varOmega =10.$ In conclusion, the binning of a future FSD should not exceed bin sizes of 0.02 eV to keep the uncertainty caused by binning of the FSD on the $10^{-4}$ level.

Fig. 7

Squared neutrino mass $m_\nu ^2$ obtained by the fit as a function of the size of the bins used for binning the FSD

6.1.6 Tritium Q value

The maximum fractional recoil $\tilde{K}$ (see Sects. 3.1 and 4.1.1) imparted by the $\upbeta$ electron and the neutrino onto the internal degrees of freedom of the molecule occurs, if the electron has maximum kinetic energy, i.e. at the end point $E_0$ of the spectrum. This value depends on the energy release in the $\upbeta$ decay and thus on the tritium Q value [15]. The uncertainty on the Q value for tritium $\upbeta$ decay is dominated by the mass difference between T and $^\text {3}$He. As mentioned in Sect. 4.1.2, the KNM1 FSD adopted the value $E_0=$18573.24 eV for the end-point energy from [15], and a corresponding mass difference between T and $^\text {3}$He of 18591.3(1) eV. Two more recent experimental values for the mass difference stem from Penning-trap measurements, by which it is determined to be ${18\, 592.01 \pm 0.07}$ eV [45] and ${18\, 592.071 \pm 0.022}$ eV [46]. The Q value specific for KATRIN which also includes plasma effects and work function differences with higher uncertainty [5] is not used in the FSD calculation, because these effects have no impact on the FSD uncertainty as they only influence the $\upbeta$-decay electron which passes the whole beamline including the spectrometer.

In order to investigate the impact of an uncertainty of the tritium Q value onto the fractional recoil and thus the calculated FSD, test FSDs with different end-point values (and thus with different fractional recoils) were generated, see also following Sect. 6.1.7). (The influence on the reduced masses was neglected, since it is orders of magnitude smaller.) The experimental end point $E_{0}$ of the Monte-Carlo data was kept constant at the value given in Sect. 2.2. The influence on the extracted value of $m_\nu ^2$ is shown in Fig. 8. Due to the discrepancy of the mass difference chosen for the FSD and the result from Penning-trap measurements, an uncertainty on the Q value of 1 eV is assumed. This uncertainty induces an uncertainty on $m_\nu ^2$ of $2 \times 10^{-5}~\text {eV}^{2}/\text {c}^{4}.$

Fig. 8

Influence of the choice of the tritium Q value on $m_\nu ^2$ for FSDs with $\varOmega = 10$. The tritium end point E$_{0}$ used in the FSD calculation is marked as vertical line, together with an uncertainty band marking the assumed end-point uncertainty of 1 eV. The obtained slope for the change of $m_\nu ^2$ as a function of the end-point energy is $2 \times 10^{-5}~\text {eV}^{2}/\text {c}^{4}/\text {eV},$ obtained from a linear fit which is shown in orange

6.1.7 Fractional-recoil variation over the fit interval

The effect of the Q-value uncertainty is similar to the effect of approximating the fractional recoil to be constant over the whole fit interval (see Sect. 4.1.1). Another extreme assumption would be the use of the fractional recoil corresponding to the end of the fit interval, i.e. 40 eV below the end point $E_0$ in the present case. A more reasonable estimate within the constant-fractional-recoil approximation is obtained by adopting the fractional recoil that arises at the mean excitation energy (within the fit interval). Since the fractional recoil depends only on the end-point energy of the isotopologue in question, the uncertainty can be determined with the help of the slope from Fig. 8. Using the mean excitation energy for the ${\textrm{T}_2}$ pseudo-KNM1 FSD $({10.74\,\mathrm{\text {eV}}})$ this yields an estimated uncertainty on $m_\nu ^2$ due to the constant-fractional-recoil approximation of $2.2 \times 10^{-4}~\text {eV}^{2}/\text {c}^{4}.$²³

6.1.8 Corrections to the sudden approximation

The first-order correction to the sudden approximation was not only derived and discussed in [17], but even explicitly evaluated for ${\textrm{T}_2}.$ For a rough estimate of the uncertainty due to the use of the sudden approximation, the total transition probabilities including the first-order correction for the electronic ground and 5 lowest-lying excited states given in Table II (column 5) in [17] were used for generating test FSDs. In more detail, based on the literature values the relative changes of the transition probabilities due to the inclusion of the first-order correction were obtained for the 6 lowest states of ${^3\textrm{HeT}^+}.$ The contributions of these states to the test FSDs that were generated within the sudden approximation were then correspondingly re-scaled with the respective correction factors, yielding new test FSDs. The obtained $m_\nu ^2$ extracted from the fits are shown in Fig. 9. The uncertainty due to the use of the sudden approximation is found to be $<4 \times 10^{-4}~\text {eV}^{2}/\text {c}^{4}$ at $\varOmega = 10.$

Fig. 9

Convergence study for estimating the effects of the corrections to the sudden approximation onto $m_\nu ^2$

6.1.9 Relativistic, radiative, and adiabatic corrections

The potential curves obtained within the non-relativistic Born–Oppenheimer approximation can be improved by taking various corrections perturbatively into account (see Sect. 4.1.1 for details). In the KNM1 FSD such corrections were only adopted for the electronic ground states of the parent and daughter molecules. Since the initial states of the parent molecule enter the calculation of all transition matrix elements and more than half of the total transition probability goes into the electronic ground state of the daughter molecule, it is expected that corrections are most important for these states. Since the largest corrections are available in literature and are assumed to be sufficiently accurate, they were adopted in the KNM1 FSD as well as in the uncertainty analysis presented here. If these adopted corrections are only available for a finite set of internuclear separations, they were taken as is, spline interpolated, and applied to the corresponding potential curves for the FSDs of this work. This procedure offers the flexibility to add the corrections to any of the potential curves generated for creating test FSDs, even if they are not available for the same values of the internuclear separation R.

The most important correction to the non-relativistic Born–Oppenheimer potential curve is the already mentioned adiabatic (or diagonal) correction, i.e. the matrix element of the nuclear kinetic-energy operator between the same electronic wavefunctions in bra and ket. Since the adiabatic correction originates from the nuclear motion, it is mass-dependent and has to be rescaled for the different isotopologues accordingly. For the parent isotopologues the values from [30] were taken. For the daughter isotopologues the data were taken from [47]. The correction from [47] was interpolated with a polynomial, as proposed in that work. For distances $R > 6\, a_0$ the value was assumed to be identical to that at $R=\infty .$ The reason for this simplified treatment is twofold. First, the R dependence of the correction at large internuclear separations is small, since the molecular wavefunctions become almost identical to the ones of two independent atoms and thus approximately R independent. Second, due to the Franck–Condon principle the transition probability (in the present case the FSD) depends only on the R interval in which the initial-state wavefunction (here the one of the electronic ground state of the parent molecule) has a non-negligible density. The non-adiabatic corrections that are the off-diagonal matrix elements of the kinetic-energy operator of the nuclei are neglected as is explained in Sect. 4.1.1.

The (leading) relativistic correction is to a good approximation nuclear-mass independent as it only describes the effect of the spin and the velocity of the electrons, since they are much faster. The smallest of the corrections adopted here is the radiative correction, describing the interaction of the electrons with the quantised electromagnetic vacuum field. Both the relativistic and radiative corrections were taken from [30], where they are provided for the parent isotopologues. To our knowledge there are no relativistic and radiative corrections in literature for the daughter isotopologues. Therefore, and because their effect for the parent molecule was found to be very small (see below), they were neglected in the present uncertainty study.

Since the corrections discussed in this subsection were not re-calculated in this work, they are $\varOmega$ independent. The adiabatic and relativistic corrections of [30] can be compared with the more recent and more accurate adiabatic corrections in [48] as well as the relativistic corrections in [38]. The relative error between the adiabatic corrections used in the KNM1 FSD and given in [48] is at most of the order of $10^{-4},$ the absolute magnitude of the correction itself being only $\approx 5.5 \cdot 10^{-4} {\textrm{E}}_{\textrm{H}}.$ For the relativistic corrections used in the KNM1 FSD and given in [38] the relative error is at most of the order of $10^{-3},$ the absolute magnitude of the correction itself being $\approx 1.5 \cdot 10^{-5} {\textrm{E}}_{\textrm{H}}.$ Therefore, the accuracy of those corrections was clearly sufficient for the first two measurement campaigns and they were adopted in both the KNM1 FSD and the pseudo-KNM1 FSD calculation.

In order to estimate the uncertainty on $m_\nu ^2$ due to the adiabatic, relativistic, and radiative corrections and the limited accuracy with which they were calculated, the extreme cases are considered in which the corrections are either all included or excluded. The results, given as a function of the basis-set convergence parameter $\varOmega ,$ are shown in Fig. 10. The complete omission of the corrections yields a constant shift of $-4 \times 10^{-3}~\text {eV}^{2}/\text {c}^{4}$ for the extracted $m_\nu ^2$ independent of $\varOmega .$ As a very conservative estimate for the uncertainty of $m_\nu ^2$ due to the omission of the corrections for the electronically excited states (and the limited accuracy of the adopted literature values for those corrections), 30% of the absolute value of the uncertainty obtained when completely omitting these corrections for the electronic ground state is used in the total uncertainty budget, since within the fit interval roughly 30% of the FSD goes into the electronically excited states.²⁴

Fig. 10

Comparison of the fitted squared neutrino mass $m_\nu ^2$ obtained if corrections to the Born–Oppenheimer potential curves are applied or not. For the orange curve (label “pseudo-KNM1 Born–Oppenheimer”), the non-relativistic Born–Oppenheimer potential curves are used for the electronic ground states of parent and daughter molecules in the FSD calculation. For the blue curve (label “pseudo-KNM1”), the adiabatic, relativistic, and radiative corrections adopted in the KNM1 FSD are included

6.2 New FSD uncertainty for the first KATRIN science campaign

Table 6 summarises all systematic uncertainties on the fitted neutrino mass square $m^2_\nu$ from the calculated FSD used for the KNM1 analysis derived in the studies presented in Sect. 6.1. The first three contributions, “Basis-set $\varOmega$ convergence”, “Bases for electronically excited states” and “Ground-state energies”, are not independent of each other, because they are connected via the variational principle. The corresponding uncertainties were thus added linearly in order to obtain a conservative estimate. Since the other contributions are independent, they were added quadratically, yielding $\varDelta m^2_\nu = 5.3 \cdot 10^{-4}$ eV$^2/c^4$ for the FSD uncertainties. Finally, the assumed uncertainties of the theoretical corrections (adiabatic, relativistic, radiative) were also added quadratically, yielding the total uncertainty of $\varDelta m^2_\nu = 1.3 \cdot 10^{-3}$ eV$^2/c^4$ given in the last row of Table 6. While the interference of different effects cannot be excluded, the resulting small size of the effects allows to add them one-by-one in the uncertainty estimation. The new uncertainty obtained this way is well below the limit of $7.5 \times 10^{-3}~\text {eV}^{2}/\text {c}^{4}$ given in the KATRIN design report [1].

It is worth mentioning that the temperature fluctuations of $\pm 1$ K of the KATRIN experiment lead to an additional uncertainty to $m_\nu ^2,$ which is, however, not related to the FSD calculation itself. A similar argument holds for the gas composition and the ortho-para ratio (only in the case of ${\textrm{T}_2})$ in the tritium source. An estimate of these experimental uncertainties can be found in Appendix C. Further deficiencies of the present uncertainty analysis that are due to the use of an imperfect pseudo-KNM1 FSD, namely the inaccuracy due to the discretisation of the dissociation continuum and the incomplete reconstruction of the electronic overlaps, are discussed in Appendix B.

Table 6 Systematic contributions to the FSD-caused uncertainty of $m_\nu ^2$ for the measurement conditions of the first KATRIN science campaign (see text for details on how the total uncertainty is obtained.) The KNM1 bins listed in this table are $2 \times 10^{-2}~\text {eV}$ wide bins in the range $-0.5$ to 5 eV and $2 \times 10^{-1}~\text {eV}$ wide bins in the range 19 to 40 eV

7 Summary and outlook

Motivated by the increased sensitivity of the upcoming measurement campaigns of the KATRIN experiment, the uncertainty of the molecular final-states distribution (FSD) entering the fit of the neutrino mass square $m_\nu ^2$ – up to now conservatively estimated by a fully phenomenological approach – has been revisited. Studying the impact of the individual sources of uncertainties on the extracted value of $m_\nu ^2$ is proposed as a direct and controllable way of obtaining an uncertainty budget that is adapted to the experimental conditions. At the same time this method provides a clear understanding of which approximations in the FSD calculations are the limiting ones and thus leads to a priority list of which aspects of the FSD calculation need to be improved on most urgently. A second ingredient of the new scheme is a systematic increase of the basis-set size allowing for a straightforward convergence study.

As a proof of principle the new uncertainty analysis was applied to the FSD used in the analyses of the first two measurement campaigns (KNM1 and KNM2) of the KATRIN experiment, the KNM1 FSD, but considering only the experimental conditions of the KNM1 campaign. Although this KNM1 FSD does not allow for a fully consistent application of the uncertainty-analysis procedure introduced in this work, as it required to introduce and use a pseudo-KNM1 FSD instead, only in this way a direct comparison to the outcome of the uncertainty-analysis estimate adopted earlier was possible. This motivated the rather time consuming efforts that were needed in order to generate the pseudo-KNM1 FSD that is nearly identical to the KNM1 FSD, but allows for the application of the new method including the convergence study based on the size of the molecular basis-set. The study presented here provides important and detailed insight into both the absolute size and the relative importance of the various factors and approximations entering the FSD evaluation. In fact, it is a rather unexpected result that especially the adiabatic corrections are presently contributing most to the FSD uncertainty. Corresponding new calculations of these corrections are thus currently in progress. Most importantly, it is confirmed that the KNM1 FSD is already very accurate with its uncertainty of $\varDelta m^2_\nu = 1.3 \cdot 10^{-3}$ eV$^2/c^4$ derived by this study when applied to the first KATRIN campaign, though necessarily adopting the pseudo-KNM1 FSD instead of the KNM1 FSD. Considering the similarity in the experimental parameters of KNM1 and KNM2, the KNM1 FSD was confirmed to be sufficiently accurate for the analyses of the experimental data obtained in the first two neutrino-mass measurement campaigns of KATRIN.

In contrast to the previously adopted uncertainty estimates, the uncertainty-analysis procedure introduced in this work accounts for the fact that the uncertainty imparted by the FSD on the fitted squared neutrino mass depends on the experimental parameters. Consequently, a new FSD-uncertainty analysis is required for every measurement campaign, even if an identical FSD is used.

On the other hand, the procedure itself can directly be applied to completely different experimental setups, including, e.g., Project 8 [49, 50] that uses (at least in the initial phase) also molecular tritium, but is based on cyclotron radiation emission spectroscopy. Of course, adopting FSDs different than the one of molecular tritium the procedure can also be used for PTOLEMY [51] or the planned final phase of Project 8 that plan to adopt tritium atoms immersed in graphene or atomic tritium, respectively. Conceptually, the here proposed uncertainty analysis can be used in neutrino-mass experiments based on electron-capture like ECHo [52] or HOLMES [53], since – although being an experiment which measures the total released energy but that of the neutrino – also there a theoretically calculated FSD enters the analysis indirectly via the phase space, see the discussion on the $^{187}$Re $\upbeta$-decay spectrum in [54].

However, it is important that the FSDs used in such experiments need to be constructed in a way that allows for a systematic uncertainty analysis. This includes the possibility of systematic basis-set convergence studies and thus usually prohibits the use of an FSD that is a compilation of best possible individual results available in literature. Therefore, a new FSD for molecular tritium has recently been generated that fulfills the requirements of the new uncertainty-analysis procedure and is used in the analysis of the KATRIN measurement campaigns that followed KNM2. This new FSD (together with the corresponding analysis of its uncertainty) will be presented in a forthcoming publication. The new FSD will also be extended in order to allow analyses of the tritium $\upbeta$-decay spectrum for larger energy intervals, e.g., 60 eV below the end point, in the future. Since the present work shows the significance of the various correction terms, especially the adiabatic and the non-adiabatic ones, they are currently recalculated.

Appendices

Appendix A: Glossary

Table 7 Original (left hand side of the table) and modified bases (right hand side of the table) with re-scaled $\bar{\beta }$ to be compatible with the H2SOLV code. The re-scaling was performed according to $\bar{\beta } = 0.01 \cdot R$

• KNM1 FSD was used for the evaluation of the first and second measurement campaigns of KATRIN. It is based on literature data and its generation is described in [10].

• pseudo-KNM1 FSD is an FSD that is produced for the uncertainty study of the KNM1 FSD, because the here proposed uncertainty analysis cannot be performed with the KNM1 FSD itself. The pseudo-KNM1 FSD contains intentionally some further (though small) modifications compared to the KNM1 FSD, as is described in Sect. 5.3.

• test FSD is a modification of the pseudo-KNM1 FSD describing a specific uncertainty (see Eq. (2)). This uncertainty can be a specific limitation of the calculation (like the finite basis set), an adopted approximation (like the sudden approximation), or some input error.

• Fackler FSD was produced by Fackler et al. [22].

• Saenz FSD was produced by Saenz et al. [7]. The main differences to the earlier produced Fackler FSD were the use of the relativistic recoil and the way the electronic continuum was treated.

• basis set is defined by all parameters specifying the basis functions used to expand the electronic wavefunction, i.e. it is a combination of a base (defined by four non-integer numbers) and a list of quintuples, a quintuple being a set of five integers. Instead of a list of quintuples alternatively a single parameter, the rank of the basis set $\varOmega ,$ is used. In the latter case all quintuples that fulfill the condition that their sum is equal or smaller than $\varOmega$ are included in the basis set.

• basis is used synonymously for basis set.

• base parameters are the non-integer exponential parameters $\alpha ,$ $\bar{\alpha },$ $\beta$ and $\bar{\beta }$ in the Kolos–Wolniewicz basis functions, see Eq. (16)

• base is a complete set of all four base parameters $\{\alpha , \bar{\alpha }, \beta , \bar{\beta }\}.$

• energy scales Note, there is a confusing issue related to the used energy scale depending on context. When discussing the $\upbeta$-decay spectrum, energies are generally defined in relation to the end point on a negative scale, e.g., the lower limit of the fit interval is defined as $E_{0}-{40\,\mathrm{\text {eV}}}.$ In contrary, energies related to the FSD are generally expressed on a positive scale in relation to the chosen zero energy (the transition from the ground state of ${\textrm{T}_2}$ to the ground state of ${^3\textrm{HeT}^+})$. Thus, both scales are antiparallel to each other and any directional terms (like above and below) are context dependent.

Appendix B: Parameters entering the FSD calculation

1.1 B.1 Potential curves and electronic transition matrix elements

1.1.1 B.1.1 ${\textrm{H}_2}$ ground state bases

The bases used for computing the Born–Oppenheimer potential curve of ${\textrm{H}_2}$ up to $R = 4.8 \, a_0$ were taken from [55]. For the bases between $R = 4.8 \,a_0$ and $R = 12.0 \,a_0$ a linear interpolation of the re-optimised bases for larger internuclear separations as in [30] was used. Together with the exponents from [55] the correctness of the linear interpolation and the bases were checked by recomputing the electronic energies.

For finding the energy eigenvalues the canonical orthonormalisation (see, e.g. [30]) was used. As in [30] the variable $\epsilon$ is introduced as a cutoff, i.e. eigenvectors corresponding to eigenvalues smaller than $\epsilon$ are omitted. The value $\epsilon = 10^{-12}$ reported in [30] was, depending on the internuclear separation R, found to be too small or too big. The ordering of the basis functions (quintuples), which is unknown, can change the resulting eigenvalues. Therefore, another strategy was finally chosen: the number of basis functions in the canonical orthonormalisation was reduced (compared to an eigenvalue cutoff with $\epsilon )$ until the lowest energy eigenvalue was judged to be “satisfactory”, i.e. it did not violate the variational principle. This does not necessarily mean the eigenvalues closest to the ones from [30] are obtained, but should be rather consistent for the present ordering of the exponents.

Table 8 Table with the reconstructed KNM1 bases for the electronic ground state of the ${^3\textrm{HeH}^+}$ molecule. The internuclear separations R are given in units of Bohr $(a_0)$

Note, the quintuples (basis functions) given in [55] contain twice the same quintuple $(i = 131$ and $i = 215)$, so one of them was replaced with $\{1,0,2,2,2\}$ as this quintuple was not yet contained. Following [30] also the quintuple $\{1,3,1,1,1\}$ was changed into $\{0,3,1,1,1\}.$ Of course, it may be expected that the ordering of the quintuples (basis functions) is irrelevant. However, this is not the case here due to the finite numerical precision used in the Kołos code.

For all the distances above $R = 0.8\, a_0$ results agreeing with the values in [55] within a precision of $10^{-9}$ to $10^{-8}$ were achieved. Keeping in mind that the original ordering of the basis functions (quintuples) is unknown, this is rather good. For internuclear separations $R < 0.8 \, a_0$ the results are orders of magnitudes worse, despite the knowledge of the bases. However, they do not contribute notably to the FSD.

The literature bases cannot be used in a fully unchanged manner in the convergence study, because of the inability of the H2SOLV code [21] to process $|a| < 0.01 \cdot R$ where a is any of the four non-linear parameters inside the base and R is the internuclear separation. This leads to the problem that in between $R = 3.4 \, a_0$ and $R = 4.2 \, a_0$ the original values of $\bar{\beta }$ cannot be used. Instead, a scaling of $\bar{\beta }$ to the minimum value required by the H2SOLV code is performed. This leads to the bases shown in Table 7.

1.1.2 B.1.2 ${^3\textrm{HeH}^+}$ ground-state bases

The Born–Oppenheimer potential curve for the ground state of ${^3\textrm{HeH}^+}$ was taken from [19, 56, 57]. Additionally, an improvement of the energies of the potential curve is added in form of an interpolated shift, as proposed in [47]. Unfortunately, only [47] provides the bases parameters, and they are only given for a few selected internuclear separations $R \in \{ 0.9, 1.46, 1.8, 3.0, 4.5, 6.0\}.$ Of course, there is no guarantee that any of the other three publications actually used these bases.

The six bases from [47] were interpolated with a polynomial of degree five, thus adopting the same procedure used in [47] for generating bases for $R = 1.2 \,a_0$ and $R = 2.4 \, a_0.$

A way to determine the quintuples defining the basis functions is also given in [47]. These quintuples were checked at the internuclear separations where the bases are given. The achieved agreement with the literature values is $10^{-8}$ or even $10^{-9},$ that is to all given digits. An exception is the energy at $R = 6.0 \, a_0$ where only a precision of $10^{-7}$ is achieved. Still the quintuples should at least be correct, but may not necessarily be in the correct order. With these quintuples the interpolated bases for the distances between $R = 0.9 \,a_0$ and $R = 6.0\, a_0$ were checked and an agreement of $10^{-6}$ to $10^{-7}$ was reached. It should be noted that the energies to which the comparison is made are not given explicitly, but rather had to be reconstructed from the aforementioned set of four publications [19, 47, 56, 57].

The most severe problem is the missing base for $R = 8 \, a_0$ and thus the extension of the bases beyond $R = 6 \,a_0.$ It was tried to use the extension of the interpolating polynomial of degree 5 from before, but this failed, especially for larger internuclear distances R. The results from two different linear interpolations were compared, once using only the bases at $R = 4.5 \, a_0$ and $R = 6 \, a_0$ and once using all given bases. Furthermore, a simple scaling of the bases at $R = 6 \, a_0$ with R was tested. All three methods give comparable results and are closer to each other than to the original values. The linear interpolation with only two points $(R = 4.5 a_0$ and $R = 6 a_0)$ was finally chosen to extrapolate to larger distances. The bases finally constructed from the polynomial interpolation and the linear extrapolation are reported in Table 8.

1.1.3 B.1.3 ${^3\textrm{HeH}^+}$ excited-state bases

For the excited states of ${^3\textrm{HeH}^+}$ the potential curves were taken from [29]. These constitute a compromise of the energies and overlaps obtained with four different bases. Two of the bases are given in [58] and in accordance to later publications the naming of Base 1 and Base 2 was swapped respectively (named Basis 1 and Basis 2 in [58]). The third base (Base 3) is not known and was therefore reconstructed. The potential curves from [29] were used together with the knowledge of Base 1 and Base 2 to determine the points that had been computed with Base 3. With these points it was possible to reconstruct Base 3 as $\{\alpha = 2.08543,$ $\bar{\alpha } = 0.68235,$ $\beta = 0.79449,$ $\bar{\beta } = 0.54546\}$ which gives results within a precision of $10^{-7}$ to $10^{-8}$ compared to [29]. The energies and overlaps for $R = 3 \, a_0$ and $R = 4 \, a_0$ in [29] were taken from [19], where, however, no bases or quintuples for their computation are given. This means the fourth base is not known and cannot be reconstructed in a reasonable time. Instead, the parameters for the ground state of ${^3\textrm{HeH}^+}$ (see Appendix B.1.2) were taken at $R = 3 a_0$ and $R = 4 a_0,$ as these should give at least comparable results for the excited states.

1.1.4 B.1.4 Electronic overlaps

For the computation of the electronic overlaps the bases for the ground state of ${\textrm{H}_2}$ and all states of ${^3\textrm{HeH}^+}$ (contributing in the fit interval) are needed. As the electronic overlaps used in the KNM1 FSD are taken from [19] (which predates [55] from which the ${\textrm{H}_2}$ bases were taken), it cannot be assumed that in both cases the same bases were used. For ${^3\textrm{HeH}^+}$ it is assumed that the same bases have been used, because they were taken from [47], which acknowledges Kołos for these bases. For the ${\textrm{H}_2}$ bases the values from [59] up to and including $R = 2.0 \,a_0$ were taken. For larger distances the bases used in [29]²⁵ were taken, which are listed in Table 9. For the electronic overlaps with the ${\textrm{H}_2}$ ground state, only the bases at $R \in \{3.0, 4.0\}\, a_0$ from Table 9 were used. For the other internuclear separations $R \in \{0.6, 0.8, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0\}\, a_0$ the bases were taken from [59].

Table 9 Additional ${\textrm{H}_2}$ bases used in the electronic overlap calculation, complementing the ones given in [59]

Since different bases for ${\textrm{H}_2}$ were chosen in the computation of the electronic overlaps than for the computation of the potential curves, an inconsistency is introduced. This affects the ground states and the excited states and thus, in principle, the computation of the complete FSD. As the electronic overlaps for the ground states use an even lower number of internuclear separations R than for the excited states and the ground state makes up roughly 57% of the total probability, the effect of using the ${\textrm{H}_2}$ bases from [30] was investigated. These should be more accurate, as they use more points (not all points were used, but rather a selection of 24 points between $R = 0.6$ and $R = 10.0)$. In Fig. 11 the absolute difference between the convergence curves using the overlaps used in the convergence studies and the more appropriately chosen overlaps, i.e. where the ${\textrm{H}_2}$ bases from [30] were used (which are also used for the computation of the potential curves), is shown.

Fig. 11

Absolute difference between the $m_\nu ^2$ fit results obtained with the overlaps used in the convergence studies and the more appropriately chosen overlaps. Here $m^2_{\nu ,co}$ stands for the shift obtained with the overlaps from the convergence study and $m^2_{\nu ,ap}$ is the shift obtained with more appropriately chosen ${\textrm{H}_2}$ bases

As shown in Fig. 11, the difference in the fitted $m_\nu ^2$ is at worst about $< 2 \times 10^{-5}~\text {eV}^{2}/\text {c}^{4},$ at $\varOmega = 5.$ So the effect of using inferior bases for the calculation of the electronic overlaps is negligible compared to the other uncertainties.

1.2 B.2 Description of the dissociation continuum

As mentioned in Sect. 3.4, all solutions of the nuclear-motion problem for the electronically excited states of ${^3\textrm{HeT}^+}$ describe dissociative and thus continuum states. In the evaluation of the KNM1 FSD this problem was treated with a discretisation approach. Adopting a very large B-spline basis and fixed-boundary conditions results in a very large number of discretised continuum states. This is, however, computationally very expensive, since a very large eigenvalue problem needs to be solved. This has to be repeated for all the electronically excited states obtained with a given two-electron basis, in the present case depending on the value of $\varOmega ,$ as discussed above in Sect. 3.3.

Computationally less expensive is the so-called density approach in which the discretised continuum states are continuum re-normalised as to yield probability densities per energy (at a discrete set of energy values). These densities can be interpolated and binned to obtain the FSD. In order to save computational resources and time, all the FSDs calculated in this work were obtained with the aid of the density approach, since this reduces the computation time by roughly a factor of 6. Clearly, it needs to be checked that this does not introduce some non-negligible error. Therefore, a separate study was made in which all other parameters are chosen as they were used in the KNM1 FSD calculation, but performing the calculation once with the original discretisation approach and once with the density approach. Noteworthy, this is a check of the calculation of the test and the pseudo-KNM1 FSDs obtained in this work and whether uncertainties found when comparing to the KNM1 FSD stem from the different way of treating the dissociation continuum of the electronically excited states. Therefore, the result does not influence the results obtained in the analysis of the first science campaign of KATRIN adopting the KNM1 FSD calculated with the more time-consuming discretisation approach and thus does not enter the uncertainty budget, but is only a check whether the conclusions of the present work are biased by using a different (computationally less expensive) way of treating the dissociative continuum of the electronically excited states. The impact of using the discretised approach or the density approach is evaluated in the following way: An additional pseudo-KNM1 FSD is calculated, where the discrete instead of the density approach is used. The Monte-Carlo data set is fitted with this newly calculated FSD, thus delivering the difference between the two approaches. The found discrepancy is $1 \times 10^{-7}~\text {eV}^{2}/\text {c}^{4},$ which means this difference is negligible and thus the here adopted density approach does not bias the findings of this uncertainty study.

Appendix C: Influence of experimental uncertainties entering the FSD calculation

Temperature The tritium source of KATRIN features a temperature stability of $\pm {1.5\,\mathrm{\text {m}\text {K}\text {h}^{-1}}}$ at 30 K, corresponding to $\pm {4.5\,\mathrm{\text {m}\text {K}}}$ during one run of typically 3 h duration, and a temperature homogeneity of 0.85 K [60], leading to a total temperature uncertainty of $<{1\,\mathrm{\text {K}}}.$ The FSD-related effect of temperature deviations on $m_\nu ^2$ was investigated by creating FSDs with angular momenta weighted for different temperatures and using them for a fit on the reference Monte-Carlo data with the pseudo-KNM1 FSD generated at 30 K. As demonstrated in Fig. 12, the absolute effect on $m_\nu ^2$ caused by temperature variations of $\pm {1\,\mathrm{\text {K}}}$ is $<5 \times 10^{-4}~\text {eV}^{2}/\text {c}^{4}.$

Fig. 12

Impact of source temperature on $m_\nu ^2.$ The slope at ${30\,\mathrm{\text {K}}}$ is $4.8 \times 10^{-4}~\text {eV}^{2}/\text {c}^{4}/\text {K}$

Isotopologue composition The isotopologue composition of the tritium gas within the WGTS is monitored using Raman Spectroscopy [12]. The effect of the gas composition on $m_\nu ^2$ was probed via the $m_\nu ^2$ shift induced by another gas composition used for the fit model applied to the pseudo-KNM1 Monte-Carlo data set which is generated with the KNM1 gas composition. Both the T$_2$ purity as well as the HT/DT-ratio were tested. Based on [12], an uncertainty of 0.002 on the tritium purity and 0.04 on the HT/DT-ratio were assumed. The obtained shift in $m_\nu ^2$ is $4 \times 10^{-5}~\text {eV}^{2}/\text {c}^{4}$ due to the tritium purity and $1 \times 10^{-6}~\text {eV}^{2}/\text {c}^{4}$ due to the HT/DT-ratio. It can be concluded that the uncertainty on $m_\nu ^2$ originating from the gas composition is negligibly small.

Ortho-para ratio For the KATRIN tritium source, an ortho-to-para-tritium ratio $\lambda =0.75,$ corresponding to a ratio of 3 : 1,  is assumed, which is the equilibrium ratio at room temperature [9]. The equilibrium ratio at ${30\,\mathrm{\text {K}}}$ is $\lambda =0.57.$ However, since the equilibration process is slow relative to the short retention time of ${\mathcal {O}}(\text {s})$ of the tritium in the WGTS, the ${30\,\mathrm{\text {K}}}$ equilibrium is not reached. Estimations based on catalytic conversion times for tritium²⁶ [61] lead to a worst-case upper limit on the ortho-para conversion in the WGTS of ${3\,\mathrm{\%}}$ and therefore $\lambda =0.7275.$

The impact of a ${3\,\mathrm{\%}}$ difference in the ortho-para ratio was investigated by generating a test FSD composed for an ortho-para ratio of 0.7275 and using it for the fit model applied to the pseudo-KNM1 Monte-Carlo data set. The Monte-Carlo data set assumed an ortho-para ratio of 0.75. The observed shift in $m_\nu ^2$ is $7.1 \times 10^{-4}~\text {eV}^{2}/\text {c}^{4}$ which can be seen as a worst-case shift caused by the ortho-para ratio.

Appendix D: Comparison of results for $m_\nu ^2$ using KNM1 FSD and pseudo-KNM1 FSD

A fit using the KNM1 FSD on the reference MC data set with the pseudo-KNM1 FSD yields a shift in $m_\nu ^2$ of $-3.5 \times 10^{-5}~\text {eV}^{2}/\text {c}^{4}.$ Since the calculation of the masses for the KNM1 FSD and the pseudo-KNM1 FSD differs, it would at first glance be expected to see the shift of $1.3 \times 10^{-4}~\text {eV}^{2}/\text {c}^{4}$ between the two FSDs which is demonstrated in Fig. 5. However, there is an additional shift between the two FSDs which counteracts the shift from the different masses: the shift of $-1.7 \times 10^{-4}~\text {eV}^{2}/\text {c}^{4}$ from the unconverged potential curves of the KNM1 FSD which is shown in Fig. 3. The change from the discrete approach for the description of the dissociation continuum in the KNM1 FSD to the density approach for the pseudo-KNM1 FSD causes another shift of $1 \times 10^{-7}~\text {eV}^{2}/\text {c}^{4}.$ This leaves a discrepancy of $5 \times 10^{-6}~\text {eV}^{2}/\text {c}^{4}$ between the KNM1 FSD and pseudo-KNM1 FSD, which can be assigned to the imperfect reproducibility of the KNM1 FSD due to missing computational details in the adopted literature values.