Abstract
The objective of the Karlsruhe Tritium Neutrino (KATRIN) experiment is to determine the effective electron neutrino mass \(m(\upnu _\text {e})\) with an unprecedented sensitivity of \(0.2 \hbox {eV}/\hbox {c}^2\) (\(90 \%\,\hbox {C.L.}\)) by precision electron spectroscopy close to the endpoint of the \(\upbeta \)decay of tritium. We present a consistent theoretical description of the \(\upbeta \)electron energy spectrum in the endpoint region, an accurate model of the apparatus response function, and the statistical approaches suited to interpret and analyze tritium \(\upbeta \)decay data observed with KATRIN with the envisaged precision. In addition to providing detailed analytical expressions for all formulae used in the presented model framework with the necessary detail of derivation, we discuss and quantify the impact of theoretical and experimental corrections on the measured \(m(\upnu _\text {e})\). Finally, we outline the statistical methods for parameter inference and the construction of confidence intervals that are appropriate for a neutrino mass measurement with KATRIN. In this context, we briefly discuss the choice of the \(\upbeta \)energy analysis interval and the distribution of measuring time within that range.
Introduction
While neutrino oscillation experiments [1,2,3] have provided unambiguous evidence of nonzero neutrino masses, the absolute neutrino mass scale remains an open question. The primary objective of the Karlsruhe Tritium Neutrino (KATRIN) experiment is to probe this scale in a direct kinematic measurement at an unprecedented sensitivity of \(0.2 \hbox {eV}/\hbox {c}^2\) (\(90 \%\,\hbox {C.L.}\)) [4]. The measurement principle is based on a shape analysis of the tritium \(\upbeta \)decay spectrum by high precision electron spectroscopy. A nonzero neutrino mass will cause a distortion in the observed spectrum, which is most pronounced close to the endpoint energy of 18.6 keV. This technique has been successfully established by the direct neutrino mass experiments in Mainz and Troitsk, which place the most stringent direct upper limit on the effective electron neutrino mass [5,6,7,8]:
Improving this limit in \(m(\upnu _\text {e})\) by a factor of 10 demands an enhancement in statistical and systematic precision of the effective observable \(m^2(\upnu _\text {e})\) by a factor of 100. This requires both an indepth understanding of the theoretical electron \(\upbeta \)decay spectrum and an accurate knowledge of the experimental response in measuring the spectral shape. In Sect. 3 we explain the KATRIN setup in more detail.
It is the goal of this work to provide a complete and uptodate model of the experiment, such that it can be used as either a prescription or reference for upcoming analyses of tritium \(\upbeta \)decay data observed with KATRIN. For established aspects of this model, we refer to the appropriate publications. For those not yet published at all or not in the required detail, we provide the necessary derivations. The later will mostly be the case for the description of the experimental response function, which has been considerably refined during recent commissioning phases.
In this work we first present a detailed account of the theoretical \(\upbeta \) spectrum of tritium, with an emphasis on molecular effects in \(\mathrm {T_2}\) (Sect. 2). We then outline the experimental configuration of KATRIN (Sect. 3), before we elaborate on the individual characteristics that define the response of our instrument in Sect. 4. The statistical techniques suited to determine the effective neutrino mass from a fit of the modeled \(\upbeta \) spectrum to the measured data are treated in Sect. 5. A summary of this work is given in Sect. 6.
Throughout this article we use natural units (\(c = \hbar = 1\)) for better readability, except for Sects. 4.7 and 4.8 where we use SI units instead.
Theoretical description of the differential \(\upbeta \)decay spectrum
In this section we compile a comprehensive analytical description of the differential \(\upbeta \)decay spectrum, with specific focus on gaseous molecular tritium \(\mathrm {T_2}\), the \(\upbeta \) emitter used by KATRIN. We will also evaluate the relevance of various theoretical correction terms on the neutrino mass analysis.
In the following, we use the shorthand notation \( {m_{\upnu }} = m(\upnu _\text {e})\) for better readability. Furthermore, we assume there is no difference between the masses of the neutrinos and the antineutrinos, i.e. \( {m_{\upnu }} = m(\upnu _\text {e}) = m({\bar{\upnu }}_\text {e})\).
In the \(\upbeta \)decay of atomic tritium, the surplus energy Q is shared between the electron’s kinetic energy E, the total neutrino energy and the recoil energy \(E_\text {rec}\) of the much heavier daughter nucleus:
In the case of a vanishing neutrino mass, the electron spectrum would terminate at the endpoint energy
Fermi theory
The differential decay rate of a tritium nucleus can be described with Fermi’s Golden Rule as [9]
The Fermi coupling constant \(G_\text {F}\) is projected onto the (u, d) coupling by the Cabibbo angle \(\theta _\text {C}\) with \(V_\text {ud} = \cos {\theta _\text {C}} = 0.97425\pm 0.00022\) [8].
For tritium \(\upbeta \)decay – a superallowed transition – the nuclear transition matrix element \(M_\text {nuc}\) is independent of the electron energy. It can be divided into a vector (Fermi) part and an axial (Gamow–Teller) part
with the vector coupling constant \(g_\text {V} = 1\) and the axialvector coupling constant defined by \(g_\text {A}/g_\text {V} = 1.2646\pm 0.0035\) in tritium [10].
The classical Fermi function F(Z, E) accounts for the Coulomb interaction between the outgoing electron and the daughter nucleus with atomic charge Z (here \(Z = 2\)):
with the Sommerfeld parameter \(\eta = \alpha Z/\beta \); \(\alpha \) is the fine structure constant and \(\beta = v/c\) is the electron velocity relative to speed of light. Here F(Z, E) is written in the nonrelativistic approximation; the relativistic \(F(Z,E)_\text {rel}\) and its commonlyused approximation is given in Appendix A.1.
The full spectrum is an incoherent sum over the three known neutrino mass eigenstates \(m_i\) (\(i=1,2,3\)) with the intensity of each component defined by the squared magnitude of the neutrino mixing matrix elements \(U_{\text {e}i}^2\) [11].
The phasespace factor of the outgoing electron with momentum p is given by the factor \(p\, (E+ {m_\text {e}} )\). The phase space of the emitted neutrino is the product of the neutrino energy \(\epsilon = E_0  E\) and the neutrino momentum \(\sqrt{ \epsilon ^2  m_i^2}\), which determines the shape of the \(\upbeta \)electron spectrum near the tritium endpoint \(E_0\). The Heaviside step function \(\varTheta \) ensures that the kinetic energy cannot become negative.
The full \(\upbeta \)decay spectrum is shown in Fig. 1. The dependence of the spectral shape on the effective neutrino mass close to the endpoint is depicted in Fig. 2.
Neutrino mass eigenstate splittings
In the KATRIN sensitivity range we can simplify the analysis by considering the effective electron neutrino mass square \( {m_{\upnu }^2} \) of a quasidegenerate model in Eq. (4), given by an incoherent sum as
Calculations have shown this approximation of the \(\upbeta \)decay spectrum to be valid, both for the normal and inverted mass hierarchies [12, 13].
Molecular tritium \(\mathrm {T_2}\)
When we consider the \(\upbeta \)decay of gaseous molecular tritium \(\mathrm {T_2}\),
the released energy Q has to be corrected for the differences in electronic binding energies between the atomic and actual molecular systems (see [9] for a detailed explanation). The nuclear recoil also excites a spectrum of rotational and vibrational final states in the daughter molecular system, and generates excitations of its electronic shell. The neutrino energy in Eq. (4) has to be corrected by
with the endpoint \(E_0(\mathrm {T_2}) = (18574.00\pm 0.07)\hbox { eV}\) for molecular tritium [9, 14]. The recoil energy reaches a maximum of \(E_\text {rec} = 1.72\hbox { eV}\) at the \(\upbeta \)endpoint, which gives a fixed endpoint energy \(E_0(\mathrm {T_2}) = Q(\mathrm {T_2})  E_\text {rec}\) [9]. The differential decay rate, with the additional summation over each final state f with energy \(V_f\) and weighing by the transitional probability \(P_f\) to a state f in the daughter molecule, is then:
Excited molecular final states
After the decay, the daughter molecular system is left in an excited rotational, vibrational and electronic state. According to theoretical calculations, about 57 % of all \(\mathrm {T_2}\) \(\upbeta \)decays result in the rovibronicallybroadened electronic ground state with an average excitation energy of about 1.7 eV, while the others go to the excited electronic states [17]. Each discrete final state effectively branches into its own \(\upbeta \) spectrum with a distinct endpoint energy.
The accuracy of a neutrino mass measurement critically depends on the knowledge of the distribution of these final states, which have to be taken from theory. Precise calculations of the final state distributions of the hydrogen isotopologues (\(\mathrm {T_2 \rightarrow HeT^+}\), \(\mathrm {DT \rightarrow HeD^+}\) and \(\mathrm {HT \rightarrow HeH^+}\)) have been performed in the endpoint region [15, 16]. The discrete energy states and their transition probabilities have been determined below the dissociation threshold, while continuous distributions are available above the threshold. A comprehensive review of the theory of the tritium finalstate spectrum and current validation efforts can be found in [18].
Figure 3 gives a comparison of the finalstate distributions of \(\mathrm {HeT^+}\) and \(\mathrm {HeD^+}\). The differences in their distributions arise from the mass difference; thus, a precise knowledge of the source gas isotopological composition and its stabilization on the 0.1 %level are necessary. Laser Raman spectroscopy [19] provides two important input parameters for our source model: the tritium purity \(\epsilon _\mathrm {T}\) denoting the fraction of tritium nuclei,^{Footnote 1} and \(\kappa \) denoting the ratio of \(\mathrm {DT}\) versus \(\mathrm {HT}\).
In the calculations provided by Doss et al. [15, 16] and Saenz et al. [20], the higher recoil energies of the lighter isotopologues are incorporated into their respective energy spectra that are given relative to the recoil energy of \(\mathrm {HeT^+}\). That way, the finalstate distributions of each isotopologue can be summed and weighted according to its abundance in the source gas. Furthermore, these calculations provide separate distributions for each initial quantum state of molecular angular momentum, denoted by the quantum number J. These must be weighted according to the population of their respective J states before the \(\upbeta \)decay, which is given by a Boltzmann distribution
where T is the local temperature of the source gas, \(k_B\) the Boltzmann constant and \(\varDelta E_J\) the energy to the electronic ground state. The rotational degeneracy of the distribution is given by the factor \(g_J = (2J+1)\), whereas \(g_s\) accounts for the spin degeneracy of the nuclei. It is \(g_s = 1\) for heteronuclear molecules (\(\mathrm {DT}\), \(\mathrm {HT}\)) without spin coupling. For \(\mathrm {T_2}\) as a homonuclear molecule, it is given by the ratio \(\lambda \) of molecules in an ortho (parallel nuclei spins) state or the ratio \(1\lambda \) in the para states (antiparallel nuclei spins). Hence, \(g_s = \lambda \) for ortho states with odd J and \(g_s = 1\lambda \) for para states with even J [21]. In the KATRIN tritium circulation system the source gas is forced into thermal equilibrium at \(T = 300\hbox { K}\) by a permeator membrane,^{Footnote 2} resulting in \(\lambda \simeq 0.75\) [18].
Exact relativistic threebody calculation
The \(\upbeta \) spectrum formalism outlined above contains approximations to the exact relativistic calculations of the threebody phase space density [22, 23]. In deriving Eq. (10), the dependence of the daughter molecule’s recoil energy \(E_\text {rec}\) on the neutrino mass \(m_i\) and the finalstate spectrum \(V_f\) is neglected. This approximation results in a minute shift of the maximum electron energy, which is on the order of 0.1 meV [23], as depicted in Fig. 4. In the neutrino mass analysis, such a shift in the energy scale is compensated by the external constraint of the endpoint \(E_0\); thus, the effective twobody representation of Eq. (10) is an adequate approximation in the energy region of interest (also see Table 1). A summary of the energydependent, higherorder correction terms is given in Sect. 2.6.
Additional correction terms
In addition to the Fermi function (F(Z, E)) correction factors arising from other nuclear and atomic physics effects must be evaluated and applied multiplicatively. The formulae and the references to these effects are given in Appendix A.1. The following is a synopsis.

Radiative corrections: In addition to the Coulomb interaction described by F(Z, E), electromagnetic effects involving contributions from virtual and real photons give rise to a correction factor \(G(E,E_0)\).

Screening: The unscreened F(Z, E), which describes the Coulomb interaction between the daughter nucleus and the departing \(\upbeta \)electron, must be corrected by a factor S(Z, E) that accounts for the screening effect on the Coulomb field by the \(\text {1s}\)orbital electrons left behind by the parent molecule.

Recoil effects: In the relativistic elementary particle treatment of the \(\upbeta \)decay (see for instance [23, 24]), energydependent recoil effects on the order of 1 / M can be calculated, with M being the mass of \(\mathrm {^3He}\). These effects – spectrum shape modification due to a threebody phase space, weak magnetism and \(VA\) interference – are typically combined into a common factor \(R(E, E_0, M)\).

Finite structure of the nucleus: Because the \(\mathrm {^3He^+}\) daughter nucleus is not a pointlike object, the Coulomb field does not scale with an inversesquared relationship within the radius, leading to a correction factor L(Z, E). A proper convolution of the electron and neutrino wave functions with the nucleonic wave function throughout the nuclear volume leads to another factor C(Z, E).

Recoiling Coulomb field: The departing electron does not propagate in the field of a stationary charge, but one which is itself recoiling from the electron emission. This effect introduces another correction factor \(Q(Z, E, E_0, M)\).

Orbitalelectron interactions: A correction factor I(Z, E) is introduced to account for possible quantum mechanical interactions between the departing \(\upbeta \)electron and the \(\text {1s}\)orbital electrons.
The differential \(\upbeta \) spectrum, including all the theoretical correction factors discussed above, can be written as follows:
The corrections connected to the recoil of the daughter nucleus, namely R and Q, and the radiative corrections G, depend on the endpoint energy and the phase space of a specific excited final state. This dependency is reflected in Eq. (12), as these factors are summed over the possible final states.
In Fig. 5, a graphical overview of these correction factors in the energy interval 30 eV below the tritium endpoint is given. The radiative corrections have the most significant effect with a pronounced energy dependence, as they deplete the spectrum completely towards the endpoint. Most other corrections are negligible in the neutrino mass analysis, as further detailed in Sect. 4.12 and Table 1.
The KATRIN experiment
The experimental setup of KATRIN combines a highluminosity windowless gaseous molecular tritium source (WGTS) with an integrating electrostatic spectrometer of MACE filter (magnetic adiabatic collimation with electrostatic filter) type [25,26,27], offering a narrow filter width and a wide solidangle acceptance at the same time.
The apparatus depicted in Fig. 6 features several major subsystems. The isotopological composition, temperature, and density fluctuations of the tritium source are monitored by a set of calibration devices housed in the rear section (a). The windowless gaseous tritium source (b) contains a beam tube of length \(L = 10\hbox { m}\) and diameter \(d = 90\hbox { mm}\), residing in a nominal magnetic field of 3.6 T, where repurified molecular tritium (\(\mathrm {T_2}\)) is continuously circulated by injection at the center and pumping at both ends through a closed loop system [28,29,30]. To prevent tritiated gas from entering the spectrometer section, the transport section (c) combines differential pumping with cryogenic pumping to reduce the tritium flow by 14 orders of magnitude [31, 32]. The \(\upbeta \)electrons are guided through the entire beamline by a magnetic field [33] into the prespectrometer (d), which acts as a prefilter that blocks the lowenergy electrons of the \(\upbeta \)spectrum [34]. The energy analysis around the endpoint region takes place in the main spectrometer (e), which is operated under ultrahigh vacuum conditions [35] at a retarding voltage of about \(18.6\hbox { kV}\). Both spectrometers are designed as MACE filters, and the main spectrometer achieves a very narrow filter width (\(\lesssim 1\hbox { eV}\)) [9] while providing high luminosity for the \(\upbeta \)electrons. Electrons with sufficient energy pass both the MACE filters and are then counted at a segmented silicon PIN diode detector (f) [36] with 148 individual pixels. An integrated \(\upbeta \)spectrum is recorded by scanning the retarding voltage in the endpoint region.
MACE filter principle
The electrons emitted isotropically from tritium \(\upbeta \)decay in the gaseous source are guided adiabatically by magnetic fields. In the forward direction the \(\upbeta \)electrons are confined in cyclotron motion along the magnetic field lines towards the MACE filter. Along their path to the analyzing plane (central plane) of the spectrometer, the magnetic field strength decreases by several orders of magnitude.^{Footnote 3} Due to the conservation of magnetic moment in a slowly varying field, most of the electrons’ transverse momentum is adiabatically transformed into longitudinal momentum. With a high negative potential (\(U \approx 18.6\hbox { kV}\), corresponding to the endpoint energy of tritium) at its center and most of the electron momentum being parallel to the magnetic field lines, the MACE filter acts as an electrostatic highpass energy filter. Only electrons with positive longitudinal energy (the kinetic energy in direction of the magnetic field line) along their entire trajectory are transmitted, while the others are reflected and reaccelerated towards the entrance of the spectrometer.
The residual transverse energy, which cannot be analyzed by the filter, is defined by the ratio of the maximum \(B_\text {max}\) to the minimum magnetic field \(B_\mathrm {min} = B_\text {A}\). This key characteristic of the MACE filter is commonly called the filter width (or sometimes energy resolution)
with E being the electron kinetic energy and \(\gamma = \frac{E}{m_\text {e}} + 1\) the relativistic gamma factor with the electron rest mass \(m_\text {e}\).
Response function of the KATRIN experiment
In the KATRIN experiment, the energy of the \(\upbeta \)electrons is analyzed using the MACE filter technique as described in Sect. 3. For a specific electrostatic retardation potential U, the count rate of electrons at the detector can be calculated, given the probability of an electron with a starting energy E to traverse the whole apparatus and hit the detector. This probability is described by the socalled transmission function T(E, U). Additional modifications arise from energy loss and scattering in the source, and reflection of signal electrons propagating from their point of origin until detection. These effects are incorporated together with the transmission function into the response function R(E, U), which is vital for the neutrino mass analysis as it describes the propagation of signal electrons that contribute to the integrated \(\upbeta \)spectrum.
For illustrative purposes, we first consider a source containing a given number of tritium nuclei (\(N_\mathrm {T}\)) that decay with an isotropic angular distribution.^{Footnote 4} The emitted electrons are guided by magnetic fields through the spectrometer. The detection rate at the detector for a given spectrometer potential U can be expressed as:
where the factor of \(\frac{1}{2}\) incorporates the fact that the response function R(E, U) only considers electrons emitted in the forward direction.
In the following, an analytical description of the response function of the KATRIN experiment will be laid out. At first, we derive the transmission function of the MACE filter that is implemented by the main spectrometer (Sect. 4.1). In Sect. 4.2 we consider energy loss in the source and develop a first description of the response function. Inhomogeneities in the MACE filter (Sect. 4.3) and the source (Sect. 4.4) requires extension of the model by a segmentation of the source and spectrometer volume. Further modifications to the response function arise from considering the effective source column density which an individual \(\upbeta \)electron traverses (Sect. 4.5), changes to the electron angular distribution (Sect. 4.6), thermal motion of the source gas (Sect. 4.7), and energy loss by cyclotron radiation (Sect. 4.8). After discussing these contributions, in Sect. 4.9 we arrive at a description of the integrated spectrum that is measured by the KATRIN experiment. We close the discussion with a general note on experimental energy uncertainties (Sect. 4.11) and give a quantitative overview of theoretical corrections and systematic effects (Sect. 4.12) on the neutrino mass analysis.
Transmission function of the MACE filter
The transmission of \(\upbeta \)electrons through the MACE filter is an important characteristic of the measurement and a significant part of the response function. In the simplest case, one can assume that electrons enter the MACE filter with an isotropic angular distribution and propagate adiabatically towards the detector. In the discussion here we apply the adiabatic approximation (see Eq. (15) below), which is fulfilled in the case of KATRIN.
In general, an electron from the source will reach the detector if the momentum \(p_\parallel \) parallel to the magnetic field lines (or the corresponding fraction \(E_\parallel \) of the kinetic energy) is always positive. The transformation of transverse to parallel momentum and back in a slowly varying magnetic field B is governed by the following adiabatic invariant (which corresponds to the conserved orbital momentum \(\mu = E_\perp / B\) in the nonrelativistic limit):
In the following discussion we use the general relation between the transverse momentum \(p_\perp \) of an electron with its transverse kinetic energy \(E_\perp \):
with the relativistic gamma factor \(\gamma = \frac{E}{ {m_\text {e}} } + 1\), and thereby define the transverse kinetic energy as:
Similarly, we define the longitudinal kinetic energy as \(E_\parallel = E \; \cos ^2 \theta \). The polar angle \(\theta = \angle (\mathbf {p},\mathbf {B})\) of an electron momentum to the magnetic field is called the pitch angle.
We can now define the adiabatic transmission condition for an electron starting at the position \(z_\text {S}\) with a magnetic field \(B_\text {S} = B(z_\text {S})\), an electrostatic potential \(U_\text {S} = U(z_\text {S})\), a kinetic energy \(E = E(z_\text {S})\) with a corresponding gamma factor \(\gamma \), and a pitch angle \(\theta = \theta (z_\text {S})\). The transmission condition then reads for all longitudinal positions z:
where \(\gamma (z)\) corresponds to the gamma factor at an arbitrary position z along the beam line where the electron has a kinetic energy \(E(z) = E_\parallel (z) + E_\perp (z)\) at a magnetic field B(z) and an electrostatic potential U(z).
Usually in a MACE filter the highest retarding potential U and at the same time the smallest magnetic field \(B_\text {A}\) is reached in the analyzing plane (located at \(z_\text {ap} = 0\) in our definition). Secondly we can assume the electrical potential \(U_\text {S}\) at the start to be zero and the relativistic factor in the analyzing plane at the largest retardation (minimum kinetic energy) to equal one, \(\gamma (z_\text {ap}) = 1\). Therefore the transmission condition in Eq. (18) simplifies to
For a given electric potential and magnetic field configuration of the MACE filter, the transmission condition \({\mathcal {T}}\) is thus just governed by the starting energy E, the starting angle \(\theta \) and the retarding voltage U.
For an isotropically emitting electron source with angular distribution \(\omega (\theta ) \, \,\mathrm {d}\theta = \sin \theta \, \,\mathrm {d}\theta \), we can integrate \({\mathcal {T}}(E,\theta ,U)\) over the angle \(\theta \) and define a response or transmission function. From here on we associate the remaining energy in the analyzing plane of the MACE filter – the surplus energy – with the expression \({\mathcal {E}}= E  qU\).
In the KATRIN setup the maximum magnetic field \(B_\text {max}\) is larger than \(B_\text {S}\), so that \(\upbeta \)electrons emitted at large pitch angles in the source are reflected magnetically before reaching the detector. The magnetic reflection occurs at the pinch magnet (with \(B = B_\text {max}\) and zero potential), and in the source the electric potential is zero. The maximum pitch angle of the transmitted electrons is therefore independent of the electron energy and given by:
For the standard operating parameters of KATRIN (see Table 2), \(\theta _\text {max}\) evaluates to about \(50.8^{\circ }\). This reflection is desired by design, since \(\upbeta \)electrons emitted with larger pitch angles have to traverse a longer effective column of source gas and are therefore more likely to scatter and undergo energy loss, as detailed in the following sections.
With this additional magnetic reflection after the analyzing plane, the transmission function is given by:
with the filter width \(\varDelta E\) from Eq. (13). In Fig. 7, the transmission function is shown for the nominal KATRIN operating parameters and for the case \(B_\text {S} = B_\text {max}\). The magnetic reflection imposes an upper limit on the pitch angle, which reduces the effective width of the transmission function. As indicated in Fig. 7, this improves the filter width of the spectrometer to 0.93 eV, compared with 1.55 eV for \(\theta _\mathrm {max} = 90 ^{\circ }\) without magnetic reflection.
Response function and energy loss
In the next step we consider the energy loss when the electron traverses the gaseous source. The dominant energy loss process is the scattering of electrons on gas molecules within the source. Because the pressure decreases rapidly outside the source, scattering processes in the transport section or thereafter are of no concern.
Two ingredients are required to appropriately treat electron scattering in the source. First, the energy loss function \({\tilde{f}}(\epsilon , \delta \vartheta )\) describes the probability for a certain energy loss \(\epsilon \) and scattering angle \(\delta \vartheta \) of the \(\upbeta \)electrons to occur in a scattering process. Because the scattering angles \(\delta \vartheta \) are small,^{Footnote 5} we will neglect them in the following formulae and describe the scattering energy losses by the function \(f(\epsilon )\). Here we do not consider a dependence of f or \(P_s\) on the incident kinetic energy E of the electrons, since for the KATRIN experiment the energy range of interest amounts to a very narrow interval of a few times 10 eV below the tritium endpoint only, where these functions can be considered as independent of E. The other important ingredients are the scattering probability functions \(P_s(\theta )\) for an electron with pitch angle \(\theta \) to scatter s times before leaving the source. These scattering probabilities depend on \(\theta \), since electrons with a larger pitch angle must traverse a longer path, meaning a larger effective column density, and are thus likely to scatter more often.
With these considerations, the response function no longer comprises only the transmission function, but is modified as follows:
Electrons leaving the source without scattering \((s = 0)\) do not lose any energy, hence \(f_0(\epsilon ) = \delta (\epsilon )\). For sfold scattering, \(f_s(\epsilon )\) is obtained by convolving the energy loss function \(f(\epsilon )\) s times with itself.
The scattering cross section can be divided into an elastic and an inelastic component. The inelastic cross section and the energy loss function for electrons with kinetic energies of \(\approx 18.6\hbox { keV}\) scattering from tritium molecules have both been measured in [40, 41]. In this work, the inelastic scattering cross section was determined to be \(\sigma _\text {inel} = (3.40 \pm 0.07) \times 10^{18}\,\hbox {cm}^2\) and an empirical model was fit to the energy loss spectrum.
The latter is parameterized by a lowenergy Gaussian and a highenergy Lorentzian part:
with \(A_1 = (0.204 \pm 0.001)\,\text {eV}^{1}\), \(A_2 = (0.0556 \pm 0.0003)\,\text {eV}^{1}\), \(\omega _1 = (1.85 \pm 0.02)\,\text {eV}\), \(\omega _2 = (12.5 \pm 0.1)\,\text {eV}\), \(\epsilon _2 = (14.30 \pm 0.02)\,\text {eV}\) and a fixed \(\epsilon _1 = 12.6\,\text {eV}\). To obtain a continuous transition between the two parts of \(f(\epsilon )\), a value \(\epsilon _c = 14.09\,\text {eV}\) was chosen. The Gaussian part summarizes the energy loss due to (discrete) excitation processes, while the Lorentzian part describes the energy loss due to ionization of tritium molecules.
This parameterization of the energy loss function is used for the response model presented in this paper. However, the parameters are not precise enough for KATRIN to meet its physics goals. Dedicated electron gun measurements with the full experimental KATRIN setup have been planned for the determination of the inelastic scattering cross section and the energy loss function with higher precision; the analysis of these data will involve a sophisticated deconvolution technique [42].
At \(\sigma _\mathrm {el} = 0.29 \times 10^{18}\,\mathrm {cm}^{2}\), the total cross section of elastic scattering of 18.6 keV electrons with molecular hydrogen isotopologues is smaller than that for inelastic scattering by an order of magnitude [43, 44]. In addition, the elastically scattered electrons are strongly forward peaked with a median scattering angle of \({{\overline{\theta }}}_\text {scat} = 2.1 ^{\circ }\) near the tritium endpoint energy. The energy loss due to elastic scattering is given by the relation
With an angular distribution for elastic scattering of molecular hydrogen by electron impact based on [45], the corresponding median energy loss amounts to \(\overline{\varDelta E} = 2.3\hbox { meV}\). The energy loss function, containing the elastic and inelastic components weighted by their individual cross section, is shown in Fig. 8.
The elastic energy loss component can be accurately calculated. Due to its narrow width and steep slope, \(\sim \hbox {meV}\) binning is required for incorporating it accurately in the response function, thereby increasing computational cost considerably. We will neglect the elastic scattering component in neutrino mass measurements as the associated systematic error on an \( {m_{\upnu }^2} \) is minute (\(\sim 5.10^{5}\hbox {eV}^2\), see Table 1).
Radial inhomogeneity of the electromagnetic field
To calculate the transmission and response functions of the KATRIN setup as explained in Sects. 4.1 and 4.2, it is in principle sufficient to only consider the axial position of an electron to identify the initial conditions such as electromagnetic fields or scattering probabilities. In the case of the main spectrometer, radial dependencies must be incorporated in the description of the magnetic field and the electrostatic potential in the analyzing plane. Additional radial dependencies in the source are discussed in Sect. 4.4; these are then incorporated into the model together with the spectrometer effects.
In order to achieve a MACE filter width in the eVregime, a reduction of the magnetic field strength in the analyzing plane on the order of \(\frac{B_\text {A}}{B_\text {max}} \approx \frac{\varDelta E}{E} \approx 10^{4}\) is required (see Eq. (13)). Consequently the diameter of the fluxtube area A is drastically increased due to the conservation of magnetic flux \(\varPhi = \mathrm {const} \approx B \cdot A\). When nominal field settings are applied (see Table 2), the projection of the detector surface with radius \(r_\mathrm {det} = 4.5\hbox { cm}\) has a radius of about 4 m in the analyzing plane. A larger (smaller) magnetic field in the analyzing plane \(B_\mathrm {A}\) shifts the transmission edge to a larger (lower) energy, see Eq. (20). This effect is even more pronounced for larger electron pitch angles. Consequently, the transmission function (see Eq. (22)) is also widened or narrowed. Utilizing a set of magnetic field compensation coils, operated with an optimal current distribution, around the spectrometer vessel, the spread of the radial inhomogeneity of the magnetic field is minimized to a few \({\upmu \hbox { T}}\) when an optimized current distribution is applied [37, 38]. The resulting variation in the filter width in the analyzing plane due to the magnetic field inhomogeneity is thus reduced to about 10 meV [46].
In the case of the electrostatic potential, unavoidable radial variation arises from the design of the spectrometer. To fulfill the transmission condition in Eq. (19), the electrode segments at the entrance and exit are operated on a more positive potential than in the central region close to the analyzing plane.^{Footnote 6} Depending on the final potential setting, the radial potential variation in the analyzing plane is expected to be of order 1 V [39]. In comparison, azimuthal variations are negligible. It is possible to considerably reduce the radial potential inhomogeneity by operating the MACE filter at larger \(B_\mathrm {A}\). However, this would require better knowledge of the magnetic field in the analyzing plane [46] and also increase the filter width.
Even with these optimizations of the setup, the small radial variations in the electromagnetic fields at the analyzing plane, as shown in Fig. 9, cannot be neglected. The segmentation of the KATRIN main detector into annuli of pixels allows us to incorporate such radial variations in the response function model for each individual detector pixel. Because the tritium source also features radial variations of certain parameters, this segmentation is combined with a full segmentation of the source volume as described in Sect. 4.4. Dependencies of the electromagnetic field are typically averaged over the surface area of a pixel. The specific detector geometry with thinner annuli towards outer radii (each with equal surface area) helps minimize the potential variation within individual annuli, despite the increasing steepness of the potential.
Source volume segmentation and effects
In addition to radial dependencies of the analyzing plane parameters that govern the energy analysis of the \(\upbeta \)electrons (Sect. 4.3), the tritium source also features radial and axial dependencies of its parameters. In the following, we will briefly outline the most relevant source parameters that are required to accurately model the differential \(\upbeta \) spectrum and the response function. These parameters include the beam tube temperature \(T_\text {bt}\), the magnetic field strength \(B_\text {S}\), plasma potentials \(U_\text {P}\), the particle density \(\rho \) and the bulk velocity u of the gas, all of which may vary slightly in longitudinal, radial and azimuthal directions. The complex gas dynamic simulations, which are needed to calculate these local source parameters, are described in comprehensive detail in [47, 48].
In order to model accurately these effects for each individual detector pixel, the simulation source model is partitioned to match the detector geometry. It is partitioned longitudinally into \(N_L\) slices and segmented radially into \(N_R\) annuli (rings) of \(N_S\) segments each, resulting in a total of \(N_L \cdot N_R \cdot N_S\) segments (see Fig. 10). The geometry of these segments is chosen in such a way, that a longitudinal stack of segments is magnetically projected^{Footnote 7} onto a corresponding detector pixel. Note that all detector pixels have identical surface area, which leads to broader annuli at the center and thinner annuli towards larger radii. In the following, we index the longitudinal slices by the subscript i and radial/azimuthal segments with their corresponding detector pixel by the subscript j.
At a retarding potential U, the detection rate for a specific detector pixel j can then be stated as
where \(N_{\text {T},i}\) is the number of tritium nuclei (assuming that the gas density has no radial or azimuthal dependence). The response function \(R_{i,j}(E, U)\) depends on the index i (i.e. the axial position) and the index j (i.e. the radial/azimuthal position) of the source segment. With the indices i, j we can describe the dependence on local source parameters such as the magnetic field. The most significant effect on the response is caused by the scattering probabilities, as detailed in Sect. 4.2. The index j further describes nonuniformities of the retarding potential U and the magnetic field \(B_\text {A}\) in the spectrometer (see Fig. 9).
Scattering probabilities
As discussed in Sect. 4.2, inelastic scattering results in an energy loss that directly affects the energy analysis of the signal electrons, and needs to be incorporated accurately into the analytical description. Changes to the angular distribution of the emitted electrons due to scattering processes, which also modify the response function, are discussed in Sect. 4.6.
The scattering probability for \(\upbeta \)electrons is considerably different depending on their starting position in the 10 m long source beam tube, as visualized in Fig. 11. The longitudinal segmentation of the source volume in our model allows us to incorporate this behavior. The probability \(P_s\) for an electron to leave the source after scattering exactly s times depends on the total cross section \(\sigma \) and the effective column density \(\mathcal {N}_\text {eff}\) that the electron traverses. This effective column density depends not only on the electron’s starting position z inside the source and the axial density distribution \(\rho (z)\), but also on the starting pitch angle \(\theta \) in the source (Eq. (21)):
L denotes the length of the source beam tube with \(L/2 \le z \le L/2\). The nominal column density is then given by \(\mathcal {N}= \mathcal {N}_\text {eff}(z=L/2,\ \theta =0)\).
Because of the low probability to scatter off a single tritium molecule, the number of scatterings during propagation can be calculated according to a Poisson distribution:
The mean scattering probabilities for a specific position z can be calculated using the isotropic angular distribution \(\omega (\theta ) = \sin {\theta }\) and the maximum pitch angle \(\theta _\text {max}\):
This integration assumes that the angular distribution is not significantly affected by the small angular change in the discussed scattering processes. A higher total column density \(\mathcal {N}\), as well as a larger \(\theta _\text {max}\), would provide a larger number of \(\upbeta \)electrons at the exit of the source and at the detector. However, they also raise the proportion of scattered over unscattered electrons, thereby increasing the systematic uncertainties due to energy loss, and at some point, limiting the \(\upbeta \)electron detection rate close to the endpoint. The optimal design values of \(\mathcal {N}= 5.10^{17}\hbox { cm}^{2}\) and \(\theta _\text {max} = 50.8 ^{\circ }\) [4] balance these effects.
Response function for nonscattered electrons
The transmission function in Eq. (22) describes the transmission probability of isotropically emitted electrons. Even if we consider only nonscattered electrons, the \(\upbeta \)electrons do not follow an isotropic angular distribution before entering the spectrometer due to the pitch angle dependence of the sfold scattering probabilities \(P_s(z,\theta )\) in the source (see Sect. 4.5).
Following the description in [39], the zeroscattering (\(s=0\)) transmission function needs to be modified to the form:
The zeroscattering probability \(P_0(\theta )\) is computed by averaging \(P_0(z,\theta )\) over z. Figure 12 illustrates the resulting difference in the response function. The surplus energy range \({\mathcal {E}}< 10\hbox { eV}\) corresponds to the steep increase in the response function at low energies as shown in Fig. 11, where energy loss from inelastic scattering does not contribute.
Doppler effect
The thermal translational motion and the bulk gas flow of the \(\upbeta \)emitting tritium molecules in the WGTS lead to a Doppler broadening of the electron energy spectrum, which further modifies the response function model that was derived in Sect. 4.2 and thereafter. These two effects can be expressed as a convolution of the differential spectrum \(\frac{\,\mathrm {d}\varGamma }{\,\mathrm {d}E}\) with a broadening kernel g, denoted by the subscript \(\text {D}\):
with \(E_\text {cms}\) being the electron kinetic energy in the \(\upbeta \)emitter’s rest frame (which is approximately the centerofmass system), and \(E_\text {lab}\) the electron energy in the laboratory frame.
The magnitude of the thermal tritium gas velocity follows a MaxwellBoltzmann distribution. However, considering only the velocity component \(v_\text {M}\) that is parallel to the electron emission direction, the thermal velocity distribution of the tritium isotopologue mass M is described by a Gaussian
which centers around \(v_\text {M}= 0\) with a standard deviation \(\sigma _v = \sqrt{k_\text {B}T_\text {bt} / M}\). For the component of the bulk gas velocity u that is parallel to the electron emission direction with pitch angle \(\theta \), the mean \(v_\text {M}\) is shifted by \(\cos \theta \cdot u\). Integrating over all emission directions up to \(\theta _\text {max}\), the expression expands to
Using the Gaussian error function this expression can be rewritten as
Finally, the tritium gas velocity distribution \(g(v_\text {M})\) can be translated into an electron energy distribution \(g(E_\text {cms}, E_\text {lab})\). Using the Lorentz factors and the electron velocities defined in the CMS and lab frames, we can write
with
The standard deviation of this convolution kernel evaluates to
With \(\sigma _v \approx 203\hbox { m/s}\) for \(\mathrm {T_2}\) molecules at \(T_\text {bt}=30\hbox { K}\) and the weighted mean bulk velocity at nominal source conditions being \({\bar{u}} \approx 13\hbox { m/s}\), thermal Doppler broadening clearly is a dominating effect. The standard deviation of the broadening function \(g(E_\text {cms}, E_\text {lab})\) at a fixed bulk velocity \(u = 0\) for \(T_\text {bt} = 30\hbox { K}\) and \(E \approx E_0\) evaluates to \(\sigma _E \approx 94\hbox { meV}\) (also see Fig. 13). This value can be interpreted as a significant smearing of the energy scale. Its implication for the neutrino mass measurement is shown in Table 1.
Cyclotron radiation
As electrons move from the source to the spectrometer section in KATRIN, they lose energy through cyclotron radiation. In contrast to energy loss due to scattering with tritium gas (Sect. 4.5), this energy loss process applies to the entire trajectory of an electron as it traverses the experimental beamline [49].
For a particle with kinetic energy E spending a time \(\varDelta t\) in a fixed magnetic field B, the cyclotron energy loss is (in SI units):
In general, cyclotron radiation reduces the transverse momentum component of the particle.^{Footnote 8} Consequently, the losses are maximal for large pitch angles and vanish completely at \(\theta = 0 ^{\circ }\).
For complex geometric and magnetic field configurations as in the KATRIN experiment, the overall cyclotron energy loss can be computed using a particle tracking simulation framework such as Kassiopeia [50]. By this means, the cyclotron energy loss from the source to the analyzing point in the main spectrometer can be obtained as a function of the electron’s starting position z and pitch angle \(\theta \). Particles starting in the rear of the source will lose more energy due to their longer path through the whole setup. The total cyclotron energy loss can be up to 85 meV for electrons with the maximum pitch angle \(\theta _\text {max} = 50.8 ^{\circ }\).
Because the resulting decrease in the angle \(\varDelta \theta \) due to the loss of transverse momentum is of order \(10^{6}\) or less, it can be neglected. We thus consider the loss of cyclotron energy \(\varDelta E^\text {cycl}(\theta ,z)\) to be a decrease in the total electron kinetic energy E. Essentially, this effect causes a shift of the electron transmission condition (see Eq. (20))
with the index i denoting the longitudinal slice where the electron starts from the source position z (see Fig. 10).
The influence of the cyclotron energy loss on the averaged response function is shown in Fig. 14.
Expected integrated spectrum signal rate
Earlier in this section we have laid out the different contributions to the response function of the experiment, which describes the probability for \(\upbeta \)electrons to arrive at the detector where they contribute to the measured integrated spectrum. The response function describes the energy analysis at the spectrometer (Sects. 4.1 and 4.3), energy loss caused by scattering in the tritium source (Sects. 4.2 and 4.5), and additional corrections (Sect. 4.6 and following).
Combining the response function with the description of the differential spectrum that was developed in Sect. 2, the integrated spectrum signal rate observed on a single detector pixel j for a retarding potential setting U can finally be expressed as
This expression incorporates all theoretical corrections (see Eq. (12) with subscript C) and the Doppler broadening (see Eq. (33) with subscript D) of the differential spectrum \(\frac{\,\mathrm {d}\varGamma }{\,\mathrm {d}E}\) (see Eq. (10)), and the full response function which incorporates the energy loss as a result of source scattering and cyclotron radiation:
The response function depends on the path traversed by the \(\upbeta \)electron between its origin in source segment (i, j) and the target detector pixel j (see Fig. 10 for the segmentation schema). The detection efficiency \(\epsilon _{\text {det},j}\) is an energydependent quantity, which needs to be measured for each pixel j. Its value is between \(\approx 90 \%\) and \(95 \%\) [36].
To first order (due to nearly constant magnetic field and tritium concentration in the source), the integrated signal rate in Eq. (41) depends on \(\mathcal {N}\sigma \) – which can be accurately determined by calibration measurements with a photoelectron source – but is independent of the longitudinal gas density profile \(\rho (z)\) which cannot be measured directly (see [47, 48] for simulation results).
Scan of the integrated spectrum
A scan of the integrated \(\upbeta \) spectrum comprises a set of detector pixel event counts \(N_j(U_k)\), observed at various retarding potential settings \(U_k\) for the duration of \(\varDelta t_k\) each, with \(k \in \{1 \ldots n_k\}\). In the following, the indices j and k are condensed by writing \(N_{jk} = N_j(U_k)\), with \(N_{jk}\) denoting the event count on a single detector pixel j for a specific retarding potential setting k.
The observed event count \(N^\text {obs}_{jk}\) is a Poissondistributed quantity with the expectation value given by
where \(\dot{N}^\text {bg}_j\) is an energyindependent background rate component (possibly with a radial dependency indicated by the index j).
KATRIN will be operated for a duration of 5 calendar years in order to collect 3 live years of spectrum data over multiple runs.
Energy uncertainties
At the end of this section we will briefly discuss the influence of energy uncertainties on the neutrino mass measurement. In general, any fluctuation with variance \(\sigma ^2\) induces a spectrum shape deformation which – if not considered in the analysis – is indistinguishable to first order from a shift of the measured value of \( {m_{\upnu }^2} \) in the negative direction with \(\varDelta {m_{\upnu }^2} = 2 \sigma ^2\) [11]. This shift of \(\varDelta {m_{\upnu }^2} \) also holds if an accounted fluctuation or distribution of true variance \(\sigma ^2_\mathrm {true}\) is described wrongly in the analysis by the variance \(\sigma ^2_\mathrm {ana} = \sigma ^2_\mathrm {true}  \sigma ^2\).
Different sources of fluctuations and distributions with uncertainties can be distinguished. One group comprises \(\upbeta \)decay and source physics, such as molecular final states, scattering processes and the Doppler effect (all discussed in this work). Others are experimental systematics originating in the energy measurement, which have to be studied during commissioning of the setup and then incorporated into the model. An example is the distortion of the spectrometer transmission function due to retardingvoltage fluctuations [51, 52].
Impact of theoretical and experimental corrections
In Table 1 we review and quantify the impact of theoretical corrections to the differential \(\upbeta \)spectrum, discussed in Sect. 2, and of experimental corrections which have been introduced above. Many individual model components can be safely neglected, while others need to be considered more accurately, such as the radial dependence of retarding potentials (Sect. 4.3), energy loss due to cyclotron radiation (Sect. 4.8) or the Doppler effect (Sect. 4.7).
Measurement of the neutrino mass
Having compiled a complete description of the theoretical \(\upbeta \)decay spectrum and the response function of KATRIN into a parameterizable model, we will now outline the statistical terms and methods required for actual neutrino mass measurements. In the next (Sect. 5.1 and 5.2) we review the process of parameter inference (model fitting) and the construction of confidence intervals in the case of a KATRIN neutrino mass analysis, and we explain the relation between observed data, fit parameters and their uncertainties. After introducing Frequentist methods of inferring \( {m_{\upnu }^2} \) we give an example of a Bayesian approach in Sect. 5.3. We briefly list statistical and systematic uncertainty contributors for KATRIN in Sect. 5.4 and in that context discuss the relevance of the choice of the energy analysis interval in Sect. 5.5 and the distribution of accounted measuring time among that interval in Sect. 5.6. In Sect. 5.7 we give an explanation of negative \( {m_{\upnu }^2} \) estimates and provide a nonphysical extension of the \(\upbeta \)decay spectrum model.
Parameter inference
The statistical technique for analyzing \(\upbeta \)decay spectrum data is well established. By comparing the observed number of counts \(N^\text {obs}_{jk}\) on each pixel j for each experimental setting k with the prediction from the spectrum and response model \(N_{jk}(U_k, {m_{\upnu }^2} , E_0, \dots )\) (see Eqs. (41) and (43)), \( {m_{\upnu }^2} \) and other unknown model parameters can be inferred. In the case of a KATRINlike neutrino mass measurement, a continuous model that depends on \( {m_{\upnu }^2} \) is fit to unbinned spectral shape data. The method of least squares is most commonly applied.
The probability to have an observed outcome \(\varvec{N}^\text {obs} = \left( N^\text {obs}_{1,1} \ldots N^\text {obs}_{n_j,n_k}\right) \), given the predicted number of counts \(\varvec{N}^\text {pre}(\varvec{\theta })\) defined by a set of model parameters \(\varvec{\theta } = (m_\upnu ^2, E_0, \dots )\), is the likelihood function
A set of parameter point estimates \({{\hat{\varvec{\theta }}}}\) is obtained by maximizing the likelihood L. Equivalently, a minimization of the negative loglikelihood \( \ln L\) can be performed, which is often more practical numerically.
If the number of observed events \(N_{jk}^\text {obs}\) is large enough (\(\gtrsim 25\)), so that the Poisson distribution can be approximated by a Gaussian, that expression is approximately a \(\chi ^2\) function:
In case of \(\sigma _{jk} = \sqrt{N_{jk}^\text {pre}}\), the above \(\chi ^2\) equals the Pearson’s chisquare statistic [53].
Our parameter of interest is \( {m_{\upnu }^2} \), which distorts the spectrum shape close to the endpoint. Because the fitted \(\upbeta \)spectrum shape essentially only depends on \( {m_{\upnu }^2} \), with \(\chi ^2\) being approximately parabolic in \( {m_{\upnu }^2} \), it is the preferred fit parameter over \( {m_{\upnu }} \) [54].
Other model parameters are nuisance parameters. In KATRINlike experiments typically three such quantities are treated as free fit parameters:

The tritium endpoint energy \(E_0\), the maximum electron energy assuming a vanishing neutrino mass, has to be estimated from the data, due to uncertainties in the measured \(\mathrm {T^+}\)/\(\mathrm {^3He^+}\) mass difference [55] and in the experimental energy scale.

The signal amplitude \(A_\text {sig}\), a multiplicative factor close to 1, is applied to the predicted signal rate^{Footnote 9} \(\dot{N}_j^\text {sig}\) to correct for any energyindependent model uncertainty. \(E_0\) and \(A_\text {sig}\) are estimated from the slope of the spectrum at lower energies of the analysis interval (\(\approx 30  40 \hbox { eV}\) below the endpoint), where the absolute signal rate is highest.

The background rate amplitude \(A_\text {bg}\) is another normalization factor, which is applied to the background model component \(\dot{N}_j^\text {bg}\). It is estimated using the data from retarding potentials above the tritium endpoint, where no signal is expected. Note that we assume a constant background rate without retarding potential dependence in the energy interval near the tritium endpoint. However, such an energy dependence could be incorporated into the model using additional data above the endpoint.
Considering only the aforementioned four model parameters, the predicted number of electrons on a detector pixel j for a retarding potential setting k in a counting period \(\varDelta t_k\) is given by
A point estimate for this set of parameters, obtained from maximizing the likelihood (or minimizing \(\chi ^2\)) is denoted in the following as \(( \widehat{m_{\upnu }^2} , \widehat{E_0}, \widehat{A_\text {sig}}, \widehat{A_\text {bg}})\).
Depending on the method of treating systematic uncertainties, the number of free (or constrained) model parameters can be higher.
Confidence intervals
Due to the stochastic nature of the observed data, a single parameter point estimate by itself cannot relate to the unknown true value of a parameter. In parameter inference, a confidence interval defines an interval of parameter values that contain the true value of the parameter to a certain proportion (confidence level), assuming an infinite number of independent experiments. Various methods of constructing such intervals exist.
Using the Neyman construction [56] (a Frequentist method), ensembles of pseudoexperiments are sampled for a range of true values of \( {m_{\upnu }^2} \), leading to the construction of a confidence belt (see Fig. 15). Incorporating an ordering principle proposed by Feldman and Cousins [57], empty confidence intervals for nonphysical estimates of \( {m_{\upnu }^2} \) can be avoided, while ensuring correct Frequentist coverage.
When parameter point estimates are constructed following the maximum likelihood ordering principle, the profile likelihood ratio [58] can be used to estimate their uncertainties. With this method the \(1 \sigma \) uncertainty of a parameter estimate is identified by those parameter values where the likelihood has decreased to half its maximum value, while profiling (maximizing) with respect to any involved nuisance parameter. Equivalently, a chisquare curve can be scanned for parameter values with \(\varDelta \chi ^2 = 1\), again profiling over nuisance parameters.
Bayesian statistics
Bayesian inference is typically based on the posterior PDF (probability density function) of a parameter of interest. Using Bayes’ theorem, the posterior distribution \(p(\varvec{\theta })\) of a set of parameters \(\varvec{\theta }\) is given by the likelihood \(L(\varvec{\theta })\) and a prior probability \(\pi (\varvec{\theta })\):
In contrast to Frequentist approaches, which make a statement about the repeatability of an experiment, Bayesian statistics inevitably introduce the concepts of probability, belief and credibility. The prior probability \(\pi (\varvec{\theta })\) has to be chosen by the analyst, based on prior belief. In the case of \( {m_{\upnu }^2} \), an objective option is the flat uniform prior (possibly zero for \( {m_{\upnu }^2} < 0\hbox { eV}^2\)), or a normalizable Gaussian distribution that reflects the results from previous measurements.
Fortunately, KATRIN’s \( {m_{\upnu }^2} \) posterior PDF is rather insensitive to the choice of prior on \( {m_{\upnu }^2} \). Assuming, for instance, a true value of \( {m_{\upnu }^2} = 0\hbox { eV}^2\), a Gaussian prior with mean \(\mu _\pi = 0\hbox { eV}^2\) and \(\sigma _\pi = 1\hbox { eV}^2\) (or a value on the order of the Mainz or Troitsk upper limits) will be outweighed by the KATRIN likelihood function. It will thus have no significant effect on the derived Bayesian upper limit compared to a prior that is flat in \( {m_{\upnu }^2} \). This underlines the improved sensitivity of the experiment.
The posterior distributions can be obtained practically with Markovchain Monte Carlo (MCMC) methods [59]. With proper adjustments, this class of algorithms is capable of efficiently traversing highdimensional parameter spaces and sampling from posterior probability distributions of an unknown quantity such as \( {m_{\upnu }^2} \). From these distributions, any choice of credibility interval \([\theta _1, \theta _2]\), with \(P = \int _{\theta _1}^{\theta _2} p(\varvec{\theta }) \,\mathrm {d}\theta \) being the confidence level, can be constructed.
When considering the distribution of only a subspace of all parameters, one speaks of a marginal posterior distribution. To determine the onedimensional posterior distribution of \( {m_{\upnu }^2} \), the fourdimensional posterior distribution of \(( {m_{\upnu }^2} , E_0, A_\text {sig}, A_\text {bg})\) is marginalized over the three nuisance parameters.
Figure 16 shows the result of a MCMC sampling of the posterior distribution that uses the basic MetropolisHastings [60] algorithm. The underlying model is based on Eq. (44) with its standard four model parameters \(( {m_{\upnu }^2} , E_0, A_\text {sig}, A_\text {bg})\), using flat priors and the constraint \( {m_{\upnu }^2} \ge 0\hbox { eV}^2\). In this representation, the correlations between these parameters can be assessed easily. The correlation matrix of this particular example evaluates to:
\( {m_{\upnu }^2} \)  \(E_0\)  \(A_\text {sig}\)  \(A_\text {bg}\)  

\( {m_{\upnu }^2} \)  1  
\(E_0\)  0.698  1  
\(A_\text {sig}\)  \(\)0.581  \(\)0.953  1  
\(A_\text {bg}\)  0.396  \(\)0.022  0.077  1 
A comparison of Bayesian and Frequentist confidence intervals for various estimates of \( {m_{\upnu }^2} \) is given in Fig. 17. For positive estimates, the different methods yield similar results.
Statistical and systematic uncertainties
Traditionally, the statistical uncertainty \(\sigma _\text {stat}( {m_{\upnu }^2} )\) is identified with the spread of an \( {m_{\upnu }^2} \) estimate caused by the randomness of the observed data (spectrum count rates \(N^\text {obs}_{k}\)), and usually decreases when data are taken (as \(1/\sqrt{N_k}\) or \(1/\sqrt{\varDelta t_k}\)). A systematic uncertainty \(\sigma _\text {syst}( {m_{\upnu }^2} )\), by contrast, represents an uncertainty in the \( {m_{\upnu }^2} \) estimate due to an uncertainty in the spectrum or response model which does not scale with the amount of data taken in general.
Providing a comprehensive review of all systematics of KATRIN – some of which are not adequately quantifiable until final commissioning and characterization of the experimental apparatus – is beyond the scope of this article. Among the major systematic contributors are the final state distribution (Sect. 2.4), the shape of the energy loss function and the inelastic scattering cross section (Sect. 4.2), the sourcegas column density (Sect. 4.4), and highvoltage fluctuations (Sect. 4.11).
The total systematics budget of KATRIN is conservatively evaluated to a maximum value of \(\sigma _\text {syst}( {m_{\upnu }^2} ) \approx 0.017\hbox { eV}^2\) [4]. Accordingly, KATRIN’s setup and configuration are chosen in such a way that the statistical uncertainty, after an envisaged datataking period of five calendar years, reaches \(\sigma _\text {stat}( {m_{\upnu }^2} ) \approx \sigma _\text {syst}( {m_{\upnu }^2} ) \approx 0.017\hbox { eV}^2\), as depicted in Fig. 18. These values are commonly translated into a \(90 \%\,\hbox {C.L.}\) sensitivity of
with the total uncertainty on \( {m_{\upnu }^2} \)
Choice of the analysis energy interval
The optimal choice of the lower spectrum energy threshold for analysis is primarily determined by the ratio of the statistical and systematic uncertainties. Neither one should dominate. With the differential spectrum rising quadratically as the filter energy qU is lowered (for \(E_0E \gg {m_{\upnu }} \)), the statistical uncertainty on the observed number of signal electrons \(\sigma _\text {stat}\left( N_{j}^\text {sig}(qU_k)\right) \) decreases. On the other hand, systematic uncertainties due to energyloss processes or electronic excitations of the daughter molecule increase at lower energies. Assuming the design operational configuration of KATRIN (see Table 2), a lower threshold of \(E_030\hbox { eV}\) will lead to the desired alignment of statistical and total systematic uncertainties (\(\sigma _\text {stat}( {m_{\upnu }^2} ) \approx \sigma _\text {syst}( {m_{\upnu }^2} )\) ). As shown in Fig. 19, the spectrum in this energy range is mainly populated with electrons that have scattered off the source gas at most once.
Measuring time distribution
The distribution of measuring time \(\varDelta t_k\) over a range of retarding potentials is of particular importance. Because the statistical uncertainties of the observed Poissonian rates are given by
more measuring time should be allocated to those regions of the spectrum that are most effective for estimating the parameters of interest and the correlated nuisance parameters.
Figure 20 illustrates the relative spectrum rates with a measuring time distribution in the energy interval of \([E_030\hbox { eV}, E_0+5\hbox { eV}]\). In the case of \( \widehat{m_{\upnu }^2} \), sufficient measuring time must be spent on the region slightly below the endpoint, where the spectral distortion due to a nonzero \( {m_{\upnu }} \) is most prominent. This is also the region with a signaltobackground ratio between 2:1 and 1:1. Accordingly, for scenarios of elevated background, this feature of the measuring time distribution must be adapted and shifted to slightly lower energies.
The measuring time distribution can be further optimized to provide even better statistical leverage on the model parameters fit to the spectrum shape (see Sect. 5.1), reducing the statistical uncertainty \(\sigma _\text {stat}^\text {opti}\left( {m_{\upnu }^2} \right) < 0.015\hbox { eV}^2\) for nominal experimental conditions [62]. An example is shown in Fig. 21, which describes a rather sparse measuring time distribution with only four features, covering distinct retarding energy regions qU. The peak at the lower end of the analysis energy interval (\(\approx 30\hbox { eV}\)) is best suited to measure \(E_0\) and \(A_\text {sig}\) due to the higher absolute spectrum rates. At \(qUE_0\approx 14.0\hbox { eV}\) the correlation between \(E_0\) and \(A_\text {sig}\) is broken. \( {m_{\upnu }^2} \) is measured through the \(\upbeta \) spectrum shape distortion around \(qUE_0\approx 4.5\hbox { eV}\), where about one third of the overall measuring time is invested. \(A_\text {bg}\) is measured using data beyond the endpoint energy \(E_0\), where no \(\upbeta \)decay signal is expected. Note that all four of these parameters are correlated, so the measuring time cannot be shifted arbitrarily between these four regions of retarding energy.
This more focused model allows a lower statistical uncertainty of the measured \( {m_{\upnu }^2} \), however, it bears a higher risk of overseeing unexpected spectrum shape distortions in the neglected regions of the \(\upbeta \)decay spectrum. To safeguard against such spectral deviations from the model and against unexpected systematics, a more uniform distribution, such as the one first shown in Fig. 20, seems more appropriate, at least for the initial datataking period.
Negative \( {m_{\upnu }^2} \) estimates
The true value of \( {m_{\upnu }^2} \) is expected to be very close to \( {m_{\upnu }^2} = 0\hbox { eV}^2\) [63]. Assuming nontachyonic neutrinos, the physical lower limit of the effective neutrino mass squared is given by the neutrino mass eigenstate splittings, measured by neutrino oscillation experiments [11].
In order to allow the estimator \( \widehat{m_{\upnu }^2} \) to follow statistical fluctuations of the data beyond the physical boundary in a \(\chi ^2\) parameter fit, a nonphysical continuous extension of the spectrum model can be introduced. It modifies the differential \(\upbeta \) spectrum in Eq. (12) by
with \(\mu = k \, \sqrt{  {m_{\upnu }^2} }\) for \( {m_{\upnu }^2} < 0\) and \(\mu = 0\) for \( {m_{\upnu }^2} > 0\) (see Fig. 22). The factor \(k \approx 0.72\) is adjusted based on numerical calculations to make the \(\chi ^2( {m_{\upnu }^2} )\) function (and negative loglikelihood respectively) symmetric around its minimum. A similar extrapolation scheme was used in the analysis of the Mainz and Troitsk neutrino mass experiments [5, 6].
For the construction of physical \( {m_{\upnu }} \) confidence intervals such a nonphysical continuation of the model is not required. The unified approach ensures correct Frequentist coverage while allowing to respect parameter boundaries in the fit [57]. In a Bayesian framework the physical constraint is typically realized through a prior \(\pi ( {m_{\upnu }^2} ) = 0\) for \( {m_{\upnu }^2} < 0\hbox { eV}^2\).
Conclusion
Using \(\upbeta \) spectroscopy, the KATRIN experiment aims to probe the absolute neutrino mass scale with an unprecedented subeV sensitivity. Both the statistical and systematic uncertainties of the model parameter of interest, the squared electron neutrino mass \( {m_{\upnu }^2} \), are required to be on the order of \({\mathcal {O}}(0.01\hbox { eV}^2)\). This demands a solid understanding and consistent implementation of the theoretical \(\upbeta \)decay spectrum model and the experimental response function.
With this work, an effort was made to summarize the \(\upbeta \) spectrum calculation with all known theoretical corrections relevant for spectroscopy in the endpoint region. Furthermore, a response function model of the KATRIN experiment was outlined, including its dependencies on sourcegas dynamics and the spectrometer electromagnetic configuration. Finally, the statistical methods applicable to the intended measurement were investigated and concrete examples of their application to the KATRIN neutrino mass measurement were given.
In Sect. 4.12, an overview of the impact of various model components on the measured squared neutrino mass was given. The purpose is to provide a quantitative measure of their relative importance, indicating components that are negligible in the neutrino mass analysis. Among the most important effects are the radial dependencies of analyzing magnetic field and retarding potential, energy loss of signal electrons due to cyclotron motion and the Doppler broadening of the electron \(\upbeta \)spectrum due to the source gas thermal motion.
The calculations presented here are implemented as part of a common C++ simulation and analysis software framework called Kasper, which is used by the KATRIN collaboration to investigate the effect of model corrections and possible systematics, and to optimize the operational parameters of the setup for the neutrino mass measurement [39, 62, 64,65,66].
During the ongoing commissioning measurement campaign of the KATRIN experiment, many aspects of the current response model will be verified with experimental data. The results of recent investigations are described in [38, 49, 67]. This thorough characterization of the complex setup will allow a quantitative evaluation of the systematic effects in the neutrino mass analysis at KATRIN.
Data Availability Statement
This manuscript has no associated data or the data will not be deposited. [Authors’ comment: The authors confirm that the data supporting the findings of this study are available within the article and cited references.]
Notes
If we denote the fraction of all hydrogen isotopologues X by c(X) with \(\sum _X c(X) = 1\), then the tritium purity is given by \(\epsilon _\mathrm {T}= c(\mathrm {T_2}) + c(\mathrm {DT}) / 2 + c(\mathrm {HT}) / 2\).
The gas is then injected into the source beam tube and rapidly cooled down to 30 K. Because the gas spends only a short time (\(\lesssim 1.5\hbox { s}\)) at this temperature, the rotational states cannot equilibrate again.
At a temperature of 30 K and a magnetic field strength of 3.6 T, the polarization of the tritium nuclei can be neglected.
As investigated in [39], the direct angular change of \(\upbeta \)electrons due to elastic and inelastic scattering has only negligible effect on the response function shape.
It is required that \(E_\parallel \) reaches its global minimum in the analyzing plane, which is achieved by optimizing the electromagnetic conditions in the spectrometer. See [37] for details.
The \(\upbeta \)electrons are guided from source to detector by magnetic field lines, so each detector pixel maps a certain stack of source segments.
In the nonrelativistic case, the power loss due to cyclotron radiation amounts to \({\dot{E}}_\perp = 0.39 /\hbox {s T}^2 \cdot E_\perp \cdot B^2\).
Deviations from unity arise mainly from incomplete knowledge of the tritium column density and the detector efficiency (see Eq. (41)).
References
Y. Fukuda et al., Phys. Rev. Lett. 81, 1562 (1998). https://doi.org/10.1103/PhysRevLett.81.1562
Q.R. Ahmad et al., Phys. Rev. Lett. 89, 011301 (2002). https://doi.org/10.1103/PhysRevLett.89.011301
S. Abe et al., Phys. Rev. Lett. 100, 221803 (2008). https://doi.org/10.1103/PhysRevLett.100.221803
KATRIN collaboration, KATRIN design report. FZKA scientific report 7090 (2005). http://bibliothek.fzk.de/zb/berichte/FZKA7090.pdf
C. Kraus, B. Bornschein et al., Eur. Phys. J. C Part. Fields 40(4), 447 (2005). https://doi.org/10.1140/epjc/s2005021397
V.N. Aseev, A.I. Belesev et al., Phys. Rev. D 84, 112003 (2011). https://doi.org/10.1103/PhysRevD.84.112003
V.N. Aseev et al., Phys. Atom. Nuclei 75(4), 464 (2012). https://doi.org/10.1134/S1063778812030027
K.A. Olive et al., Chin. Phys. C 38, 090001 (2014). https://doi.org/10.1088/16741137/38/9/090001
E.W. Otten, C. Weinheimer, Rep. Prog. Phys. 71(8), 086201 (2008). https://doi.org/10.1088/00344885/71/8/086201
Y. Akulov, B. Mamyrin, Phys. Lett. B 610(1), 45 (2005). https://doi.org/10.1016/j.physletb.2005.01.094
R.G.H. Robertson, D.A. Knapp, Ann. Rev. Nucl. Part. Sci. 38(1), 185 (1988). https://doi.org/10.1146/annurev.ns.38.120188.001153
R.E. Shrock, Phys. Lett. B 96(1), 159 (1980). https://doi.org/10.1016/03702693(80)90235X
Y. Farzan, A.Y. Smirnov, Phys. Lett. B 557(3), 224 (2003). https://doi.org/10.1016/S03702693(03)002077
E.G. Myers, A. Wagner et al., Phys. Rev. Lett. 114, 013003 (2015). https://doi.org/10.1103/PhysRevLett.114.013003
N. Doss, J. Tennyson et al., Phys. Rev. C 73, 025502 (2006). https://doi.org/10.1103/PhysRevC.73.025502
N. Doss, J. Tennyson, J. Phys. B 41(12), 125701+ (2008). https://doi.org/10.1088/09534075/41/12/125701
S. Jonsell, A. Saenz, P. Froelich, Phys. Rev. C 60, 034601 (1999). https://doi.org/10.1103/PhysRevC.60.034601
L.I. Bodine, D.S. Parno, R.G.H. Robertson, Phys. Rev. C 91(3), 035505 (2015). https://doi.org/10.1103/PhysRevC.91.035505
S. Fischer, M. Sturm et al., Nucl. Phys. B Proc. Suppl. 229–232, 492 (2012). https://doi.org/10.1016/j.nuclphysbps.2012.09.129
A. Saenz, S. Jonsell, P. Froelich, Phys. Rev. Lett. 84, 242 (2000). https://doi.org/10.1103/PhysRevLett.84.242
P.C. Souers, Hydrogen Properties for Fusion Energy (University of California Press, Berkeley, 1986)
F. Simkovic, R. Dvornicky, A. Faessler, Phys. Rev. C 77, 055502 (2008). https://doi.org/10.1103/PhysRevC.77.055502
S.S. Masood, S. Nasri et al., Phys. Rev. C 76, 045501 (2007). https://doi.org/10.1103/PhysRevC.76.045501
C.E. Wu, W.W. Repko, Phys. Rev. C 27, 1754 (1983). https://doi.org/10.1103/PhysRevC.27.1754
G. Beamson, H.Q. Porter, D.W. Turner, J. Phys. E Sci. Instrum. 13(1), 64 (1980). https://doi.org/10.1088/00223735/13/1/018
V.M. Lobashev, P.E. Spivak, Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrom. Detect. Assoc. Equip. 240(2), 305 (1985). https://doi.org/10.1016/01689002(85)906400
A. Picard, H. Backe et al., Nucl. Instrum. Methods Phys. Res. Sect. B Beam Interact. Mater. Atoms 63(3), 345 (1992). https://doi.org/10.1016/0168583X(92)95119C
M. Sturm, M. Schlösser et al., Laser Phys. 20(2), 493 (2010). https://doi.org/10.1134/S1054660X10030163
M. Schlösser, S. Rupp et al., J. Mol. Struct. 1044, 61 (2013). https://doi.org/10.1016/j.molstruc.2012.11.022
F. Priester, M. Sturm, B. Bornschein, Vacuum 116, 42 (2015). https://doi.org/10.1016/j.vacuum.2015.02.030
W. Gil, J. Bonn et al., IEEE Trans. Appl. Superconduct. 20(3), 316 (2010). https://doi.org/10.1109/TASC.2009.2038581
S. Lukić, B. Bornschein et al., Vacuum 86(8), 1126 (2012). https://doi.org/10.1016/j.vacuum.2011.10.017
M. Arenz et al., JINST 13(08), T08005 (2018). https://doi.org/10.1088/17480221/13/08/T08005
M. Prall, P. Renschler et al., New J. Phys. 14(7), 073054 (2012). https://doi.org/10.1088/13672630/14/7/073054
M. Arenz, M. Babutzka et al., J. Instrum. 11, P04011 (2016). https://doi.org/10.1088/17480221/11/04/P04011. arXiv:1603.01014
J. Amsbaugh, J. Barrett et al., Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrom. Detect. Assoc. Equip. 778, 40 (2015). https://doi.org/10.1016/j.nima.2014.12.116
F. Glück, G. Drexlin et al., New J. Phys. 15(8), 083025 (2013). https://doi.org/10.1088/13672630/15/8/083025
M. Erhard, J. Behrens et al., J. Instrum. 13(02), P02003 (2018). https://doi.org/10.1088/17480221/13/02/P02003
Groh, S.: Modeling of the response function and measurement of transmission properties of the KATRIN experiment. Ph.D. thesis, Karlsruher Institut für Technologie (KIT) (2015). http://nbnresolving.org/urn:nbn:de:swb:90465464
V.N. Aseev, A.I. Belesev et al., Eur. Phys. J. D 10, 39 (2000). https://doi.org/10.1007/s100530050525
D. Abdurashitov, A. Belesev et al., Phys. Part. Nuclei Lett. 14(6), 892 (2017). https://doi.org/10.1134/S1547477117060024
V. Hannen, I. Heese et al., Astropart. Phys. 89, 30 (2017). https://doi.org/10.1016/j.astropartphys.2017.01.010
J. Geiger, Z. Phys. 181(4), 413 (1964). https://doi.org/10.1007/BF01380873
J.W. Liu, Phys. Rev. A 35, 591 (1987). https://doi.org/10.1103/PhysRevA.35.591
W. Kołos, H.J. Monkhorst, K. Szalewicz, J. Chem. Phys. 77(3), 1323 (1982). https://doi.org/10.1063/1.443955
M.G. Erhard, Influence of the magnetic field on the transmission characteristics and neutrino mass systematic of the KATRIN experiment. Ph.D. thesis, Karlsruher Institut für Technologie (2016). http://nbnresolving.org/urn:nbn:de:swb:90650034
L. Kuckert, The windowless gaseous tritium source of the KATRIN experiment—characterisation of gas dynamical and plasma properties. Ph.D. thesis, Karlsruher Institut für Technologie (KIT) (2016). http://nbnresolving.org/urn:nbn:de:swb:90650776
L. Kuckert, F. Heizmann, et al. (2018). arXiv:1805.05313
M. Arenz, et al. 78(5), 368 (2018). https://doi.org/10.1140/epjc/s100520185832y
D. Furse, S. Groh et al., New J. Phys. 19(5), 053012 (2017). https://doi.org/10.1088/13672630/aa6950
M. Kraus, Energyscale systematics at the KATRIN main spectrometer. Ph.D. thesis, Karlsruher Institut für Technologie (2016). http://nbnresolving.org/urn:nbn:de:swb:90544471
M. Slezák, Monitoring of the energy scale in the KATRIN neutrino experiment. Ph.D. thesis, Charles University in Prague (2015)
R.L. Plackett, Int. Stat. Rev. 51(1), 59 (1983). https://doi.org/10.2307/1402731
E. Holzschuh, Rep. Prog. Phys. 55(7), 1035 (1992). https://doi.org/10.1088/00344885/55/7/004
E.G. Myers, A. Wagner et al., Phys. Rev. Lett. 114, 013003 (2015). https://doi.org/10.1103/PhysRevLett.114.013003
J. Neyman, Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Sci. 236(767), 333 (1937). http://www.jstor.org/stable/91337
G.J. Feldman, R.D. Cousins, Phys. Rev. D 57, 3873 (1998). https://doi.org/10.1103/PhysRevD.57.3873
W.A. Rolke, A.M. López, J. Conrad, Nucl. Instrum. Methods A 551(2–3), 493 (2005). https://doi.org/10.1016/j.nima.2005.05.068
C.P. Robert, G. Casella, Monte Carlo Statistical Methods (Springer Texts in Statistics) (Springer, Secaucus, 2005)
N. Metropolis, A.W. Rosenbluth et al., J. Chem. Phys. 21(6), 1087 (1953). https://doi.org/10.1063/1.1699114
G. Cowan, K. Cranmer et al., Eur. Phys. J. C 71, 1554 (2011). https://doi.org/10.1140/epjc/s1005201115540
M. Kleesiek, A dataanalysis and sensitivityoptimization framework for the KATRIN experiment. Ph.D. thesis, Karlsruher Institut für Technologie (KIT) (2014). http://nbnresolving.org/urn:nbn:de:swb:90433013
G. Drexlin, V. Hannen et al., Adv. High Energy Phys. 2013, (2013). https://doi.org/10.1155/2013/293986
T. Corona, Methodology and application of high performance electrostatic field simulation in the KATRIN experiment. Ph.D. thesis, University of North Carolina at Chapel Hill (2014). https://cdr.lib.unc.edu/record/uuid:6f44a9c2f053404ab726b960d5772619
D. Furse, Techniques for direct neutrino mass measurement utilizing tritium \(\beta \)decay. Ph.D. thesis, Massachusetts Institute of Technology (2015). http://hdl.handle.net/1721.1/99313
J.D. Behrens, Design and commissioning of a monoenergetic photoelectron source and active background reduction by magnetic pulse at the KATRIN spectrometers. Ph.D. thesis, Westfälische WilhelmsUniversität Münster (2016). http://www.katrin.kit.edu/publikationen/phd_behrens.pdf
M. Arenz, et al., 13(04), P04020 (2018). https://doi.org/10.1088/17480221/13/04/P04020
D.H. Wilkinson, Nucl. Phys. A 526, 131 (1991). https://doi.org/10.1016/03759474(91)90301L
L. Elton, Nucl. Phys. 5, 173 (1958). https://doi.org/10.1016/00295582(58)900166
J.J. Simpson, Phys. Rev. D 23, 649 (1981). https://doi.org/10.1103/PhysRevD.23.649
W. Repko, C.E. Wu, Phys. Rev. C 28, 2433 (1983). https://doi.org/10.1103/physrevc.28.2433
H. Behrens, W. Bühring, Electron Radial Wave Functions and Nuclear BetaDecay, The International Series of Monographs on Physics Series (Clarendon Press, New York, 1982)
C.K. Hargrove, D.J. Paterson, I.S. Batkin, Phys. Rev. C 60, 034608 (1999). https://doi.org/10.1103/PhysRevC.60.034608
W.C. Haxton, Phys. Rev. Lett. 55, 807 (1985). https://doi.org/10.1103/PhysRevLett.55.807
S.M. Bilen’kii, R.M. Ryndin et al., Sov. Phys. JETP 10, p1241 (1960)
D. Wilkinson, Nucl. Instrum. Methods A 290(2–3), 509 (1990). https://doi.org/10.1016/01689002(90)90570V
D. Wilkinson, Nucl. Phys. A 377(2), 474 (1982). https://doi.org/10.1016/03759474(82)900513
Acknowledgements
We acknowledge the support of Helmholtz Association (HGF), Ministry for Education and Research BMBF (05A17PM3, 05A17VK2 and 05A17WO3), Helmholtz Alliance for Astroparticle Physics (HAP), and Helmholtz Young Investigator Group (VHNG1055) in Germany; and the Department of Energy through Grants DEAC0205CH11231 and DESC0011091 in the United States. We thank T. Lasserre, V. Sibille, N. Trost and D. Vénos for contributive discussions, and D. Parno for her careful review and suggestions.
Author information
Affiliations
Corresponding author
Additional information
M. Kleesiek: né M. Haag.
Appendix
Appendix
Theoretical corrections to the \(\upbeta \) spectrum shape
The calculation of many theoretical corrections follows the comprehensive summary of [68]. Consequently, a similar nomenclature was chosen in this article.
Nomenclature
Natural units (\(\hbar = c = 1\)) are used unless stated otherwise.
The nuclear radius \(R_\text {n}\) of \(\mathrm {^3He}\) is given by the Elton formula [69]. The value for \(\lambda _t\) is derived from the halflife of tritium by Simkovic et al. [22].
Fermi function
A fully relativistic description of the Fermi function is given by
with the complex Gamma function \(\varGamma \). A commonly used approximate, yet sufficiently accurate for our purpose, expression for Eq. (A.1) is [70]
with F(Z, W) denoting the classical Fermi function (equation (6)).
Radiative corrections due to virtual and real photons
Radiative corrections, denoted by the multiplicative factor G, are implemented according to equation 20 of [71]:
with
Screening by the Coulomb field of the daughter nucleus
The calculation of the screening correction factor S follows [72]:
where
with the nuclear screening potential \(V_0 = 76(10)\hbox { eV}\) of the finalstate orbital electron cloud of the daughter \(\mathrm {^3He}\) atom after \(\upbeta \)decay, as determined by Hargrove et al. [73].
Exchange with the orbital 1s electron
The effect of an orbital electron exchange I is calculated according to Haxton [74]. Considering only the ground state of the daughter \(\mathrm {^3He^+}\) ion:
where
with \(\tau = 2 \alpha / p\).
Recoil effects
In the relativistic description of the threebody phase space, the spectral change due to recoil effects, including those from weak magnetism and \(VA\) interference, is reflected by the correction factor R [75]:
where
with \(\mu = 5.107\) being the difference between the magnetic moments of helion and triton.
Finite nuclear extension
The two correction factors L and C, considering the finite structure of the daughter nucleus, are given by Wilkinson [76]. L accounts for the scaling of the Coulomb field within the nucleus:
The convolution of the electron and neutrino wave functions with the nucleonic wave function throughout the nuclear volume leads to C:
with
Recoiling Coulomb field
The correction factor Q, describing the recoil of the charge distribution by the emitted lepton, is calculated according to Wilkinson [77]:
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Funded by SCOAP^{3}
About this article
Cite this article
Kleesiek, M., Behrens, J., Drexlin, G. et al. \(\upbeta \)Decay spectrum, response function and statistical model for neutrino mass measurements with the KATRIN experiment. Eur. Phys. J. C 79, 204 (2019). https://doi.org/10.1140/epjc/s1005201966867
Received:
Accepted:
Published:
DOI: https://doi.org/10.1140/epjc/s1005201966867