1 Introduction

The double-\(\beta \) decay is a nuclear process that changes the charge of a nucleus by two units through the simultaneous \(\beta \)-decay of two constituent neutrons to protons. The two-neutrino double-\(\beta \) decay (\(2\nu \beta \beta \)) is a rare second-order weak interaction process occurring with emission of two electrons and two antineutrinos. It was observed for several nuclear isotopes [1, 2].

The community’s interest in the double-\(\beta \) decay is particularly motivated by the search for its hypothetical neutrinoless mode (\(0\nu \beta \beta \)) [3]. This process violates the lepton number conservation and is only possible if the neutrino has mass and is a Majorana particle [4], i.e. \(\nu \equiv {\bar{\nu }}\). The discovery of \(0\nu \beta \beta \) would indicate the physics beyond the Standard Model (BSM).

The rates of two-neutrino and neutrinoless double-\(\beta \) decay may be expressed as

$$\begin{aligned} 1/T^{2\nu }_{1/2}&= G^{2\nu } {g}^4_A |M^{2\nu }|^2~, \end{aligned}$$
(1)
$$\begin{aligned} 1/T^{0\nu }_{1/2}&= G^{0\nu } {g}^4_A \vert M^{0\nu } \vert ^{2} \langle \eta \rangle ^{2} \,, \end{aligned}$$
(2)

where \(G^{2\nu ,0\nu }\) are the phase space factors, \(g_A\) is the axial vector coupling constant, \(M^{2\nu ,0\nu }\) are the nuclear matrix elements (NMEs) for the corresponding decay modes, and \(\langle \eta \rangle \) is a parameter of the underlying BSM physics model (in the case of the commonly considered mass mechanism of the \(0\nu \beta \beta \) decay, the exchange of a light Majorana neutrino, \(\langle \eta \rangle \) is the effective neutrino mass). The phase space factors can be accurately calculated while the model-dependent NME calculations have a substantial theoretical uncertainty. The measurement of the \(2\nu \beta \beta \) decay half-life provides valuable information for nuclear structure models used in NME calculations.

The double-\(\beta \) decay can proceed through transitions either to the ground state or to excited states of the daughter nucleus. The latter occurs at a lower rate because of its smaller transition energy \(Q_{\beta \beta }\) leading to the correspondingly suppressed phase space factor. Nevertheless, the measurement of the \(2\nu \beta \beta \) decay to excited states provides supplementary information for nuclear models. Additionally, in the case of the \(0\nu \beta \beta \) decay discovery, the ratio of half-lives for transitions to the 0\(^+\) first excited state and the ground state may allow the dominant decay mechanism to be determined [5]. Information on the results of experiments on the \(\beta \beta \)-decay to excited states of daughter nuclei can be found in [6].

The isotope \(^{150}\)Nd is one of the best candidates for neutrinoless \(\beta \beta \)-decay searches because of its high transition energy \(Q_{\beta \beta }\) = 3371 keV and highest phase space factor [7]. However, a modest isotopic abundance of 5.638(28)% and difficulties in isotopic enrichment limit its use in large-scale experiments.

The decay scheme of \(^{150}\)Nd to the 2\(^+_1\) and 0\(^+_1\) excited states is shown in Fig. 1. The half-life for the transition to the first 0\(^+\) excited state was first measured in 2004 [8]. These data were subsequently re-analysed, with the final result published in [9]. The measurements of this decay were obtained with \(\gamma \)-ray spectrometry using high-purity germanium detectors [9,10,11]; none of the previous measurements detected a signal with a 5\(\sigma \) statistical significance. For other excited states, only lower limits on the half-life were established; the best available limit for the transition to the \(2^+_1\) state set at 90% confidence level (C.L.) is \(T_{1/2}^{\beta \beta }(2^+_1) > 2.2\times 10^{20}\,\hbox {year}\) [9].

The most precise measurement for the \(\beta \beta \)-decay of \(^{150}\)Nd to the ground state was performed by NEMO-3 [12]:

$$\begin{aligned} T_{1/2}^{2\nu \beta \beta }(0^+_{\textsf {g.s.}}) = [9.34\pm 0.22 \,(\hbox {stat}) ^{+0.62}_{-0.60}(\hbox {syst})] \times 10^{18}\,\hbox {year}. \end{aligned}$$
(3)

It is based on the data recorded for 5.25 years with 36.6 g of \(^{150}\)Nd. The same data set is used in this analysis.

Fig. 1
figure 1

Scheme of the \(^{150}\)Nd \(\beta \beta \)-decay to the lowest excited states of \(^{150}\)Sm

2 NEMO-3 detector

The NEMO-3 experiment in the Modane Underground Laboratory (LSM) took data from February 2003 to January 2011. The NEMO-3 detector, designed to search for the \(0\nu \beta \beta \) decay, uses both a tracking device and a calorimeter, which enables the direct detection of electrons, positrons, photons, and \(\alpha \)-particles.

A schematic view of the NEMO-3 detector is shown in Fig. 2. The detector was a hollow cylinder with a diameter of 5 m and a height of 3 m and was composed of 20 equal sectors. These hosted thin source foils of 7 different enriched \(\beta \beta \)-decaying isotopes (\(^{100}\)Mo, \(^{82}\)Se, \(^{116}\)Cd, \(^{130}\)Te, \(^{150}\)Nd, \(^{96}\)Zr, and \(^{48}\)Ca) with a total mass of about 10 kg.

Fig. 2
figure 2

Schematic view of the NEMO-3 detector with the source foils (1), scintillators (2), photomultipliers (3), and wire chamber (4)

The source foils were suspended vertically between two concentric cylindrical tracker volumes, parallel to the wires of the tracking detector. The tracking detector was composed of 6180 open octagonal drift cells arranged in 18 concentric layers, with 9 layers in each of the two volumes. The tracker was filled with a gas mixture of helium (94.9%), ethyl alcohol (4%), argon (1%), and water vapour (0.1%) at 7 mbar above atmospheric pressure. The drift cells operating in the Geiger mode enabled three-dimensional measurements of trajectories and decay vertices of charged particles. The average Geiger cell resolution was 0.5 mm in the horizontal plane and 8 mm in the vertical direction (parallel to the wires).

The tracking chamber was surrounded by a calorimeter composed of 1940 plastic scintillator blocks coupled to low-radioactivity 3-inch and 5-inch photomultiplier tubes (PMTs). The calorimeter provided both time and energy measurements. The energy resolution of the calorimeter was \(\sigma \) = (5.8 – 7.2)%, and the time resolution was \(\sigma \) = 250 ps for 1-MeV electrons.

A vertical magnetic field of 25 Gauss inside the wire chamber was provided by a solenoidal coil. The detector was surrounded by the passive shielding consisting of 19-cm-thick iron plates to suppress the external \(\gamma \)-ray flux and also of borated water, paraffin, and wood to thermalize and absorb environmental neutrons. The experimental hall is located at a depth of 4800 m.w.e., to reduce the cosmic-ray flux.

The \(^{150}\)Nd foil was manufactured using Nd\(_2\)O\(_3\) powder provided by the Institute for Nuclear Research of RAS in Moscow. Neodymium was enriched by electromagnetic separation to (91.0±0.5)% of the isotope \(^{150}\)Nd and chemically purified. A total of 46.64 g of Nd\(_2\)O\(_3\) powder mixed with a concentration of 8% PVA glue was uniformly distributed between two layers of mylar to produce a composite foil with a total mass of 56.68 g. The foil was 2484 mm long and 65 mm wide. The total mass of the isotope \(^{150}\)Nd in the foil was 36.6±0.2 g [13]. The \(^{150}\)Nd composite foil was located in Sector 5 of the NEMO-3 detector between a foil of \(^{100}\)Mo and a foil containing \(^{96}\)Zr and \(^{48}\)Ca.

A more detailed description of the NEMO-3 detector, its calibration and performance can be found in [13, 14].

3 Analysis and results

The \(\beta \beta \)-decay of \(^{150}\)Nd to the lowest (2\(^+_1\) and 0\(^+_1\)) excited states of \(^{150}\)Sm has been investigated. The contribution from the higher excited states was neglected. According to the decay scheme in Fig. 1, two electrons from the \(\beta \beta \)-decay are accompanied by one \(\gamma \) in the case of the transition to the 2\(^+_1\) excited state and by two photons in the transition to the 0\(^+_1\) excited state. We therefore select for this analysis two-electron one-\(\gamma \) (\(ee\gamma \)) and two-electron two-\(\gamma \) (\(ee\gamma \gamma \)) event topologies. After the event selection, the \(\beta \beta \)-decay signal is identified by an excess in the data over the expected background.

Both a measurement of the two-neutrino \(\beta \beta \)-decay and a search for the neutrinoless \(\beta \beta \)-decay to the 0\(^+_1\) excited state are carried out in the \(ee\gamma \gamma \) and \(ee\gamma \) channels. The \(\beta \beta \)-decays to the 2\(^+_1\) excited state are explored in the \(ee\gamma \) channel.

A multivariate analysis improves the separation of the signal from the background. To this end the Boosted Decision Tree (BDT) method is used. The analysis employs a BDT algorithm with adaptive boosting, part of the ROOT [15] TMVA package [16].

Where no evidence of a signal in the data is found, a limit on the corresponding decay half-life is set. The 90% C.L. limit is calculated using the CL\(_s\) method employing the modified frequentist approach [17,18,19].

3.1 Event selection

In this analysis, the \(ee\gamma \gamma \) and \(ee\gamma \) event topologies are used. Events are selected by requiring two reconstructed electron tracks coming from the source foil, with each depositing in a separate scintillator block an energy greater than 150 keV. Extrapolating each track to the source foil gives the position of its decay vertex, and extrapolating to the calorimeter associates the track with the scintillator block of an optical module for energy and time measurement. A scintillator hit associated with a track must be isolated, i.e. no hits should be found in neighboring scintillator blocks.

Each of the two electron tracks must have a length greater than 50 cm and originate from a common vertex in the \(^{150}\)Nd source foil: the separation between the two individually reconstructed track vertices is required to be less than 4 cm in the horizontal plane and less than 8 cm in the vertical direction. An event is excluded if its vertex is found in one of the regions of the enhanced activity in the foil corresponding to the localized contamination from \(^{234m}\)Pa and \(^{207}\)Bi (hot spots). The locations of the hot spots, which amount to 7% of the \(^{150}\)Nd foil area, were determined in [12].

To ensure that an event corresponded to the simultaneous emission of two electrons from a common vertex, the corresponding time-of-flight (TOF) probability is required to be higher than 5%. The TOF probability is calculated using energy and time measurements from the calorimeter and the distances travelled by particles in the event; see [13, 14] for details.

A \(\gamma \)-ray is identified as either a single calorimeter hit or a cluster of neighbouring hits that are not associated with any track. A minimum threshold of 100 keV for the energy deposited in each of these calorimeter blocks is required. It is also required that no prompt Geiger hits are detected within 20 cm of any scintillator block attributed to a \(\gamma \)-ray. Events are rejected if the TOF probability exceeds 1% for the hypothesis that the event originates from an external \(\gamma \)-ray. The probability for the hypothesis that the photon(s) originated from the event vertex simultaneously with two electrons is required to be higher than 5%.

An event is rejected if it contains a recognized delayed alpha-particle track, as described in [14], to reduce the background from \(^{214}\)Bi decays.

As shown in Table 1, a total of 142 \(ee\gamma \gamma \) and 571 \(ee\gamma \) events are selected from the full data set.

Table 1 Expected number of events from different sources of the background with statistical and systematic uncertainties and the number of the observed events in the \(ee\gamma \gamma \) and \(ee\gamma \) channels after the event selection

3.2 Background model

The main source of background events is trace amounts of naturally occurring radioactive isotopes that come from the \(^{238}\)U and \(^{232}\)Th radioactive series. The most important of them are (\(\beta ,\gamma \))-emitting isotopes with high \(Q_\beta \) values, such as \(^{208}\)Tl (\(Q_\beta \) = 4.99 MeV) and \(^{214}\)Bi (\(Q_\beta \) = 3.27 MeV). According to their origin with respect to the source foil, the background events are classified as internal or external ones.

Fig. 3
figure 3

Distributions of the measured quantities for the \(ee\gamma \gamma \) events from the \(^{150}\)Nd foil after the preliminary event selection: energy sum of two electrons \(E_{2e}\), minimal electron energy \(E_{e}^{\text {min}}\), minimal \(\gamma \) energy \(E_{\gamma }^{\text {min}}\), maximal electron energy \(E_{e}^{\text {max}}\), maximal \(\gamma \) energy \(E_{\gamma }^{\text {max}}\), total measured energy \(E_\text {TOT}\), cosine of the angle between two electrons \(\cos (ee)\), between two photons \(\cos (\gamma \gamma )\), and between electron and \(\gamma \) \(\cos (e\gamma )\) for all \(e\gamma \) combinations. Data are compared to the MC prediction with the resulting number of \(0^{+}_{1}\) signal events obtained by background subtraction

The largest background contribution comes from the internal contamination of the source foil. The decay of a \(\beta \)-emitting isotope inside the foil can mimic the \(\beta \beta \)-decay signal through several different mechanisms, such as a single \(\beta \)-decay combined with Møller scattering or a single \(\beta \)-decay to an excited state of the daughter nucleus followed by the emission of a conversion electron or a \(\gamma \)-ray that undergoes Compton scattering in the foil. From these mechanisms, additional \(\gamma \)-rays could be produced by bremsstrahlung or from a decay to an excited state. In addition to the radioactive impurities, a decay in a neighbouring NEMO-3 source foil can be misinterpreted to have its vertex in the \(^{150}\)Nd foil. The \(^{150}\)Nd \(\beta \beta \)-decay to the ground state also contributes to the background for the excited-state measurement; two electrons are produced in the decay, and one or two \(\gamma \)-rays could be emitted via bremsstrahlung.

The external background is there due to the radioactivity outside of the source foil. Radioactive decays within the detector components (mainly PMT glass), the shielding and rock, surrounding the laboratory, generate the external \(\gamma \)-ray flux. \(\gamma \)-ray interactions with the source foil can cause electron-positron pair production, a Compton interaction followed by Møller scattering, or double Compton scattering. In the case of electron-positron pair production, two photons can be produced through annihilation of the positron.

A subset of the external background is induced by radon. Radon is a highly diffusive gas and is outgassed into the air from the rock walls of the LSM laboratory. It is present in the tracker volume due to diffusion from laboratory air through detector seals and emanation from detector materials. The decay of radon progenies (mainly \(^{214}\)Bi) near the source foil can produce signal-like events similar to internal background decays.

Details of the background model and measured values of activities that are used in this analysis are provided in [12]. The DECAY0 event generator [20] is used to simulate the signal and backgrounds, and particles are tracked through a detailed GEANT3-based detector simulation [21]. Both the data and Monte Carlo (MC) events are processed by the same reconstruction and selection algorithm. The number of the expected background events with the \(ee\gamma \gamma \) and \(ee\gamma \) topologies is given in Table 1.

3.3 Measurement of \(\varvec{2\nu \beta \beta }\) decay to \(\varvec{0^+_1}\) excited state

An excess in the data over the total expected background is observed both in the \(ee\gamma \gamma \) and \(ee\gamma \) channels (see Table 1) and can be attributed to the signal of the \(\beta \beta \)-decay to the excited states of the daughter nucleus \(^{150}\)Sm.

3.3.1 Use of \(\varvec{ee\gamma \gamma }\) events

The \(ee\gamma \gamma \) event topology is the best one for measuring the transition to the 0\(^+_1\) excited state when both electrons, produced in the \(\beta \beta \)-decay, and both photons from deexcitation of \(^{150}\)Sm are detected.

Distributions of measured quantities for the selected \(ee\gamma \gamma \) events are demonstrated in Fig. 3. The number of the 0\(^+_1\) signal events \(S = N - B = 56.6 \pm 11.9\) is obtained by subtracting the expected background from the number of events observed. This corresponds to the signal-to-background ratio

\(S/B = 0.66\) and to the statistical signal significance \(N{\sigma } = S/\sqrt{S+B} = 4.8\). The 0\(^+_1\) signal efficiency is \(\epsilon = 0.87 \%\). This corresponds to the following half-life estimation:

$$\begin{aligned} T_{1/2}^{2\nu \beta \beta }(0^{+}_{1}) = \left[ 8.18 ^{+2.18}_{-1.42}(\text {stat})\right] \times 10^{19}~\hbox {year}. \end{aligned}$$
(4)

In order to suppress the background and maximize the signal significance, the event classification employed the BDT method. Using MC of the 0\(^+_1\) signal and the background, the BDT training is performed on the set of observables shown in Fig. 3, with the total measured energy \(E_{\text {TOT}}\) and the maximal \(\gamma \) energy \(E_{\gamma }^{\text {max}}\) being the most important variables. After training, both the data and MC were processed by the BDT algorithm which assigned a BDT score to each event to aid discrimination of the signal from the background. The BDT score is a continuous variable with lower values for more background-like events and higher values for more signal-like events. The resulting BDT score distribution for the signal of the \(^{150}\)Nd \(2\nu \beta \beta \) decay to the 0\(^+_1\) excited state in the \(ee\gamma \gamma \) channel is presented in Fig. 4. The vertical dashed line in this figure denotes the optimal cut on the BDT score to maximize the signal significance.

After rejecting the events with lower BDT score values, we are left with 53 data events and a total expected background of 13.9 events, see Table 2. This requirement suppresses the background by a factor of 6.1 and reduces the signal efficiency by a factor of 0.88 to \(\epsilon \) = 0.76%. After background subtraction, 39.1 events attributed to the signal remain. This provides the signal-to-background ratio \(S/B = 2.8\) and the signal statistical significance \(N{\sigma } = S/\sqrt{S+B} = 5.4\). The corresponding half-life is estimated to be

$$\begin{aligned} T_{1/2}^{2\nu \beta \beta }(0^+_1) = \left[ 1.04 ^{+0.24}_{-0.16} \,\left( \hbox {stat}\right) ^{+0.12}_{-0.11}\,\left( \hbox {syst}\right) \right] \times 10^{20}~\hbox {year}. \end{aligned}$$
(5)

This half-life value statistically agrees within 1\(\sigma \) with the result in Eq. (4) obtained after preliminary event selection, but is more precise.

Fig. 4
figure 4

BDT score distribution for the \(ee\gamma \gamma \) events used for the signal of the \(^{150}\)Nd \(2\nu \beta \beta \) decay to the 0\(^+_1\) excited state. The vertical dashed line denotes the optimal cut position maximizing the signal significance

Table 2 Number of the expected events from different sources of the background with statistical and systematic uncertainties and the number of the observed events in the \(ee\gamma \gamma \) and \(ee\gamma \) channels after the BDT cut
Fig. 5
figure 5

Distributions of the two-electron two-\(\gamma \) events from the \(^{150}\)Nd foil after the cut on BDT score: energy sum of two electrons \(E_{2e}\), minimal electron energy \(E_{e}^{\text {min}}\), minimal \(\gamma \) energy \(E_{\gamma }^{\text {min}}\), maximal electron energy \(E_{e}^{\text {max}}\), maximal \(\gamma \) energy \(E_{\gamma }^{\text {max}}\), total measured energy \(E_\text {TOT}\), cosine of the angle between two electrons \(\cos (ee)\), between two photons \(\cos (\gamma \gamma )\), and between electron and \(\gamma \) \(\cos (e\gamma )\) for all \(e\gamma \) combinations. The 0\(^+_1\) signal contribution is defined by performing background subtraction

The distributions of the measured kinematic variables after the cut on the BDT score are shown in Fig. 5. The data show good agreement with MC for the measured quantities. The Kolmogorov-Smirnov probability values obtained for them are in the range from 10% to 98%. In particular, this probability is equal to 29% for the \(\cos (\gamma \gamma \)) distribution representing the \(\gamma \gamma \) angular correlation. This correlation is measured for this decay for the first time and is found to agree well with \(1-3\text {cos}^{2}\theta +4\text {cos}^4\theta \) behaviour characterizing the \(0^{+} \rightarrow 2^{+} \rightarrow 0^{+}\) cascade [22].

Fig. 6
figure 6

Distributions for the two-electron one-\(\gamma \) events from the \(^{150}\)Nd foil after the preliminary selection: energy sum of two electrons \(E_{2e}\), \(\gamma \) energy \(E_{\gamma }\), minimal and maximal electron energy \(E_{e}^{\text {min}}\), \(E_{e}^{\text {max}}\), total measured energy \(E_\text {TOT}\), cosine of the angle between electron, and \(\gamma \) \(\cos (e\gamma )\) for both \(e\gamma \) combinations. Data are compared to the MC prediction with the number of \(0^{+}_{1}\) signal events obtained by background subtraction

3.3.2 Use of \(\varvec{ee\gamma }\) events

The two-electron one-\(\gamma \) events are also used to measure the decay to the 0\(^+_1\) excited state since one of two emitted photons can remain undetected. The 0\(^+_1\) signal efficiency in this channel \(\epsilon \) = 2.2% is higher than in the \(ee\gamma \gamma \) channel. With \(N = 571\) data events and the total expected background of \(B = 512.5\) events (see Table 1), for the 0\(^+_1\) signal contribution defined by background subtraction \(S = N-B = 58.5\pm 23.9\) events, we obtain the half-life estimation

$$\begin{aligned} T_{1/2}^{2\nu \beta \beta }(0^{+}_{1}) = \left[ 1.98 ^{+1.37}_{-0.58} (\text {stat}) \right] \times 10^{20}~\hbox {year}. \end{aligned}$$
(6)

This is less precise than the estimate for the \(ee\gamma \gamma \) channel due to the worse signal-to-background ratio \(S/B = 0.11\) and the low signal statistical significance \(N{\sigma } = S/\sqrt{N} = 2.5\) in this channel. The effect of systematic uncertainty on background rates also becomes more significant with larger backgrounds. Nevertheless, this estimation is statistically compatible with the measurement obtained using the \(ee\gamma \gamma \) events.

The background decomposition for the selected events is presented in Table 1 and the measured energy and angular distributions are shown in Fig. 6. A possible contribution from the \(\beta \beta \)-decay of \(^{150}\)Nd to the 2\(^+_1\) level is neglected. If the \(0^+_1\) contribution is normalized to the half-life value obtained in the \(ee\gamma \gamma \) channel, the resulting data deficit does not leave space for the 2\(^+_1\) contribution.

Fig. 7
figure 7

BDT score distribution for the \(2\nu \beta \beta \) decay to the 0\(^+_1\) excited state in the \(ee\gamma \) channel. The vertical dashed line denotes the optimal cut position maximizing the signal significance

The photon energy \(E_{\gamma }\) and the total measured energy \(E_\text {TOT}\) are the most important variables in the set of observables used for BDT training in this channel. The BDT score distribution obtained after the event classification is shown in Fig. 7. The position of the optimal BDT cut to maximize the 0\(^+_1\) signal significance is marked by a vertical dashed line. After the BDT cut, 85 data events remain, with \(B = 45.9\) expected background events (see Table 2). Subtracting the background leaves \(S =39.1\) events attributed to the 0\(^+_1\) signal. The resulting signal-to-background ratio is \(S/B = 0.85\), with a statistical signal significance of \(N{\sigma } = S/\sqrt{S+B} = 4.2\). The distributions of the measured kinematic variables for these events are shown in Fig. 8. The signal efficiency after the BDT cut is \(\epsilon = 0.88\%\) and the half-life estimate is

$$\begin{aligned} T_{1/2}^{2\nu \beta \beta }(0^+_1) = \left[ 1.21 ^{+0.37}_{-0.23} \,\left( \hbox {stat}\right) ^{+0.26}_{-0.20}\,\left( \hbox {syst}\right) \right] \times 10^{20}~\hbox {year}. \end{aligned}$$
(7)

3.3.3 Systematic uncertainties

Several sources of systematic uncertainty were investigated, with the most significant effect on the precision of the measured decay rate being the uncertainty on the event selection efficiency. It is estimated by measuring the calibrated \(^{232}\)U source activities. These activities, measured by the NEMO-3 detector for \(ee\gamma \gamma \) and \(ee\gamma \) events, are found to be in agreement with true values within 7.8%. This value, taken into account both for signal and background events, leads to the decay rate uncertainty of (+11.5, \(-\)9.8)% in the \(ee\gamma \gamma \) and (+18.4, \(-\)15.7)% in the \(ee\gamma \) channel.

The systematic uncertainty on the number of background events (see Tables 1, 2) was calculated from those of the individual background components’ activities estimated in [12]. It contributes to the decay rate uncertainty of (+2.3, \(-\)2.5)% in the \(ee\gamma \gamma \) and (+6.6, \(-\)6.8)% in the \(ee\gamma \) channel.

The effect of the limited accuracy in simulation of ionization energy loss and of bremsstrahlung in the foil on the measured decay rate was studied. This was done by generating additional MC data samples varying the relevant parameters within their expected uncertainty. The decay rate uncertainty due to ionization energy loss was evaluated to be ±1.6% in the \(ee\gamma \gamma \) and ±2.8% in the \(ee\gamma \) channel. The uncertainty due to bremsstrahlung is ±1.2% in the \(ee\gamma \gamma \) and ±4.4% in the \(ee\gamma \) channel.

The effect of energy calibration accuracy was studied by altering measured energies according to the energy scale uncertainty; it yields a systematic uncertainty of ±1% on the decay rate measurement in the \(ee\gamma \gamma \) and ±1.6% in the \(ee\gamma \) channel.

Fig. 8
figure 8

Distributions for the two-electron one-\(\gamma \) events from the \(^{150}\)Nd foil after the cut on the BDT score value: energy sum of two electrons \(E_{2e}\), \(\gamma \) energy \(E_{\gamma }\), minimal and maximal electron energy \(E_{e}^{\text {min}}\), \(E_{e}^{\text {max}}\), total measured energy \(E_\text {TOT}\), cosine of the angle between the electron and \(\gamma \) \(\cos (e\gamma )\) for both \(e\gamma \) combinations. The 0\(^+_1\) signal contribution is defined as the difference between the data and the background

Finally, a ±0.5% uncertainty on the mass of \(^{150}\)Nd translates into the same uncertainty on the measured decay rate.

All these contributions are summarized in Table 3, with the total uncertainty calculated by summing the individual contributions in quadrature.

Table 3 Sources of systematic uncertainty on the rate \(\Gamma \) of \(2\nu \beta \beta \) decay to the 0\(^+_1\) excited state measured with \(ee\gamma \gamma \) and \(ee\gamma \) events

3.3.4 Mean half-life from \(\varvec{ee\gamma \gamma }\) and \(\varvec{ee\gamma }\) channels

The individual half-life estimates from the \(ee\gamma \gamma \) and \(ee\gamma \) channels in Eqs. (5) and (7) are in good agreement. The mean value of the two measurements was calculated using their statistical weights:

$$\begin{aligned} T_{1/2}^{2\nu \beta \beta }(0^+_1) = \left[ 1.11 ^{+0.19}_{-0.14} \,\left( \hbox {stat}\right) ^{+0.17}_{-0.15}\,\left( \hbox {syst}\right) \right] \times 10^{20}~\hbox {year}. \end{aligned}$$
(8)

The systematic uncertainty of the mean value was obtained by calculating the mean of the two measurements coherently increased/decreased by their individual systematic uncertainties. The mean value corresponds to the signal-to-background ratio \(S/B = 1.3\) and the statistical signal significance \(N\sigma _{\text {stat}}\) = 6.8. Using the total error determined by summing the statistical and systematic errors in quadrature, we obtain the 0\(^+_1\) signal significance \(N\sigma _\text {tot}\) = 5. This half-life value is compared with the results of the previous measurements in Table 4. There is a good overall agreement between the results. The value obtained in this work is most precise.

Table 4 Comparison of our results for \(^{150}\)Nd \(\beta \beta \) decay to the \(0^{+}_{1}\) and \(2^{+}_{1}\) excited states of \(^{150}\)Sm with the previous results. Limits are given at 90% CL

This half-life value may be used to extract the experimental value of the corresponding NME according to Eq. (1). Using the phase space factor value \(G = 4.116 \times 10^{-18}~\hbox {year}^{-1}\) [24] and \(g_A\) = 1.2756 [1], one obtains the NME value (scaled by the electron rest mass) for the 2\(\nu \beta \beta \) transition to the 0\(^+_1\) excited state:

$$\begin{aligned} |M^{2\nu }(0^+_1)| = 0.0288^{+0.0032}_{-0.0028}. \end{aligned}$$
(9)

One can compare this value with the NME value for the 2\(\nu \beta \beta \) transition to the ground state of \(^{150}\)Sm:

$$\begin{aligned} |M^{2\nu }(0^+_{\textsf {g.s.}})| = 0.0338^{+0.0013}_{-0.0011}, \end{aligned}$$
(10)

obtained using the half-life of Eq. 3 from [9] and \(G = 3.540 \times 10^{-17} year^{-1}\) from [24]). The relative difference between these two NMEs is \((15^{+10}_{-9})\)%. One can see that the values of these matrix elements are very close (in the first approximation they are the same) and that \(M^{2\nu }(0^+_{\textsf {g.s.}})\) is \(\sim \)15% greater than \(M^{2\nu }(0^+_{1})\). This is consistent with the conclusions in [8, 25]. The same conclusions were drawn for \(^{100}\)Mo (see, for example, [25]). Thus, it looks like a certain regularity. It seems important and interesting to understand this regularity and to give it some theoretical explanation. Finally, this can help to clarify the situation with the calculations of nuclear matrix elements.

3.4 Search for \(\varvec{2\nu \beta \beta }\) decay to \(\varvec{2^+_1}\) excited state

As stated above, there is no excess in the data above the expected background that could be attributed to the \(2\nu \beta \beta \) decay to the 2\(^+_1\) excited state. A limit on this transition was set using the \(ee\gamma \) events. A BDT classification was performed for the 2\(^+_1\) signal against backgrounds, including all the contributions listed in Table 1 plus the \(2\nu \beta \beta \) decay to the 0\(^+_1\) excited state normalized according to Eq. (8).

Fig. 9
figure 9

BDT score distribution for the \(ee\gamma \) events used to set the limit on the \(^{150}\)Nd \(2\nu \beta \beta \) decay to the 2\(^+_1\) excited state of \(^{150}\)Sm

The BDT score distribution in Fig. 9 was used for calculating the limit. Hereafter, the systematic uncertainties discussed above are taken into account. Systematic uncertainties are treated as uncertainties on the expected numbers of events and are folded into the signal and background expectations via the Gaussian distribution. The systematic uncertainties considered for each of the background component normalizations are given in Table 1. The uncertainty on the normalization of the \(^{150}\)Nd 2\(\nu \beta \beta \) decay to the 0\(^+_1\) excited state is constrained by the statistical uncertainty of the measurement given in Eq. (8). The obtained result

$$\begin{aligned} T^{2\nu \beta \beta }_{1/2}(2^+_1) > 2.42 \times 10^{20}~\hbox {year at } 90\%\hbox { C.L.} \end{aligned}$$
(11)

is slightly more restrictive than the limit set in [9].

Fig. 10
figure 10

Two-electron energy sum distributions for the \(ee\gamma \gamma \) and \(ee\gamma \) events with the superimposed signal expected for the 0\(\nu \beta \beta \) decay to the 0\(^+_1\) excited state

Fig. 11
figure 11

BDT score distribution for the \(0\nu \beta \beta \) decay to the 0\(^+_1\) excited state in the \(ee\gamma \gamma \) and \(ee\gamma \) channels

3.5 Search for \(\varvec{0\nu \beta \beta }\) decay to \(\varvec{0^+_1}\) excited state

The \(ee\gamma \gamma \) and \(ee\gamma \) events were used to search for the signal of the 0\(\nu \beta \beta \) decay to the 0\(^+_1\) excited state. No significant data excess is observed for the \(0\nu \beta \beta \) signal in the distribution of the summed kinetic energy of two electrons (see Fig. 10). There are 4.2 (16.1) events expected in the region from 2 MeV to 2.8 MeV under the 0\(\nu \beta \beta \) peak, and 3 (16) data events are observed in the \(ee\gamma \gamma \) (\(ee\gamma \)) channel. The BDT score distribution is shown in Fig. 11.

In the \(ee\gamma \gamma \) channel, the observed half-life limit is

$$\begin{aligned} T_{1/2}^{0\nu \beta \beta }(0^+_1) > 5.20 \times 10^{21}~\hbox {year at } 90\% \hbox { C.L.} ~ ~ , \end{aligned}$$
(12)

and in the \(ee\gamma \) channel it is

$$\begin{aligned} T_{1/2}^{0\nu \beta \beta }(0^+_1) > 9.97 \times 10^{21}~\hbox {year at } 90\% \hbox { C.L.} \end{aligned}$$
(13)

When the combination of the two channels is used for calculating the limit, the result is

$$\begin{aligned} T_{1/2}^{0\nu \beta \beta }(0^+_1) > 13.6 \times 10^{21}~\hbox {year at } 90\% \hbox { C.L.} \end{aligned}$$
(14)

This limit is much stronger than that from [23].

Fig. 12
figure 12

Two-electron energy sum distribution for the \(ee\gamma \) events with the signal expected for the 0\(\nu \beta \beta \) decay to the 2\(^+_1\) excited state superimposed

Fig. 13
figure 13

BDT distribution for the \(0\nu \beta \beta \) decay to the 2\(^+_1\) excited state

3.6 Search for \(\varvec{0\nu \beta \beta }\) decay to \(\varvec{2^+_1}\) excited state

The search for the \(0\nu \beta \beta \) decay to the 2\(^+_1\) excited state was performed using the \(ee\gamma \) events. The distribution of the summed kinetic energy of two electrons for these events is shown in Fig. 12. The 0\(\nu \beta \beta \) signal in this distribution would be manifested in a peak located in the energy interval from 2.4 to 3.2 MeV. There are 12 data events in this interval in good agreement with the MC expectation of 11.6 events, meaning that no evidence of the \(0\nu \beta \beta \) decay is observed. The BDT score distribution for this decay mode is presented in Fig. 13. With this distribution, we obtain the following half-life limit:

$$\begin{aligned} T_{1/2}^{0\nu \beta \beta }(2^+_1) > 1.26 \times 10^{22}~\hbox {year at } 90\% \hbox { C.L.} \end{aligned}$$
(15)

that is much more stringent than the result of [23], see Table 4.

4 Summary

The two-neutrino double-\(\beta \) decay of \(^{150}\)Nd to the 0\(^+_1\) excited state of \(^{150}\)Sm has been measured with a statistical signal significance of 6.8 standard deviations and the signal-to-background ratio \(S/B = 1.3\). The most precise measurement of the half-life for this process to date has been performed:

$$\begin{aligned} T_{1/2}^{2\nu \beta \beta }(0^+_1) = \left[ 1.11 ^{+0.19}_{-0.14} \,\left( \hbox {stat}\right) ^{+0.17}_{-0.15}\,\left( \hbox {syst}\right) \right] \times 10^{20}~\hbox {year}. \end{aligned}$$
(16)

The \(\gamma \gamma \) angular correlation for this decay, measured for the first time, is found to be consistent with the expected behaviour characterizing the \(0^{+} \rightarrow 2^{+} \rightarrow 0^{+}\) cascade.

No signal evidence was found for the \(2\nu \beta \beta \) decay to the 2\(^+_1\) excited state nor for the \(0\nu \beta \beta \) decay to the 0\(^+_1\) or 2\(^+_1\) excited states. The corresponding 90% confidence level limits have been established. The obtained half-life limit on the \(2\nu \beta \beta \) decay to the 2\(^+_1\) excited state

$$\begin{aligned} T_{1/2}^{2\nu \beta \beta }(2^+_1) > 2.4 \times 10^{20}~\hbox {year at } 90\% \hbox { C.L.} \end{aligned}$$
(17)

is slightly better than the best previous result of [9], but still far short of the theoretical predictions \(T_{1/2} = 2.6\times 10^{23}~\hbox {year}\) [26], \(T_{1/2} = 7.2\times 10^{24}~\hbox {year}\) [27].

The limits set on the neutrinoless decay half-life have been significantly improved.