1 Introduction

Several observational evidences, including galactic rotation curves [1], gravitational lensing [2], and cosmological modeling [3], indicate the presence of a non-relativistic, gravitationally interacting, and stable matter in the universe. Traditionally referred to as Dark Matter (DM), it is expected to be responsible for 26% of the mass-energy density of our universe [4] and has not been conclusively detected. A prominent, NaI(Tl) based direct detection experiment is DAMA/LIBRA [5,6,7,8], which is located in the Laboratori Nazionali del Gran Sasso (LNGS) under 1400 m of rock overburden (3600 m.w.e) [9]. For the past 27 years, the DAMA collaboration has observed an annually modulating event rate whose phase and period match with the expected DM signal in an earth-bound detector [8]. However, under the “standard” DM halo assumptions [10], numerous null results of various other dark matter searches exclude the region of parameter space explored by DAMA/LIBRA [11]. These direct detection experiments employ different target materials, which makes any model-independent comparison difficult. To resolve this decade-long tension, many experiments using NaI as target material have begun taking data (ANAIS-112 [12] and COSINE-100 [13]), are currently under construction (COSINUS [14, 15] and SABRE [16]) or are in the R &D phase (ASTAROTH [17] and PICO-LON [18]).

Uniquely, the COSINUS (Cryogenic Observatory for SIgnatures seen in Next-generation Underground Searches) experiment will operate the NaI crystals as cryogenic scintillating calorimeters [15]. The NaI detectors will be cooled to milli-kelvin temperatures, and both the scintillation light and phonon signal will be read out. The phonon signal is measured using the novel remote transition edge sensor (remoTES) setup [19,20,21], while scintillation light will be detected by a silicon beaker enclosing the NaI crystal and instrumented with a TES. This dual channel capability will allow for the discrimination between electromagnetic and nuclear recoil interactions on an event-by-event basis. Assuming a nuclear recoil threshold of 1 keV, a background rate similar to what is discussed in [15], and an exposure of 1000 kg\(\cdot \)days, COSINUS will provide a model-independent cross-check of the DM interpretation of the DAMA/LIBRA signal as generated by DM-nuclei scattering events [22].

This work details the results of a comprehensive Monte Carlo simulation to optimize the design of the active water Cherenkov muon veto for the COSINUS experiment. An optical simulation was performed to test the effect of the muon veto efficiency (probability of tagging an incident muon) with different photomultiplier tube (PMT) arrangements and trigger conditions. Additionally, a study was done on how the size of an optically invisible region in the water tank (dead layer) would affect the overall muon veto capability and PMT trigger rate.

2 Muon-induced backgrounds and conceptual veto design

COSINUS is currently being built in Hall B at LNGS in Italy, with construction expected to be finished by the middle of 2024. The cooling of the detector modules is provided by a dry dilution refrigerator with the detector volume residing at its base. The cryostat is held in a stainless steel cylinder, known as the dry-well, which is contained in a larger, \(\sim 7\times 7\) m, cylindrical steel tank filled with ultra-pure water that acts as a passive shield against ambient radiation. Additionally, copper shielding is placed above and around the detector volume to reduce the radiogenic background from internal components. The optimization of the passive shielding was discussed in detail in [23]. Because of the event-type discrimination capability of the COSINUS experiment, neutrons are the most important background to mitigate.

Fig. 1
figure 1

Schematic of the simulated muon veto setup of COSINUS. The exact dimensions used in the muon veto simulation can be found in [23]

In underground laboratories, neutron flux occurs from either natural radioactivity or cosmic rays. Spontaneous fission and (\(\alpha ,\) n) interactions from decaying isotopes of the \(^{238}\)U, \(^{235}\)U  and \(^{232}\)Th chains are the processes responsible for the natural radioactivity (ambient and radiogenic neutron flux) emitted from the rock around the laboratory and detector materials. The energy spectrum of these neutrons peaks at a few MeV [24] and the passive shielding discussed in [23] would reduce these events to less than one neutron per year in the detector volume (\(\sim 1\) kg) of the COSINUS experiment.

Conversely, neutrons that are produced as a result of cosmic-ray muon interactions are much more challenging to moderate. At LNGS the cosmic muon flux was measured to be 3.4 \(\times \) 10\(^{-4}\) m\(^{-2}\cdot \)s\(^{-1}\) [25, 26]. As a result of the passive shielding of COSINUS, the dominant contribution to the neutron background will come from cosmogenics, with an expected rate of 3.5 ± 0.7 cts\(\cdot \)kg\(^{-1}\cdot \)year\(^{-1}.\) Muon-induced neutrons can be produced through four different processes: (1) negative muon capture, (2) muon-induced spallation reactions, (3) neutron production by hadrons from muon-induced hadronic cascades, and (4) neutron production by photons from muon-induced electromagnetic cascades [27]. The muon-induced neutron energy spectrum can be approximated by a 1/E relationship up to 0.5 GeV and a \(1/E^2\) relationship beyond [28]. The spectrum (see Fig. 5c in [23]) will extend up to a few GeV allowing these neutrons to travel through the passive shielding of the COSINUS experiment and into the detector volume. Additionally, these neutrons can be produced past the shielding, inside the setup, and transfer \(>1\) keV energies to nuclei, which will be above the threshold for the experiment. Muon-induced neutrons are typically accompanied by a large number of secondary particles. These neutrons can be detected simultaneously with the primary muon themselves or one of the secondaries created.

2.1 Water Cherenkov muon veto conceptual design

In the COSINUS experiment the muon veto will consist of a cylindrical water tank, with PMTs mounted on the wall and bottom, and a reflector encasing the PMTs. The PMTs will detect the Cherenkov radiation (with wavelengths in the optical range) produced from the muons or their products that enter the tank. A schematic of the veto setup can be seen in Fig. 1.

COSINUS has selected the 8-inch diameter R5912-30 PMT from Hamamatsu [29]. The PMT consists of a bialkali photocathode material with borosilicate glass. A maximum of 28 PMTs will be deployed in the setup, each providing a typical gain of \(\times 10^{6}\) at an operating voltage of 1100 V. The quantum efficiency of the R5912-30 can be seen in Fig. 2. The efficiency of the light collection will be enhanced by the addition of a reflector, which will be placed around the interior of the water tank. The additional reflectivity will elongate the photon path increasing the probability it will be collected in a PMT.

Fig. 2
figure 2

a Hamamatsu R5912-30 photomultiplier tube, and b its quantum efficiency as a function of wavelength [30]

2.1.1 Background triggering of the PMTs

Spurious triggers can be induced by thermionic emissions in the cathode (dark counts) and the ambient gamma background. This, in turn, can lead to random coincidences with a signal observed in the NaI detector and ultimately the rejection of a real event. Dark counts will be independent of the PMTs and depend on the specific PMT model. The nominal dark count rate for the R5912-30 is 6000 Hz, but experiments that have utilized the same PMT have measured a much lower rate of 1200 Hz [31].

In Hall B at LNGS, the total gamma flux between 7.4–2734.2 keV was measured to be 0.23 \(\gamma \cdot \)cm\(^{-2}\cdot \)s\(^{-1}\) [32]. For the COSINUS experiment, this would yield a total rate through the tank of \(\sim 5\times 10^{5}\) \(\gamma \cdot \)s\(^{-1},\) a factor significantly higher than the dark count rate. Although not every gamma will trigger a PMT, the effect of the ambient gammas can be reduced by creating an optically dead layer in the periphery of the tank. In [23], the gamma flux was shown to decrease by an order of magnitude for each 50 cm of water added to the radius of the tank. The idea would therefore be to split the water volume into a passive border (which will not be instrumented by the PMTs, referred to as the dead layer or optical invisible region) and an active central region. In practice, this will be achieved by placing the reflector and PMTs at a certain distance from the tank walls. The effect of the different dead layer lengths is studied in Sect. 4.4.

3 Description of the Monte Carlo simulation

For this work, a detailed Monte Carlo simulation was performed using a Geant4 [33,34,35] based software toolkit called ImpCRESST [36], which was initially developed within the CRESST DM search experiment [37] and later also within COSINUS. An ImpCRESST version equipped with Geant4 v10.2.3 and root v6-08-06 [38] was used for this work. The ImpCRESST simulation results consist of information such as particle tracking, energy deposition, particle type, interaction mechanism, and timing information. For this particular study, the propagation of the optical photons was also recorded. For optimization of the muon veto design, the geometry of the set-up has been simulated. In the simulation, the experimental geometry is placed in an empty hall surrounded by 2.6 m thick rock on the top, bottom, and one side of the water tank. The simulated geometry of the muon veto setup is shown in Fig. 1.

3.1 Physics models

The simulation’s outcome relies on the processes and models that are activated and implemented through the physics list utilized. ImpCRESST employs a set of physics models adapted from the Geant4 Shielding physics list [39], which is suitable for deep shielding and neutron transport simulation.

G4EmStandardPhysics is responsible for standard electromagnetic interaction in Geant4. In ImpCRESST, G4EmStandardPhysics_option4 is used, which is tailored with the most accurate models for standard and low-energy processes, with a minimum energy threshold of 100 eV (not recommended to lower below 250 eV). At the energy range of interest for DM searches, the implementation of EM physics processes happens via the various Geant4 classes. For example, Muon interactions are employed by classes such as G4MuIonisation and G4MuMultipleScattering, while interactions of \(\alpha \)-particles and heavier ions are governed by G4hMultipleScattering and G4ionIonisation.

G4EmExtraPhysics enables gamma-nuclear processes for simulating electromagnetic showers from high-energy muons. G4HadronElasticPhysicsHP employs high-precision models for elastic neutron scattering below 20 MeV. G4HadronPhysicsShielding manages inelastic hadron processes with HP models for neutrons below 20 MeV. G4NeutronHPThermalScattering refines elastic neutron scattering below 4 eV. G4StoppingPhysics captures charged particles at rest, including negative muons. G4IonQMDPhysics manages inelastic processes of ions above certain energy thresholds. G4ProductionCutsTable sets energy thresholds for secondary particle production, adjusted to 250 eV in ImpCRESST. G4OpticalPhysics was activated for this work, which enables the simulation of optical processes and the creation of optical photons.

3.2 Cosmogenic muons

The initial position, energy, and angular distribution of cosmogenic muons underground were generated using the MUon Simulations UNderground (MUSUN) [40] production code. MUSUN uses the results of muon transport through rock/water to generate muons in or around underground laboratories taking into account their initial energy spectrum, angular distribution, and the overburdened rock profile. For the current work, a customized version of the code was used that incorporates the complex mountain landscape of the LNGS site [24].

Drawing from the MUSUN-generated events, originating from a \(12\times 12\times 13~\text {m}^{3}\) cuboid surface, 30 million muons were simulated through the COSINUS geometry. This corresponds to an exposure of \(\sim 13\) years which is approximately 4 times the planned measurement length of COSINUS. If a muon produces a neutron that enters the dry-well of the experiment this event is classified as dangerous. Out of 30 million muons, 11,682 were tagged as dangerous with their initial energy and angular distribution shown in Fig. 3. These dangerous muons are further classified into two categories: (1) muon events where the muon passes directly through the water tank itself generating Cherenkov radiation and (2) shower events where the muon misses the tank, interacts in the rock or cryostat components creating secondary particles that produce the Cherenkov radiation. The simulations show that shower events will make up \(\sim 5\%\) of all dangerous events to reach the cryostat. The optical simulations were conducted exclusively on dangerous events to evaluate the impact of the different muon veto configurations on reducing the cosmogenic neutron background.

Fig. 3
figure 3

Initial a energy, b zenith angle, and c azimuthal angle for the MUSUN simulated events. The normalized distributions are also shown for the subset of events that produce a neutron in the dry-well of the COSINUS experiment (i.e. the dangerous events). See text for more details

3.3 Ambient gamma background

As discussed in Sect. 2.1.1, ambient gammas contribute significantly to the overall PMT trigger rate. This necessitates an optical simulation in order to evaluate the best possible configuration to reduce the ambient gamma trigger rate. For this work, 20 million ambient gammas are simulated, corresponding to an exposure of 40 s. The sampled energy distribution is shown in Table 1 which is measured in Hall B of LNGS [32]. Ambient gammas are generated from the internal surface of a cylindrical volume, positioned at a distance of 1 mm outside the water tank, with gamma rays directed inward towards the tank. Section 4.4.1 presents a detailed description of the results of the simulation.

Table 1 Ambient gamma flux as a function of energy in Hall B of LNGS.

4 Simulation results

The following chapter details the results of the design study to optimize the tagging efficiency of a water Cherenkov muon veto. In Sect. 4.1, the optical photon maps along the tank walls from the incident cosmogenic muons are presented. That information is then used to test the efficiency of different PMT arrangements, reflector type, and trigger conditions in Sects. 4.2 and 4.3. To reduce the ambient background trigger rate various optical dead layers were studied in Sect. 4.4. The information found in the following sections was then used to recommend the final veto setup, found in Sect. 4.6.

4.1 Cherenkov illumination of the water tank

To study the veto efficiency with different PMT configurations, the interaction points of the Cherenkov photons, produced from muon (or shower) events, were mapped along the surface of the COSINUS water tank. This is shown for both the bottom (Fig. 4) and the wall (Fig. 5) of the tank. In the simulation, the tank is layered with a thin polyethylene material with a reflectivity of 95%. The contrast between the photon maps of muon and shower events is due to the topology of the event and how the particles are absorbed/transported through the water tank. Most muon events will traverse through the water tank, producing copious amounts of Cherenkov radiation as they pass. This will tend to give a more uniform spread where any unique, visible features in the illumination map will be due to the directionality of the muons and the LNGS mountain rock profile (taken into account by the MUSUN simulation). The products of shower events, however, will get absorbed closer to the surface of the water tank, inducing a larger collection of photons around the border (see Fig. 4b). Occasionally, shower products can produce intense, localized traces, such as the ones seen in Figs. 4b and 5b.

Fig. 4
figure 4

Photon illumination maps on the bottom of the water tank generated by muon (a) and shower (b) events. Binning corresponds to the surface area of an 8-inch diameter PMT. The Z-axis is the average number of photons that interact in a specific bin area on the bottom surface

The inside of the COSINUS water tank has very little material and the main shadowing is done by the dry-well containing the cryostat. This produces the dull circle seen in the middle of Fig. 4a and b. Qualitatively these photon maps provide a guide on how the PMTs should be placed. A majority of the PMTs should be on the bottom of the water tank as this corresponds to a higher concentration of photons. Additionally, the bottom PMTs can be arranged in two concentric circles, one with a smaller diameter (\(\sim 1.5\) m) for the muon events and one with a diameter closer to the border region for shower events. Determining the position of the wall PMTs from the photon illumination map is more ambiguous and specific PMT arrangements are discussed in Sect. 4.2, along with different trigger conditions.

Fig. 5
figure 5

Muon (a) and shower (b) event photon illumination map of the wall of the water tank. Binning corresponds to the surface area of the 8-inch diameter PMT. The Z-axis is the average number of photons that interact in a specific bin area on the wall surface

4.2 Photomultiplier arrangement and trigger conditions

The PMTs employed in the COSINUS muon veto, will be placed both on the bottom and the wall of the water tank. Following the qualitative arguments presented in Sect. 4.1 a standard arrangement can be seen in Fig. 6. PMTs are represented along the surface as a \(\sim 18\times 18~\text {cm}^{2}\) square which gives them an equivalent surface area to the 8-inch diameter PMT. A collection efficiency of 90% is assumed based on the effective area of the R5912-30 PMT. The quantum efficiency value is taken from Fig. 2. When a photon interacts with the PMT surface it is subjected to a quantum efficiency and collection efficiency test. Passing both of these, the photon counts as a photoelectron hit for that PMT. The PMT is then triggered if the total number of photoelectron hits for a given event equals or exceeds a predetermined threshold. The final veto condition is then set by requiring an L-fold triggering of PMTs for a given event.

Fig. 6
figure 6

Example 28 PMT arrangement along the bottom (a) and wall (b) of the water tank

Using the dangerous muons from Sect. 3.2 as the initial particle source and the PMT arrangement shown in Fig. 6, the veto efficiency is evaluated at different trigger conditions. This is shown in Fig. 7 for both muon and shower events. In general, Fig. 7 shows that both a high photoelectric threshold or number of PMTs in coincidence will result in a decrease of vetoing power. The large amount of Cherenkov light produced during a muon event will still result in a large veto efficiency of at least 85%, even with the strictest of triggering requirements. Conversely, due to the lack of Cherenkov photons, shower events experience a large drop-off in vetoing power with an increased photoelectron threshold. This necessitates a low photoelectron number threshold in order to be as efficient as possible in tagging the shower events, and even then only \(\sim 40\%\) of shower events will be vetoed.

Fig. 7
figure 7

Muon (a) and shower (b) veto efficiency as a function of photoelectron threshold (PE) per PMT. For a given photoelectron trigger, the shower veto efficiency does not change by more than 10% over the required number of PMTs in coincidence. The muon veto efficiency variance, however, depends strongly on the number of photoelectrons to trigger.The uncertainty shown was determine using the bootstrap method [41]

The other parameter to be investigated is the time width of the coincidence window. The time difference between the triggering of the first and last PMT is shown in Fig. 8. These distributions contain both muon and shower events, and it is not expected that the event type will have an effect on the overall shape. By integrating over the spectra, the required coincidence window can be determined in order to successfully tag the event. For a 6-fold PMT coincidence and a photoelectric threshold per PMT of 1, 95% of events will be tagged with a coincidence window of 410 ns.

Fig. 8
figure 8

Distribution of the time difference between photons arriving at the first and final triggered PMT. A various number of PMT coincidence requirements are shown. With the 2-fold PMT coincidence defined by \(t_2,\) 3-fold by \(t_3,\) etc. The 95% event integral for each PMT fold coincidence is denoted by the corresponding dotted lines

4.3 Experimental design considerations

The reflector can play a large role in the strength of the veto, as the photon survival probability and thus PMT hit rate will increase with reflectivity. The muon event veto efficiency was studied for a reflectivity of 0, 50, 90, 95, and 100% in Fig. 9. This figure shows that at a low photoelectron threshold, the reflectivity will have a minimal effect on veto efficiency and the veto power will only severely deteriorate with a high threshold. The surface type (specular vs. diffuse) of the reflector was also studied. It was found that, in all trigger conditions, a diffuse reflector resulted in a drop of 5% efficiency in vetoing shower events, while the muon event veto efficiency remained unchanged at low photoelectron thresholds. As such, a specular reflector is recommended.

Fig. 9
figure 9

Effect of the reflectivity of the reflector on the muon veto efficiency. The uncertainty shown was determine using the bootstrap method [41]

An assortment of PMT arrangements was tested in order to search for an optimal configuration that maximizes the muon and shower event veto efficiency. This included varying the parameters:

  • inner ring radius along the bottom of the water tank

  • outer ring radius along the bottom of the water tank

  • number of rows for the wall PMTs

  • total number of PMTs along the bottom of the water tank

  • total number of PMTs along the top of the water tank

  • vertical position shifting of the wall PMT rows.

However, the only factor that significantly (\(> 1\%\) change) altered the efficiency was the total number of PMTs along the bottom or the wall of the tank. A linear relationship was found between the veto efficiency and the number of PMTs but, as COSINUS is restricted to a total number of 28 PMTs (to balance efficiency and cost) this cannot be exploited further. Therefore, based on construction restrictions, the configuration shown in Fig. 6 (or some small variation to account for the dead layer) will be the one used for the rest of this study.

4.4 Optimization of an optical dead layer

One design goal of the water Cherenkov veto is to maximize the muon tagging efficiency, while minimizing the PMT trigger rate from non-muon sources. Based on the discussion in Sects. 4.2 and 4.3 there are three primary parameters that can be tuned in the optimization: (1) the size of the optical dead layer, (2) number of PMTs required to trigger to classify an event and (3) the PE-threshold. The following sections discuss the optimization of these for the COSINUS water Cherenkov muon veto.

4.4.1 Ambient gamma background rate

As discussed in Sect. 2.1.1 the PMTs can be triggered by other sources than cosmic ray muons. Outside of dark counts, the primary background will be from ambient gammas. Following the procedure outlined in Sect. 3.3 an optical simulation of the ambient gamma background was performed. The reflectivity was chosen to be 95% and the PMTs were arrayed along the wall and bottom of the water tank, as shown in Fig. 6. Using the same triggering method as described in Sect. 4.2 and a single photoelectron threshold it was found that the rate of triggering two PMTs, caused by the ambient gammas, was 359 ± 3 Hz. The probability of having a spurious coincidence over a specified time window \(\Delta T\) with a given background rate \(R_{B}\) is given by

$$\begin{aligned} p(\Delta T) = \int ^{\Delta T}_{0} R_B e^{-R_B T} dT = 1-e^{-R_B \Delta T}. \end{aligned}$$
(1)

A typical time window around a cryogenic event is \(\sim 10\) ms [19]. In order to have a \(<1\%\) chance of a spurious coincidence the ambient gamma trigger rate, employing Eq. (1), would need to be \(<1\) Hz.

The ambient gamma trigger rate can be reduced by creating an optical dead layer (described in Sect. 2.1.1) or by requiring a higher-fold PMT coincidence. An optical dead layer will decrease the overall ambient gamma flux present in the active volume of the water tank, reducing the total trigger rate of the PMTs. Additionally, due to the small amount of light produced from the (\(\sim \) 100 keV) \(\gamma \)-rays, requiring a higher PMT coincidence will also reduce the overall trigger rate. It is possible to reduce the ambient gamma trigger rate with a larger photoelectron threshold, but, as shown in Fig. 7, this will unnecessarily reduce the effective muon and shower veto efficiency. For six different dead layer thicknesses (0, 10, 20, 30, 40, 70, and 100 cm), the ambient gamma background was simulated using the same reflector and PMT arrangement as discussed above. The results can be seen in Fig. 10, where the veto configurations above the dashed-red line are those that reduce the ambient gamma trigger rate below 1 Hz. Figure 10 shows that for each additional PMT that is required to be in coincidence, the ambient gamma trigger rate will drop by approximately an order of magnitude. Conversely, for every 10 cm of dead layer, the rate will decrease by a factor of two. Additional dead layer increases are less effective above 40 cm.

Unfortunately, it is not possible to increase the dead layer or PMT coincidence requirement without reducing the overall veto efficiency. Following the same procedure outlined in Sect. 4.2 the veto efficiency was evaluated for different dead layer thicknesses and PMT trigger conditions, shown in Fig. 11. Again a single photoelectron threshold was used to maximize the efficiency. The results clearly show that increasing the dead layer thickness results in a decrease in overall veto efficiency. The overall reduction in efficiency is vastly different when comparing muon and shower events. Muon events, which generate a large amount of light, are not affected by higher PMT coincidence requirements and only lose a few percent in efficiency with a large, 100 cm dead layer. Shower events will see a few percent reductions in efficiency for each increase in PMT coincidence requirement but will be dramatically reduced by an increase in the dead layer. When five PMTs are required to be in coincidence there is a drop of 36% (from 56 to 19%) when comparing 0 to 100 cm dead layers. This drop is due to the fact, discussed in Sect. 4.1, that the distribution of light from the shower event falls primarily near the tank’s surface. From Figs. 10 and 11 it is clear that a higher fold PMT requirement is more effective than an increased dead layer at reducing the false trigger rate while maintaining the overall veto power.

Fig. 10
figure 10

Rate of PMT triggers caused by the ambient gammas when considering different dead layers. Above the dashed red line denotes configurations in which the trigger rate is less than 1 Hz. The rates shown have been corrected with respect to the raw simulated rates as described in Appendix A.1

Fig. 11
figure 11

Muon (a) and shower (b) event veto efficiency under the effect of different dead layers. Multiple L-fold PMT triggering is evaluated. The same dashed red line from Fig. 10 is overlayed, indicating which configurations would result in an ambient background less than 1 Hz

Taking into consideration that the ambient gamma trigger rate must be below 1 Hz the optimal configuration that maximizes the veto efficiency was found to be: a single photoelectron threshold, a 5-fold PMT coincidence requirement, and no dead layer. This gives a muon event veto efficiency of 99.89 ± 0.05% and a shower event veto efficiency of 56.19 ± 4.8%.

It should be noted that the systematic uncertainties in the muon propagation and interaction mechanisms in GEANT4 and MUSUN are not included in this paper as that is beyond the scope of this work.

4.4.2 Inclusion of thermionic emissions in the total background rate

Thermionic emissions inside the photocathodes of the PMTs can induce triggers that must be folded into the ambient gamma background event rate from Sect. 4.4.1. By treating the dark counts as a single independent rate they can be added, using combinatorial probabilities, with the ambient gamma rate. This gives the total background rate experienced by the muon veto system. The procedure to calculate the total background rate is outlined in Appendix A. Based on the results from Fig. 8 a conservative time window of 500 ns was assumed. Three different dark count rates were considered: \(R_d=1200\) Hz which is the rate found by the XENON-1T [31] experiment for the same PMTs, \(R_d=6000\) Hz which is the typical dark rate “quoted” by Hamamatsu and \(R_d=10{,}000\) Hz which is the maximal dark count rate quoted by Hamamatsu [29]. The total rate, which includes both dark counts and ambient gamma triggers, is shown in Fig. 12 as a function of dead layer and PMT coincidence. The configurations above the dashed red lines represent cases where the total rate is below 1 Hz.

Fig. 12
figure 12

Total background trigger rate for different configurations. Three different dark count rates are used a 1200 Hz, b 6000 Hz, c 10,000 Hz. Above the dashed red line denotes configurations in which the trigger rate is less than 1 Hz

Comparing Figs. 10, 11 and 12 shows that the addition of the dark counts limits the allowed configurations. In the most restrictive case (\(R_d=10{,}000\) Hz) the configuration that optimizes the veto efficiency is a 6-fold PMT coincidence requirement and 0 cm dead layer. Again, while contrasting Figs. 11 and 12 it is clear that a higher-fold PMT coincidence will be more effective than an increase in dead layer for reducing the total background rate while maintaining an acceptable efficiency. However, adopting a 30 cm dead layer is a more realistic and practical option as this accounts for the physical length of the PMT and ensures that the required scaffolding used to support the PMT is behind the reflector. The corresponding muon veto efficiency, for the highest dark count rate and 30 cm dead layer, is then 99.58 ± 0.16 %, and the shower veto efficiency is 41.2 ± 5.6%. The muon veto efficiency has remained effectively unchanged from the value determined in Sect. 4.4.1 but the shower efficiency has decreased by 15%. The radiogenic contribution to the PMT background rate from the steel tank is expected to be a few orders of magnitude below the ambient gamma contribution and was not included in this analysis. The count rate is expected to be minimally affected by the Rn contamination in the water, as the level of Rn contamination in COSINUS is expected to be comparable to \(10^{-6}\) Bq/kg as reported by Borexino [42]. This similarity is expected as COSINUS plans to use similar water distillation methods to those used in [42]. The total radiogenic background recorded from the selected PMT is 7.3 Bq per PMT, with contributions from K-40 at 3.84 Bq per PMT, U-series at 1.89 Bq per PMT, and Th-series at 1.57 Bq per PMT, which is insignificant.

Future work will involve characterizing the experimental dark rate of the PMTs for the COSINUS experiment.

4.5 Systematic uncertainties

The uncertainties quoted in this manuscript, so far, represent the statistical error found during the Monte Carlo evaluation. This section discusses the systematic uncertainties due to the physics models employed in various code bases. Similar reasoning has been discussed in a previous COSINUS article about the passive shielding simulations [23].

The MUSUN code, which generates the initial energy, position, and direction of the underground muons, was validated against the measured data from the LVD experiment [40]. The average energy of muons produced from MUSUN was 273 GeV which agrees with the measured LVD energy of 270 ± 3 (stat.) ± 18 (syst.) GeV. Additionally, Fig. 7 of Ref. [40] shows an identical azimuthal angle comparison. The systematic uncertainty of the computed amount, energy spectrum, and angular distribution of muons reaching the LNGS underground halls calculated from MUSUN can be treated as negligible for this study.

There also exist large uncertainties in Geant4 for the yield of muon-induced neutrons. In Ref. [43], muon-induced neutron yields are compared between measurements and simulations. For an LNGS-like environment the quoted agreement within 25% can be an estimate of the systematic uncertainty. For the transport of neutrons (\(<20\) MeV) in materials a high-precision, data-driven model is used in Geant4. An agreement of 20% between Geant4 and MCNPX [44] was found [45] and can be assumed to be an upper limit on the systematic error on low-energy neutron transport. Once the muon veto is operational there will be a dedicated paper comparing the experimental results to the simulations and evaluating the accuracy of these programs for use in underground muon production.

4.6 Final configuration

Based on the study performed in Sects. 4.14.4 the following configuration is recommended:

  • 28 PMTs (8-inch diameter R5912-30 Hamamatsu)

  • Specular reflector with a reflectivity of \(> 90\%.\)

  • Optical dead layer on all sides of the tank (top, bottom, and wall) between 30–40 cm (final size to be decided by engineering considerations) in order to reduce the ambient gamma rate, allow for lower fold PMT coincidence triggering, and account for the practical length of the PMT.

  • Trigger at a single PE threshold with a greater than 4-fold PMT coincidence (depending on the eventual dark count rate) and a coincidence time window of at least 500 ns.

The above conditions, assuming a dark count rate of 1200 Hz, will give a veto efficiency of 99.63 ± 0.16% for muon events and 44.4 ± 5.6% for shower events. As shower events will make up five percent of the total cosmogenic background this gives a total veto efficiency of 97.0 ± 0.3 %. Therefore, the proposed water Cherenkov detector of the COSINUS experiment will reduce the total cosmogenic neutron background from 3.5 ± 0.7 cts\(\cdot \)kg\(^{-1}\cdot \)year\(^{-1}\) to an acceptable rate 0.11 ± 0.02 cts\(\cdot \)kg\(^{-1}\cdot \)year\(^{-1}.\) To provide a model-independent check of the DAMA/LIBRA experimental result an exposure of 1000 kg\(\cdot \)days is required, with the inclusion of the muon veto, the cosmogenic neutron background will be less than one event during this exposure.

5 Conclusion

COSINUS uses NaI crystals as cryogenic calorimeters to search for a direct dark matter signal. An extremely low background rate is required for any successful rare-event search experiment and COSINUS employs both passive and active shielding to achieve this. In this work, a detailed Monte Carlo simulation was performed to design an optimal cosmogenic muon veto. Different experimental configurations, trigger conditions, and reflector types were studied in order to maximize the efficiency of the veto at tagging neutrons produced by cosmic ray muons while maintaining acceptable background trigger rates in the PMTs. With the final configuration presented in Sect. 4.6 the obtained veto efficiency in tagging muon and shower events is found to be 99.63 ± 0.16% and 44.4 ± 5.6% respectively which gives a total veto efficiency of \(97.0\pm 0.3\%.\) This allows the experiment to reach a muon-induced neutron background of 0.11 ± 0.02 cts\(\cdot \)kg\(^{-1}\cdot \)year\(^{-1}.\) For the proposed COSINUS-\(1\pi \) exposure of 1000 kg\(\cdot \)days this will result in less than one background event in the region of interest.