Observed charged particle multiplicity distribution
Following the implementation of the signal extraction procedure and verification through closure test on MC events, we execute the same maximum likelihood fit on data. Table 4 lists the values of the fit parameters obtained for the data; and Table 9 lists the number of neutrino events in different multiplicity bins for the data. While our method does not require this to be the case, we note that the fitted PH and MCS test probabilities P(PH), Q(PH), P(MCS), and Q(MCS) agree in data and simulation within statistical uncertainties. This provides evidence that the simulation correctly describes the muon PH and MCS tests used in the analysis.
Area normalized, bin-by-bin fitted multiplicity distributions from three different GENIE predictions overlaid on data are presented in Fig. 11 where data error bars include statistical uncertainties obtained from the fit and the MC error bands include MC statistical and systematic uncertainties that are listed in Table 6 added in quadrature.
In general the three GENIE models agree within uncertainties with one another, and agree qualitatively with the data. There are indications that GENIE overestimates the mean charged particle multiplicity relative to the data. We emphasize that no tuning or fitting has been performed to this or any of the other kinematic distributions.
Observed kinematic distributions
A key technical feature of our analysis entails performing tests on the pulse height and multiple Coulomb scattering behavior of hits on the long contained track in each event. This allows a categorization of events in each multiplicity into four categories according to whether the long track passes or fails the PH and MCS tests: \(\left( PASS,PASS\right) \), \(\left( PASS,FAIL\right) \), \(\left( FAIL,PASS\right) \), and \(\left( FAIL,FAIL\right) \). We have shown that the \(\left( PASS,PASS\right) \) category is “neutrino-enriched” and the \(\left( FAIL,FAIL\right) \) category is cosmic-ray-dominated. The mixed cases \( \left( PASS,FAIL\right) \) and \(\left( FAIL,PASS\right) \) provide samples with intermediate signal-to-background ratios.
Our fit to the distribution of the eight event categories in on-beam and off-beam data allows us to estimate the number of neutrino events \(\hat{N}_{\nu i}\) and the number of corresponding background CR events \(\hat{N}_{CRi}\) for each observed multiplicity i. Once \(\hat{N}_{\nu n}\) and \(\hat{N}_{CRn}\) are established, we can obtain a prediction for the content of any bin k of any kinematic quantity \(X_{ij}\) associated with track j in an observed multiplicity i event in any \(\left( PH,MCS\right) \) test combination:
$$\begin{aligned} \text {model}\left( X_{ij},PH,MCS\right) _{k}= & {} \hat{N}_{\nu i} \hat{x}_{\nu ,ij}\left( PH,MCS\right) _{k}\nonumber \\&+\hat{N}_{CRi} \hat{x}_{CR,ij}\left( PH,MCS\right) _{k}.\nonumber \\ \end{aligned}$$
(28)
Here \(\hat{x}_{\nu ,ij}\left( PH,MCS\right) _{k}\) is an area-normalized histogram of \(X_{ij}\) for “true neutrino events” in a given category obtained from a “MC” sample, and \(\hat{x} _{CR,ij}\left( PH,MCS\right) _{k}\) is an area-normalized histogram of \(X_{ij} \) for CR events obtained from off-beam data. This distribution can be compared to the corresponding one for data in each category, data \(\left( X_{ij},PH,MCS\right) _{k}\).
In short, we assume that the observed distribution of events consists of a mix of neutrino events plus CR events. The proportions of the mix in each category are fixed by the output of our fit, which, by construction, constrains the normalization of the model to equal that of the data. We emphasize that only the PH and MCS tests have been used to extract the neutrino interaction signal sample; no information from any quantity \(X_{ij}\) is used.
Checks on distributions lacking dynamical significance
Several kinematic properties of neutrino interactions depend only weakly on the neutrino interaction model; these include the reconstructed vertex positions, the initial and final coordinates of the long track, and the azimuthal angles of individual tracks. These distributions provide checks on the overall signal-to-background separation provided by the test-category fits and flux and detector modeling. They also test for differences between the modeling of neutrino events, which depend on the GEANT detector simulation, and CR events, which use the off-beam data and thus do not depend on detector simulation.
As an example, we show the observed distributions for the selected vertex y position for the candidate muon track from the full selected sample in Fig. 12. For this and all subsequent distributions, the on-beam data events are indicated by plotted points with statistical error bars. The model prediction is shown by a colored band (red for GENIE default, green for GENIE+TEM, and blue for GENIE + MEC) with the width of the band indicating the correlated statistical plus efficiency systematic uncertainty from using common \(N_{\nu ,n},N_{CR,n}\) values for all bins of all distributions of a given multiplicity bin n. The CR contribution to a distribution in a given category is shown by the shaded cyan region. For example, Fig. 12 compares the on-beam data to GENIE default MC sample and also shows the CR background.
The signal-enriched \(\left( PASS,PASS\right) \) category for vertex y has the nearly flat distribution expected for a neutrino event sample with a small CR background. Note that in our selection, we only allow candidate muon tracks initial y position \(<70\) cm. This cut rejects many cosmic rays that produce a downward trajectory in the final selected sample. The remaining background is dominated by cosmic rays with an apparent upward trajectory. This can be seen in the background-enriched sample \(\left( FAIL,FAIL\right) \) in the vertex y distribution where a peak at negative y values corresponds to “upwards-going” CR.
Figure 13 shows the distribution of azimuthal angle \(\phi \), defined in the plane perpendicular to the beam direction, of the muon candidate track for the full selected sample. The CR-dominated \(\left( FAIL,FAIL\right) \) category shows the expected peaking at \(\phi =\pm \pi /2\) from the mainly vertically-oriented CR. The asymmetry in the peak’s structure is due to the requirement on vertex y position described previously in Sect. 5.4. By contrast the signal-enriched \(\left( PASS,PASS\right) \) category has the nearly flat distribution expected for a neutrino event sample with a small CR background.
Similar levels of agreement exist between data and simulation for distributions of the event vertex x and z positions, for the \(\left( x,y,z\right) \) position of the end point of the muon track candidate, and for the azimuthal angles of individual tracks in multiplicity 2 and 3 topologies. We thus conclude that the simulation and reconstruction chain augmented by our method for estimated CR backgrounds satisfactorily describes features of the data that have no dependence on the neutrino interaction model.
Dynamically significant distributions
Events with N reconstructed tracks have potentially 4N dynamically significant variables-the components of each particle 4-vector-which will have distributions that depend on the neutrino interaction model. Azimuthal symmetry of the beam eliminates one of these, leaving 3, 7, 11, and 15 dynamically significant variables for multiplicities 1–4, respectively. In the following, we use the notation \( X_{ij}\) to label a dynamical variable x associated with track j in an observed multiplicity i event. For example, \(\cos \theta _{11}\) describes the cosine of polar angle distribution of the only track in multiplicity 1 events, while \(L_{22}\) would describe the length of the second (short) track in multiplicity 2 events. The notation with three subscripts, \(X_{ijk}\), represents a distribution of the difference in variable x associated with tracks j and k in an observed multiplicity i event.
Table 10 \(\chi ^{2}\) test results for dynamically significant variables for all three GENIE models. Only the uncorrelated statistical uncertainties from data are used in forming the \(\chi ^{2}\). Contributions from systematic uncertainties are not included. The last five listed distributions are not included in the total \(\chi ^{2}/DOF\) since these quantities can be expressed in terms of others
For one-track events, three variables exist. We use the observed length \( L_{11}\) as a proxy for kinetic energy, and the cosine of the scattering angle with respect to the neutrino beam direction \(\cos \theta _{11}\). The azimuthal angle \(\phi _{11}\) has no dynamical significance and must be uniformly distributed due to the cylindrical symmetry of the neutrino beam.
Since the particle mass is not determined in our analysis, we are free to introduce a third dynamically significant quantity that is sensitive to particle mass, which we take to be
$$\begin{aligned} \sin \Theta _{11}=\left| \hat{s}_{11}\times \hat{t}_{11}\right| , \end{aligned}$$
(29)
where \(\hat{s}_{11}\) is a unit vector parallel to the track direction at the event vertex, and \(\hat{t}_{11}\) is a unit vector that points from the start of the track to the end of the track in the detector. The variable \( \Theta _{ij}\) measures the angular deflection of a track over its length due to multiple Coulomb scattering. Its dependence on track momentum and energy differs from that of track length. For most of the MicroBooNE kinematic range, we expect light particles (\(\pi \) and \(\mu \)) to scatter more, and thus produce a broader \(\sin \Theta _{ij}\) distribution, than protons over the same track length.
Figure 14 shows the distributions of \(L_{11}\), \(\cos \theta _{11}\), and \(\sin \Theta _{11}\), from the neutrino-enriched sample compared to the GENIE default model. Figure 15 presents the \(L_{11}\) distribution for the GENIE + MEC and GENIE + TEM models. This is the distribution where the agreement between data and GENIE + TEM model, compared to the agreement between data and the other two models, is largest. Figure 16 presents the \(\cos \theta _{11}\) distribution for GENIE + MEC and GENIE + TEM models. This is the distribution where the agreement between data and the GENIE default compares least favorably than to the GENIE+MEC and GENIE+TEM models.
For brevity in the following, except where noted, we only show comparisons of data to predictions from the GENIE default model. Comparisons to GENIE + TEM and GENIE + MEC show qualitatively similar levels of agreement. Differences for specific distributions can be examined in terms of the \( \chi ^{2}\) test statistic values in Table 10.
For two track events, seven dynamic variables exist. These include properties of the long track that parallel the choices for one-track events, \(L_{21}\), \(\cos \theta _{21}\), and \(\sin \Theta _{21}\), similar quantities for the second track, \(L_{22}\), \(\cos \theta _{22}\), and \(\sin \Theta _{22}\), plus a quantity that describes the correlation between the two tracks in the event, which we take to be the difference in azimuthal variables \( \phi _{221}=\phi _{22}-\phi _{21}\). Since track 2 can exit the detector, the meaning of \(L_{22}\) and \(\sin \Theta _{22}\) differ somewhat from \(L_{21}\) and \(\sin \Theta _{21}\). Two other two-track correlated variables of interest, which are not independent, are the cosine of the opening angle,
$$\begin{aligned} \cos \Omega _{221}=\cos \theta _{21}\cos \theta _{22}+\sin \theta _{21}\sin \theta _{22}\cos \left( \phi _{22}-\phi _{21}\right) , \end{aligned}$$
(30)
and the cosine of the acoplanarity angle
$$\begin{aligned} \cos \theta _{A}&=\frac{\hat{s}_{21}\cdot \left( \hat{z}\times \hat{s} _{21}\right) }{\left| \hat{z}\times \hat{s}_{21}\right| } \end{aligned}$$
(31)
$$\begin{aligned}&=\sin \theta _{21}\sin \left( \phi _{22}-\phi _{21}\right) , \end{aligned}$$
(32)
with \(\hat{z}\) a unit vector in the neutrino beam direction and \(\hat{s}_{21} \) is a unit vector parallel to the first track direction at the event vertex. For the scattering of two initial state particles into two final state particles \(\left( 2\rightarrow 2\right) \), one expects from momentum conservation \(\phi _{221}=\pm \pi \) and \(\cos \theta _{A}=0\). Deviations of \( \phi _{221}\) from \(\pm \pi \) or of \(\cos \theta _{A}\) from 0 could be caused by undetected tracks in the final state, from NC events in the sample, or from effects of final state interactions in CC events.
The opening angle serves a useful role in identifying spurious two-track events that result from the tracking algorithm “breaking” a single track into two tracks, most commonly in cosmic ray events. Broken tracks produce values of \(\cos \Omega _{221}\) very close to \(-1\). Figures 17 and 18 show the distributions of (\(L_{21}\) and \(L_{22}\)) and (\(\cos \theta _{21}\) and \(\cos \theta _{22}\)) from the neutrino-enriched sample, compared to the GENIE default model. Figure 19 presents the distributions of \(\cos \theta _{21}\) using GENIE + MEC and GENIE + TEM models. This is the distribution where the agreement between data and GENIE default model, compared to the agreement between data and the other two models, is largest. Figures 20 and 21 show the distributions of (\(\sin \Theta _{21}\) and \(\sin \Theta _{22}\)) and (\(\phi _{22}-\phi _{21}\) and \(\cos \Omega _{221}\)) from the neutrino-enriched sample, compared to the GENIE default model. We have performed a test where we remove all events in the first bin of Fig. 21; we see no changes in the level of agreement between data or model in other kinematic distribution comparisons, and no statistically significant shifts in the observed multiplicity distributions.
For three-track events, eleven dynamic variables exist. A straightforward continuation of the previous choices leads to the choice of \(L_{31}\), \(\cos \theta _{31}\), \( \sin \Theta _{31}\), \(L_{32}\), \(\cos \theta _{32}\), \(\sin \Theta _{32}\), \(\phi _{32}-\phi _{31}\), \(L_{33}\), \(\cos \theta _{33}\), \(\sin \Theta _{33}\), and \( \phi _{33}-\phi _{31}\) as the eleven variables. Other azimuthal angle difference such as
$$\begin{aligned} \phi _{32}-\phi _{33}=\left( \phi _{32}-\phi _{31}\right) -\left( \phi _{33}-\phi _{31}\right) \end{aligned}$$
(33)
are not independent. Figures 22, 23, 24, 25, 26, and 27 show the distributions of (\( L_{31}\), \(L_{32}\), and \(L_{33}\)), (\(\cos \theta _{31}\), \(\cos \theta _{32}\), and \(\cos \theta _{33}\)), (\(\sin \Theta _{31}\), \(\sin \Theta _{32}\), and \(\sin \Theta _{33}\)), (\(\phi _{32}-\phi _{31}\) and \(\cos \Omega _{321}\)), (\( \phi _{33}-\phi _{31}\) and \(\cos \Omega _{331}\)), (\(\phi _{32}-\phi _{33}\) and \( \cos \Omega _{323}\)) from the neutrino-enriched sample, compared to the GENIE default model.
\(\chi ^{2}\) tests for kinematic distributions
We quantify agreement between model and observation through use of \(\chi ^{2}\) tests on the kinematic distributions described in Sect. 8.4. Ensemble tests have established the validity of the use of the \(\chi ^{2}\) criterion. We use only the “neutrino-enriched” sample of events in which the candidate muon passes both the PH and MCS tests. Data are binned into histograms, with a bin k for a variable \(x_{ij}\), \(d_{ijk}\), and compared to model predictions constructed by assuming that the number of events in a bin k of a variable \(X_{ij}\), \(m_{ijk}\), consists of contributions from neutrino and CR background contributions. We shorten the notation in Eq. 28 to
$$\begin{aligned} m_{ijk}=M_{\nu ,i}\hat{x}_{\nu ,ijk}+M_{CR,i}\hat{x}_{CR,ijk}, \end{aligned}$$
(34)
where \(M_{\nu ,i}\) and \(M_{CR,i}\) are the number of neutrino and CR events, respectively, predicted to be in the neutrino-enriched category for multiplicity i (as described in Sect. 8.2); and \(\hat{x}_{\nu ,ijk}\) and \(\hat{x}_{CR,ijk}\) the fraction of neutrino and CR events, respectively, falling in the k bin for variable \(x_{ij}\) as predicted by the GENIE model and the off-beam CR sample, respectively. The \(\hat{x}_{\nu ,ijk}\) and \(\hat{x}_{CR,ijk}\) are shape distributions normalized to one:
$$\begin{aligned} \sum _{k=1}^{bins}\hat{x}_{\nu ,ijk}=\sum _{k=1}^{bins}\hat{x}_{CR,ijk}=1. \end{aligned}$$
(35)
We then construct a \(\chi ^{2}\) for \(x_{ij}\) using a Poisson form appropriate for the low statistics in many bins:
$$\begin{aligned} \chi _{ij}^{2}=2\sum _{k=1}^{bins}\left( m_{ijk}-d_{ijk}-d_{ijk}\ln m_{ijk}+d_{ijk}\ln d_{ijk}\right) . \end{aligned}$$
(36)
Table 10 summarizes the results of these \(\chi ^{2}\) comparison tests for 21 independent kinematic variables to the three GENIE models. Only bins with at least one data event and one model event were used in the calculation of \(\chi ^{2}\). The number of degrees of freedom associated with the \(\chi ^{2}\) test was set equal to the number of bins used for that histogram minus one to account for the overall normalization adjustment. We note here that these tests for consistency are defined at the level of statistical uncertainties only; systematic uncertainties are not incorporated into the \(\chi ^{2}\) terms.
We summarize here salient features of Table 10 as follows: all three models consistently describe the data, with summed \(\chi ^{2}\) per degree-of-freedom (\(\chi ^{2}/DOF\)) of 228.1 / 216, 216.9 / 216, and 229.6 / 216, respectively, and corresponding p values of \( P_{\chi ^{2}}=27\%\), \(47\%\), and \(25\%\) for GENIE default, MEC, and TEM, respectively. The total \(\chi ^{2}\) after including all dynamic and non-dynamic variable distributions is 714 / 652. No tune of GENIE is superior to any other with any meaningful statistical significance for the distributions we have considered. The acceptable values of \(\chi ^{2}\) are consistent with the hypothesis that the combination of a GENIE event generator, the MicroBooNE BNB flux model, and the MicroBooNE GEANT-based detector simulation satisfactorily describe the properties of neutrino events examined in this analysis in a shape comparison. All elements of the MicroBooNE analysis chain thus appear to be performing satisfactorily; and no evidence exists for missing systematic effects that would produce data-model discrepancies outside the present level of statistics.
Aggregating Table 10 different ways uncovers no significant discrepancies. The \(\chi ^{2}\) tests on leading track \( \cos \theta \) and \(\sin \Theta \) yield satisfactory results for all multiplicities. Combined \(\chi ^{2}/DOF\) for all distributions associated with a particular multiplicity likewise exhibit adequate agreement. The most poorly described single distribution is that for the length of the muon candidate in multiplicity 1 events. The \(P_{\chi ^{2}}\), while acceptable, are \(16\%\) and \(8\%\) for the GENIE default and GENIE + MEC, respectively. The GENIE+TEM model has \(P_{\chi ^{2}}=53\%\).
The \(\chi ^{2}\) values for different distributions in a given multiplicity are calculated using the same events, which gives rise to concerns about correlations between different distributions. We have performed studies that verify that the \(\chi ^{2}\) values would be highly correlated if the model and data disagreed by an overall normalization, but that otherwise the \( \chi ^{2}\) tests on different distributions exhibit independent behavior, even when the same events are used. The \(P_{\chi ^{2}}\) values for different distributions do not cluster near 0 or 1, which is consistent with the view that the projections display approximately independent statistical behavior.
In summary, all GENIE models successfully describe, through \(\chi ^{2}\) tests, the shapes of a complete set of dynamically significant kinematic variables for observed charged particle multiplicity distributions 1, 2, and 3. The statistical power-the highest precision afforded by the available statistics with which the predictions can be tested-of these tests from the overall data statistics available corresponds to approximately \(4\%\), \(7\%\), and \(20\%\) for multiplicity 1, 2, and 3, respectively.
\(\chi ^{2}\) tests for multiplicity distribution
While we find satisfactory agreement between GENIE models and kinematic distribution shapes using \(\chi ^{2}\) tests that incorporate only statistical uncertainties, the situation differs for the overall multiplicity distribution. Here, we find statistical \(\chi _{M}^{2}/\)DOF\(=30/4\), 22 / 4, and 28 / 4 for the default, MEC, and TEM GENIE models, respectively. However, in the case of multiplicity, a significant systematic uncertainty exists for tracking efficiency that must be taken into account before any conclusion can be drawn.
We incorporate a tracking efficiency contribution to the \(\chi ^{2}\) test by defining
$$\begin{aligned} \chi _{M}^{2}&=\sum _{i=1}^{2}\frac{\left( D_{i}-K\hat{M}\left( \delta \right) _{i}\right) ^{2}}{\sigma _{i}^{2}}+\sum _{i=3}^{5}2\left[ K\hat{M}\left( \delta \right) _{i}\right. \nonumber \\&\quad \left. -D_{i}\ln \left( K\hat{M}\left( \delta \right) _{i}\right) -D_{i}+D_{i}\ln \left( D_{i}\right) \right] \nonumber \\&\quad +\sum _{i=1}^{5}2\left[ \hat{M}\left( 0\right) _{i}-M_{i}\ln \left( \hat{M}\left( 0\right) _{i}\right) -M_{i}+M_{i}\ln \left( M_{i}\right) \right] \nonumber \\&\quad +\left( \frac{\delta }{0.15}\right) ^{2}. \end{aligned}$$
(37)
Here \(D_{i}\) is the number of neutrino events estimated by the signal extraction procedure (Sect. 8.2), and \(\sigma _{i}\) is the estimated uncertainty on \(D_{i}\) using the signal extraction procedure. For multiplicity 3 and higher, the uncertainty on \(D_{i}\) is purely statistical as the CR background becomes negligible. The quantities \(M_{i}\) are the number of events in the MC sample with multiplicity i. Finite statistics in the MC sample are incorporated by interpreting the \(M_{i}\) as Poisson fluctuations about their true values \(\hat{M}\left( 0\right) _{i}\) in the third term of Eq. 37. This analysis does not absolutely normalize MC to data, hence the relative normalization of data to MC is allowed to float via the parameter K in the first term of Eq. 37. The normalization constant K, while not used directly in the model test, is consistent with the predicted value from the default GENIE model.
As discussed in Sect. 7.2, changing the per-track efficiency by a constant fraction \(\delta \) in the model would shift events between multiplicities according to
$$\begin{aligned} \hat{M}\left( \delta \right) _{4}&=\left[ \hat{M}\left( 0\right) _{4}\right] \left( 1-\delta \right) ^{3}, \end{aligned}$$
(38)
$$\begin{aligned} \hat{M}\left( \delta \right) _{3}&=\left[ \hat{M}\left( 0\right) _{3}+3\hat{ M}\left( 0\right) _{4}\delta \right] \left( 1-\delta \right) ^{2}, \end{aligned}$$
(39)
$$\begin{aligned} \hat{M}\left( \delta \right) _{2}&=\left[ \hat{M}\left( 0\right) _{2}+2\hat{ M}\left( 0\right) _{3}\delta +3\hat{M}\left( 0\right) _{4}\delta ^{2}\right] \left( 1-\delta \right) , \end{aligned}$$
(40)
$$\begin{aligned} \hat{M}\left( \delta \right) _{1}&=\left[ \hat{M}\left( 0\right) _{1}+\hat{M }\left( 0\right) _{2}\delta +\hat{M}\left( 0\right) _{3}\delta ^{2}+\hat{M} \left( 0\right) _{4}\delta ^{3}\right] . \end{aligned}$$
(41)
For the nominal model used in the MC simulation \(\delta =0\). As discussed in Sect. 7.2 we estimate the uncertainty on \(\delta \) to be \(15\%\), and we introduce this into \(\chi _{M}^{2}\) through the “pull term” \(\left( \delta /0.15\right) ^{2}\).
We minimize \(\chi _{M}^{2}\) with respect to the tracking efficiency pull parameter \(\delta \), the MC-to-data normalization K, and the five MC statistical quantities \(\hat{M}\left( 0\right) _{i}\), \(i=1-5\). This procedure yields
$$\begin{aligned} \chi _{M}^{2}/\text {DOF}&=6.4/3\text { (default), }4.3/3\text { (MEC), }5.8/3 \text { (TEM),} \end{aligned}$$
(42)
$$\begin{aligned} \delta&=0.32\text { (default)},0.27\text { (MEC), }0.32\text { (TEM).} \end{aligned}$$
(43)
We find that a satisfactory \(\chi ^{2}\) value can be obtained for the multiplicity distribution itself, albeit at the cost of a \(\approx 2\sigma \) pull in the parameter \(\delta \).