1 Introduction

All measurements of \(C\!P\) violation performed so far are consistent with the predictions of the Standard Model (SM) [1]. Nonvanishing \(C\!P\)-violating asymmetries have been observed in the decays of both K and \(B \) mesons [2]. In contrast, \(C\!P\) violation has not been observed in baryon decays, although some indications for nonvanishing \(C\!P\) asymmetries in \(b \)-flavoured baryon decays have been reported by the LHCb collaboration [3,4,5,6].

The abundant production of \({\varLambda } ^0_{b} \) and \({\varXi } ^0_{b} \) baryonsFootnote 1 in proton-proton collisions at the Large Hadron Collider (LHC) gives the LHCb experiment the opportunity to study multibody charmless decays of \(b \)-flavoured baryons. In particular, \({\varLambda } ^0_{b} \) and \({\varXi } ^0_{b} \) baryon decays to charmless four-body final states were observed by the LHCb collaboration and their branching fractions measured [7]. Their large yields enable measurements of \(C\!P\)-violating asymmetries to be performed with a precision at the level of a few percent.

This search follows the successful path of the observation of large \(C\!P\)-violating asymmetries in multibody charmless decays of charged and neutral \(B \) mesons by LHCb  [8,9,10,11]. These decays proceed simultaneously through the charged-current \({b} \rightarrow {u} \) transition and neutral-current \({b} \rightarrow {s},{d} \) transitions, and the resulting interference exhibits a weak-phase difference. Furthermore, and analogously to the aforementioned charmless multibody \(B \)-meson decays, charmless multibody decays of \(b \)-flavoured baryons contain rich resonance structures, both in the two- or three-body baryonic invariant-mass spectra (i.e. \(p K ^-\), \(p \pi ^-\), \(p \pi ^+\), \(p \pi ^+\pi ^-\) and \(p K ^-\pi ^+\)) and in the two- or three-body nonbaryonic ones (i.e. the \({\pi } ^+{\pi } ^-\), \({K} ^\pm \) \({\pi } ^\mp \), \({{K} ^+} {{K} ^-} \), \(\pi ^+\pi ^{-}\pi ^{\pm }\) and \(K^{\pm }\pi ^+\pi ^-\)). Consequently, \(C\!P\) asymmetries might be enhanced due to the strong-phase differences induced by the interference patterns between these transitions in the mass regions around resonances. The charmless \(b \)-baryon decays studied in this paper are hence well suited for a potential first observation of \(C\!P\) violation in the baryon sector. However, the presence of these strong phases, that are difficult to predict, would make a potential observation of \(C\!P\) violation difficult to interpret in terms of the weak phase of the Cabibbo-Kobayashi-Maskawa (CKM) quark-mixing matrix [12, 13].

This work focuses on a search for \(C\!P\) violation in \({X ^0_{b}} \rightarrow p h h 'h ''\) charmless decays, where \(X ^0_{b} \) stands either for \({\varLambda } ^0_{b} \) or \({\varXi } ^0_{b} \) and \(h^{(\prime ,\, \prime \prime )}\) stand either for a pion or a kaon. Six decays are studied, namely \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-\), \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\), \({{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+\pi ^-\), \({{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+K ^-\), \({{\varXi } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\) and \({{\varXi } ^0_{b}} \rightarrow p K ^-\pi ^+K ^-\). The \(C\!P\) asymmetry is defined as

$$\begin{aligned} {{\mathcal {A}}^{{C\!P}}} \equiv \frac{{\Gamma } ({X ^0_{b}} \rightarrow f) - {\Gamma }({\overline{X}{}^0_{b}} \rightarrow \overline{f})}{{\Gamma } ({X ^0_{b}} \rightarrow f) + {\Gamma }({\overline{X}{}^0_{b}} \rightarrow \overline{f})}, \end{aligned}$$
(1)

where \({\Gamma }({X ^0_{b}} \rightarrow f)\) is the partial width of the given decay. The \(C\!P\) asymmetry measurement relies on counting the number of reconstructed particle and antiparticle decays and includes therefore experimental charge-asymmetric effects such as the track detection efficiency or b-baryon production asymmetries. They are cancelled out to first order by comparing the \(C\!P\) asymmetries of the signal modes to those of charmed decays that lead to the same or very similar final states and for which no measurable \(C\!P\) violation is expected in the SM [14]. The decays \({{\varLambda } ^0_{b}} \rightarrow ({{\varLambda } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\), \({{\varLambda } ^0_{b}} \rightarrow ({{\varLambda } ^+_{c}} \rightarrow p \pi ^-\pi ^+)\pi ^-\) and \({{\varXi } ^0_{b}} \rightarrow ({{\varXi } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\) are thus reconstructed with the same selection as the corresponding charmless signals. The \(C\!P\)-violating observable considered in this work is then referred to as \({\varDelta {{\mathcal {A}}^{{C\!P}}}} \equiv {\mathcal{A}^{{C\!P}}_\mathrm{no\text {-}c}}- {\mathcal{A}^{{C\!P}}_\mathrm{c}} \), where \({\mathcal{A}^{{C\!P}}_\mathrm{no\text {-}c}} \) (\({\mathcal{A}^{{C\!P}}_\mathrm{c}} \)) is the asymmetry measured in the charmless (charmed) decays. The decays of interest are reported in Table 1.

Table 1 Four-body charmless and charmed decays considered in this analysis. The difference of \(C\!P\)-asymmetries measured for the charmless modes and for the control channels results in \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) measurements. For each observable, the choice of the control channel is aiming at cancelling at first order production and detection asymmetries. Given the data samples at hand, it is not possible to meet both criteria for the signal decay \({{\varXi } ^0_{b}} \rightarrow p K ^-\pi ^+K ^-\): the choice of the Cabibbo-favoured decay \({{\varXi } ^0_{b}} \rightarrow ({{\varXi } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\) as a control channel requires in turn to correct the corresponding \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) for the kaon-detection asymmetry

In addition to \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) measurements integrated over all of the four-body phase space, specific regions of the space are studied in order to search for local \(C\!P\) asymmetries.

The same final states have been used by the LHCb experiment to search for \(C\!P\) violation using triple product asymmetries [5, 15]. The latter technique and the \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) measurements exhibit different sensitivity to \(C\!P\) violation [16], which makes the two approaches complementary.

2 Detector and data set

The analysis is performed using pp collision data recorded with the LHCb detector, corresponding to an integrated luminosity of 1.0\(\text { fb} ^{-1}\) at a centre-of-mass energy of 7\(\text { TeV}\) in 2011 and 2.0\(\text { fb} ^{-1}\) at a centre-of-mass energy of 8\(\text { TeV}\) in 2012. The LHCb detector [17, 18] is a single-arm forward spectrometer covering the pseudorapidity range \(2<\eta <5\), designed for the study of particles containing \(b \) or \(c \) quarks. The detector includes a high-precision tracking system consisting of a silicon-strip vertex detector surrounding the pp interaction region, a large-area silicon-strip detector located upstream of a dipole magnet with a bending power of about \(4{\mathrm {\,Tm}}\), and three stations of silicon-strip detectors and straw drift tubes placed downstream of the magnet. The tracking system provides a measurement of the momentum, \(p\), of charged particles with a relative uncertainty that varies from 0.5% at low momentum to 1.0% at 200\(\text { GeV/}c\). The minimum distance of a track to a primary vertex (PV), the impact parameter (IP), is measured with a resolution of \((15+29/p_{\mathrm {T}})\,\upmu \text {m} \), where \(p_{\mathrm {T}}\) is the component of the momentum transverse to the beam, in \(\text { GeV/}c\). Different types of charged hadrons are distinguished using information from two ring-imaging Cherenkov detectors. Photons, electrons and hadrons are identified by a calorimeter system consisting of scintillating-pad and preshower detectors, an electromagnetic and a hadronic calorimeter. Muons are identified by a system composed of alternating layers of iron and multiwire proportional chambers.

Simulation is used to investigate backgrounds from other \(b \)-hadron decays and also to study the detection and reconstruction efficiencies of the signals. In the simulation, pp collisions are generated using Pythia  [19, 20] with a specific LHCb configuration [21]. Decays of hadronic particles are described by EvtGen  [22] in which final-state radiation is generated using Photos  [23]. The interactions of the generated particles with the detector, and its response, are implemented using the Geant4 toolkit [24, 25] as described in Ref. [26].

3 Trigger and selection requirements

The selection follows most of the strategy described in Ref. [7]. The online event selection is performed by a trigger[27] that consists of a hardware stage, based on information from the calorimeter and muon systems, followed by a software stage, in which all charged particles with \(p_{\mathrm {T}} >500\,(300)\text { MeV/}c \) are reconstructed for 2011 (2012) data. At the hardware-trigger stage, events are required to include a muon or a dimuon with high transverse momentum or a hadron, photon or electron with high transverse energy. The software trigger reconstructs charged particles with transverse momentum \(p_{\mathrm {T}} >500\,(300)\text { MeV/}c \) for 2011 (2012) data and requires a two-, three- or four-track secondary vertex with significant displacement from all primary pp interaction vertices. At least one charged particle must have transverse momentum \(p_{\mathrm {T}} > 1.7\,(1.6)\text { GeV/}c \) for 2011 (2012) data and be inconsistent with originating from any PV. A multivariate algorithm [28] is used for the identification of secondary vertices consistent with the decay of a \(b \) hadron. In the offline selection, trigger signals are associated with reconstructed particles. Selection requirements can therefore be made on the trigger selection itself and on whether the decision was due to the signal candidate, other particles produced in the pp collision, or a combination of both.

The events passing the trigger requirements are filtered in two stages. Initial requirements are applied to further reduce the size of the data sample before a multivariate selection is applied. Selection requirements based on topological variables, such as the flight distance of the \(b \) baryon, are used as the main discriminants. In order to preserve the phase space of the decays of interest, only loose requirements are placed on the transverse momenta of the decay products, \(p_{\mathrm {T}} >250\text { MeV/}c \).

Neutral \(b \)-baryon candidates, hereafter denoted as \(X ^0_{b} \), are formed from a proton candidate selected with particle identification (PID) requirements and three additional charged tracks. When more than one PV is reconstructed, the \(X ^0_{b} \) candidate is associated to the PV with the smallest value of \(\chi ^2_{\text {IP}} \), where \(\chi ^2_{\text {IP}} \) is the difference in \(\chi ^2 \) of a given PV reconstructed with and without the considered candidate. Each of the four tracks of the final state is required to have \(\chi ^2_{\text {IP}} > 16\) and \(3<p <100\text { GeV/}c \). Beyond 100 \(\text { GeV/}c\), there is little pion/kaon/proton discrimination. The \(X ^0_{b} \) candidates are then required to form a vertex with a fit quality \(\chi ^2_{\text {vtx}} < 20\) and to be significantly separated from any PV with \({\chi ^2_\mathrm{FD}} > 50\), where \({\chi ^2_\mathrm{FD}} \) is the square of the flight-distance significance. To remove backgrounds from higher-multiplicity decays, the difference in \(\chi ^2_{\text {vtx}}\) when adding any other track must be greater than 4. The \(X ^0_{b} \) candidates must have a transverse momentum \(p_\mathrm{T}({X ^0_{b}})\) greater than \(1.5\text { GeV/}c \) and an invariant mass within the range \(5340< m(phh^{\prime }h^{\prime \prime }) < 6400 \text { MeV/}c^2 \). They are further required to be consistent with originating from a PV, quantified by both \(\chi ^2_{\text {IP}} < 16\) and the cosine of the angle \(\theta _\mathrm{DIR}\) between the reconstructed momentum of the b hadron and the vector defined by the associated PV and the decay vertex be greater than 0.999. Finally, PID requirements are applied to provide discrimination between kaons and pions in order to assign the candidates to one of the five different final-state hypotheses \(p \pi ^-\pi ^+\pi ^-\), \(p K ^-\pi ^+\pi ^-\), \(p K ^-K ^+\pi ^-\), \(p K ^-\pi ^+K ^-\) and \(p K ^-K ^+K ^-\).

There are three main categories of background that contribute significantly in the selected invariant-mass regions: the so-called signal cross-feed background, resulting from a misidentification of one or more final-state particles in a charmless baryon decay, which can therefore be reconstructed as another charmless decay with a different mass hypothesis; the charmless decays of neutral \(B \) mesons to final states containing four charged mesons, where a pion or a kaon is misidentified as a proton; and the combinatorial background, which results from a random association of unrelated tracks. The pion and kaon PID requirements, that define mutually exclusive samples, are optimised to reduce the cross-feed background, and hence to maximise the significance of the signal. The charmless \(B \)-meson decays are identified by reconstructing the invariant-mass distributions of candidates using the pion or kaon mass instead of the proton mass hypothesis, in the high-mass sidebands defined as \(m_\mathrm{sideband}< m(phh^{\prime }h^{\prime \prime }) < 6400 \text { MeV/}c^2 \), where \(m_\mathrm{sideband} = 5680 \text { MeV/}c^2 \) for \(p \pi ^-\pi ^+\pi ^-\) and \(p K ^-K ^+\pi ^-\) final states, and \(m_\mathrm{sideband} = 5840 \text { MeV/}c^2 \) for \(p K ^-\pi ^+\pi ^-\), \(p K ^-\pi ^+K ^-\) and \(p K ^-K ^+K ^-\) final states. This background contribution is reduced by the optimisation of the proton PID requirement.

To reject combinatorial background, multivariate discriminants based on a boosted decision tree (BDT) [29] with the AdaBoost algorithm [30] have been designed. Candidates from simulated \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-\) decays and the high-mass sideband are used as the signal and background training samples, respectively. This high-mass sideband region is chosen such that the sample is free of cross-feed background. The samples are divided into two data-taking periods and further subdivided into two equally sized subsamples. Each subsample is then used to train an independent discriminant. The BDT trained on one subsample is used to select candidates from the other subsample, in order to avoid a possible bias in the selection.

The BDTs have the following quantities as inputs: \(p_{\mathrm {T}}\), \(\eta \), \(\chi ^2_{\text {IP}}\), \(\chi ^2_\mathrm{FD}\), \(\cos \theta _\mathrm{DIR}\), and \(\chi ^2_{\text {vtx}}\) of the \(X ^0_{b} \) candidate; the smallest change in the \(b \)-baryon \(\chi ^2_{\text {vtx}}\) when adding any other track from the event; the sum of the \(\chi ^2_{\text {IP}}\) of the four tracks of the final state; and the \(p_{\mathrm {T}}\) asymmetry

$$\begin{aligned} p_\mathrm{T}^{\mathrm{asym}} = \frac{p_\mathrm{T}({X ^0_{b}}) - p_\mathrm{T}^\mathrm{cone}}{p_\mathrm{T}({X ^0_{b}}) + p_\mathrm{T}^\mathrm{cone}} \,, \end{aligned}$$
(2)

where \(p_\mathrm{T}^\mathrm{cone}\) is the transverse component of the vector sum of all particle momenta inside a cone around the \(b \)-baryon candidate direction, of radius \(R \equiv \sqrt{\delta \eta ^2 +\delta \phi ^2} =1.5\), where \(\delta \eta \) and \(\delta \phi \) are the difference in pseudorapidity and azimuthal angle (expressed in radians) around the beam direction, between the momentum vector of the track under consideration and that of the b-hadron candidate. The distribution of \(p_\mathrm{T}^{\mathrm{asym}}\) for the signal candidates is enhanced towards high values. The BDT output is determined to be uncorrelated with the position in phase space of the decays of interest. The selection requirement placed on the output of the BDTs is optimised for the six decays of interest by minimising the uncertainties on the \(C\!P\)-asymmetry differences.

A number of background contributions consisting of fully reconstructed \(b \)-baryon decays into the two-body \({{\varLambda } ^+_{c}} h \), \({{\varXi } ^+_{c}} h \), three-body \({D} p h \) or \(({c} {\overline{{c}}}) p h \) combinations, where \(({c} {\overline{{c}}})\) represents a charmonium resonance, may produce the same final state as the signal. Hence, they have similar invariant-mass distribution of the \(b \)-baryon candidate as the signal along with a similar selection efficiency. The presence of a misidentified hadron in the \(D \), \({\varLambda } ^+_{c} \) and \({\varXi } ^+_{c} \) decay also produces peaking background under the signal. Therefore, the following decay channels are explicitly reconstructed under the relevant particle hypotheses and vetoed by means of a requirement on the resulting invariant mass, in all spectra: \({{\varLambda } ^+_{c}} \rightarrow pK^-\pi ^+\), \({{\varLambda } ^+_{c}} \rightarrow p\pi ^+\pi ^-\), \({{\varLambda } ^+_{c}} \rightarrow pK^+K^-\), \({{\varXi } ^+_{c}} \rightarrow pK^-\pi ^+\), \({{D} ^+} \rightarrow K^-\pi ^+\pi ^+\), \({{D} ^+_{s}} \rightarrow K^-K^+\pi ^+\), \({{D} ^0} \rightarrow K^{-}\pi ^{+}\), \({{D} ^0} \rightarrow \pi ^+\pi ^-\), \({{D} ^0} \rightarrow K^+K^-\), \({{J /\psi }} \rightarrow \pi ^+\pi ^-\) and \({{J /\psi }} \rightarrow K^+K^-\). The decays of other possible broad charmonium resonances to \(\pi ^+\pi ^-\) and \(K^+K^-\) are retained as potential interfering amplitudes with the charmless amplitudes under study.

The same set of trigger, \(\mathrm {PID}\) and BDT requirements is applied to the control modes \({{\varLambda } ^0_{b}} \rightarrow ({{\varLambda } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\), \({{\varLambda } ^0_{b}} \rightarrow ({{\varLambda } ^+_{c}} \rightarrow p \pi ^-\pi ^+)\pi ^-\) and \({{\varXi } ^0_{b}} \rightarrow ({{\varXi } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\) to cancel out most of the systematic effects related to the selection criteria. Candidates whose \(pK^-\pi ^+\) or \(p\pi ^-\pi ^+\) invariant mass is in the range \([2213, 2313] \text { MeV/}c^2 \) for \({{\varLambda } ^+_{c}} \) and \([2437, 2497] \text { MeV/}c^2 \) for \({{\varXi } ^+_{c}} \), are retained as control channels candidates. Events outside these intervals belong to the corresponding signal spectrum, again ensuring statistically independent samples for the simultaneous fit.

The fraction of events containing more than one candidate is below the percent level. The candidate to be retained in each event is chosen randomly and reproducibly.

4 Simultaneous fit

A simultaneous unbinned extended maximum-likelihood fit is performed to the invariant-mass distributions of the \({b} \)-hadron candidates under each of the mass hypotheses for the signal and control channel final-state tracks. The data samples are split according to the charge of the proton and to the year of data taking. Furthermore, data are split according to the hardware trigger conditions, in order to correct raw measurements for charge-asymmetric trigger efficiencies. The components of the model include, in addition to signal decays, partially reconstructed five-body \(X ^0_{b} \) decays, signal and background cross-feeds, four- and five-body decays of \(B \) mesons and combinatorial background. The independent data samples obtained for each final state are fitted simultaneously. For each sample, the likelihood is expressed as

$$\begin{aligned} \displaystyle \ln \mathcal{L} = \sum _{i} \ln \left( {\sum _{j} N_j P_{j,i}} \right) -\sum _{j}N_j \end{aligned}$$
(3)

where \(N_{j}\) is the number of events related to the component j and \(P_{j,i}\) is the probability distribution function for component j evaluated at the mass of the candidate i.

4.1 Fit model

The signal decays are modelled as the sum of two Crystal Ball (CB) functions [31] that share peak positions and widths but have independent power-law tails on opposite sides of the peak. The \({\varLambda } ^0_{b} \) mass parameter is free in the fit and shared among the \({\varLambda } ^0_{b} \) decays. The difference between the fitted \({\varXi } ^0_{b} \) and \({\varLambda } ^0_{b} \) masses is also a shared parameter and is constrained to the value reported in Ref. [2] by using a Gaussian function.

The width parameter for \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\) decays measured in the 2012 data-taking sample is found to be \(16.47 \pm 0.22 \text { MeV/}c^2 \) and is chosen as reference. The ratio of the experimental widths of the signal decay functions is constrained using Gaussian prior probability distributions multiplying the likelihood function, with parameters obtained from a fit to simulated events. The other parameters of the CB components are obtained from a simultaneous fit to simulated samples, and are fixed to those values in the fits to the data.

The cross-feed backgrounds are modelled by the sum of two CB functions, whose parameters are determined from simulated samples weighted to match the performances of the particle identification algorithm as measured in the data. All cases resulting from the misidentification of either one or two of the final-state particles are considered. The yield of each misidentified decay is constrained to the yield of the corresponding correctly identified decay and the known misidentification probabilities. These constraints are implemented using Gaussian prior probability distributions multiplying the likelihood function. Their mean values are obtained from the ratio of selection efficiencies and their widths include uncertainties originating from the finite size of the simulated events samples as well as the systematic uncertainties related to the determination of the PID efficiencies.

The backgrounds resulting from four- or five-body decays of \(B \) mesons are identified in each spectrum by a dedicated fit to the candidates in the high-mass sideband, reconstructed under the hypothesis of the kaon mass for the proton candidates. The relative yield of each decay is then constrained in the simultaneous fit from its observed abundance in the high-mass sidebands. The invariant-mass distributions are modelled by the sum of two CB functions, whose parameters are determined from simulation.

Partially reconstructed backgrounds where a neutral pion is not reconstructed, such as \({{\varLambda } ^0_{b}}, {{\varXi } ^0_{b}} \rightarrow phh^{\prime }h^{\prime \prime } {{\pi } ^0} \), are modelled by means of generalised ARGUS functions [32] convolved with a Gaussian resolution function. The Gaussian width is taken as the signal \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\) width parameter. The parameters of the ARGUS function are shared among all invariant-mass spectra and are determined directly from the fit, except for the threshold, which is given by \(m(X_b) - m(\pi ^0)\). Partially reconstructed decays with a missing photon such as \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\eta ^{\prime } \) and \({{\varLambda } ^0_{b}} \rightarrow p K ^-\eta ^{\prime } \) decays, with \(\eta ^{\prime } \rightarrow \pi ^+\pi ^-\gamma \), are modelled separately using the same functional form but where the parameters are fixed from simulation. The \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\pi ^0\) decay modes where a charged pion is misidentified as a kaon can significantly contribute to the \(p K ^-K ^+\pi ^-\) and \(p K ^-\pi ^+K ^-\) spectra. They are modelled with an empirical function determined from the partially reconstructed background candidates in the control channel.

Finally, the combinatorial background is modelled by a linear function whose slope is shared among the invariant-mass spectra.

4.2 The ensemble of measurements

The following three categories of measurements have been considered a priori (before any evaluation of the data) to search for global and local effects of \(C\!P\) violation.

  • \(C\!P\) asymmetries are measured, considering the whole selected phase space of the decay candidates.

  • \(C\!P\) asymmetries are also measured in the phase-space region of low invariant mass on the baryonic pair (i.e. \(p\pi ^{\pm }\) or \(pK^-\)) and low invariant mass on the pairing of the two other tracks. The ensemble of measurements that are performed with this phase-space selection is hereafter referred to as LBM (Low \(2\!\times \!2\)-Body Mass) measurements. The invariant mass of the baryonic pair is required to be lower than 2 \(\text { GeV/}c^2\) while the invariant-mass requirements on the two remaining tracks depends on whether it is a \(\pi ^+\pi ^-\) pair, a \(K^{\pm }\pi ^{\mp }\) or a KK pair. These values are chosen to include several known resonances, in particular \(f_0(1500)\) resonance for \(\pi ^+\pi ^-\), the broad scalar \(K_0^*(1430)^0\) resonance for \(K^+\pi ^-\) and the \(f_2^{\prime }(1525)\) resonance for \(K^+K^-\). Only the modes with the largest signal yields are considered, namely \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-\), \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\) and \({{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+K ^-\) decays. The two-body low-mass distributions are displayed in Fig. 1. Several resonant structures are observed, and correspond to baryon resonances like \(\varLambda (1520)\), \(\varDelta (1232)^{++}\) and N(1520) or meson resonances like \(K^*(892)^0\), \(\rho (770)^0 \) or \(\phi (1020) \). This phase-space selection focuses therefore on low-invariant-mass resonances (both mesonic and baryonic) as well as low-invariant-mass nonresonant components of the amplitudes. The latter have been shown to generate large \(C\!P\)-violating asymmetries in analogous \(B \)-meson decays [8].

  • \(C\!P\) asymmetries are measured for regions of the phase space that contain specific quasi-two-body decays, \({{{{\varLambda } ^0_{b}} \rightarrow p {{a_{1}}} (1260)^-}}\), \({{{{\varLambda } ^0_{b}} \rightarrow N(1520)^0\rho (770)^0}}\), \({{{{\varLambda } ^0_{b}} \rightarrow p {{K_{1}}} (1410)^-}}\), \({{{{\varLambda } ^0_{b}} \rightarrow \varLambda (1520)\rho (770)^0}}\), \({{{{\varLambda } ^0_{b}} \rightarrow N(1520)^0{{K} ^*(892)^{0}}}}\), \({{{{\varLambda } ^0_{b}} \rightarrow \varLambda (1520)\phi (1020)}}\) or three-body decays, \({{{{\varLambda } ^0_{b}} \rightarrow \varDelta (1232)^{++}\pi ^{-}\pi ^{-}}}\), \({{{{\varLambda } ^0_{b}} \rightarrow \varDelta (1232)^{++}K ^{-}\pi ^{-}}}\), and \({{{{\varLambda } ^0_{b}} \rightarrow (p K ^{-})_\text {high-mass} \phi (1020)}}\), where in the latter only the high \(pK^{-}\) mass region is selected. Invariant-mass requirements for these measurements are reported in Table 2. Only the narrower baryons or the well-known baryon and meson resonances have been considered, with the noticeable exception of the \({{a_{1}}} (1260)\) meson. Although the \({{a_{1}}} (1260)\) meson is a broad resonance, the analogous \(B \)-meson decay \({B} ^0 \rightarrow {{a_{1}}} (1260)^{\pm } \pi ^{\mp }\) has been studied at the \(B \)-factories [33, 34] and could serve as a benchmark comparison in the interpretation of the results obtained for the \({{\varLambda } ^0_{b}} \rightarrow p {{a_{1}}} (1260)^-\) decay.

Fig. 1
figure 1

Distributions of invariant masses of pairs of final-state particles for the candidates selected in the mass window of \(\pm 3\sigma \) around the measured \({\varLambda } ^0_{b} \) mass. ac show the two-body invariant-mass distributions of baryonic \(p\pi ^-\), \(p\pi ^+\) pairs from \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\) decays and \(pK^-\) pairs from \({{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+K ^-\) decays, respectively. Structures around known the masses of the N(1520), \(\varDelta (1232)^{++}\) and \(\varLambda (1520)\) baryons are observed. df show the invariant-mass distributions of \(K^-\pi ^+\), \(K^-K^+\) and \(\pi ^+\pi ^-\) pairs from \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\), \({{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+K ^-\) and \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-\) decays, respectively. Structures corresponding to the \(K^*(892)^0\), \(\phi (1020) \) and \(\rho (770)^0 \) resonances are visible. The red lines correspond to the invariant-mass requirements applied to the selection of the main quasi-two- or three-body decays analysed

Table 2 Invariant-mass requirements applied for the different phase-space selections for each final state considered

5 Corrections for experimental detection asymmetries and related systematic uncertainties

Tracking reconstruction, trigger selection and particle identification requirements can generate charge-dependent selection efficiencies of the decays of interest. Most of these charge-dependent effects are however cancelled out in the \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) observables, up to the kinematical differences between signal and control channels. The remaining impact is addressed by evaluating corrections to the \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) observables. These correction factors are determined from calibration samples as discussed in this Section. Systematic uncertainties are estimated for each correction factor and propagated to the \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) measurements. A summary of the systematic uncertainties is reported in Table 3 for all modes.

Table 3 Systematic uncertainties for each decay mode. The uncertainties related to the kaon and proton detection asymmetry, the difference of triggering efficiency, the PID asymmetries and the production asymmetry are respectively reported as \(\sigma _K\), \(\sigma _p\), \(\sigma _\mathrm{L0}\), \(\sigma _\mathrm{PID}\) and \(\sigma _{A_P}\)
  • Tracking detection efficiency: differences between the interactions of oppositely charged pions, kaons or protons in the material of the spectrometer induce detection charge asymmetries. The difference in \(\pi ^{\pm }\) and \(K^{\pm }\) tracking efficiency has been quantified with calibration samples, as a function of the transverse momentum of the tracks [35, 36]. The simulated signal and control channels kinematics is used to weight the simulation track efficiency in order to match the hadron detection efficiencies as measured in those calibration samples. The \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) values (as measured by the difference of signal yields) are then corrected for these efficiencies and the uncertainty on the detection efficiency determination itself is propagated as a systematic uncertainty to the final \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) measurements, taking into account the correlation between signal and control channel induced by the use of the same calibration samples. The systematic uncertainty arises from the size of the simulated samples used in the weighting, the statistical and systematic uncertainties on the charge asymmetry of the data calibration samples and the knowledge of the kinematical distributions generated in the simulated samples. The latter is determined by taking two different kinematic configurations of the final state (saturated by quasi two-body modes on one hand and phase-space decay on the other hand) and using the difference as the systematic estimate. The difference between the \(p\) and \(\overline{p}\) particles is not measured to date. Simulation is used to obtain the reconstruction efficiencies as a function of the momentum of the proton or antiproton track. An additional systematic uncertainty related to the knowledge of the material budget in the simulation is added, as reported in Ref. [37]. The proton detection correction follows the same procedure as \(\pi ^{\pm }\) and \(K^{\pm }\) detection asymmetry correction.

  • The same methodology is used to correct for the difference of triggering efficiency between oppositely charged hadrons of the signal candidate, at the hardware stage of the trigger system. The trigger asymmetry effects are quantified as a function of the transverse momentum of the tracks of interest, by studying the triggering efficiency of \(K^-\) and \(\pi ^+\) from the decay \(D^0 \rightarrow K^-\pi ^+\) [27] and protons from \({{\varLambda } ^0_{b}} \rightarrow ({{\varLambda } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\) decays.

  • The production asymmetry can depend on the kinematical properties of the reconstructed \(X ^0_{b} \) candidates, though the actual dependence has not been observed yet [38]. Differences between signal and control channel \(X ^0_{b} \) candidates kinematics would reflect in an incomplete cancellation of the production asymmetry in the \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) observable. This effect has been estimated by considering the \({\varLambda } ^0_{b} \) production asymmetry measured in Ref. [38] as a function of its \(p_{\mathrm {T}}\) and pseudorapidity.

  • The PID requirements set on the tracks of the final state can induce asymmetries. Efficiencies for the final-state particles are determined from \({\varLambda } ^+_{c} \) decays selected in data, and are parameterised by their momentum and electric charge. The correction factors to apply to the value of \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) are here again determined by performing a weighting of the simulated signal and control channel events to match the efficiencies measured in the data. The uncertainties coming from the finite size of the calibration samples are propagated as a systematic uncertainty for the final \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) measurements.

The first three corrections on the value of \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) are found to be at the few per mille level, commensurate with their uncertainties. The lattermost source is dominating the systematic uncertainty budget, and can reach the percent level. The correction factors are however consistent with zero. The design of the fit model and the simultaneous fit strategy allow the direct measurement of the combinatorial background and the \(B \)-meson decay asymmetries. No significant asymmetries are observed and the results are presented in Section 7. Systematic uncertainties can be induced by the fit model and the fit complexity and it is evaluated by means of pseudoexperiments reproducing the nominal fit results. No significant biases are obtained. The normalised residuals of the signal yields are computed and the uncertainties on their pull mean value are propagated as a systematic uncertainty to each relevant \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) measurement. The largest uncertainty is determined to be at the level of few \(10^{-4}\), hence negligible in comparison to the aforementioned systematic uncertainty estimate.

6 Fit results

The results of the simultaneous fits to the five experimental spectra split by year of data taking, magnet polarity and trigger conditions are discussed in this section. The fit results are reported for each final state in the following subsections, and the summary of the measured yields is reported in Table 4.

Table 4 Signal yields for each decay mode, summed over all trigger configurations and years of data taking
  • \(\varvec{p \pi ^-\pi ^+\pi ^-}\) final state: Figures 2 and 3 show the results of the simultaneous fits to the invariant-mass spectra of the \(p \pi ^-\pi ^+\pi ^-\) spectra for the inclusive, LBM and quasi two-body measurements. The high-mass region of the \(p \pi ^-\pi ^+\pi ^-\) spectrum is only populated by either \(B \)-meson decays or combinatorial background. The good agreement between the data and the fit model, especially in this region, validates the chosen modelling of these components. The same comment is in order for the fit in the different phase-space regions. The combinatorial component becomes negligible in the quasi two-body case.

  • \(\varvec{p K ^-\pi ^+\pi ^-}\) final state: Figures 4 and 5 show the results of the simultaneous fits to the \(p K ^-\pi ^+\pi ^-\) mass spectrum for the inclusive, LBM and quasi two-body measurements. The fit model provides also in this case a satisfactory description of the data, despite the very different background contributions depending on the phase-space selection. Raw asymmetries at the level of several percent are observed.

  • \(\varvec{p K ^-K ^+K ^-}\) final state: Figure 6 shows the results of the simultaneous fits to the reconstructed \(p K ^-K ^+K ^-\) mass spectrum for the inclusive, LBM and quasi two-body measurements. Negligible raw asymmetries are obtained.

  • \(\varvec{p K ^-K ^+\pi ^-}\) and \(\varvec{p K ^-\pi ^+K ^-}\) final states: The simultaneous fit results for the two remaining final states are shown in Fig. 7. The result of the fit for the control channel \({{\varXi } ^0_{b}} \rightarrow ({{\varXi } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\) is also displayed and shows a good description of the spectrum. This control channel is used to account for the production asymmetry of the \({\varXi } ^0_{b} \) modes.

Fig. 2
figure 2

Invariant \(p \pi ^-\pi ^+\pi ^-\) mass distributions with the results of the fit superimposed: (first row) full phase space, (second row) LBM and (third row) \({{\varLambda } ^0_{b}} \rightarrow ({{\varLambda } ^+_{c}} \rightarrow p \pi ^-\pi ^+)\pi ^-\) control channel. The two columns correspond to the charge-conjugate final states: (left) baryon, (right) antibaryon. The different components employed in the fit model are indicated in the legends. The \({{\varLambda } ^0_{b}} \rightarrow \) five-body legend includes two components: the partially reconstructed \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\eta ^{\prime } \) and \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-\pi ^0\) decays where a \(\gamma \) or \(\pi ^0\) is not reconstructed. The latter has a lower-mass endpoint

Fig. 3
figure 3

Invariant \(p \pi ^-\pi ^+\pi ^-\) mass distributions with the results of the fit superimposed: region of the phase space containing (first row) \({{\varLambda } ^0_{b}} \rightarrow p {{a_{1}}} (1260)^-\), (second row) \({{\varLambda } ^0_{b}} \rightarrow \varDelta (1232)^{++}\pi ^{-}\pi ^{-}\) and (third row) \({{\varLambda } ^0_{b}} \rightarrow N(1520)^0\rho (770)^0 \) quasi two-body decays. The two columns correspond to the charge-conjugate final states: (left) baryon, (right) antibaryon. The different components employed in the fit model are indicated in the legends. The \({{\varLambda } ^0_{b}} \rightarrow \) five-body legend includes two components: the partially reconstructed \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\eta ^{\prime } \) and \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-\pi ^0\) decays where a \(\gamma \) or \(\pi ^0\) is not reconstructed. The latter has a lower-mass endpoint

Fig. 4
figure 4

Invariant \(p K ^-\pi ^+\pi ^-\) mass distributions with the results of the fit superimposed: (first row) full phase space, (second row) LBM and (third row) \({{\varLambda } ^0_{b}} \rightarrow ({{\varLambda } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\) control channel. The two columns correspond to the charge-conjugate final states: (left) baryon, (right) antibaryon. The different components employed in the fit are indicated in the legends. The \({{\varLambda } ^0_{b}} \rightarrow \) five-body legend includes two components: the partially reconstructed \({{\varLambda } ^0_{b}} \rightarrow p K ^-\eta ^{\prime } \) and \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\pi ^0\) decays where a \(\gamma \) or \(\pi ^0\) is not reconstructed. The latter has a lower-mass endpoint

Fig. 5
figure 5

Invariant \(p K ^-\pi ^+\pi ^-\) distributions, with the results of the fit superimposed: region of the phase space containing (first row) \({{\varLambda } ^0_{b}} \rightarrow p {{K_{1}}} (1410)^-\), (second row) \({{\varLambda } ^0_{b}} \rightarrow \varDelta (1232)^{++}K ^{-}\pi ^{-}\), (third row) \({{\varLambda } ^0_{b}} \rightarrow \varLambda (1520)\rho (770)^0 \) and (last row) \({{\varLambda } ^0_{b}} \rightarrow N(1520)^0{{K} ^*(892)^{0}} \) quasi two-body decays. The two columns correspond to the charge-conjugate final states: (left) baryon, (right) antibaryon. The different components employed in the fit are indicated in the legends

Fig. 6
figure 6

Invariant \(p K ^-K ^+K ^-\) mass distributions, with the results of the fit superimposed: (first row) full phase space and (second row) LBM, (third row) \({{\varLambda } ^0_{b}} \rightarrow \varLambda (1520)\phi (1020) \) and (fourth row) \({{\varLambda } ^0_{b}} \rightarrow (p K ^{-})_\text {high-mass} \phi (1020) \). The two columns correspond to the charge-conjugate final states: (left) baryon, (right) antibaryon. The different components employed in the fit are indicated in the legends. The \({{\varLambda } ^0_{b}} \rightarrow \) five-body legends includes two decays: partially reconstructed \({{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+K ^-{\gamma } \) and \({{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+K ^-\pi ^0\), where the \(\gamma \) and \(\pi ^0\) are not reconstructed

Fig. 7
figure 7

Invariant (first row) \(p K ^-K ^+\pi ^-\) and (second row) \(p K ^-\pi ^+K ^-\) mass distributions, with the results of the fit superimposed. The two bottom plots are the results of the fit to the \({{\varXi } ^0_{b}} \rightarrow ({{\varXi } ^+_{c}} \rightarrow p K ^-\pi ^+)\pi ^-\) control channel. The two columns correspond to the charge-conjugate final states: (left) baryon, (right) antibaryon. The different components employed in the fit are indicated in the legends. The \({{\varLambda } ^0_{b}} \rightarrow \) five-body legend includes two components where a \(\pi ^0\) is not reconstructed: the partially reconstructed background \({{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-\pi ^0\) where a pion is misidentified as a kaon and the partially reconstructed background \({{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+\pi ^-\pi ^0\)

7 Measurements of \(C\!P\) asymmetries and concluding remarks

Five charmless final states of \({\varLambda } ^0_{b} \) and \({\varXi } ^0_{b} \) four-body hadronic decays are examined in this paper. Specific regions of their phase space have been selected to search for local \(C\!P\) asymmetries in addition to the integrated \(C\!P\)-asymmetry. A total of eighteen measurements of \(C\!P\) asymmetries are reported in this paper.

A simple counting experiment allows the measurement of a \(C\!P\) asymmetry up to the corrections due to instrumental and \(b \)-baryon production asymmetries. These corrections are mitigated by establishing the differences (denoted \(\varDelta {{\mathcal {A}}^{{C\!P}}} \)) between the raw \({\mathcal {A}}^{{C\!P}}\) values of the signals and those of the decay modes with intermediate charmed baryons comprising the same final-state particles. The asymmetries \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) are further corrected for residual experimental charge asymmetries due to kinematic differences between signal and control modes. The integrated \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) asymmetry differences are measured to be

$$\begin{aligned} {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-})&= (+\,1.1 \pm 2.5 \pm 0.6)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-})&= (+\,3.2 \pm 1.1 \pm 0.6)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+\pi ^-})&= (-\,6.9 \pm 4.9 \pm 0.8 )\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+K ^-})&= (+\,0.2 \pm 1.8 \pm 0.6 )\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varXi } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-})&= (-\,17 \pm 11 \pm 1 )\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varXi } ^0_{b}} \rightarrow p K ^-\pi ^+K ^-})&= (-\,6.8 \pm 8.0 \pm 0.8)\,\%. \end{aligned}$$

The measurements for the two-body low invariant-mass regions are

$$\begin{aligned} {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-})&= (+\,3.7 \pm 4.1 \pm 0.5)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p K ^-\pi ^+\pi ^-})&= (+\,3.5 \pm 1.5 \pm 0.5)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p K ^-K ^+K ^-})&= (+\,2.7 \pm 2.3 \pm 0.6)\,\%. \end{aligned}$$

Finally, the measurements for the quasi two-body decays are

$$\begin{aligned} {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p {{a_{1}}} (1260)^-})&= (-\,1.5 \pm 4.2 \pm 0.6)\,\% \, , \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow N(1520)^0\rho (770)^0})&= (+\,2.0 \pm 4.9 \pm 0.4)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow \varDelta (1232)^{++}\pi ^{-}\pi ^{-}})&= (+\,0.1 \pm 3.2 \pm 0.6)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow p {{K_{1}}} (1410)^-})&= (+\,4.7 \pm 3.5 \pm 0.8)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow \varLambda (1520)\rho (770)^0})&= (+\,0.6 \pm 6.0 \pm 0.5)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow N(1520)^0{{K} ^*(892)^{0}}})&= (+\,5.5 \pm 2.5 \pm 0.5)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow \varDelta (1232)^{++}K ^{-}\pi ^{-}})&= (+\,4.4 \pm 2.6 \pm 0.6)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow \varLambda (1520)\phi (1020)})&= (+\,4.3 \pm 5.6 \pm 0.4)\,\%, \\ {\varDelta {{\mathcal {A}}^{{C\!P}}}} ({{{\varLambda } ^0_{b}} \rightarrow (p K ^{-})_\text {high-mass} \phi (1020)})&= (-\,0.7 \pm 3.3 \pm 0.7)\,\%. \end{aligned}$$

In all cases the first uncertainties are statistical and the second systematic. No significant \(C\!P\) violation is observed. The \(\varDelta {{\mathcal {A}}^{{C\!P}}} \) measurements for the independent samples of the two magnet polarities, the two categories of trigger requirements and the two distinct data-taking samples are found to be consistent. In addition, the measured asymmetries for the combinatorial background in all spectra are consistent with zero. The background contributions coming from \(B \)-meson decays (that could potentially exhibit nonzero \(C\!P\) violation) are also consistent with null asymmetries.

In a previous analysis, the LHCb collaboration reported evidence for \(C\!P\) violation in a specific region of the phase space of the decay \({{\varLambda } ^0_{b}} \rightarrow p \pi ^-\pi ^+\pi ^-\), by measuring triple-product asymmetries [5]. By contrast, in the present analysis, no indication of a significant \(C\!P\)-violating asymmetry is obtained with the same data sample, providing complementary insights about the origin of this potential \(C\!P\)-symmetry breaking effect. The quest for the first observation of \(C\!P\) violation in baryon decays continues. LHCb Run 2 data provides about five times larger yields allowing for a more sensitive search of smaller \(C\!P\)-violating effects.