1 Introduction

Hadrons are systems of quarks bound by the strong interaction, described at the fundamental level by quantum chromodynamics (QCD). Decays of heavy hadrons containing at least one b quark provide clean signatures to test new emergent phenomena in the realms of QCD and physics beyond the Standard Model. Effective theories testing these signatures are based on the fact that the heavy-quark mass \(m_Q\) (e.g. \(m_{b} \sim 4\text {Ge\hspace{-1.00006pt}V} \)) is much larger than the QCD scale \(\varLambda _\text {QCD}\sim 100\text {Me\hspace{-1.00006pt}V} \).Footnote 1 Processes that occur at the scale of \(m_Q\) can be described by perturbative QCD, while nonperturbative effects, including the formation of light hadrons, are suppressed by powers of \(\varLambda _\text {QCD}/m_Q\). This factorization of high-energy and low-energy processes is widely used in effective theories describing decays of heavy hadrons, such as the heavy quark effective theory (HQET) [1,2,3,4].

Beauty hadron decays to two charmed hadrons probe factorization assumptions in HQET in a regime where their application is contestable due to the presence of two charm quarks (\(m_{c} \sim 1.3\text {Ge\hspace{-1.00006pt}V} \)) in the final state. In particular, decays of the \({{\varLambda } ^0_{b}} \) baryon, which has quark content udb, are interesting as the \(({u} {d})\) diquark is in a spin-zero state, which gives rise to additional symmetries of HQET. In that sense, the \({{\varLambda } ^0_{b}} \) baryon is a simpler object than a B meson.

This article reports the measurement of the branching fraction of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \), \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \) double-open-charm decays, relative to that of the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \) decay. The dominant Feynman diagrams of these decays are shown in Fig. 1, where the left and middle diagrams contribute to \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) decays, while the right diagram corresponds to \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {D} ^{(*)-}_{s} \) decays. The color-suppressed, internal \({W} \)-emission diagram, as shown in the middle, does not exist for \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {D} ^{(*)-}_{s} \) decays. The isospin of the light diquark in the left and right diagrams is conserved, such that \({{\varLambda } ^0_{b}} \!\rightarrow {\varSigma } _{c} ^+{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) decays are suppressed, and \({{\varLambda } ^0_{b}} \!\rightarrow {\varSigma } _{c} ^+{D} ^{(*)-}_{s} \) decays are forbidden up to nonfactorizing contributions [5, 6].

Fig. 1
figure 1

Dominant Feynman diagrams for (left, middle) the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) decays and (right) the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {D} ^{(*)-}_{s} \) decays

Two-body beauty to double-open-charm decays have been theoretically studied for over three decades, and several models predict the ratio of branching fractions \(\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \right) /\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) \) in the range 0.75–2.2 [7,8,9,10,11,12,13,14,15,16,17,18,19]. Due to the complexity of the three-body system, there are no predictions for the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) channels. However, the contribution of color-suppressed amplitudes in \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) can be qualitatively assessed by comparing the relative branching fractions to those of mesons, when exchanging the ud diquark in the \({{\varLambda } ^0_{b}} \) with an antiquark [20]. For this comparison, it is convenient to define the double ratio

$$\begin{aligned} \mathcal{D}\mathcal{R}^{(*)}(M_b){} & {} \equiv \left[ \frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) }\right] \Big /\nonumber \\{} & {} \quad \times \left[ \frac{\mathcal {B}\left( M_b\rightarrow M_c {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \right) }{\mathcal {B}\left( M_b\rightarrow M_c {{D} ^-_{s}} \right) }\right] , \end{aligned}$$
(1)

where \(M_b\) or \(M_c\) is a beauty or charm meson, and the star in \(\mathcal{D}\mathcal{R}^{(*)}\) denotes the \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) ground state or the \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} ^{*}(2007)^{0}} \) vector state. Additionally, there is a large interest in exploring the resonance structure of the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) decays. The reason is that the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) and \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} \) systems are the open-charm equivalent of the \({{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} ({c} {\overline{{c}}}){p} ({u} {u} {d})\) final state, where \(P _{c} ^+({c} {\overline{{c}}} {u} {u} {d})\) pentaquark resonances have been observed [21,22,23,24]. Decays of these pentaquarks to \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) and \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} \) states are anticipated in many models, yet the predicted branching fractions relative to that into \({{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} \) vary by orders of magnitude [19, 25,26,27,28,29,30,31,32,33,34,35]. To test those predictions experimentally, two more ingredients are needed, namely the pentaquark fit fractions coming from \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) amplitude analyses \(f_{{{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}}(P _{c} ^+)\), and the branching fractions of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) decays relative to that of the \({{\varLambda } ^0_{b}} \!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} {{K} ^-} \) mode [36]. Assuming that the production mechanism for \({{\varLambda } ^0_{b}} \!\rightarrow P _{c} ^+{{K} ^-} \) is the same as for \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} {{K} ^-} \) decays, the fit fraction in the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) final states is given by

$$\begin{aligned} f_{{{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}}(P _{c} ^+)= & {} f_{{{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p}}(P _{c} ^+) \cdot \frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \right) } \nonumber \\{} & {} \cdot \frac{\mathcal {B}\left( P _{c} ^+\!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0} \right) }{\mathcal {B}\left( P _{c} ^+\!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} \right) }. \end{aligned}$$
(2)

Thus, the values of \(\frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \right) }\) that will be derived in this article, can be used to calculate sensitivities for observing \(P _{c} ^+\) in the \({{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}\) system for a given theoretical prediction of \(\frac{\mathcal {B}\left( P _{c} ^+\!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0} \right) }{\mathcal {B}\left( P _{c} ^+\!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} \right) }\).

Using 5.4\(\text {fb} ^{-1}\) of proton-proton (pp) collision data collected at \(\sqrt{s} =13\text {Te\hspace{-1.00006pt}V} \) by the LHCb detector in 2015–2018, the branching fractions of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \), \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \), and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \) decays are measured relative to that of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \) decays. The latter is chosen as a normalization channel because it has large signal yield, and its branching fraction has been previously measured [37]. The \({{\varLambda } ^0_{b}} \) decays proceed through different intermediate charm hadron decays, namely \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \), \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \!\rightarrow {{K} ^+} {{\pi } ^-} \) and/or \({{D} ^-_{s}} \!\rightarrow {{K} ^-} {{K} ^+} {{\pi } ^-} \). All decays are reconstructed with the same particles in the final state, \({p} {{K} ^-} {{K} ^-} {{K} ^+} {{\pi } ^-} {{\pi } ^+} \), which reduces or cancels various uncertainties related to the \({{\varLambda } ^0_{b}} \) production and the determination of absolute reconstruction and selection efficiencies. Neutral objects or electron-positron pairs, in the decays of \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} \) or \({{D} ^{*-}_{s}} \) mesons are not reconstructed. Therefore, the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \) decays are referred to as partially reconstructed signal.

The expressions for the ratio of branching fractions for \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \) decays are

$$\begin{aligned}&\frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) }\\&\quad = \frac{N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} }}{N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }} \frac{\epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }}{\epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} }} \frac{\mathcal {B}\left( {{D} ^-_{s}} \!\rightarrow {{K} ^-} {{K} ^+} {{\pi } ^-} \right) }{\mathcal {B}\left( {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \!\rightarrow {{K} ^+} {{\pi } ^-} \right) }, \\&\frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) } \quad = \frac{N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} }}{N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }} \frac{\epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }}{\epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} }}, \end{aligned}$$

where \(N^X\) are the observed yields and \(\epsilon ^X\) the efficiency for the decay mode X. The ratio of branching fractions \(\mathcal {B}\left( {{D} ^-_{s}} \!\rightarrow {{K} ^-} {{K} ^+} {{\pi } ^-} \right) /\mathcal {B}\left( {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \!\rightarrow {{K} ^+} {{\pi } ^-} \right) \) is taken from Ref. [38].

2 LHCb detector

The LHCb detector [39, 40] is a single-arm forward spectrometer covering the pseudorapidity range \(2<\eta <5\) at the LHC, designed for the study of particles containing \(b \) or \(c \) quarks. The detector includes a high-precision tracking system consisting of a silicon-strip vertex detector surrounding the pp interaction region [41], a large-area silicon-strip detector located upstream of a dipole magnet with a bending power of about \(4{\mathrm {\,Tm}}\), and three stations of silicon-strip detectors and straw drift tubes [42, 43] placed downstream of the magnet. The tracking system provides a measurement of the momentum, \(p\), of charged particles with a relative uncertainty that varies from about 0.5% below 20\(\text {Ge\hspace{-1.00006pt}V}\) to 1.0% at 200\(\text {Ge\hspace{-1.00006pt}V}\). The minimum distance of a track to a primary vertex (PV), the impact parameter (IP), is measured with a resolution of \((15+29/p_{\textrm{T}})\,\upmu \text {m} \), where \(p_{\textrm{T}}\) is the component of the momentum transverse to the beam, in \(\text {Ge\hspace{-1.00006pt}V}\). Different types of charged hadrons are distinguished using information from two ring-imaging Cherenkov (RICH) detectors [44]. Hadrons are identified by a calorimeter system consisting of scintillating-pad and preshower detectors, an electromagnetic and a hadronic calorimeter. Muons are identified by a system composed of alternating layers of iron and multiwire proportional chambers [45].

The online event selection is performed by a trigger [46, 47], which consists of a hardware stage, based on information from the calorimeter and muon systems, followed by a software stage, which applies a full event reconstruction.

Simulation is required to calculate reconstruction and selection efficiencies, and to determine the shapes of partially reconstructed components in the invariant mass distributions. Some of these components are modeled by the fast simulation package RapidSim [48] and the AmpGen generator [49]. In the full detector simulation, pp collisions are generated using Pythia [50] with a specific LHCb configuration [51]. Decays of unstable particles are described by EvtGen [52], in which final-state radiation is generated using Photos [53]. The interaction of the generated particles with the detector, and its response, are implemented using the Geant4 toolkit [54, 55] as described in Ref. [56]. The underlying pp interaction is reused multiple times, with an independently generated signal decay for each simulated event [57].

3 Dataset and selection

Data are selected first by an online trigger system, consisting of a hardware- and two software stages, further filtered in an offline selection. Online, data are selected by a sequence of inclusive trigger decisions. The majority of candidates are selected by criteria based on the topology of b-hadron decays [58, 59]. In the offline selection, candidates that fulfill characteristics of the exclusive \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) or \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \) decay are selected using tracks with hits in at least the vertex tracker and the three downstream tracking stations. Further selection is applied on momentum, transverse momentum, track quality and displacement from any PV. Intermediate charm hadrons are selected using the distance of closest approach of their decay products, the decay vertex fit \(\chi ^2\), and the displacement of the decay vertex from any PV. Combined information from particle identification (PID) detectors is used to reject topologically similar background contributions. The \({{\varLambda } ^0_{b}} \) candidates are reconstructed by combining a \({{\varLambda } ^+_{c}} \) candidate with either a \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) candidate and a companion kaon or a \({{D} ^-_{s}} \) candidate. Kinematic and topological variables are used to suppress combinatorial background: namely the sum of transverse momenta of all the final-state particles, the presence of at least one track with large displacement from any PV and high momentum and high transverse momentum, the angle between the reconstructed momentum direction of the \({{\varLambda } ^0_{b}} \) candidate and its flight direction and distance, determined from the production and decay vertices.

A clean sample of \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) and \({{D} ^-_{s}} \) mesons is selected using topological and kinematic criteria, and a requirement on the product of the probabilities of final-state kaons and pions to be correctly identified. These probabilities correspond to the response of a neural network combining PID information from the full detector [40]. Even though the \({{\varLambda } ^+_{c}} \) decays produce high-momentum protons that can be cleanly reconstructed and identified, the shorter lifetime compared to \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) and \({{D} ^-_{s}} \) mesons makes it more difficult to suppress background contributions in the reconstruction of \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \) decays. Consequently, a dedicated gradient boosted decision tree (BDT) algorithm [60] for secondary \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \) decays is trained and calibrated on \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\pi } ^-} \) data, similar to the \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \) BDT classifier used in Ref. [37]. An optimization procedure is carried out, that determines a working point for for the following variables: the \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \) BDT output, the \(\chi ^2_{\text {IP}} \) of the \({{\varLambda } ^0_{b}} \) candidate, where \(\chi ^2_{\text {IP}} \) is defined as the difference in the vertex-fit \(\chi ^2\) of a given PV reconstructed with and without the \({{\varLambda } ^0_{b}} \) candidate, and the probability of the companion kaon to be correctly identified in the \({\varLambda } ^+_{c} \) \({\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0\) \({K} ^-\) channel. The optimization is carried out in form of a grid search, maximizing the approximate signal significance (\(S/\sqrt{S+B}\)) multiplied by the purity (\(S/(S+B)\)). Here, S and B represent the signal and background yields in a 2 \(\sigma \) window around the exclusive \({{\varLambda } ^0_{b}} \) signal mass peak. This optimization is done independently for signal and normalization channels.

Candidates for which the opening angle between any track pair is smaller than 0.2 mrad are rejected, which removes artifacts from matching track segments reconstructed with the downstream tracking stations and the vertex locator. Fiducial regions are selected in the phase-space later used for weighting simulated events, see Sect. 5. This concerns the transverse momentum and pseudorapidity of the \({{\varLambda } ^0_{b}} \) candidate, as well as the total number of tracks in the event.

Sources of peaking background candidates from particle misidentification are studied by statistically subtracting the combinatorial background contribution [61], identified in a preliminary fit to the invariant mass of the reconstructed system, described in Sect. 4. For the background-subtracted sample, invariant-mass distributions with swapped particle hypotheses are investigated to identify and remove candidates with misidentified particles. To increase selection efficiencies, only those candidates that also fail at least one tighter particle identification requirement, are removed as summarized in Table 1.

Table 1 Explicitly rejected physics backgrounds. Some background contributions are present only in the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) or \({{\varLambda } ^+_{c}} {{D} ^-_{s}} \) systems, while others in both. A particle, \(M_\textrm{misID}\), that decays through a real particle a, which is reconstructed as a particle with different mass hypothesis b, is denoted as \(M_\textrm{misID}\!\rightarrow \{a \leftarrow b\}X \), where X corresponds to the rest of the decay. As there are two \({{K} ^-} \) mesons in the final state, the subscripts “\({{\varLambda } ^+_{c}} \)” and “com” (for companion) or “\({{D} ^-_{s}} \)” denote the assignment in the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) or \({{\varLambda } ^+_{c}} {{D} ^-_{s}} \) reconstruction chain. Cases where the proton is misidentified as a pion or kaon and combined into a \({{D} ^0} \) candidate, are marked with the \({{D} ^0} \) subscript

Background events from a wrong combination of candidates in the sample of \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) decays, such as \({{\varLambda } ^0_{b}} \!\rightarrow {p} {{D} ^0} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {K} _{{{\varLambda } ^+_{c}}}^{-} \), with \({{D} ^0} \!\rightarrow K^-_\textrm{com}{{\pi } ^+} ({{\pi } ^0}) \), where the subscripts “\({{\varLambda } ^+_{c}} \)” and “com” denote the nominal assignment in the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) reconstruction chain. are suppressed by requiring a small IP and \(\chi ^2_{\text {IP}} \) of the companion kaon \(K^-_\textrm{com}\), with respect to the \({{\varLambda } ^0_{b}} \) decay vertex.

For each species of charm hadron, an invariant-mass requirement is defined such that the central 95% of the individual charm-candidate signal is retained. These are determined from a three-dimensional fit to the invariant masses of the \({{\varLambda } ^0_{b}} \) candidate and the two intermediate charm-hadron candidates. To select at most one \({{\varLambda } ^0_{b}} \) candidate for a given LHC bunch crossing, a final selection step randomly removes all but one candidate from each event. This concerns 2.4% of the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) candidates and 1.1% of the \({{\varLambda } ^+_{c}} {{D} ^-_{s}} \) candidates.

4 Invariant-mass fits

Unbinned maximum-likelihood fits are first carried out in three mass dimensions, given by the invariant mass of \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) or \({{\varLambda } ^+_{c}} {{D} ^-_{s}} \) decays, and the two charm hadrons. The invariant masses of beauty candidates are defined as \(m({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-})\equiv m({p} {{K} ^-} {{\pi } ^+} {{K} ^+} {{\pi } ^-} {{K} ^-})-m({p} {{K} ^-} {{\pi } ^+})-m({{K} ^+} {{\pi } ^-})+M_{{{\varLambda } ^+_{c}}}+M_{{{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0}}\) and \(m({{\varLambda } ^+_{c}} {{D} ^-_{s}})\equiv m({p} {{K} ^-} {{\pi } ^+} {{K} ^-} {{K} ^+} {{\pi } ^-})-m({p} {{K} ^-} {{\pi } ^+})-m({{K} ^-} {{K} ^+} {{\pi } ^-})+M_{{{\varLambda } ^+_{c}}}+M_{{{D} ^-_{s}}}\), where \(M_X\) is the known value of the mass of particle X from Ref. [38].

This fit is restricted to narrow regions around the exclusively reconstructed \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \) signals in any of the three mass dimensions as shown in Fig. 2. The figures illustrate that the three-dimensional fit directly measures the normalization of single-charm and charmless backgrounds. The central interval containing 95% of the signal component determines the mass selection of the charm candidates, which removes backgrounds of different origin that would complicate model building in a three-dimensional fit in the full mass range.

Fig. 2
figure 2

Distributions of (upper left) \(m({{\varLambda } ^+_{c}} {{D} ^-_{s}})\), (upper right) \(m({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-})\), (middle) \(m(p{{K} ^-} {{\pi } ^+})\), (lower left) \(m({{K} ^-} {{K} ^+} {{\pi } ^-})\) and (lower right) \(m({{K} ^+} {{\pi } ^-})\) for the (left) \({{\varLambda } ^+_{c}} {{D} ^-_{s}} \) and (right) \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) candidates, with the fit projections overlaid

Exclusive signal contributions in all mass dimensions are modeled by two-sided Hypatia functions, which are convolved with a Gaussian function [62]. The core width of the Hypatia and the mass parameter are free in the fits, the parameters which determine the transitioning points from the generalized hyperbolic resolution model to the exponential tails are constrained and the remaining parameters of the signal models are fixed. Constraining and fixing parameters in the signal model and other components is validated with simulation and pseudoexperiments to ensure unbiased parameter estimation and ensure valid coverage properties. Combinatorial backgrounds are described by Chebychev polynomial functions up to order three; their coefficients allowed to float free in the fit. The order chosen for an individual fit is the one minimizing a likelihood that has been corrected for the number of degrees of freedom in the fit model, accounting for constraints and the number of parameters [63]. This likelihood is also used to select a baseline fit model among the various possibilities discussed in Sect. 6.

The only significant single-charm or charmless background contribution comes from the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{K} ^+} {{\pi } ^-} {{K} ^-} \) decay. Its three-dimensional probability density function (PDF) is composed of a signal component in \(m({p} {{K} ^-} {{\pi } ^+})\), a dedicated linear background in \(m({{K} ^+} {{\pi } ^-})\) or \(m({{K} ^-} {{K} ^+} {{\pi } ^-})\), and a conditional Gaussian PDF in \(m({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-})\) or \(m({{\varLambda } ^+_{c}} {{D} ^-_{s}})\). The mean of the conditional Gaussian PDF depends linearly on \(m({{K} ^+} {{\pi } ^-})\) or \(m({{K} ^-} {{K} ^+} {{\pi } ^-})\), since a \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) or \({{D} ^-_{s}} \) mass constraint is used in the definition of \(m({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-})\) or \(m({{\varLambda } ^+_{c}} {{D} ^-_{s}})\). In total, \(84\pm 13\) (\(990\pm 50\)) \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{K} ^+} {{\pi } ^-} {{K} ^-} \) candidates remain in the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) (\({{\varLambda } ^+_{c}} {{D} ^-_{s}} \)) dataset used for the one-dimensional baseline fit. A subsequent one-dimensional fit, referred to as the baseline fit, which is independent from the three-dimensional fit, is used to determine the exclusive and partially reconstructed signal yields, that are needed for measuring the branching fractions. In the baseline fit, the models of signal and combinatorial background are the same as in the three-dimensional fit, following the same strategy of floating, fixed and constrained parameters described earlier. The yields of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{K} ^+} {{\pi } ^-} {{K} ^-} \) decays are fixed in the baseline fit, and their uncertainties, including the anti-correlation with the signal yields, are propagated from the fit result of the three-dimensional fit to the statistical uncertainty of the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) branching fraction.

The partially reconstructed quasi-two-body decays \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \), \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {D} ^*_{{s} 0}(2317)^- \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {D} _{{s} 1}(2460)^- \) are described analytically. Their kinematic endpoints, which define the domain of the corresponding PDF, are fully determined by the masses of the decay products [38]. Each PDF is a superposition of Gaussian functions convolved with a two-sided step-function with a sloped plateau (box-like), and an upward- or downward-open parabola [64, 65]. These PDFs model the spin structure of the respective decays, and additionally describe a linear drop of efficiency towards lower invariant masses, which is constrained to the value extracted from simulation.

In particular, the quasi-two-body decays through vector mesons, \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{D} ^-_{s}} {\gamma } \right] _{{{D} ^{*-}_{s}}} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{D} ^-_{s}} {{\pi } ^0} \right] _{{{D} ^{*-}_{s}}} \) are modeled by parabolic shapes facing downward and upward respectively, and due to the spin-1/2 initial state also involve a box-like component. Their relative normalization is constrained by the branching fraction of \(\mathcal {B}({{D} ^{*-}_{s}} \!\rightarrow {{D} ^-_{s}} {{\pi } ^0} )\). Quasi-two-body decays through the spin-0 meson \({D} ^*_{{s} 0}(2317)^-\!\rightarrow {{D} ^-_{s}} {{\pi } ^0} \), and the spin-1 meson \({D} _{{s} 1}(2460)^-\!\rightarrow {{D} ^-_{s}} {\gamma } \) are fully described by a box-like PDF. The latter is an effective description, since only about 20% of the high mass tail of the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{D} ^-_{s}} {\gamma } \right] _{{D} _{{s} 1}(2460)^-} \) decay can be reconstructed in the chosen mass range, and variations of the spin structure in simulation do not significantly alter the shape. The normalization of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{D} ^-_{s}} {{\pi } ^0} \right] _{{D} ^*_{{s} 0}(2317)^-} \) and also \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{D} ^-_{s}} {\gamma } \right] _{{D} _{{s} 1}(2460)^-} \) decays is loosely constrained relative to that of the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \) component by means of the average of branching fraction measurements from the corresponding meson decays, their simulated integral in the chosen invariant-mass range, and a correction accounting for the additional degrees of freedom in the spin structure of baryon decays [6]. Further decays to excited \({{D} ^-_{s}} \) mesons are either kinematically forbidden or contribute at a significantly lower rate.

The shapes of partially reconstructed multibody decays, i.e. \({{\varLambda } ^0_{b}} \!\rightarrow \left[ {{\varLambda } ^+_{c}} \pi \pi \right] _{{\varLambda } _{c} ^{*+}}{{D} ^-_{s}} \) and partially reconstructed decays in \(m({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-})\) cannot be modelled analytically unless their corresponding three-body dynamics is known and modeled. Shapes of those decays are thus derived from simulation in terms of a nonparametric PDF using a kernel density estimation method (KDE) [66]. Fast simulation samples, either generated with AmpGen [49] or RapidSim [48], are used to model and cross-validate the shapes of \({{\varLambda } ^0_{b}} \!\rightarrow \left[ {{\varLambda } ^+_{c}} \pi \pi \right] _{{\varLambda } _{c} ^{*+}}{{D} ^-_{s}} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {\varSigma } _{c} ^+{{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) decays. As the shape of the partially reconstructed \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) decay depends on the three-body dynamics, a dedicated simulation sample including the most prominent \({{D} ^+_{s}} \) resonances, \({D} _{{s} 1}(2536)^-\) and \({D} ^*_{{s} 1}(2700)^-\), as well as a small contribution from \({\varXi } _{c} (2790)^0\) is used. The same sample is employed for the efficiency correction described in Sect. 5. The composition of simulated resonances is adapted to what is observed in the data. An effective correction for unconsidered three-body dynamics or efficiency effects is obtained by multiplying the KDE template for the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) contribution with a first order polynomial function with a freely varying coefficient. That coefficient is anticorrelated (\(-0.38\)) to the leading coefficient of the polynomial describing the combinatorial background, and shared between the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{\pi } ^0} \right] _{{{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}}}{{K} ^-} \) and the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {\gamma } \right] _{{{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}}}{{K} ^-} \) components. The normalization of those components is a free parameter in the fit, found to be consistent with the world-average value of \(\mathcal {B}\left( {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \!\rightarrow {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{\pi } ^0} \right) /\mathcal {B}\left( {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \!\rightarrow {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {\gamma } \right) \) [38].

The baseline fits for the signal and normalization channel are shown in Fig. 3. The measured signal yields are

$$\begin{aligned}&N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} } = 4010\pm 70,\quad N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} } = 10\,560^{+310}_{-290}, \\&N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} } = 35\,450 \pm 200,\quad N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} } = 46\,400 \pm 500, \end{aligned}$$

where the uncertainties are statistical, and the asymmetric uncertainties of the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) signal yield are driven by an anticorrelation with the \({{\varLambda } ^0_{b}} \!\rightarrow {\varSigma } _{c} ^+{{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) component.

Fig. 3
figure 3

Invariant-mass distributions of (top) \({{\varLambda } ^+_{c}} {{D} ^-_{s}} \) and (bottom) \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) candidates with the results of the baseline fit overlaid

5 Efficiency correction

Efficiency ratios are calculated as the product of four factorizing terms:

  • Efficiencies of selections before the GEANT4 step of the simulation, referred to as generator level efficiencies hereafter.

  • Trigger, reconstruction, and selection efficiencies are taken from simulation, up to but excluding the final selection step.

  • Selection efficiencies of the final step are computed from simulation that has been weighted to match data.

  • \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \) BDT efficiencies are evaluated using \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\pi } ^-} \) data.

Generator level efficiency ratios comprise the angular coverage of LHCb, as well as loose kinematic and geometric selections of the generated candidates. Efficiencies are calculated only from those events for which a new underlying pp interaction has been simulated.

The efficiency correction of the final selection step handles variables that are more difficult to model, like those that combine PID information [67]. These variables depend on detector multiplicity, production and decay kinematics, hereafter referred to as calibration variables.

Simulated events are weighted to match signal distributions in the phase-space of the calibration variables. The set of 5 or 7 variables chosen for calibration are the track multiplicity, transverse momentum and pseudorapidity of the \({{\varLambda } ^0_{b}} \) candidate for the production kinematics and the two square Dalitz variables of \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \), as well as those of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) for \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) candidates. It is not necessary to weight the simulated kinematics of the \({{D} ^+_{s}} \!\rightarrow {{K} ^+} {{K} ^-} {{\pi } ^+} \) decay in \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {D} ^{(*)-}_{s} \), due to the good agreement of the EvtGen model with the data [68]. The convention used for the square Dalitz variables follows that of Ref. [69]:

$$\begin{aligned} m'\equiv \frac{1}{\pi } \arccos \left( 2\frac{m_{12}-m_{12}^\textrm{min}}{m^\textrm{max}_{12}-m_{12}^\textrm{min}}-1\right) \quad \textrm{and} \quad \theta '\equiv \frac{1}{\pi } \theta _{12} \, \end{aligned}$$

where \(m^\textrm{max}_{12}=m_P-m_{c_3}\) and \(m_{12}^\textrm{min}=m_{c_1}+m_{c_2}\) are the kinematic limits of \(m_{12}\) in the \(P\!\rightarrow c_1 c_2 c_3 \) decay, while \(\theta _{12}\) is the helicity angle between \(c_1\) and \(c_3\) in the rest frame of \(c_1 c_2\). In the squared Dalitz plot of the \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \), P corresponds to \({{\varLambda } ^+_{c}} \), \(c_1\) to the proton, \(c_2\) to the kaon and \(c_3\) to the pion; while P corresponds to \({{\varLambda } ^0_{b}} \), \(c_1\) to \({{\varLambda } ^+_{c}} \), \(c_2\) to \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) and \(c_3\) to the companion kaon in the square Dalitz plot for the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \).

In the weighting, the 7 (5) dimensional calibration variable space is factorized into one 1D and three (two) 2D spaces for \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \)(\({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {D} ^{(*)-}_{s} \)). This speeds up the weighting algorithm and allows for finer bins/partitions. For weighting, the Meerkat library [70] is used to create kernel density profiles of calibration variable distributions from reconstructed simulation and signal data. This method is validated against an adaptive binning algorithm, that split the calibration spaces in equally populated bins. Several configurations of the adaptive binning returned negligible differences with respect to the nominal kernel density method. The assumption of factorization is verified with control plots that show that weighting in a certain subspace does not affect the other calibration variables. It is further validated by testing the weighting procedure with a method that does not factorize the space of calibration variables, but partitions the full phase space using gradient boosted decision trees [71]. Without weighting, individual efficiencies are found to be larger across all studied decays, up to 2.5%. In the ratio, the difference with respect to the weighted evaluation is between 0.6 and 1.8%.

The efficiency ratio for the \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \) BDT response is obtained in a data-driven manner using the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\pi } ^-} \) calibration channel. Efficiencies are obtained by fitting the calibration samples, weighted to match \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) or \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \) data, simultaneously in “pass” and “fail” categories. For validation tests, the requirements on the BDT response are equalized between signal and normalization channels. The systematic uncertainty associated with the \({{\varLambda } ^+_{c}} \!\rightarrow {p} {{K} ^-} {{\pi } ^+} \) BDT efficiency correction is negligible due to the large size of the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\pi } ^-} \) samples, but also due to the similarity of selection requirements between this channel and the signal.

The overall efficiency ratios are found to be

$$\begin{aligned} \epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} }/\epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }&= 0.809\pm 0.006, \\ \epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} }/\epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }&= 0.689\pm 0.005, \\ \epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} }/\epsilon ^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }&= 0.785\pm 0.005, \end{aligned}$$

where the uncertainties are statistical.

6 Systematic uncertainties

6.1 Invariant-mass fits

As described in Sect. 4, the baseline model is the one that minimizes the likelihood corrected for the number of degrees of freedom in the fit model. Nine alternative models, returning comparable corrected likelihood values, are used in a discrete profiling method [63] to evaluate systematic uncertainties of the fit model. Each of these alternative models concerns a single aspect of the baseline model. For example an exponential instead of a linear function is used as the combinatorial background description. For each alternative, all other variations are tested in conjunction with the given variation, and the model that minimizes the corrected likelihood is taken into consideration for the discrete profiling. Changes to the baseline model are described below:

  • For signal, the baseline Gaussian-convolved Hypatia function is replaced with a double-sided Crystal Ball function [72] with common mean. The double-sided Crystal Ball model improves the corrected likelihood in the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) channel, but not in the normalization channel. For consistency, the Hypatia model is quoted as the baseline result, while the uncertainty calculation uses the alternative model as best fit to calculate the likelihood envelope of the discrete profiling method.

  • Combinatorial background models are either Chebychev polynomials or exponential functions. It is found that either a first or second order polynomial function describes the combinatorial background best, even though higher-order functions can improve the uncorrected likelihood.

  • In the baseline fit, the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) decays are modeled with a KDE approach, in which the \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} \) decay modes are separated. An alternative model is a KDE template of the combined \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) decay where the relative normalization of individual \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} \) decay modes is fixed at the level of event generation.

  • The \({{\varLambda } ^0_{b}} \!\rightarrow {\varSigma } _{c} ^+{{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) decays are modeled with KDE templates from AmpGen simulation, numerically convolved with a Gaussian function. One variation multiplies the shape with a first-order polynomial, to effectively correct for unconsidered three-body dynamics or efficiency effects, another one adds contributions from \({\varSigma } _{c} (2520)^+\) decays. When allowing the normalization of this component to float free in the fit, it is compatible with zero yield. A model is employed, that loosely constrains the ratio of yields of the components modeling the decays through \({\varSigma } _{c} (2520)^+\) and \({\varSigma } _{c} (2455)^+\) states. The value of the constraint is based on the approximate ratios observed in \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{K} ^-} {{K} ^+} {{\pi } ^-} \) [73] and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\pi } ^-} {{\pi } ^+} {{\pi } ^-} \) [74] decays.

  • The normalization fraction of box-like shapes with respect to parabolic shapes are separated for the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{D} ^-_{s}} {{\pi } ^0} \right] _{{{D} ^{*-}_{s}}} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} \left[ {{D} ^-_{s}} {\gamma } \right] _{{{D} ^{*-}_{s}}} \) components. In addition, the slope parameter that models the efficiency gradient and the \({{D} ^{*-}_{s}} \) branching fractions is fixed, instead of constrained, to values obtained from simulation.

  • The \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \pi \pi \) decays are effectively modeled by a KDE template derived from an AmpGen simulation sample where the decay occurs through the \({\varLambda } _{c} (2625)^+\) resonance. An alternative model is the decay through the \({\varLambda } _{c} (2595)^+\) state.

To compute the envelope for the discrete profiling method analytically, likelihoods are approximated with bifurcated parabolas, which account for asymmetric uncertainties. From this method, the yield ratios obtained, including their statistical and systematic uncertainties due to the fit model, are

$$\begin{aligned} \begin{aligned} N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} }/N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }&= 0.1132 {}_{-0.0020}^{+0.0021} {}_{-0.0007}^{+0.0006}, \\ N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} }/N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }&= 0.298 {}_{-0.008}^{+0.009} {}_{-0.009}^{+0.008}, \\ N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} }/N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} }&= 1.309 \pm 0.017 {}_{-0.043}^{+0.047}\ . \end{aligned} \end{aligned}$$

The dominating systematic effect comes from the signal shape variation in the determination of the exclusive \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) yields. For the measurement of \(N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} }\), the dominant source of uncertainty is the multiplication of the \({{\varLambda } ^0_{b}} \!\rightarrow {\varSigma } _{c} ^+{{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) component by a first-order polynomial function. Changing the combinatorial background description, and separating the normalization fraction of the contributions in the description of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \) decays, are the largest contributions to the uncertainty on \(N^{{{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} }\).

6.2 Parameters of the simulation weighting

Efficiency ratios are calculated using five different settings of the initial phase-space binning to generate the adaptive KDE profiles. The results are found to be very stable against changing this parameter from the default 80 bins to 20, 40, 160 and 320 bins. Their standard deviation with respect to the baseline result is taken as the systematic uncertainty.

6.3 Multiple candidates

Multiple candidates are randomly removed from the sample, as they are mainly composed of candidates where a signal track has been swapped with a combinatorial track that happens to have similar kinematics. In the case of \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \), another source of multiple candidates exists, swapping the companion \({{K} ^-} \) with that from the \({{\varLambda } ^+_{c}} \) decay. Removing this background entirely is inefficient, but since the fraction of multiple candidates in the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) channel is larger (2.4% compared to 1.1% in \({\varLambda } ^+_{c} \) \({D} ^-_{s} \)), further methods to remove multiple candidates are studied: namely using the minimal \(\chi ^2/\)ndf of a kinematic fit of the decay chain [75], or the maximum sum of PID variables to select the best out of the multiple candidates. The resulting maximum deviation from the baseline result is small, but assigned as systematic uncertainty.

6.4 Simulation and control sample sizes

Statistical uncertainties of the generator level efficiency ratios, and simulation and calibration sample sizes are propagated to the branching ratio measurement, as summarized in Table 2. For generator level efficiencies, only those events for which a new underlying pp interaction has been simulated can be taken into account for the uncertainty calculation. As the underlying pp interaction is reused 100 times [57], with an independently generated signal decay for each simulated event, the uncertainty on the generator level efficiency ratio is treated independently of the statistical uncertainty of the final simulation sample size.

6.5 Summary

Table 2 summarizes the systematic uncertainties for the three measured ratios of branching fractions.

Table 2 Systematic uncertainties relative to the branching fraction ratio measurements. The relative statistical uncertainty is shown as a reference. Values are given in percent

The choice of the fit model is found to dominate the uncertainty for partially reconstructed decays, while the systematic uncertainties for \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) are small compared to their corresponding statistical uncertainty.

7 Conclusion

The ratio of branching fractions for the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \), \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \) and \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \) decays, relative to that of the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \) decay, are measured in \({p} {p} \) collisions at \(\sqrt{s} =13\text {Te\hspace{-1.00006pt}V} \) corresponding to an integrated luminosity of 5.4\(\text {fb} ^{-1}\) collected with the LHCb detector. The results are found to be

$$\begin{aligned} \begin{aligned}&\frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) }\cdot \frac{\mathcal {B}\left( {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \!\rightarrow {{K} ^+} {{\pi } ^-} \right) }{\mathcal {B}\left( {{D} ^-_{s}} \!\rightarrow {{K} ^-} {{K} ^+} {{\pi } ^-} \right) }\\&= 0.1400 {}_{-0.0025}^{+0.0026} {}_{-0.0013}^{+0.0012}, \\&\frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) }\cdot \frac{\mathcal {B}\left( {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \!\rightarrow {{K} ^+} {{\pi } ^-} \right) }{\mathcal {B}\left( {{D} ^-_{s}} \!\rightarrow {{K} ^-} {{K} ^+} {{\pi } ^-} \right) }\\&= 0.432 {}_{-0.012}^{+0.013} \pm 0.013, \end{aligned} \end{aligned}$$

where the first uncertainties are statistical and the second systematic. In the \({{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \) channel, the ratio of branching fractions of \({{\varLambda } ^0_{b}} \) decays proceeding through the excited compared to the ground state \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \) is measured to be

$$\begin{aligned} \frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \right) }=3.09^{+0.11}_{-0.10}{}^{+0.09}_{-0.10}, \end{aligned}$$

where correlations between uncertainties are taken into account, but are found to be small. Including the known values of the \({D} \) meson branching fractions from Ref. [38], the ratios of branching fractions are

$$\begin{aligned} \begin{aligned} \frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) }&= 0.1908 {}_{-0.0034}^{+0.0036} {}_{-0.0018}^{+0.0016} \pm 0.0038, \\ \frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) }&= 0.589 {}_{-0.017}^{+0.018} {}_{-0.018}^{+0.017} \pm 0.012, \\ \frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) }&= 1.668 \pm 0.022 {}_{-0.055}^{+0.061}, \end{aligned} \end{aligned}$$

where the third uncertainties are due to the uncertainty of the branching fractions of \({{D} ^-_{s}} \!\rightarrow {{K} ^-} {{K} ^+} {{\pi } ^-} \) and \({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} \!\rightarrow {{K} ^+} {{\pi } ^-} \) decays.

The result obtained for \(\frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^{*-}_{s}} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{D} ^-_{s}} \right) }\) is compatible with several predictions [9,10,11, 13, 15,16,17].

To probe factorization approaches in the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) decays, the following values for the double ratios \(\mathcal{D}\mathcal{R}^{(*)}\), defined in Eq. (1), are obtained

$$\begin{aligned} \mathcal{D}\mathcal{R}({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}B}} {}^0})&=1.29\pm 0.20,&\mathcal{D}\mathcal{R}^{*}({{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}B}} {}^0})&=1.28\pm 0.19,\\ \mathcal{D}\mathcal{R}({{{B} ^-}})&=1.20\pm 0.30,&\mathcal{D}\mathcal{R}^{*}({{{B} ^-}})&=0.87\pm 0.12,\\ \mathcal{D}\mathcal{R}({{B} _{c} ^-})&=1.3\pm 0.5,&\mathcal{D}\mathcal{R}^{*}({{B} _{c} ^-})&=0.8\pm 0.4, \end{aligned}$$

assuming uncorrelated uncertainties, and taking known values for the mesonic branching fractions from Ref. [38]. Larger baryonic branching fractions are expected, because of an additional color-suppressed amplitude (see Fig. 1) in the \({{\varLambda } ^0_{b}} \) decay, which does not exist for mesons, however the measured ratios are still inconclusive.

The ratios of branching fractions that are relevant for pentaquark searches, see Eq. (2), are

$$\begin{aligned} \begin{aligned} \frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^0} {{K} ^-} \right) }&=0.152^{+0.032}_{-0.028}, \\ \frac{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} {{K} ^-} \right) }{\mathcal {B}\left( {{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {{\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{*0}} {{K} ^-} \right) }&=0.049^{+0.011}_{-0.009}, \end{aligned} \end{aligned}$$

using results from Ref. [36], and assuming that uncertainties are uncorrelated. A future search for pentaquarks in the \({{\varLambda } ^0_{b}} \!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}{{K} ^-} \) decays will be able to determine the fit fractions \(f_{{{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0}}(P _{c} ^+)\), which can be used to test model predictions of the ratio of \(P _{c} ^+\) branching fractions \(\mathcal {B}(P _{c} ^+\!\rightarrow {{\varLambda } ^+_{c}} {\hspace{1.79993pt}\overline{\hspace{-1.79993pt}D}} {}^{(*)0})/\mathcal {B}(P _{c} ^+\!\rightarrow {{J \hspace{-1.66656pt}/\hspace{-1.111pt}\psi }} {p} )\).