1 Introduction

Quantum spin liquid (QSL) is a long-range entangled “quantum liquid” state of interacting spins [13], which is believed not only to be a platform for obtaining the original driving force of the high-temperature superconductivity but also to be a basic unit for the quantum data storage and computation. Although there has been much theoretical effort on different models of QSL, a firm experimental establishment of this exciting and exotic phase in a real compound is still lacking. Although limited behaviors have been confirmed and actively investigated on a few spin liquid candidate compounds, such as the organic materials [47] κ-(BEDT-TTF)2Cu2(CN)3 and EtMe3Sb[Pd(dmit)2]2, kagomé herbertsmithite [8, 9] ZnCu3(OH)6Cl2, the cluster Mott insulator [10, 11] 1T-TaS2 in the commensurate charge density wave phase, the honeycomb Kitaev materia [1216], and the pyrochlore spin ice materials [1727], there is always some unexpected factors observed in these candidate materials, such as the distorted lattice, relatively low-energy scale of the spin interactions, anti-site ionic disordering, and restricted experimental resolution, and the intrinsic magnetic behaviors might be shaded. Therefore, investigating an ideal compound and revealing the intrinsic quantum and magnetic properties related to QSLs is crucial to both material and physics research.

Previously, part of us and many others have worked on a rare-earth triangular lattice magnet [2848] where the Yb3+ ions form a perfect triangular lattice with spin-orbit-entangled effective spin-1/2 local moments. Thus, is likely to be the first spin liquid candidate in the strong spin-orbit-coupled Mott-insulators with odd electron fillings [29, 30, 49] and goes beyond the conventional Oshikawa-Hastings-Lieb-Schultz-Mattis theorem [5053] that requires the U(1) spin rotational symmetry. However, the potential issue of is the Mg-Ga disorder in the non-magnetic layers. To what extent the non-magnetic disorder in the non-magnetic layer would impact the intrinsic magnetic properties of the system may require further scrutiny. Instead of delving with , we turn to a different route. This is motivated by the robustness of QSLs and the principle of universality. If the QSL ground state is relevant for , similar QSL state could likely be realized in other triangular lattice rare-earth magnets. On the other hand, other triangular lattice rare-earth magnets could potentially eliminate the non-magnetic disorder issue that was suggested for . Indeed, a series of rare-earth chalcogenides was recently identified [54, 5667] as candidates for QSLs, and apparently, there is no structural disorder like the Mg/Ga ions in YbMgGaO4. Therefore, we combine the experimental measurements and theoretical analysis and propose NaYbS2 likely to be a gapless QSL candidate. The field- and temperature-dependence of magnetization, specific heat, neutron diffraction, and inelastic neutron scattering measurements demonstrate several field-induced magnetic transitions in this QSL ground state system. Moreover, we provide extra theoretical results and suggestions for further experimental verification of the QSL state in the isostructural compounds.

2 Samples and experimental techniques

The polycrystalline and single crystals of were synthesized by the high-temperature solid-state reaction and the -flux method, respectively [54]. Around 0.5 g single crystals of NaYbS2 were co-aligned in the (HHL) scattering plane on a copper plate, and the single-crystal neutron diffraction data were collected at the Australia Nuclear Science and Technology Organisation (ANSTO), using a cold neutron triple-axis diffractometer, Sika [55]. An 8 T vertical magnetic field cryostat with a dilution refrigerator insert was applied to collect magnetic field data at 0 T, 3 T, 4 T, 5 T, and 8 T at 0.1 K. The magnetic field was applied along [1 −1 0], and the data were collected with final energy as \(\mathrm{E}_{f}\sim 5\) meV. Rietveld refinement was performed with the SARAh software and the magnetic symmetry approach at the Bilbao Crystallographic Server. The fitted data are shown in Fig. 1 at 300 K and 1.6 K. The inelastic neutron-scattering powder data were collected at the Swiss Spallation Neutron Source (SINQ), the Paul Scherrer Institut (PSI), using a disc chopper spectrometer, FOCUS, in a 10 T magnetic field with a dilution insert. Incident neutrons with 5 Åwavelength were used with the medium-resolution chopper setting.

Figure 1
figure 1

Crystal structure and powder diffractions of NaYbS2. (a) The lattice structure and the different triangular-layers of NaYbS2. The three-layers of triangles are marked with solid lines, dot lines and broken lines, respectively. (b) The comparison of the neutron powder diffractions at 0.05 K and 0.3 K under 0 T with the wavelength of 3.2 Å−1. The lattice diffractions have been labeled and the olive line presents the difference. (c) The comparison of the neutron reflections at \((1/3,1/3,L)\) (L: 0, 1, 2, and 3) between 0 T and 6 T near 0.1 K and the magnetic field-dependence of \((1/3,1/3,0)\) and \((1/3,1/3,3)\), respectively. The vertical error bars indicate the statistical errors of one standard deviation. The reflections of \((1/3,1/3,0)\) and \((1/3,1/3,3)\) are clearly obtained at 6 T, while \((1/3,1/3,1)\) and \((1/3,1/3,2)\) are too weak to be observed. (d) Schematics of the Brillouin zones with equivalent K-points denoted by red dots, and the magnetic structure of NaYbS2 between 3 T and 6 T

Specific heat measurements were performed for sintered pellets of down to 0.3 K at zero-field on a physical property measurement system (PPMS, Quantum Design), and the He-3 refrigerator insert was applied for temperatures below 1.8 K. Moreover, data under 3 T, 4 T, 5 T, 6 T, 7 T, 8 T, and 9 T applied fields were also collected. The magnetic signal of specific heat was determined by subtracting the data of non-magnetic compound between 1.8 K and 30 K. The calculated magnetic entropy was obtained by integrating Cmag/T between 0.3 K and 20 K.

Magnetic properties were measured by using several different instruments. Isothermal DC magnetization up to 9 T was collected on a PPMS with a vibrating sample magnetometer insert, and isothermal AC susceptibility in applied fields up to 7 T from 0.5 K to 20 K was obtained on a magnetic property measurement system (MPMS3, Quantum Design). The DC magnetic susceptibilities versus H up to 18 T at 0.8 K were collected at Wuhan National High Magnetic Field Center. Torque magnetometry was performed at the University of Michigan using a self-built capacitive cantilever setup mounted inside a dilution refrigerator.

3 Neutron diffraction and magnetic structure

The rare-earth ions in the chalcogenides (Ch = O, S, Se) have an ideal triangular lattice structure with the magnetic ions localized at the center of chalcogenide octahedra, Fig. 1(a), and the triangular layers are well separated by the non-magnetic octahedra. Since the difference between the and ionic sizes is large, the anti-site disorder is eliminated. The single-crystal XRD data have confirmed that the anti-site disorder between them is less than 4%. has a perfect triangular lattice, and the intrinsic properties of the QSL, if there exists any, would be more clearly presented. We perform the neutron powder diffraction on NaYbS2 and compare the results at 50 mK and 300 mK, Fig. 1(b). From the labeled lattice reflections, the overplotting agreement suggests no long-range magnetic ordering down to 50 mK in this system. The space groups of is \(R\bar{3}\)m, which is same as and different to the P-lattice type triangular oxides with a rotation axis \(C_{3}\) or \(C_{6}\) of the \(Z_{3}\) symmetry, such as and [68, 69]. Meanwhile, the external magnetic field could break the degenerated ground state and induce complicated magnetic structures ([57] and see Additional file 1).

To determine the magnetic structure of new phases under the field, single-crystal neutron diffraction was applied, Fig. 1(c). Although up-up-down (uud) plateau phase has been reported to be induced by the magnetic field in the isostructure [57] and has the similar powder neutron diffraction spectra except an extra reflection at \(\boldsymbol{q}\approx 1.2429\) Å−1 (see Additional file 1), surprisingly, the reflections of \((1/3,1/3,1)\) and \((1/3,1/3,2)\) were too weak to be observed under the field, which is different to the powder data of ([57] and see Additional file 1). Based on the program SARAh [70] and the magnetic symmetry approach at the Bilbao Crystallographic Server [71], the magnetic Yb3+ ions spiral upwards with a 120° state and the equivalent reflections of \((2/3,-1/3,1)\) and \((2/3,-1/3,-2)\) are instead of \((1/3,1/3,1)\) and \((1/3,1/3,2)\), Fig. 1(d). Therefore, the magnetic wave vector is \((1/3,1/3,0)\). As the magnetic field increased, the reflections of \((1/3,1/3,0)\) and \((1/3,1/3,3)\) disappeared above 8 T. Since this high field phase does not have the wave vector of \((1/3,1/3)\) in ab-plane, it is disappointed to minimize the possibility of reported \(uud\) plateau in .

Compared to the M(H) measurements with the van Vleck correction, the saturated magnetization is 1.0\(\mu _{B}\) per ion and the moment of the intermediate phase is 0.5\(\mu _{B}\) per ion with a constant dM/dH, Fig. 2(a). The plateau phase has been predicted by Ye et al. as an up-up-up-down (uuud) state and breaks either \(Z_{3}\) orientational symmetry or \(Z_{4}\) sublattice symmetry with the competition of the magnetic field and quantum fluctuations [72]. Therefore, the subsequent state above the 120° state is an uuud state, which has been realized on the triangular lattice antiferromagnet Fe1/3NbS2 [73] and [74], and the stabilization of such a state requires further neighbor exchange interaction. A recent study on the diamond lattice antiferromagnet does suggest the importance of the second neighbor exchange interaction [75]. Moreover, the magnetic wave vector of \(uuud\) phase shifts from \((1/3,1/3,0)\) to \((0,1/2,0)\) in [76, 77]. On further increasing the magnetic field, the \(uuud\) phase undergoes a second-order phase transition to a high-field oblique phase, commonly to be observed in triangular lattices [72].

Figure 2
figure 2

The magnetization, torque magnetometry, specific heat and the phase diagram with H//ab-plane of NaYbS2. (a) DC magnetic susceptibility versus H and the related derivative dM/dH versus H at 0.8 K. The anomaly fields are ∼ 3.3 T, 6.1 T, 10.2 T and 14.8 T. The dash lines present the saturated magnetization of 1.0\(\mu _{B}\) per Yb ion with van Vleck correction and the intermediate Ms/2 anomaly-phase of 0.5\(\mu _{B}\) per Yb ion, respectively. (b) The temperature-dependence of the torque magnetometry in DC field. The fields marked by the black arrows are suggested as the magnetic phase transition fields. (c) The specific heat of NaYbS2 and NaLuS2, respectively. Inset presents the linear relationship of Cp/T vs T below 0.7 K. (d) The magnetic field-dependence of specific heat divided by T as a function of temperature and the related magnetic entropy. (e) The proposed magnetic phase diagram of temperature versus magnetic field for NaYbS2 with different techniques

4 Thermodynamic data at finite temperature and phase diagram

To reveal the magnetic phases in detail, the DC and AC magnetizations and the torque magnetometry were measured. For zero field, the Curie-Weiss temperatures of \({\Theta _{\text{CW},\perp}= -13.5 \text{ K}}\) and \({\Theta _{\text{CW},\parallel}=-4.5 \text{ K}}\) for the in-plane and out-of-plane susceptibility measurements [56] present the related frustration parameters as \(f>|\Theta _{\text{CW},\perp}|/(0.05 \text{ K})\approx 270\) for the in-plane value and \({f>|\Theta _{\text{CW},\parallel}|/(0.05 \text{ K})\approx 90}\) for the out-of-plane value. Meanwhile, the larger in-plane Curie-Weiss temperature expects to enhance the spin correlation between the in-plane spin components and contribute mostly to the diffusive scattering peak. The exchange energy scale of NaYbS2 is a couple times larger than the ones in YbMgGaO4, consistent with the enhanced Curie-Weiss temperatures. This advantage allows a broader temperature window to explore the QSL physics in this system. Furthermore, it is noted that the magnetic susceptibility in the zero field limit is constant. Although it seems to be consistent with a spinon Fermi surface state, this is expected from the fact that the total magnetization is not a conserved quantity. Thus, the constant spin susceptibility cannot be used to identify the QSL ground state for this material and other materials. Unlike NaYbO2, NaYbS2 demonstrates quite rich phases with a higher saturated magnetization field (up to ∼ 16.0 T), Fig. 2: there are four transition fields for DC susceptibility at 0.8 K, ∼ 3.3 T, 6.1 T, 10.2 T, and 14.8 T, respectively, and are consistent with the transition fields by the AC susceptibility very well (see Additional file 1). Figure 2(b) presented the torque magnetometry measurements by applying the field in the ab-plane. The dM/dH curves between 30 mK and 600 mK show two peaks on the lower and upper critical fields. As the temperature increases, this derivative becomes weaker and finally disappears around \({T=1}\) K. Finally, the sublattice spins are aligned along the applied field and saturated.

The low-temperature specific heat measurements on were performed down to 0.3 K at zero-field, Fig. 2(c). As the neutron powder diffraction measurement, no signature of magnetic transition is observed, and a broad peak occurs at about 1 K, a typical phenomenon of the QSL candidate materials. The lattice phonon contribution to the specific heat is obtained from the isostructural non-magnetic material and could be almost negligible below 3 K, Fig. 2(c). The entropy release for from 0.3 K to 20 K saturates up to 92% of the \(R\ln 2\) entropy (where R is the ideal gas constant), and an effective spin-1/2 description of the local moments is obtained. Unlike , \(C_{P}/T\) of NaYbS2 does not diverge at low temperatures, while the divergence in is interpreted as the signature of U(1) gauge fluctuations [29, 30]. The intercept of \(C_{P}/T\) on the \({T=0}\) axis is slightly larger than the one for , and the specific heat fits well with a \(T^{2}\) behavior below 0.7 K, the inset in Fig. 2(c), which is consistent with a gapless QSL and the Lorentz invariance guarantees. A residual density of states that contribute to the small intercept could be regarded [78] as the tail of the contributions of nuclear spin and becomes more and more significant below 0.1 K.

As the external magnetic field is applied, the specific heat demonstrates a series of highly tunable states, which relate to the quantum and thermal fluctuations, Fig. 2(d): a broad peak starts to be observed at 3 T, then a sharp peak appears at 5 T around ∼ 1.3 K and turns to be suppressed gradually with the increasing field. Finally another broad peak is split from the sharp peak and shifts to the lower temperature at 7 T. Those transitions could also be recognized by the magnetic entropy, which decreases from 92% (0 T) to 86% (3 T), then increases 88% (5 T), and drops again to 83% (9 T). Meanwhile, the low-temperature specific heat below the peak temperature is strongly suppressed, and \(C_{P}/T\) actually goes to zero in the zero temperature limit. Hence, the magnetic field breaks the QSL ground state and induces a range of intermediate states before the magnetic ions in are fully saturated.

The temperature versus magnetic field diagram is plotted in Fig. 2(e). The phase transitions are determined by combining different techniques, and the various exotic states from 120° to the \(uuud\) Ms/2 anomaly-phase to oblique phases were decided. Especially, the thermal effect is realized in the system even at low temperatures, ∼ 0.8 K, with the S-shape transition between the Ms/2- or Ms/3- anomaly phase and oblique phases.

5 Inelastic neutron scattering and gapless spin liquid

We perform the inelastic neutron scattering (INS) measurements on both at \(H=0 \text{ T}\) and 6 T at 50 mK. This measurement contains the dynamical energy-momentum information about the magnetic excitations in the system. In Fig. 3(a), highly dispersive signals are revealed. It is proposed a U(1) spinon Fermi surface spin liquid in with a clear V-shape at Γ point [30]. Several key features can be lost for the powder sample of due to the lack of angular momentum information. However, there still exits a cone feature that can be distinguished at \(|\boldsymbol{Q}|\approx 1.24\) Å−1 from Fig. 3(a), which should correspond to the cone-like feature of the Dirac spin liquid. The inter-Dirac cone scattering and intra-Dirac cone scattering processes would present these characters at low energies. Meanwhile, for the spinon Fermi surface states, a large amount of low-energy intensity is expected in a wider momentum range, which is clearly incompatible with the experimental data. For example, [79] is a quantum spin liquid candidate with spinon Fermi surface states. With magnetic field \(H=6 \text{ T}\), as depicted in Fig. 3(b) and the insets of constant-energy and -moment cuts in Fig. 3(c) and (d), the low-energy spectral weight is mostly transferred to higher energies, consistent with specific heat data that this field-induced state should be gapped due to the anisotropic spin interactions between the local moments, which is different to with the bigger ion on the Ch-site. Moreover, the temperature dependence of the energy- and moment-integrated intensities, Fig. 3(c) and (d), clearly presented that the V-shape magnetic signals start to be observed around 8 K.

Figure 3
figure 3

Inelastic neutron scattering of NaYbS2. (a), (b) Powder-averaged inelastic neutron scattering spectra of NaYbS2 at \({T = 0.05}\) K, \({H = 0}\) T and 6 T, respectively. There is no signal observed below 1 meV at \({H = 6}\) T and an energy gap is suggested. (c), (d) The temperature dependence of the energy-integrated intensity (from 0.2 meV to 1.0 meV) and momentum-integrated intensity (from 0.9 Å−1 to 1.3 Å−1), respectively. Although there is no magnetic long-range ordering down to 50 mK at 0 T, the magnetic dynamics starts around 8 K. The insets present the related magnetic field dependence of NaYbS2 at 0.05 K. The constant-energy cut (c) demonstrates that the magnetic signal at 6 T is still between 0.9 Å−1 and 1.3 Å−1; the constant-Q cut (d) demonstrates that there is no magnetic signal below 0.8 meV at 6 T and the gap is generated. Error bars correspond to one standard error, and the high intensity at the upper energy limits is the artefact from the background of the spectrometer and magnet

Furthermore, was also suggested as a gapless quantum spin liquid by longitudinal field (LF) muon spin relaxation (μSR) [80]. At 0.1 K, an indicator as the spin relaxation rate, \(\lambda _{LF}\), was applied to describe the spin dynamics and obtained from LF-μSR experimental data of . As the magnetic field increased to 1000 G, \(\lambda _{LF}\) was almost close to zero, significantly different from the LF-μSR spin relaxation rate \(\lambda _{LF}\) of quantum spin liquids with spinon Fermi surface state. For example, the quantum spin liquid material demonstrated a spinon Fermi surface state [79], and the spin relaxation rate \(\lambda _{LF}\) maintained a constant value of ∼0.2 \(\mu s^{-1}\) at 0.1 K and the magnetic field of 1000 G [80]. Although both and had magnetic ground states with quantum spin liquid states, and the crystallographic structures were close, the spinon excitation characteristics were significantly different. Therefore, the substitution of the oxychloride element effect was clearly demonstrated, and the crystal electric field played an important role in regulating quantum spin liquids [81].

6 Summary

Comparing to , the bigger ion on the Ch-site not only prevents crystallographic site-mixing, but also inceases the interlayer distances via the mediating cation, therefore the layer distance along the c axis is reduced significantly in with the slightly larger a axis and leads to the \(c/a\) ratio for (5.1) is more closer to (7.4) than (4.9). Rather than a pure two-dimensional model with anisotropic exchanges, a more precise theoretical analysis should naturally include other factors, such as the interlayer couplings [57]. Although the spins in the oxide are expected to be more localized than in sulfide, the quantum effects from both spin- and thermal-fluctuations in the latter are more complicated. Additionally, the ground state of can be driven into a magnetically ordered state in intermediate magnetic fields. Due to the strong easy-plane exchange anisotropy of , the numerical studies suggest a canted 120° state or an incommensurate state rather than an \(uud\) state in , which is also confirmed by the magnetization and neutron diffraction measurements.

Our experiments demonstrate that the nearly ideal triangular lattice of Yb ions in strongly spin-orbit-coupled materials can realize various exotic ground states. Especially, both the thermodynamic and the neutron scattering measurements suggest realizes a gapless spin liquid state. According to the fermion doubling theorem, there cannot be a single Dirac cone in a lattice system, and the cones must at least come in pairs. INS experiments measure the dynamical spin structural factor, which corresponds to the particle-hole pair of spinon excitations, and the intra-cone scattering will contribute to low energy spin excitations near Γ point. In contrast, the inter-cone scattering corresponds to low energy spin excitations at finite momentum [30, 49]. Furthermore, the scenario of staggered π-flux Dirac spin liquid would double the unit cell of spinons. This results in an enhanced periodicity of the dynamical spin structure factor despite the lack of magnetic ordering, which can be identified as a sharp feature to distinguish this peculiar fractionalized state from trivial spin glass. Although the density-of-state decided the moment-dependence of the spin dynamics and no gap above 0.1 meV, the clear feature of enhanced periodicity is smeared out due to lack of angular resolution, and more experimental efforts on single crystals are highly desired to distinguish if this system really hosts this π-flux gapless (or Dirac) spin liquid state.