Introduction

Introducing magnetic order into electron band structure presents an exciting opportunity to realize topological phases otherwise unavailable in non-magnetic systems. Such phases can emerge from perturbations due to magnetism on the electronic band structure’s geometry and topology. Anomalous Hall effect (AHE) in metallic ferromagnets is the most direct consequence of such perturbations. However, the more interesting case of magnetic topological band structure lies with antiferromagnets, which provide several interesting situations and could, in principle, offer unseen properties arising simply from the enormous number of possible spin configurations1,2. One of the most remarkable properties is the sizable AHE in a metallic antiferromagnet despite its zero or vanishingly small net magnetization, which is roughly proportional to AHE in the case of ferromagnets. Thus, a different organizing principle than magnetization on its own should govern the AHE in antiferromagnets.

To understand AHE in metallic antiferromagnets, one should identify an effective order parameter that characterizes the symmetry-breaking field that leads to finite AHE. One example is AHE from non-coplanar magnetic order (also known as topological Hall effect), which can be well characterized by scalar spin chirality3,4. More interestingly, it has been recently suggested that noncollinear antiferromagnets with coplanar magnetic order can also manifest a large AHE with the aid of spin-orbit coupling5,6,7,8,9. In this case, AHE can be characterized by the cluster multipole moment10,11. Despite such an exciting organizing principle, however, there are still no relevant experimental results except for the only one known example of such: that is, the cluster octupole moments in Mn3X (X = Ge, Sn, Ir)1,2,10,11,12. In other words, the richness of noncollinear antiferromagnetic configuration space with non-trivial band structure is just a hypothetical proposition at the moment. Hence, finding another noncollinear antiferromagnet is the crucial way forward to materialize the untapped potential of antiferromagnetic metallic systems, which would give access to AHE originating from a much more diverse multipole magnetism, where unique physics may be found in each case. This may, in particular, hold for a multipole called toroidal moment. Because it requires breaking both space inversion and time-reversal symmetry (TRS), the ordering of toroidal moments (i.e., ferro-toroidal order) is distinct from that of magnetic multipoles13 and therefore can manifest some non-trivial phenomena. However, due to the scarcity of suitable materials, relatively less attention has been paid to this multipole class so far, which prevents a complete understanding of its connection to macroscopic quantities measured by experiments.

TM1/3MS2 (TM = 3d transition metals and M = Ta, Nb) are promising materials for studying metallic antiferromagnetism in a triangular lattice14,15. It consists of well-known metallic 2H-MS2 with triangular layers of TMs intercalating into a vdW gap of 2H-MS2 (Fig. 1a). Such intercalation slightly alters the atomic positions of sulfur atoms, leading to the chiral structure with a non-centrosymmetric P6322 space group (No. 182)16. The charge transfer from TM to 2H-MS2 forms divalent (TM2+) or trivalent (TM3+) TM ions, and subsequently localized magnetic moments settle down in TM sites (Fig. 1b). Interestingly, this material class exhibits various magnetic ground states depending on the choice of TM and M14. Among the combinations, Co1/3TaS2 is the only material with the noncollinear 120° magnetic order17, a natural outcome due to geometrical frustration (Fig. 1b). Therefore, Co1/3TaS2 offers a combination of chirality, triangular lattice, and 120° magnetic structure, unique among the current set of magnetic topological materials. Moreover, TM1/3MS2 can be thinned down to a few atomic layers through mechanical exfoliation18. Hence, it could also open up an entirely new route to study noncollinear antiferromagnetism and corresponding topological physics in genuine two-dimension.

Fig. 1: Structure and bulk properties of noncollinear metallic triangular antiferromagnet Co1/3TaS2.
figure 1

a A crystallographic unit cell of Co1/3TaS2, where Co atoms are intercalated into octahedral sites. The image shows Co1/3TaS2 single crystals with the solid orange line as long as 10 mm. b Noncollinear magnetic order (Γ2) formed by magnetic moments of divalent Co ions. c The magnetization of Co1/3TaS2 as a function of temperature along the three crystallographic axes. The inset shows the Curie-Weiss behavior of Co1/3TaS2. d The field dependence of the magnetization along the c-axis at 5 K. In addition to apparent hysteresis from weak ferromagnetic domains (±Hc1), there is a jump of magnetization at ±Hc2. e Temperature-dependent resistivity of Co1/3TaS2 where its first derivative (inset) demonstrates two phase transitions, consistent with the magnetization data. f Specific heat of Co1/3TaS2 as a function of temperature. The inset shows the specific heat after subtracting the non-magnetic contribution (see “Methods”).

This work reports that single-crystal Co1/3TaS2 is a metallic triangular antiferromagnet where coplanar 120° magnetic order plays a crucial role in forming electron band topology and concomitant AHE. Our density functional theory (DFT) calculations show that AHE is triggered by hourglass Weyl fermions enforced by non-symmorphic symmetry under broken TRS. Moreover, we explain how the toroidal moment, a multipole component in non-centrosymmetric chiral antiferromagnets13, can characterize AHE in Co1/3TaS2. Finally, we discuss the remarkable tunability of the toroidal moment under an out-of-plane magnetic field, which implies a profound correlation between AHE and the magnetism of Co1/3TaS2.

Results and discussion

Bulk properties

Temperature-dependent magnetization (M), resistivity \((\rho _{xx})\), and specific heat (\(C_{{{\mathrm{p}}}}\)) of Co1/3TaS2 indicate two magnetic phase transitions at TN1 = 38 K and TN2 = 26.5 K (Fig. 1c–f). The major transition occurs at TN2, where all the properties change significantly. The negative Curie-Weiss temperature \((\theta _{{{{\mathrm{CW}}}}})\) with a significant frustration factor \(f \equiv \frac{{\left| {\theta _{{{{\mathrm{CW}}}}}} \right|}}{{T_{{{{\mathrm{N}}}}2}}}\sim 4\) indicates dominant antiferromagnetic interactions with frustration, consistent with the formation of the 120° order. Interestingly, the phase below TN2 involves a weak ferromagnetic moment of 0.01 μB/Co2+ along the c-axis (\(M_{{{\mathrm{z}}}}({{{\mathbf{H}}}} = 0)\), as shown in the field dependence of Mz with hysteresis (Fig. 1d). In addition to the domain switch of \(M_{{{\mathrm{z}}}}({{{\mathbf{H}}}} = 0)\)) at ±Hc1, there is a jump at ±Hc2 due to a meta-magnetic transition. The mechanism of this meta-magnetic transition remains unclear due to limited information, which requires further studies.

Most interestingly, the AHE (\(\rho _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\)) appears below TN2, despite the antiferromagnetic nature of Co1/3TaS2 (Fig. 2a). The anomalous Hall conductivity (AHC, \(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\)) reaches a value of \(70\Omega ^{ - 1}{{{\mathrm{cm}}}}^{ - 1}\) at 2 K, comparable to those measured in ferromagnetic materials3. After excluding the linear Hall effect, the magnitude of the \(\rho _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}})\) remains nearly unchanged with varying a magnetic field, except for a jump at Hc2. Also, the observed hysteresis of AHE with a significant coercive field Hc1 is quite similar to that of the weak ferromagnetic moment (Fig. 2b), particularly in that they simultaneously flip to the opposite domain. This connection between the two quantities shall be discussed later in this paper.

Fig. 2: Anomalous Hall effect and its comparison to weak ferromagnetism.
figure 2

a, b The field dependence of Hall resistivity \(\rho _{{{{\mathrm{xy}}}}}\) and magnetization along the c-axis at several temperatures. c Temperature dependence of Hc1 (circles) and Hc2 (triangles) determined from the \(\rho _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}})\) (hollow symbols) and \(M_{{{\mathrm{z}}}}({{{\mathbf{H}}}})\) (filled symbols) curves. d, e Temperature-dependent zero-field Hall conductivity (\(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\)) and zero-field magnetization along the c-axis \((M_{{{\mathrm{z}}}}(H = 0))\), measured after a field cooling process (see “Methods”). Inset shows the critical behavior of \(\sigma _{xy}(T,{{{\mathbf{H}}}} = 0)\) and \(M_{{{\mathrm{z}}}}(T,{{{\mathbf{H}}}} = 0)\) near 26.5 K and fitted results (black dashed lines) with β = 0.499 and 0.492, respectively. f A plot of the longitudinal and zero-field anomalous Hall conductivity for several CoxTaS2 samples (0.31 < x < 0.325, see Supplementary Fig. 6), which were extracted from the \({\uprho}_{{{{\mathrm{xy}}}}}\left( {{{\mathbf{H}}}} \right)\) curves measured below 15 K.

The shape of the hysteresis itself is interesting and merits separate comments. First of all, the meta-magnetic transition at ±Hc2 already starts to appear when TN2 < T < TN1 (Supplementary Fig. 5), indicating that the meta-magnetic transition is more related to the magnetic order corresponding to TN1. Second, the domain switch at Hc1 and the meta-magnetic transition at Hc2 are not independent. While Hc2 remains nearly unchanged, Hc1 increases rapidly with decreasing temperature from TN2 and reaches almost 9 T at 2 K (Fig. 2c). Indeed, the crossover between Hc1 and Hc2 should occur somewhere below TN2, around 14 K (Fig. 2c). Surprisingly, the domain switch and the meta-magnetic transition happen simultaneously at Hc1 after the crossover (|Hc1| > |Hc2|), resulting in a peculiar shape of hysteresis below 15 K (Fig. 2a, b). Thus, the two-fold magnetic domain with respect to the c-axis, the meta-magnetic transition at ±Hc2, and the magnetic order that forms at TN1 and TN2 are deeply related to each other, which requires further studies for a complete understanding.

Moreover, the temperature evolution of \(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\) in Co1/3TaS2 behaves just like an order parameter of the phase transition at TN2 (Fig. 2d), implying that the observed AHE is closely related to the magnetic order of Co1/3TaS2. This hypothesis is further supported by the fact that the critical exponent β of \({\upsigma}_{{{{\mathrm{xy}}}}}\left( {T,{{{\mathbf{H}}}} = 0} \right)\) is nearly identical to that of \(M_{{{\mathrm{z}}}}(T,{{{\mathbf{H}}}} = 0)\), as shown in Fig. 2e. Note that the fitted β is close to 0.5, consistent with the fact that the magnetic order mediated by long-ranged RKKY interaction follows the mean-field critical behavior19. Finally, we compared the magnitude of \(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\) measured below 15 K for several samples with different Co compositions (0.310 < x < 0.325). While \(\rho _{{{{\mathrm{xx}}}}}\) and \(\rho _{{{{\mathrm{xy}}}}}\) vary depending on the samples, \(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\)) and the shape of the hysteresis remain nearly the same for all the samples (Fig. 2f and Supplementary Fig. 6).

Theoretical understanding of AHE

The observed AHC’s robustness and its close connection to the magnetic order imply the intrinsic nature of the AHE in Co1/3TaS2. The following two possibilities are not likely the situation of Co1/3TaS2, based on our findings regarding its microscopic relation to the magnetic order. First, although Co1/3TaS2 has a small weak ferromagnetic moment, the observed AHE in Co1/3TaS2 cannot be totally attributed to the weak ferromagnetic moment itself. This is because the field evolution of AHE does not match with that of Mz; see \({{{\mathrm{d}}}}M_{{{\mathrm{z}}}}/{{{\mathrm{d}}}}H\) and \({{{\mathrm{d}}}}\rho _{{{{\mathrm{xy}}}}}/{{{\mathrm{d}}}}H\) at Hc1 and Hc2. Also, \(M_{{{\mathrm{z}}}}\left( {{{{\mathbf{H}}}} = 0} \right)\sim 0.01\) μB/Co2+ itself is too small to induce \(\sigma _{{{{\mathrm{xy}}}}}\left( {{{{\mathbf{H}}}} = 0} \right)\) as large as \(\sim 70\Omega ^{ - 1}{{{\mathrm{cm}}}}^{ - 1}\). Nor could it originate from the scalar spin chirality due to the slight out-of-plane canting, as the spin chirality is usually canceled out in a triangular lattice antiferromagnet with 120° order20. Therefore, understanding AHE in Co1/3TaS2 requires a new idea related to the coplanar 120° spin order itself, analogous to the case in noncollinear kagome antiferromagnet Mn3X (X = Ge, Sn)5,7.

We first begin with symmetry arguments for the possible 120° magnetic order in Co1/3TaS2, which can formulate the existence of AHE. According to the group-theoretical analysis21, the possible magnetic ground states can be described by six irreducible representations referred to as Γ16 (Supplementary Fig. 8). Note that Γ1, Γ2, and Γ5 spin configurations can be transformed into each other through uniform spin rotation around the c-axis (θ in Fig. 3a). Notably, the two-fold rotation (C2a or C2a*) and the 63 screw symmetry combined with time-reversal (\(\equiv\)63) forbid finite \(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\) and \(M_{{{\mathrm{z}}}}({{{\mathbf{H}}}} = 0)\) in Co1/3TaS2: e.g., Γ1 (P6322) in Fig. 3a (for more explanation, see Supplementary Text). Consequently, only Γ2 (P6322) and the Γ5 (P63) can possess finite \(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\) and \(M_{{{\mathrm{z}}}}\left( {{{{\mathbf{H}}}} = 0} \right)\) and explain our experimental results. Yet Γ5 is a two-dimensional irreducible representation with much lower symmetry, and therefore, is less likely to occur in natural materials, as pointed out in previous studies of two-dimensional triangular antiferromagnets22. Thus, we focus on the Γ2 spin configuration in our analysis (Fig. 3b, c). Our data indeed support such a choice as magnetic susceptibility measured along the a-axis is slightly larger than that along the a*-axis (Fig. 1c); aligning the spins along the a-axis or its symmetrically equivalent directions is energetically more favorable (i.e., local easy-axes), which is the case of Γ2 as shown in Fig. 3b. Also, as shown in Fig. 3d, the total energy obtained from our DFT calculation reaches its minimum when θ = 0° (\(= \Gamma _2^ +\)) or 180° (\(= \Gamma _2^ -\)), which is another evidence supporting the Γ2 ground state.

Fig. 3: Interplay between a field-tunable toroidal moment and AHE.
figure 3

a The Γ1 spin configuration having a magnetic space group P6322, in which the symmetry forbids finite \(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\) and \(M_{{{\mathrm{z}}}}({{{\mathbf{H}}}} = 0)\). b, c The Γ2 spin configuration having a magnetic space group P6322. No symmetry operation forces \(\sigma _{{{{\mathrm{xy}}}}}\) and \(M_{{{\mathrm{z}}}}\) to vanish. Three sky-blue ellipses in a, b denote the three-fold easy-axes to realize the Γ1 or Γ2 ground state, respectively. d Total energy from the DFT calculation as a function of the uniform spin rotation angle \(\theta\) shown in a. e Comparison between a toroidal moment along the c-axis (a solid blue line) and calculated \(\sigma _{xy}\) for the spin configurations specified by \(\theta\) in a (black circles). f Interpreting the sign change of AHE as a switch of the toroidal moment. A blue (orange) rectangle denotes a region of the \(\Gamma _2^ +\) (\(\Gamma _2^ -\)) spin configuration.

The observed intrinsic AHE can be understood as a manifestation of novel Weyl fermions in the electron band structure of Co1/3TaS2, where non-symmorphic symmetry enforces their existence. Notably, the crystal structure of Co1/3TaS2 has the 63 screw symmetry that remains unbroken in the Γ2 representation. Such non-symmorphic symmetry could result in specific Weyl crossings in the k-space called hourglass Weyl fermion23,24,25. Figure 4a shows the schematic band dispersion representing a typical situation of hourglass Weyl fermions. Along the screw-invariant line connecting time-reversal invariant momenta (TRIM) points (say, Γ–A and M–L), there must be band crossings as guaranteed by screw symmetry. By half-translation, the eigenstates of the \(2_1\) screw symmetry operation acquire a momentum-dependent phase factor of \({{{\mathrm{e}}}}^{ - {{{\mathrm{i}}}}k_zc/2}\) (c is the lattice constant along the c-axis), and it then causes Kramers’ pair exchange (indicated by using color code) as \(k_{{{\mathrm{z}}}}\) moves from one TRIM point to another. Consequently, the pairs of bands must cross at least once within the invariant space, which naturally makes the band dispersion have the ‘hourglass’ shape and creates inevitable Weyl crossing (olive circles in Fig. 4a).

Fig. 4: Hourglass Weyl dispersion with noncollinear magnetic order.
figure 4

a A schematic band connectivity diagram for a screw-invariant line. The color scale indicates the \(2_1\) screw rotation eigenvalues of the Bloch bands. Kramers pairs with eigenvalues (i, −i) are paired at \(k_z = 0\) (Γ, M), and those with (1, 1) or (−1, −1) are paired at \(k_z = \pi /c\)(A, L). The olive circles represent the Weyl point. The lower right figure shows the Brillouin zone of Co1/3TaS2 with high symmetry points being marked. b The calculated band structure of the \(\Gamma _2^ +\) spin configuration of Co1/3TaS2. The Ta-5d portions are depicted by color code. Fermi level is set to zero energy. The yellow-colored regions correspond to the hourglass Weyl dispersion, some of which are enlarged in ce. ce The calculated band structure along the \(\overline {\Gamma {{{\mathrm{A}}}}}\), \(\overline {{{{\mathrm{ML}}}}}\) and \(\overline {{{{\mathrm{KH}}}}}\) lines, respectively (some of the yellow regions in b). Due to the broken TRS, the band degeneracy at the TRIM points is lifted. The color plot in e displays the calculated Berry curvature |\({{\Omega }}_{{{{\mathrm{xy}}}}}\)| near the Weyl point designated by an arrow in a logarithmic scale. f The calculated AHC by integrating over the whole Brillouin zone (BZ) (orange color) and only around the \(\overline {{{{\mathrm{KH}}}}}\) line (black), respectively. \(\theta\) denotes the angle of uniform spin rotation around the c-axis; see Fig. 3a.

To confirm this scenario for Co1/3TaS2, we carried out DFT calculations with the Γ2 spin configuration (Fig. 4b): we also tested other possible spin configurations with less promising results. As marked in color, certain pockets of the k-space contain the anticipated novel hourglass type dispersion enforced by the 63 screw symmetry. It is especially notable that, protected by the screw symmetry, the Weyl “crossing” point in the hourglass dispersion can be preserved even under TRS breaking26. Figure 4c, d shows the enlarged band dispersion along the \(\overline {\Gamma {{{\mathrm{A}}}}}\) and \(\overline {{{{\mathrm{ML}}}}}\) lines using the Γ2 spin configuration (Fig. 4b), where the color scale indicates the \(2_1\) screw rotation eigenvalues. Due to the absence of TRS, \(\Gamma\), \({{{\mathrm{A}}}}\), \({{{\mathrm{M}}}},\) and \({{{\mathrm{L}}}}\) are no longer TRIM points (Fig. 4c, d). Thus, the Kramers’ degeneracies are lifted at these points (indicated by gray arrows in Fig. 4d), although the degeneracy lifting is not visible at the Γ and A points in this energy scale. However, the hourglass dispersion and the Weyl crossings (the olive circles in Fig. 4c, d) are still well maintained. This is clearly shown when comparing Fig. 4c, d with Fig. 4a; the evolution of the screw operation’s eigenvalue is essentially the same as each other. We note that few theoretical studies have shown the hourglass Weyl fermions with broken TRS, which adopted simple ferromagnetic or collinear antiferromagnetic spin order27,28,29.

From the AHC’s point of view, however, the Weyl crossings along the \(\overline {\Gamma {{{\mathrm{A}}}}}\) and \(\overline {{{{\mathrm{ML}}}}}\) lines, albeit interesting, cannot be responsible for the observed AHE. It is simply because their locations are too far off from the Fermi level (see Fig. 4b). Instead, we found that the Weyl points on the \(\overline {{{{\mathrm{KH}}}}}\) line play a more critical role. Although both K and H points are not the TRIM points even with TRS, the existence of Weyl points on the \(\overline {{{{\mathrm{KH}}}}}\) line is guaranteed by the hourglass Weyl points on the \(\overline {\Gamma {{{\mathrm{A}}}}}\) and \(\overline {{{{\mathrm{ML}}}}}\) lines due to the fermion doubling theorem30,31. From the fact that Weyl points on the \(\overline {\Gamma {{{\mathrm{A}}}}}\) and \(\overline {{{{\mathrm{ML}}}}}\) line respectively have a multiplicity of 1 and 3 in half Brillouin zone (BZ), Weyl points are required to lie also on the \(\overline {{{{\mathrm{KH}}}}}\) line with a multiplicity of 2 to neutralize the chiral charge: \(\nu _{\Gamma {{{\mathrm{A}}}}} + 3\nu _{{{{\mathrm{ML}}}}} + 2\nu _{{{{\mathrm{KH}}}}} = 0\). As displayed in Fig. 4e, there are indeed multiple Weyl crossings along the \(\overline {{{{\mathrm{KH}}}}}\) line, generating Berry curvature like a magnetic monopole. Moreover, since they are located close enough to the Fermi level, they are likely to make a dominant contribution to the anomalous charge transport. The results presented in Fig. 4f constitute more convincing evidence supporting our argument. The black and orange line represents the calculated AHC obtained by integrating the whole BZ and only along the \(\overline {{{{\mathrm{KH}}}}}\) line, using several 120° spin configurations specified by \(\theta\) in Fig. 3a, i.e., using Γ1 (\(\theta = 90^\circ \,{{{\mathrm{or}}}}\,270^\circ\)), \(\Gamma _2(\theta = 0^\circ \,{{{\mathrm{or}}}}\,180^\circ )\), and Γ5(other angles) spin configurations. First, the calculated \(|\sigma _{{{{\mathrm{xy}}}}}|\) using the \(\Gamma _2\) configuration is about 90\({{\Omega }}^{ - 1}{{{\mathrm{cm}}}}^{ - 1}\), which is indeed in excellent agreement with our experimental results (Fig. 2d, f). Also, only a small difference is found between the \(\sigma _{{{{\mathrm{xy}}}}}\) calculated over the entire BZ and that obtained only along the \(\overline {{{{\mathrm{KH}}}}}\) line, implying that the most dominant contribution of \(\sigma _{{{{\mathrm{xy}}}}}\) comes from the Weyl points in this particular \(\overline {{{{\mathrm{KH}}}}}\) symmetry line. Taken together, the ‘vestige’ of hourglass Weyl fermions on the \({{{\overline {\Gamma A}}}}\) and \({{{\overline{M L}}}}\) lines, which remains intact under the 120° magnetic order, triggers the AHE in this material despite the broken TRS and the lifted degeneracy otherwise residing at Γ, A, M, and L.

AHE and toroidal moment

Figure 4f further demonstrates that the AHC strongly depends on the detailed in-plane spin configuration of Co1/3TaS2. To put it intuitively, a vortex-like arrangement inside the 120° spin configuration breaks the \(C_{2a}\) or \(C_{2a^ \ast }\) symmetry and allows finite \(\sigma _{{{{\mathrm{xy}}}}}({{{\mathbf{H}}}} = 0)\) in Co1/3TaS2. Figure 3a, b shows two clear cases: one is without the vortex-like arrangement of Γ1 and another is with the vortex-like arrangement of Γ2. Such an arrangement can be precisely quantified by the toroidal dipole moment (t)11,13,32:

$${\mathbf{t}} \equiv {\sum \limits_i} {{{\mathbf{r}}}}_i \times {{{\mathbf{S}}}}_i$$
(1)

where \({{{\mathbf{r}}}}_i\) is the position vector of the ith site from the center of the six Co atoms with hexagonal configuration (Fig. 3b). Unlike ordinary magnetic or electric multipoles, t breaks both inversion and TRS and consequently appears in a non-centrosymmetric chiral magnet with geometrical frustration13,32. Therefore, Co1/3TaS2 is a rare system that can have non-zero t, and so it is helpful for a proper understanding of its role in characterizing AHE, which has been suggested only in theory so far11. Figure 3e shows a complete comparison between tz and calculated \(\sigma _{{{{\mathrm{xy}}}}}\) for the Γ1, Γ2, and Γ5 spin configurations. As expected, it clearly reveals the aspect of tz as the order parameter characterizing the AHE.

From this point of view, we can interpret the \(\Gamma _2^ +\) (\(\theta = 0^\circ\)) and \(\Gamma _2^ -\) \((\theta = 180^\circ )\) spin configurations as a pair of ferro-toroidal states with an opposite sign of \(t_{{{\mathrm{z}}}}\) (Fig. 3b, c). Interestingly enough, the sign change of \(\sigma _{{{{\mathrm{xy}}}}}\) at ±Hc1 then corresponds to the transition between the \(\Gamma _2^ +\) and the \(\Gamma _2^ -\) configurations (Fig. 3f). In other words, an out-of-plane magnetic field switches the AHE of Co1/3TaS2 by manipulating the in-plane spin configuration. While such a consequence may appear at first sight somewhat surprising, it is the weak ferromagnetic moment that connects the out-of-plane field and the in-plane spin configuration in Co1/3TaS2. The simultaneous jump of \(\sigma _{{{{\mathrm{xy}}}}}\) and Mz seen in our experiments (Fig. 2a, b) implies that the sign of \(t_{{{\mathrm{z}}}}\) is coupled to the sign of \(M_{{{\mathrm{z}}}}({{{\mathbf{H}}}} = 0)\); i.e., the \(\Gamma _2^ +\) (\(\Gamma _2^ -\)) configuration always tends to have positive (negative) \(M_{{{\mathrm{z}}}}({{{\mathbf{H}}}} = 0)\). Consequently, the out-of-plane magnetic field can effectively control the sign of \(t_{{{\mathrm{z}}}}\) by flipping Mz(H = 0), consistent with the similar hysteretic behavior of \(\sigma _{{{{\mathrm{xy}}}}}\) to that of Mz (see Fig. 2a, b). The coexistence of the toroidal moment and the weak ferromagnetic moment in Co1/3TaS2 is not accidental. A combination of lattice chirality and toroidal spin configuration results in the broken two-fold rotation (C2a, C2a*), mirror, and TRS. Such a situation leads to the emergence of a finite vector quantity along the chiral axis (the c-axis in Co1/3TaS2), whose transformation property is equivalent to that of magnetization33,34. Of further interest, \(t_{{{\mathrm{z}}}}\) can be coupled to \(M_{{{\mathrm{z}}}}({{{\mathbf{H}}}} = 0)\) at the microscopic level, where a symmetry-allowed higher-order on-site anisotropy can clearly describe such a property in Co1/3TaS2 (see Supplementary Text and Supplementary Fig. 11).

We finally note that a two-fold crystallographic chiral domain related to left or right-handedness, which itself is another interesting topic in TM1/3MS2 (refs. 35,36), is deeply linked to the sign relation between \(t_{{{\mathrm{z}}}}\) and Mz; see Supplementary Text. Therefore, the sign relation suggested in Figs. 3, 4 is limited to one specific chiral domain. However, the presence of the chiral domain is not in conflict with our interpretation of the observed AHE, as it does not affect the measurement of \(\sigma _{{{{\mathrm{xy}}}}}\) (see Supplementary Text for more explanations).

To summarize, a combination of triangular lattice, chirality, and coplanar 120° spin order gives rise to an interesting feature in Co1/3TaS2: the interplay between the field-tunable toroidal moment and AHE. We have shown that it exhibits large AHE from the vestige of symmetry-protected hourglass Weyl fermions, which has not been reported before. Moreover, Co1/3TaS2 reported in this work is a rare cleavable antiferromagnetic metal: we confirmed that Co1/3TaS2 could be exfoliated down to a few-nm thickness (Supplementary Fig. 7). Therefore, it will further offer an exciting chance of studying noncollinear antiferromagnetism and relevant topological properties in a genuine two-dimensional limit, not to mention its rich bulk properties worth further studies.

Methods

Sample preparation and characterization

To obtain Co1/3TaS2 single crystals, we first synthesized polycrystalline Co1/3TaS2 using the solid-state reaction. A mixture of Co (Alfa Aesar, >99.99%), Ta (Sigma Aldrich, >99.99%), and S (Sigma Aldrich, >99.999%) in a molar ratio of 1.15:3:6 was well ground in an Ar-filled glove box and sintered at 900 °C for 10 days in an evacuated quartz ampoule. We measured the powder X-ray diffraction using a commercial machine (Smartlab, Rigaku Japan) to find that it forms in the expected hexagonal structure of the P6322 space group; the corresponding data are shown in Supplementary Fig. 1 and Supplementary Table 1. We grew Co1/3TaS2 single-crystal starting with the pre-reacted powder by a chemical vapor transport technique in an evacuated quartz tube with I2 as a transport agent (4.5 mg I2/cm3). The quartz tube was placed in a two-zone furnace with a temperature gradient from 940 to 860 °C for 7~10 days. Finally, we obtained shiny hexagonal crystal pieces (4~13 mm) shown in Fig. 1a.

The obtained single crystals were examined by high-resolution single-crystal x-ray diffraction (XtaLAB PRO, Cu \(K_\alpha\), Rigaku Japan) and Raman spectroscopy (XperRam Compact, Nanobase Korea); see Supplementary Figs. 2 and 3, and Supplementary Table 2. These two measurements confirmed the high quality of our samples (see Supplementary Text for more details). We determined the Co composition (x in CoxTaS2) using both energy-dispersive X-ray spectroscopy and inductively coupled plasma (ICP) spectroscopy (OPTIMA 8300, Perkin-Elmer USA). Both examinations allow us to give the exact value of x within a measurement error close to 1/3 (see Supplementary Table 3). Note that one can slightly tune the resultant stoichiometric ratio by changing excess Co in the starting mixture.

Bulk property measurements

Magnetic properties of Co1/3TaS2 were measured by the two commercial magnetometers (MPMS and PPMS-14 with the VSM option, Quantum Design USA). For the temperature-dependent weak ferromagnetic moment (Fig. 2e), we measured the magnetization of the field-cooled (9 T) sample from 2 to 80 K without a magnetic field. Specific heat of Co1/3TaS2 was measured by using the commercial setup (PPMS-14, Quantum design USA). The non-magnetic contribution to the specific heat was estimated by the summation of the Debye model and the electronic contribution linear with the temperature, where we used two Debye temperatures (\(\theta _{D1}\) = 270 K and \(\theta _{D2}\) = 611.7 K) for the fitting due to the coexistence of heavy (Ta) and light (S) elements.

Longitudinal and Hall voltage of Co1/3TaS2 were measured by a standard four-probe method using our home-built cryostat, PPMS-9 (Quantum Design, USA), and Teslatron PT (Oxford, UK). All the samples used for the transport measurements were prepared in a straight rectangular shape with a uniform thickness. Hall resistivity data were antisymmetrized to remove any accidental longitudinal signal from the contact misalignment. To observe the temperature dependence of the AHE separated from the ordinary Hall effect (Fig. 2d), we measured the Hall voltage of the field-cooled (± T) sample from 2 to 80 K without a magnetic field. We removed the longitudinal component by antisymmetrizing the two datasets measured after the field cool under +9 T and −9 T, respectively.

Density functional theory (DFT) Calculations

First-principles calculations were carried out using “Vienna ab initio simulation package”37,38,39 based on projector augmented wave potential40 and within Perdew-Burke-Ernzerhof type of GGA functional41. To properly take into account localized Co-3d orbitals, we adopted DFT + U method42,43 with the values of \(U = 4.1\,{{{\mathrm{eV}}}}\) and \(J_{Hund} = 0.8\,{{{\mathrm{eV}}}}\) as obtained by constrained RPA for CoO and Co44. We used the Γ-centered k-grid of 6 × 6 × 4 for the \(\sqrt 3 \times \sqrt 3\) magnetic supercell. The optimized crystal structure was used with the force criteria of \(1\,{{{\mathrm{meV}}}}/\mathop {{{{\mathrm{A}}}}}\limits^ \circ\). The plane-wave energy cutoff is 500 eV. The screw rotation eigenvalues of the Bloch bands were calculated using the “irvsp”45. To investigate the topological properties of electronic states, we obtained Wannier functions using the “WANNIER90” code46. The Berry curvature and anomalous Hall conductivity were calculated using the ‘WannierTools’ package47, using the following formula:

$$\Omega _{n,\alpha \beta } = 2\,{{{\mathrm{Im}}}}(\partial _{k_\alpha }u_{nk}{{{\mathrm{|}}}}\partial _{k_\beta }u_{nk})$$
(2)
$$\sigma _{n,\alpha \beta } = \frac{{e^2}}{{\left( {2{\uppi}} \right)^2h}}\mathop {\sum }\limits_n {\int} {d{{{\mathbf{k}}}}f_n\left( {{{\mathbf{k}}}} \right){{\Omega }}_{n,\alpha \beta }({{{\mathbf{k}}}})} ,$$
(3)

where \({{\Omega }}_{{{{\mathrm{n}}}},\alpha \beta }\) is Berry curvature, fn(k) is the Fermi distribution function at 11.6 K, n is the band index, α and \(\beta ( \ne \alpha )\) denote the Cartesian coordinates (xyz), and unk is the periodic part of the Bloch function. We found it essential to take a dense-enough k-grid for accurately estimating the AHC value. After the convergence test, we adopted 301 × 301 × 301.