Introduction

In general, the origin of the magnetically ordered states can be understood by considering the exchange interactions using the Heisenberg model, which accounts for a variety of magnetic orders ranging from simple collinear to complex non-collinear magnetic states. However, the energy landscape of magnetic interactions is not limited to the usual pairwise Heisenberg exchange (J), instead higher-order exchange terms corresponding to multiple order of hopping between different sites can be taken into account in suitable magnetic systems. The most notable interaction involving multi-spins in a magnetic Hamiltonian is the 4-spin exchange in the form of 4-spin-two-site (B), 4-spin-three-site (Y), and 4-spin-four-site (K) interactions (Eq. (1))1. A signature feature of the 4-spin exchange interactions is the stabilization of multi-Q states. The noncoplanar 3Q magnetic ordering detected in Mn/Cu(111) system and the unique 2Q ground state observed in Fe/Rh(111) can be well explained by considering the 4-spin exchange interaction2,3,4,5. In addition, certain noncoplanar spin structures can be characterized by scalar spin chirality (SSC), defined as Si. (Sj × Sk), which gives rise to an additional Hall response owing to the real space Berry phase. Such extraordinary Hall response has been previously observed in glassy systems6,7,8,9 and a limited number of pyrochlore lattices9,10,11,12,13.

The kagome lattice system, which can be constructed from the triangular lattice by removing one of the lattice points from the unit cell, exhibits a great prospect for the realization of various types of spin-ordering. Among them, Mn3Sn is one of the extensively studied materials due to its multiple magnetic ordered states that include a 120° triangular antiferromagnetic (AFM) state. The cluster octupole order of the triangular AFM state breaks the time-reversal symmetry, resulting in the observation of Weyl nodes14,15,16,17, which leads to the observation of large anomalous Hall effect in Mn3Sn and Mn3Ge18,19,20,21. In addition, the effect of cluster octupole order and/or Weyl nodes can also be seen from the realization of large spin/inverse spin Hall effect22,23, electrical switching of Hall signal24,25, and the Kondo effect26, among others. Interestingly, Mn3Sn displays a temperature-dependent modification of the magnetic ground state with different levels of electron doping27,28,29. However, the octupole order remains as the ground state at room temperature for all the electron-doped Mn3Sn samples. Below the room temperature, the existence of a helical modulation of the triangular spin structure along the c-axis has been reported for the near stoichiometric Mn3Sn samples27,28,30. For the higher electron-doped Mn3Sn samples, the presence of a weakly ferromagnetic and glassy magnetic state was also reported, where the glassy nature comes from the additional Mn atoms occupying the Sn 2c sites31,32. In addition, the possible existence of a canted magnetic structure at very low temperature is discussed in a recent report33. Furthermore, it has been reported that the electron doping in the form of Fe in place of Mn in Mn3−xFexSn can stabilize a ferromagnetic-like low-temperature magnetic phase without a saturated magnetic state34. Similarly, Liu et al. reports the presence of magnetic bubbles in the nearly ferromagnetic sample Mn2.1Fe0.9Sn35. The existence of multiple magnetic ground states is also proposed in the polycrystalline Mn2.5Fe0.6Sn0.936. However, the exact nature and mechanism that governs the low-temperature magnetic phase and related transport phenomena remain unclear.

Results

Effect of higher-order exchange on the magnetic ground state of Mn3Sn

The 120° triangular AFM ground state of Mn3Sn can be derived from the J with in-plane magnetic anisotropy30. Possible higher-order exchange interactions along with the nearest neighbor Heisenberg exchange and corresponding hopping paths for Mn3Sn are depicted in Fig. 1a. The Hamiltonian of the system with the 4-spin interactions can be written as,

$$\begin{array}{ll}H=\mathop{\sum}\limits_{ij}{J}_{ij}({S}_{i}.{S}_{j})+\mathop{\sum}\limits_{ij}{B}_{ij}{({S}_{i}.{S}_{j})}^{2}\\ \qquad+\,\mathop{\sum}\limits_{ijk}{Y}_{ijk}[({S}_{i}.{S}_{j})({S}_{j}.{S}_{k})+({S}_{j}.{S}_{i})({S}_{i}.{S}_{k})+({S}_{i}.{S}_{k})({S}_{k}.{S}_{j})]\\ \qquad+\,\mathop{\sum}\limits_{ijkl}{K}_{ijkl}[({S}_{i}.{S}_{j})({S}_{k}.{S}_{l})+({S}_{i}.{S}_{l})({S}_{j}.{S}_{k})-({S}_{i}.{S}_{k})({S}_{j}.{S}_{l})].\end{array}$$
(1)
Fig. 1: Higher-order magnetic interactions and scalar spin chirality.
figure 1

a Schematic diagram of the 120° in-plane triangular AFM structure of Mn3Sn. The red and blue atoms represent two layers of kagome lattice formed by Mn atoms. Hopping paths for Jij, Bij, Yijjk, and Kijkl are shown by blue, magenta, orange, and green curved arrows, respectively. b Schematic of an out-of-plane canted magnetic state with a finite scalar spin chirality (SSC), shown by the solid cone. The variation of SSC with canting angle is plotted at the bottom. c abinitio calculated change in energy (solid symbols) for Mn3−xFexSn as a function of out-of-plane canting angle ϕ. The lines represent fittings with Eq. (1). d Temperature-dependent magnetization for Mn3−xFexSn samples with field parallel and perpendicular to the c-axis.

Full size image

As discussed earlier, the magnetic ground state of Mn3Sn can be effectively modified by a small electron doping. Motivated by these previous reports, we carry out a detailed first principle electronic calculation for the electron-doped Mn3Sn samples. Using the virtual crystal approximation (VCA) technique as implemented in the Vienna Ab initio Simulation Package (VASP) code37,38,39,40, we systematically vary the electron concentration by substituting Fe in place of Mn in Mn3−xFexSn. Starting from the in-plane 120° structure, we calculate the total energy of the pristine and the electron-doped samples by subjecting all the Mn moments with equal rotation from the kagome plane towards the c-axis, as represented in Fig. 1b. An in-depth examination concerning the calculation and the impact of various parameters can be found in Supplementary Figs. 18–28. The change in energy as a function of the out-of-plane canting angle ϕ is plotted in Fig. 1c. In the case of the undoped Mn3Sn, we find a minimum at ϕ = 0, signifying an in-plane triangular AFM order as the ground state. The calculated energy curve can be fitted only with the pairwise Heisenberg exchange term. A similar trend is found for x = 0.3, albeit the stability of the triangular AFM state (ϕ = 0°) reduces significantly in comparison to the FM order (ϕ = 90°). However, a satisfactory fit of the calculated energy for x = 0.3 can only be achieved with the inclusion of 4-spin exchange terms in addition to the 2-spin Heisenberg exchange.

By further increasing the electron doping in x = 0.5, a drastic change in the nature of the energy curve is observed where a shallow minimum appears at a finite value of the out-of-plane canting angle with ϕ = 23° [Fig. 1c]. The position of the minimum further increases to about ϕ = 37° for x = 0.75. For Fe doping beyond 1.0, the FM state comes out to be the ground state. Note that the end compound Fe3Sn exhibits a FM ground state41. In contrast to the 3Q magnetic state with zero net magnetic moment found in the case of triangular lattice, here, the canted magnetic structure introduces a finite out-of-plane moment2. The energy curves for the intermediate Fe-doped samples are best fitted upon inclusion of 6-spin interaction in the form of Xijk(Si. Sj)(Sj. Sk)(Sk. Sl) along with the 4-spin and 2-spin terms42,43. The extracted values of different order exchange constants from the fitting reveals that the canted magnetic ground state in the present case is stabilized by the higher order exchange contributions (Supplementary Fig. 26). The presence of effective higher order magnetic interactions in the local moment correlation functions in the Mn3A based compounds is also mentioned by E. Mendive-Tapia et al. in their theoretical work44. It is worth mentioning that the in-plane component of the canted magnetic configuration can still retain the octupole order of the triangular AFM state15,16. The coexistence of this octupole order, along with the noncoplanar magnetic order stabilized by higher-order exchange can lead to a two-component order parameter in the system. The transport phenomena of such a magnetic ground state can be interesting. Motivated by our theoretical results, in this work, we experimentally demonstrate the existence of peculiar magnetic and AHE behaviors in the electron-doped Mn3.09Sn0.91 and Mn3−xFexSn samples.

Low-temperature magnetic transition in electron-doped samples

The structural analysis of the present materials are summarized in Supplementary Fig. 1 and Supplementary Fig. 2. The temperature dependence of magnetization [M(T)] measured with field parallel and perpendicular to the c-axis for Mn3.09Sn0.91 and Mn2.7Fe0.3Sn, and Mn2.5Fe0.5Sn are depicted in Fig. 1d (Supplementary Fig. 5). A distinguishable sharp upturn in the M(T) data appears at low temperatures for all the samples when the field is applied parallel to the c-axis. On the contrary, the M(T) curves for H⊥c show very low magnetization due to the near cancellation of the net magnetic moment in the ab-plane. The sudden increase in the magnetization at low temperatures for the H∥c case indicates a realignment of the Mn moments along the c-axis. Most importantly, the temperature corresponding to this spin-reorientation transition (TSR) and the out-of-plane magnetic moment increases gradually with Fe doping, signifying the stabilization of the out-of-plane magnetic state with Fe doping. It is worth mentioning that the TSR for the undoped Mn3Sn sample falls below 50 K33, whereas Mn2.5Fe0.5Sn exhibits TSR=170 K that further increases to near room temperature for the polycrystalline Mn2.3Fe0.7Sn sample (Supplementary Fig. 3). Further, the large irreversibility between the zero field cooled (ZFC) and the field cooled (FC) M(T) curves for H∥c indicates the presence of large out-of-plane magnetic anisotropy. The above results suggest the presence of an insurgent out-of-plane magnetic moment along with the in-plane triangular AFM state.

To further illustrate the stabilization of the out-of-plane magnetic state of the electron-doped Mn3Sn samples, isothermal magnetization [M(H)] measurements are carried out at 5 K, as shown in Fig. 2(a-c). M(H) plots for the full temperature range is shown in Supplementary Fig. 6. All three samples exhibit a large hysteretic behavior for the M(H) loops measured with H∥c, whereas linear kind of M(H) curves are found for H⊥c. In addition, the saturation magnetization for H∥c keeps on increasing and reaches a value of ≈2.0μB for Mn2.5Fe0.5Sn. In the case of the polycrystalline samples, the saturation magnetization further increases to ≈4.0μB for Mn2.0Fe1.0Sn (Supplementary Fig. 3). It is important to note here that any increase in the in-plane magnetic moment of the Mn atoms with Fe doping cannot explain the large change in the out-of-plane magnetic moment below the TSR. Moreover, a complete realignment of the Mn/Fe moments along the c-axis should give rise to a total moment of ~8μB/f.u. Hence, the systematic increase in the out-of-plane magnetic moment further corroborates the evolution of a canted magnetic state from the inverse triangular spin structure with electron doping.

Fig. 2: Magnetization and neutron diffraction study.
figure 2

Field dependent magnetization, M(H), loops measured at 5 K with field parallel and perpendicular to the c-axis for (a) Mn3.09Sn0.91, b Mn2.7Fe0.3Sn, and c Mn2.5Fe0.5Sn samples. d Rietveld refinement of the neutron diffraction (ND) pattern at 3 K for Mn2.5Fe0.5Sn. The data were recorded in a neutron wavelength λ = 2.4395 Å. We obtain the fitting parameter Rwp = 11.3 for the Rietveld refinement. The inset shows extracted canting angle at different temperatures. e Simulated powder neutron diffraction patterns corresponding to the out-of-plane canting of spins from the in-plane 120° structure for different canting angles ϕ. f Change in the relative integrated intensity of (200) and (201) peaks with respect to the (102) peak as a function of temperature. Dotted lines serve as a guide to the eye. g Expanded view of the (200) and (201) ND patterns at different temperatures, along with the Rietveld refinement for coplanar and noncoplanar models.

Full size image

Neutron diffraction experiment

To confirm the proposed non-coplanar magnetic ground state in the present system, we next carry out a temperature-dependent neutron diffraction (ND) study for the Mn2.5Fe0.5Sn sample. The 350 K ND data recorded above the magnetic ordering temperature can be well fitted by considering the hexagonal Mn3Sn structure. The magnetic structure at room temperature is found to be co-planar inverse triangular in nature. The 3 K ND data with Rietveld refinement using a non-coplanar model is shown in Fig. 2d. To analyze the low-temperature ND data, first we simulate the ND intensity considering the non-coplanar spin structure. The simulated patterns are obtained using the FullProf package, where the calculated intensities for different canting angles are exported. As it can be seen, an appreciable redistribution in the peak intensity only for the (200) and (201) reflections is found with increasing ϕ (Fig. 2e). Interestingly, the low-temperature ND data exhibit an exact trend to the simulated non-coplanar model, as shown in Fig. 2f. It can be clearly seen that the normalized integrated intensity of both (200) and (201) peaks starts increasing sharply at the onset of the TSR. The enhanced intensities can only be accommodated by considering a non-coplanar canted model of the inverse triangular spin structure. For clarity, we have also plotted the zoomed view of the ND data along with fittings corresponding to the in-plane coplanar and the canted non-coplanar spin structures around (200) and (201) peaks at selected temperatures [Fig. 2g]. It can be seen that the observed intensities can only be fitted using a canted model below 150 K. The estimated canting angle at 3 K is about 16°, which decreases to zero at 150 K [inset of Fig. 2e]. The full range ND patterns at different temperatures with Rietveld analysis and other fitting parameters are shown in the Supplementary Figs. 9–16. For further clarity, we have also plotted the normalized integrated intensity of (100), (101), and (110) peaks showing a trivial increase in the intensity below the Neel temperature of the sample. In addition, we have also simulated the ND data by systematically increasing the Mn/Fe moment (Supplementary Fig. 15). As expected, the (200) peak does not show any intensity distribution, whereas the intensity of the (201) peak decreases with an increasing moment and hence can not account for the experimental findings. These results categorically establish that the non-coplanar magnetic state stabilized by out-of-plane canting of the Mn moments is the ground state in the electron-doped Mn3Sn samples, hence, go hand in hand with our magnetization data and theoretical model. A small mismatch between the canting angle found experimentally and theoretically may arise due to slight correlation effects observed in the Mn3Sn sample, where the best result was observed by using the DFT+DMFT calculations45. In addition, the spin triangles with and without Fe atom may show different canting angles. Interestingly, although the electron doping significantly modifies the low-temperature non-coplanar magnetic state, it does not introduce any kind of helical magnetic ordering in the system. We do not observe any satellite peaks expected for helical magnetic state in the neutron diffraction measurements. In addition, the presence of finite Hall signal over all the temperatures safely excludes the presence of helical phase.

Hall signal due to dual components of magnetic order

Next, we perform Hall resistivity measurements on our single crystalline samples in different geometries to uncover the effect of noncoplanar canted magnetic state on electron transport. Our magnetization measurements indicate that the in-plane 120° triangular AFM state remains robust irrespective of the canting of the Mn moments. It is well known that the octupole domains of the in-plane AFM configuration align when a field is applied in the ab-plane15. The Hall voltage corresponding to this order can be observed in the ρzx component of the Hall resistivity. In addition, the formation of a canted magnetic state with Fe doping may give rise to a finite SSC, which in turn can generate an additional Hall signal that can be found from the ρyx measurement. Figure 3a–c show the ρzx and ρyx data measured at 5 K in a field up to 5 T. All the samples exhibit a robust ρzx signal that persists up to the magnetic ordering temperature (Supplementary Figs. 4 and 7), indicating the preservation of the octupole order. Most importantly, a large ρyx of about 14 μΩ-cm is also found for the Mn3.09Sn0.91 sample. With Fe doping, the ρyx increases to 25 μΩ-cm for Mn2.7Fe0.3Sn and further surges to an extremely large ρyx of ≈ 35 μΩ-cm for Mn2.5Fe0.5Sn. We expect that the ρyx may attain its maxima for Mn2.25Fe0.75Sn (Supplementary Fig. 4).

Fig. 3: Individual manipulation of two component Hall voltage.
figure 3

Hall resistivity data, ρyx and ρzx for (a) Mn3.09Sn0.91, b Mn2.7Fe0.3Sn, and c Mn2.5Fe0.5Sn samples measured at T = 5 K. d Individual switching of octupole and noncoplanar dipole orders for Mn2.7Fe0.3Sn measured at 5 K. The middle panel represents ρyx with H∥c. The left and right panels show both ρyx and ρzx measurements with H⊥c. The image of the sample for the transport measurement is shown in the inset of the right panel.

Full size image

Independent switching of dual components of magnetic order

Although we have demonstrated that both ρzx and ρyx coexist in the electron-doped Mn3Sn samples, it would be really interesting to study if these two different Hall signals in our samples can be manipulated simultaneously. For this purpose, we have performed AHE measurements on the Mn2.7Fe0.3Sn single crystal by rotating the sample in a magnetic field. The measurement setup is prepared in such a way that both ρzx and ρyx can be measured at the same time (inset of Fig. 3d, right panel). First, the ρyx component of AHE is measured with H∥c at 5 K, where a large SSC-induced AHE signal is clearly visible (Fig. 3d, middle panel). Subsequently, we set the ρyx to a positive maximum by applying a positive magnetic field before reducing the field to zero. In this configuration, we rotate the sample to apply in-plane magnetic fields (H⊥c) to measure the octupole order induced AHE, ρzx. At the same time, we also measure the spin chirality induced AHE, ρyx. As it can be seen from the left panel of Fig. 3d, the ρzx can be readily changed from positive to negative value by sweeping the in-plane magnetic fields while keeping the ρyx at the maximum value. The change in sign of the ρzx directly confirms a switching of the octupole order, whereas an unchanged ρyx signifies the preservation of the noncoplanar order. A similar kind of variation in the ρzx signal is observed by setting the ρyx at the negative maximum (Fig. 3d, right panel). These measurements clearly establish that the dual components of magnetic order can be individually manipulated using external stimuli.

Scaling of Hall resistivity with SSC

The observed ρyx component of the Hall resistivity can originate from the non-zero SSC induced by the non-coplanar magnetic ordering. The ρyx arising from the trivial out-of-plane magnetic moment must linearly vary with the magnetization. Whereas, the SSC-induced Hall signal depends on the canting angle of the Mn moments and exhibits oscillating nature, as shown in Fig. 1b. Hence, ρyx originating from the SSC must depend on the canting angle. Since the canting angle is maximum for Mn2.5Fe0.5Sn among the three single crystals that we present in this study, we carry out pulsed field magnetization and ρyx measurements up to 60 T on this sample, as shown in Fig. 4a, b. The magnetization of the sample increases monotonically with fields up to 60 T for all the measured temperatures (Fig. 4a), signifying a realignment of the Mn moments along the c-axis. This results in the change of canting angle from about 16° at zero field to ≈ 27° at 60 T for T = 4.2 K. Hence, we expect to see a dominating effect of SSC on the high field Hall data in comparison to the low field measurements.

Fig. 4: High field magnetic and Hall effect measurements.
figure 4

Field dependent (a) magnetization and b Hall resistivity data, ρyx measured for fields up to 60 T for Mn2.5Fe0.5Sn sample. c, d Plot of ρyx against magnetization \(M\times {\rho }_{xx}^{2}\), and SSC\(\times {\rho }_{xx}^{2}\). The lines represent a linear fit. The data in (c, d) is shifted along the y-axis for better clarity.

Full size image

The 60 T Hall resistivity (ρyx) measurement carried out on the Mn2.5Fe0.5Sn sample is shown in Fig. 4b. The ρyx increases with the field and reaches a giant value of ≈ 50 μΩ-cm at 60 T for temperature between 4.2 K and 50 K. Most importantly, ρyx data exhibit a slower increase at higher fields compared to the magnetization data. This trend can be understood by invoking the SSC contribution to the Hall signal along with the trivial magnetization contribution According to the scaling relation, the Hall resistivity for the intrinsic contribution (as the present system falls in the bad metal regime) varies as : \({\rho }_{AHE}\propto {\rho }_{xx}^{2}\times M\)9,46,47. Figure 4c shows the fitting of ρyx with \({\rho }_{xx}^{2}M\). It can be clearly seen that a good fitting is obtained at the low field range, whereas the fitting considerably deviates from the experimental data at very large fields for temperatures between 4.2 K and 100 K. This deviation at high fields (higher canting angle) originates due to the SSC contribution that becomes more nonlinear at higher canting angles. In contrast, a reasonable fitting of the experimental ρyx data is obtained at 150 K, where the zero field canting angle is almost negligible. This is also clear from the fact that the difference between the experimental data and the fitted curve is maximum at 4.2 K and decreases monotonically with increasing temperatures. To makes things more clear, we also plot ρyx as a function of SSC\(\times {\rho }_{xx}^{2}\) in Fig. 4d. In this case, a nice linear fitting is obtained for all temperatures up to 100 K, whereas the fitting deviates for 150 K. The slight mismatch in the fitting with SSC is due to the fact that although the SSC contributes predominately to the ρyx signal, a finite contribution from the trivial M component is always present. This is very clear from the fact that the experimental data can not be accounted with the SSC model at 150 K as it lies above the TSR where Hall voltage is dominated by the trivial magnetization component. Hence, our high field measurements categorically establish that SSC is the major contributor to the ρyx component of the Hall resistivity. Refer to Supplementary Fig. 8 and Supplementary Fig. 17 for scaling of AHE with temperature.

Berry curvature due to noncoplanar magnetic state

In order to investigate the impact of the canted magnetic state on the electronic band structure, we carry out DFT calculations for both the in-plane triangular AFM structure with zero canting angle in Mn3Sn and the canted non-coplanar magnetic state in Mn2.5Fe0.5Sn, as shown in Fig. 5. In case of Mn2.5Fe0.5Sn the calculation is carried out for a canting angle of 25°. The plots also depict the regions of significant Berry curvature within the band structure. To facilitate a meaningful comparison, we specifically present the Berry curvature associated solely with the ρyx component. By examining the distribution of Berry curvature represented by the contrasting red and blue colors, it becomes apparent that there are minimal regions of significant Berry curvature for Mn3Sn. This limited presence of Berry curvature naturally leads to a vanishing anomalous Hall conductivity (AHC), as clearly demonstrated in Fig. 5c. Notably, only a finite value of the σzx component is observed at the Fermi level, which is in line with our experimental findings for the in-plane magnetic structure, as illustrated in Supplementary Fig. 3. In contrast, the band structure of the Mn2.5Fe0.5Sn with a canting angle of 25° exhibits a noteworthy variation in comparison to that of Mn3Sn. Notably, the previously discussed topological characteristics, often associated with a Weyl semimetal, are absent from the band structure of the canted system. However, a distinct enhancement in the Berry curvature, specifically in the ρyx component of Hall resistivity, becomes evident. This augmentation is clearly illustrated in Fig. 5d, where we present the corresponding calculated AHC. As anticipated, a substantial value of σyx is observed at the Fermi level, accompanied by a small σzx component. Remarkably, this outcome corroborates our experimental findings (Fig. 3c), where a significant Hall signal is observed in the ρyx and a very small value is seen for ρzx component. Hence, a direct correlation is established between our experimental observations and the corresponding theoretical calculations17,20. The high magnetic field Hall resistivity measurements also provide evidence of the SSC nature of the Hall signal. Furthermore, recent calculations by Li et al.48 also reinforce our interpretation, indicating a mechanism driven by scalar spin chirality, rather than spin-orbit coupling, as the underlying cause of the Hall signal found in the present system.

Fig. 5: Theoretical calculations of band structure and anomalous Hall conductivity.
figure 5

DFT calculated band struture for (a) in-plane triangular AFM structure of Mn3Sn and b canted non-coplanar structure for Mn2.5Fe0.5Sn systems. The color shows the Berry curvature. The anomalous Hall conductivity as a function of energy for (c) in-plane triangular AFM structure of Mn3Sn and d canted non-coplanar structure for Mn2.5Fe0.5Sn systems. Fermi level is taken as E = 0.

Full size image

Discussion

After the discovery of large AHE in the triangular AFM state of Mn3Sn, a considerable amount of work has been carried out related to the role of underlying magnetic structure in determining the band topology in the system17,30,49. In all the cases, the magnetic properties of the system can be understood by invoking the 2-spin exchange interaction. It is also well known that the non-zero intra-plane Dzyaloshinskii-Moriya interaction (DMI) dictates the sense of rotation of the spins in the octupole magnetic structure and leads to the inverse triangular spin structure in Mn3Sn. As our samples exhibit the octupole order-induced Hall signal for all levels of doping in whole temperature range, it can be concluded that the role of DMI does not change in the present electron-doped Mn3Sn system. Furthermore, the centrosymmetric nature of the samples safely exclude any inter-plane DMI in the system. Hence, the present work brings foreward a different direction in the understanding of the role of higher-order interaction to stabilize the non-coplanar magnetic ground state. For the present layered kagome lattice structure, the 4-spin and 6-spin terms stabilize a tunable noncoplanar spin structure with dual order characteristics. This magnetic state gives rise to a large Hall signal related to the real space Berry curvature of the system. At the same time, the momentum space Berry curvature induced AHE can be independently manipulated without affecting the former one. This exemplifies the rich Berry phase physics of the underlying electron band structure in the electron-doped Mn3Sn system. Hence in this direction, it would be really interesting to study the effect of multi-spin interaction to the band topology of the present class of materials.

Moreover, the present finding of non-coplanar magnetic structure adds several opportunities to the possible room temperature applications of the triangular AFM-based non-collinear magnetic states in Mn3Sn. One of the most striking effects that can be easily manipulated is the finite real-space Berry curvature associated with the scalar spin chirality (SSC). This non-vanishing SSC can generate a large anomalous Hall signal even in the absence of any spin-orbit coupling (SOC). The extremely large AHE (ρyx) realized in the present case belongs to this category. Furthermore, these noncoplanar states have also been proposed to induce topological superconductivity when proximitized to trivial s-wave superconductors50. Hence, it can be safely proposed that the non-coplanar magnetic states are potential substitutes to the SOC-based AHE. Furthermore, the observation of two different sources for the present AHE is a consequence of the non-coplanar magnetic ordering in the system. Due to this dual nature of the order parameter, a storage unit in a memory device can be proposed with larger than one bit. This is similar to the multi-level cell in modern SSD storages, where one unit can store more states, such as [0, 0][0, 1][1, 0][1, 1] instead of normal [0][1]. In addition, the non-coplanar magnetic ground state stabilized by the higher-order exchange interactions, mostly ignored previously, could play a prevalent role in several other similar materials in the future studies.

In conclusion, the present study exemplifies the effects of higher order exchanges in the stabilization of ground state properties of the layered kagome lattice-based electron-doped Mn3Sn. A canted magnetic state is expected to play the primary role in the realization of a large anomalous Hall signal of topological origin. In addition to the noncoplanar magnetic state, the octupole order of the layered kagome structure can be preserved for all the samples. This results in a dual magnetic order, where both these orders can be manipulated individually using magnetic fields. We hope that the present results will motivate further studies on the effect of higher-order interactions on a variety of lattice classes. In addition, prospective device application of the dual order system can be anticipated.

Note: During the review process of the present manuscript, we came across a manuscript by Li et al.48, which discusses the modification of the electronic band structure due to a very small canting induced by the magnetic field. Our study presents a comprehensive study on the intrinsics-canted magnetic ground state itself in the electron-doped Mn3Sn system.

Methods

Single-crystal growth and characterization

Polycrystalline samples of Mn3−xFexSn were synthesized by arc melting the stoichiometric amount of high-purity constituent elements. Later, the samples were sealed in high vacuum quartz tubes and annealed at 800 °C for eight days. To synthesize single crystals, the polycrystalline samples were powdered and sealed under a high vacuum in pointed alumina crucibles. Then the samples were heated to above the melting point and slowly cooled down to ~800 °C. Single crystals of size ~3–4 mm were found to crystallize in the case of Mn3.09Sn0.91, whereas smaller size crystals were found for the Fe-doped samples. The structural phase of the samples was determined by room temperature powder X-ray diffraction (XRD) using Cu-Kα radiation (Rigaku) (Supplementary Fig. 1). The composition and homogeneity of the samples were determined by field emission scanning electron microscopy (FESEM), energy dispersive X-ray spectroscopy (EDS). The Electron Backscatter Diffraction (EBSD) characterization was used to confirm the single crystalline nature of samples (Supplementary Fig. 2).

Magnetic and transport measurements

Magnetic measurements were carried out using vibrating sample SQUID magnetometer (MPMS-3, Quantum Design). Transport measurements were carried out using Quantum Design Physical Properties Measurement System (PPMS). Powder neutron diffraction measurements were performed at the High Resolution Powder Diffractometer - Echidna51, housed at ANSTO, Australia, with a neutron wavelength of 2.4395 Å. The data analysis was performed with the FullProf Suite52. High-magnetic-field transport and magnetic measurements up to 60 T were performed at the International MegaGauss Science Laboratory of the Institute for Solid State Physics (ISSP), University of Tokyo.

Density functional theory calculations

The density functional theory (DFT) calculations are employed to study the magnetic properties of the system. The spin-polarized calculations have been carried out within the projector augmented wave (PAW) method53 as implemented in Vienna Ab initio Simulation Package (VASP)37,38,39,40. The generalized gradient approximation (GGA)is used as the exchange-correlation potential in the form of Perdew-Burke-Enzerhof (PBE)54. For the Brillouin-Zone (BZ) integration, a Γ-centered k-point grid of 8 × 8 × 9 is used with a plane-wave cut-off energy of 500 eV. The virtual crystal approximation (VCA) technique, as implemented in the Vienna Ab initio Simulation Package (VASP) code, is used to calculate the effect of Fe doping in Mn3−xFexSn system.

The DFT calculations for the band structure were performed using full-potential linearized augmented plane wave (FLAPW) method as implemented in the FLEUR code (www.flapw.de). To expand the LAPW basis functions, a plane-wave cutoff of \({k}_{\max }=4.2\) a.u.−1 was employed. Additionally, a Monkhorst-Pack k-mesh of 8 × 8 × 9 was utilized to cover the Brillouin zone (BZ). The tight-binding Hamiltonian was constructed by considering atomic orbital-like maximally-localized Wannier functions (MLWFs) derived from Mn-d, Fe-d and Sn-p states using the Wannier90 tool55,56 using a grid of 8 × 8 × 9. Subsequently, the WannierBerri Python code57 was employed to calculate the anomalous Hall conductivity (AHC) and map the Berry curvature onto the band structure. The AHC calculations were performed employing a highly dense grid of 500 × 500 × 500 points.