Introduction

Anomalous transverse transport denotes the transverse electrical (the anomalous Hall effect, AHE), thermoelectric (the anomalous Nernst effect, ANE), and thermal (the anomalous Righi–Leduc effect) responses that are not caused by the Lorentz force experienced by the electrons subjected to magnetic fields (H). The most common case of anomalous transverse transport occurs in the ferromagnets, wherein the anomalous transport coefficients are proportional to the magnetization (M); it is well established that such behavior can arise from both intrinsic (which reflects the non-zero Berry curvature in the electronic system) and extrinsic (which is related to the scattering process) origins1. In ferromagnets, the validity of both the intrinsic and extrinsic mechanisms for the anomalous transport are guaranteed by the spin-orbit coupling (SOC) effect and the broken time-reversal symmetry (TRS)2,3,4. On the other hand, in a collinear antiferromagnet, despite the breaking of TRS, the intrinsic anomalous coefficients are usually forced to be zero because the momentum-space integral of Berry curvature is nullified in the presence of a combined symmetry, i.e., the combination of time-reversal operation and other unitary space group operations such as fractional translation or inversion operation5,6. Whilst it is the conventional notion that anomalous transverse transport is less important in antiferromagnets, large anomalous terms were recently observed in the transverse responses in antiferromagnetic (AFM) materials with peculiar noncollinear or noncoplanar spin configurations and non-trivial band topology, most notably Mn3Sn7, Mn3Ge8, YbMnBi29,10, YMn6Sn611, and CoNb3S612.

In comparison to ferromagnets, a zero (or rather weak7,8,12) net magnetization at H = 0 in antiferromagnets provides good opportunities for realizing high-performance integrated devices without perturbing stray fields. Most importantly, even with much smaller amplitudes of low-field M, several AFM compounds exhibit remarkable AHE and ANE that are not proportional to M; this behavior is in sharp contrast to the cases in most ferromagnets1 and the heavy-fermion antiferromagnets wherein the extrinsic anomalous terms dominate13. The possible origins of such “unconventional” anomalous transport include the “topological” transverse responses stemming from long-period noncoplanar chiral spin textures (e.g., Skyrmions) that introduces real-space Berry curvature14; the field-tuning of topological band crossings (e.g., Weyl nodes) and anticrossings in the electronic structures15,16, which are main sources contributing to the momentum-space Berry curvature; interplay between SOC and special crystal symmetry17,18; domain wall effects19, and so on. Overall, the correspondence between the symmetry of the system (depending on magnetic configurations) and the topological characterizations of the electronic bands (determining the behavior of itinerant electrons) plays an essential role in provoking the enriched phenomena of anomalous transport in topological antiferromagnets, consequently opens up a broad avenue in the design and fine control of advanced devices.

In this work, we present a combined transport, angle-resolved photoemission spectroscopy (ARPES), and first-principles study in a recently discovered20 member of the Mn-based RMnPn2 family, namely SmMnBi2; where R is a divalent rare-earth or alkaline-earth metal ion, Pn is Sb or Bi21,22,23,24. The RMnPn2 series of materials have been established as prototypical topological semimetals with AFM order of the Mn spins (Mn2+, 3d5, S = 5/2), hence serving as good candidates for investigating the interplay between magnetism and band topology23,24,25,26. In particular, it has been pointed out that in YbMnBi2, a small canting of the Mn spins in the C-type AFM configuration lifts the twofold band degeneracy, resulting in a Weyl semimetal state that is characterized by considerably large AHE and ANE6,9,24. Compared to other well-studied members in the RMnPn2 series, SmMnBi2 is unique: although it also hosts the Pn (here Pn = Bi) square sheets that engender the topological nontriviality in this series21,22,24,27, it crystallizes in a tetragonal crystal structure different from all other Mn-112 materials20, which leads to a more complicated electronic band structure.

We report, to the best of our knowledge, for the first time the observation of unconventional AHE (UAHE) and large ANE in SmMnBi2. These effects are detected in a configuration (Hc) in which the anomalous terms in YbMnBi2 disappear9, suggesting that their occurrence is closely related to the details of magnetic structures. Based on the experimental results and first-principles calculations, we propose that SmMnBi2 is an AFM topological metal in the magnetically ordered phase below TN = 235 K; the UAHE arises from the non-zero momentum-space Berry curvature invoked by the canted spin configuration in the AFM state, whereas the anomalous terms persisting above TN can originate from magnetic fluctuations and/or peculiar extrinsic mechanisms. The ANE also contains extrinsic contributions which render its value much larger than the simple expectation for an intrinsic ANE. The coexistence of intrinsic and extrinsic mechanisms for anomalous transverse transport and their intricate relationship with magnetism make SmMnBi2 a fertile playground to explore the rich physics in magnetic topological phases of matter.

Results

Sample characterization

High-quality SmMnBi2 single crystals were synthesized by a self-flux method (Experimental section). The single-crystal X-ray diffraction (XRD) pattern (Supplementary Fig. 1) confirms that the shiny surface of our plate-shaped samples (top panel of Fig. 1a) corresponds to the crystal plane (001). Analysis of the results of single crystal XRD experiment that utilizes a four-circle goniometer provides detailed structure information for SmMnBi2 (Supplementary Tables 1 and 2). It is shown that SmMnBi2 crystallizes into a tetragonal structure (space group I4/mmm, No.139) as illustrated in Fig. 1a, with the lattice constants a = b = 4.506 Å, c = 20.597 Å; these results are consistent with previous work20. Compared with other members in the RMnPn2 family22,23,24,26, SmMnBi2 shares similar structural features, such as the periodic stacking of -Mn/SmBi/Bi/SmBi- layers along the crystallographic c-axis. It is believed that the Bi/Sb square net gives rise to the topologically nontrivial electronic state in RMnPn2, which is controlled by the spin configuration of Mn sublattice; in the cases of a collinear AFM order, the electronic states near the Fermi energy EF are characterized by the presence of highly anisotropic Dirac cones21,27. Nonetheless, the crystal structure of SmMnBi2 exhibits remarkable difference from other RMnPn2 compounds in the sense that it shows a MnBi4 square-planar coordination geometry (highlighted by the purple dashed lines in Fig. 1a), in stark contrast to the typical tetrahedra coordination in all other members. With such structural discrepancy, distinct electronic structure and physical properties are also expected for SmMnBi2.

Fig. 1: Crystal structure and basic physical properties of SmMnBi2.
figure 1

a A schematic illustration of the crystal structure of SmMnBi2. The top panel shows an optical image of the as-grown single crystals. b Zero-field-cooled (ZFC, open symbols) and field-cooled (FC, solid symbols) magnetic susceptibility χ plotted against temperature (T). Data were measured under a magnetic field μ0H = 0.1 T applied along the c-axis (blue) and within the ab-plane (red). c Isothermal magnetization curves of SmMnBi2 collected at several temperatures with μ0H up to 7 T applied along the c-axis. d T-dependent longitudinal resistivity (green) and thermopower (blue) measured in a SmMnBi2 crystal at zero field, with current and temperature gradient applied within the ab-plane. e The specific heat of SmMnBi2. The anomaly marked out by an arrow manifests the onset of magnetic ordering.

We first look into the magnetic properties. The temperature (T)-dependent magnetic susceptibility (χ) curves displayed in Fig. 1b show that the onset of irreversibility between the zero-field-cooled (ZFC) and field-cooled (FC) curves at T = 235 K is accompanied by a weak kink on the Hc curves (no evidence for additional magnetic transitions can be resolved from χ(T) up to 400 K, see Supplementary Fig. 2a). Such features match those of the AFM transition temperature TN previously reported for other Mn-112 materials21,26,28,29, thus we conclude that TN = 235 K in SmMnBi2. A small jump in the specific heat at the same temperature (Fig. 1e) supports this identification. Based on the χ(T) and the isothermal magnetization (MH) data (Fig. 1c and Supplementary Fig. 2b), we can further comment on the nature of magnetism in SmMnBi2. First, the difference between ZFC and FC magnetizations, together with the nonlinear MH profile, point towards a canted AFM order below TN, similar to the case in YbMnBi29,24,30, SrMnSb222,31, and Ca1−xNaxMnBi232; the magnetic moment per Mn atom is only 0.1–0.15 μB (μB is Bohr magneton) at μ0H = 7 T at the lowest T for both Hc (Fig. 1c) and Hc (Supplementary Fig. 2b), suggesting that the polarized/ferromagnetic (FM) component remains small compared to the total moment for Mn2+ (typically 3–5μB). The more prominent kink at ~75 K in χ(T) for Hc (Fig. 1b) may arise from a variation in spin canting angle. Second, the larger suppression of χc compared to χab at low T, which is widely observed in other Mn-112 compounds21,31, implies that the Mn spins are aligned predominantly along the c-axis (with the above referred small canting-induced FM component lying in the ab-plane). The tiny hysteresis loops appearing at low H and high temperatures (inset of Fig. 1c) are probably attributed to residual spin clusters formed above TN, or some impurity-related extrinsic effects31.

The T dependence of zero-field resistivity ρxx displayed in Fig. 1d exhibits metallic behavior with a relatively high residual resistivity [ρxx(0 K) = ρ0 ≃ 110 μΩ cm]. The Seebeck coefficient Sxx is negative between 10 and 300 K (Fig. 1d), indicative of dominant electron-type carriers. The linear-in-T behavior of Sxx below ~ 30 K is consistent with the diffusion thermopower in metals; the nonmonotonic T dependence at higher temperatures suggests multiband transport, as also seen in SrMnBi233. Sxx reaches ~6 μV K−1 near room temperature (300 K). The magnetoresistance (MR) ρxx(H) and magnetothermopower (MTP) Sxx(H) (Supplementary Figs. 3a, b) both show features that can be understood within a single-carrier-type picture: the positive MR and MTP decline rapidly with increasing H and evolves into a saturation at moderate fields34,35. Such behaviors, together with the relatively large ρ0 and small MR, depict SmMnBi2 as a multiband metal with low carrier mobilities and poor electron-hole compensation.

Anomalous Hall and anomalous Nernst effects

Most intriguingly, the Hall resistivity ρxy of SmMnBi2 develops a striking nonmonotonic H-profile that is distinct from common metals. As shown in Fig. 2a, the amplitude of negative ρxy increases promptly at low H, reaching its maximum at a magnetic field that gradually shifts to higher values with increasing T (μ0H ≃ 0.06 T at 10 K and μ0H ≃ 1.3 T at 300 K); above this maximum, ρxy decreases in a trend roughly tracking the inverse magnetic field (1/μ0H), forming a skewed (or tilted) peak that becomes broader upon warming. This behavior of ρxy is unlikely to reflect the ordinary band-filling contribution. Neither it can be ascribed to the conventional AHE since it does not trace the profile of M(H) (Fig. 1c). To clarify the origin of the unusual Hall signal, we plot the Hall conductivity σxy [\(=-{\rho }_{xy}/({\rho }_{xx}^{2}+{\rho }_{xy}^{2})\)] against H in Fig. 2b. σxy(H) exhibits a notable low-H resonance-like peak that also undergoes broadening with increasing T. A resonant feature in σxy has been discussed in the framework of the single-band Drude model, wherein the reciprocal of the peak position in H provides an estimation of carrier mobility36,37. For SmMnBi2, such approach yields a very high mobility of ~104 cm2 V−1 s−1 (Supplementary Fig. 4a), in sharp contrast to the low mobility in SmMnBi2 indicated by the low Hall angle (Supplementary Fig. 4b). We conclude that the low-H tilted-peak-like features in ρxy and σxy are not caused by Lorentz force-induced orbital motion of highly mobile band electrons (see further discussions in Supplementary Note 1 for the explicit preclusion of ordinary contribution).

Fig. 2: Anomalous transverse transport effects in SmMnBi2.
figure 2

a Magnetic-field (H) dependence of the Hall resistivity ρxy measured in a SmMnBi2 single crystal with Hc. Data curves taken at selected temperatures as shown. Inset is an expanded view of the low-H data (between −1 T and +1 T). b The Hall conductivity σxy as a function of H. Inset is an expanded view of the low-H data (between −2 T and +2 T). c The maximal value of ρxy, \({\rho }_{xy}^{\max }\), plotted against square of the zero-field longitudinal resistivity, \({\rho }_{xx}^{2}\), measured at the same T. Blue shaded area highlights the deviation above TN. Error bars denote the maximum experimental error in Hall measurements, which is smaller than 0.01 μΩ cm (Methods). The red dashed line is a linear fit: \({\rho }_{xy}^{\max }\propto {\rho }_{xx}^{2}\). Data points fall onto this line up to T ~ TN. Inset: T dependence of the peak value of the Hall conductivity σxy. d The H-dependent Nernst signal Sxy in SmMnBi2 measured at varying temperatures. The temperature gradient ∇ T and field H are applied along the ab-plane and the c-axis, respectively.

Alternatively, it is more reasonable to assign these features to a rare case of intrinsic UAHE that is exceptionally sensitive to the application of a weak H. For the intrinsic mechanism, the anomalous Hall conductivity \({\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\) is directly given by the integral of Berry curvature in the momentum space, thus it is usually considered as an inherent property of the electronic bands that is independent of or weakly depends on T1. Consequently, a scaling relation between the AHE and the square of longitudinal resistivity, \({\rho }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\propto {\rho }_{xx}^{2}\), often holds empirically for intrinsic AHE according to a simple Drude analysis1. We test this scaling in Fig. 2c, and it confirms that the maximal value of ρxy, \({\rho }_{xy}^{\max }\), is proportional to \({\rho }_{xx}^{2}\) in a broad range of T up to T ≃ TN. The validity of \({\rho }_{xy}-{\rho }_{xx}^{2}\) scaling, together with the weakly-T-dependent peak value of σxy (inset of Fig. 2c), imply an intrinsic AHE originating from nonvanishing Berry curvature of the electronic bands in SmMnBi2, at least for T < TN. (We note that the overall behavior of UAHE in SmMnBi2 resembles the spin-canting-induced intrinsic AHE observed in another AFM metal, EuCd2As238). Such a large Berry curvature effect will leave its footprints in the transverse thermoelectric response, i.e., a sizable ANE4. This is exactly what we observed with Hc and an in-plane thermal gradient: the Nernst signal Sxy evolves in H with a unique skewed-peak profile similar to ρxy, showing a sharp increase in weak H and a continuous reduction at higher H (Fig. 2d). The amplitude of Nernst response in SmMnBi2 reaches 1.8 μV K−1 at its maximum, a remarkably large value for an AFM metal; we attribute it to a dominant ANE term (see Supplementary Note 1 and Supplementary Fig. 4d for arguments against normal Nernst). The observation of both intrinsic UAHE and ANE in SmMnBi2 is highly suggestive of a topologically nontrivial electronic structure that has an intimate connection with the field-tunable magnetism.

ARPES measurements and first-principles calculations

We investigate the electronic band structure of SmMnBi2 by methods of ARPES and first-principles calculations based on the density functional theory (DFT, see Supplementary Notes 2 and 3 for the details of calculation). The Fermi surface (FS) contour in the surface Brillouin zone (BZ) (Fig. 3a) probed by ARPES is presented in Fig. 3b. It reveals a few notable characteristics: a diamond-shaped contour around \(\overline{\Gamma }\), a diamond-like loop consisting of elongated FS pockets between \(\overline{\Gamma }\) and \(\overline{{{{{{{{\rm{M}}}}}}}}}\), and spot-like FS pockets around the \(\overline{{{{{{{{\rm{X}}}}}}}}}\) points. We note that the overall features of FS contours share many similarities with the ARPES results of other members in the RMnPn2 series21,27,39; in particular, the diamond-like FS loop in most of the Mn-112 compounds contains four anisotropic FSs possessing massive Dirac holes, whereas the small pockets at the X/Y points can also show Dirac-type dispersions. The massive Dirac fermions hosted by these FSs contribute to most of the intriguing physical properties in RMnPn2 materials21,23. For SmMnBi2, a direct identification of Dirac(-like) bands by ARPES measurements is hampered by the terraced surface of the cleaved single crystals. As shown in Fig. 3d–i, the ARPES-resolved bands largely track the dispersions given by DFT calculations along the \(\overline{\Gamma }-\overline{{{{{{{{\rm{M}}}}}}}}}\) and \(\overline{\Gamma }-\overline{{{{{{{{\rm{X}}}}}}}}}\) paths (see also Supplementary Fig. 8); in particular, the electron Fermi surface pockets centered at \(\overline{\Gamma }\) are clearly observed (Fig. 3d, e). On the other hand, ARPES and DFT results show some discrepancies along the \(\overline{{{{{{{{\rm{M}}}}}}}}}-\overline{{{{{{{{\rm{X}}}}}}}}}\) direction (Fig. 3f, i). A detailed discussion on the comparison between calculations and ARPES data is presented in Supplementary Note 4.

Fig. 3: Electronic structure of SmMnBi2 resolved by ARPES.
figure 3

a Schematics showing the three-dimensional (3D) Brillouin zone (BZ) of SmMnBi2 and the projected 2D BZ on the (001) surface. Green dashed lines and red solid lines denote high-symmetry directions in the 3D and 2D BZ, respectively. b A Fermi surface (FS) map of SmMnBi2 probed by ARPES measurement. The 2D BZ is shown in black dashed lines. Red dashed line indicate the FS contours. c DFT-calculated FS contour for the high-symmetry plane kz = π. df Band dispersion of SmMnBi2 along the high-symmetry paths of \(\overline{{{{{{{{\rm{M}}}}}}}}}-\overline{\Gamma }\) (d), \(\overline{{{{{{{{\rm{X}}}}}}}}}-\overline{\Gamma }-\overline{{{{{{{{\rm{X}}}}}}}}}\) (e), and \(\overline{{{{{{{{\rm{M}}}}}}}}}-\overline{{{{{{{{\rm{X}}}}}}}}}-\overline{{{{{{{{\rm{M}}}}}}}}}\) (f) in the surface 2D BZ, respectively, measured with the linear horizontal (LH) polarization. The DFT calculation results are appended to the corresponding panels. gi Same as df, but measured with the linear vertical (LV) polarization. All data were collected at 6.5 K using the photon energy of 82 eV.

The band structure attained from the first-principles calculations relies on the spin configurations. Here, we consider two types of canted AFM orderings with Mn spins aligning predominantly along the c-axis but maintaining a small in-plane FM component, as revealed by the magnetization measurements (Fig. 1b, c). Illustrations of the two orderings, i.e., C-type22,24 and G-type21,23 AFM structures, are presented in Supplementary Fig. 5. The FM component is fixed to the [100] direction in our calculations. Whilst both C-type and G-type AFM orders can be established in the RMnPn2 series, we can further identify that the canted G-type order is a more plausible spin configuration for SmMnBi2. This argument is based on the observations of nonzero \({\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\) (Fig. 2b) and anomalous Nernst conductivity \({\alpha }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\) (Supplementary Fig. 3c): in the case of canted C-type AFM order, a glide plane perpendicular to the in-plane FM moments (and parallel to c) is preserved; the in-plane AHC (\({\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\)) is forced to zero by the corresponding mirror operation (Supplementary Fig. 5e); by contrast, no mirror or glide plane parallel to c-axis exists in the canted G-type AFM structure and a nonzero AHC is thus permitted (see Supplementary Note 3). The DFT calculation results of the band structure in SmMnBi2 considering the canted G-type AFM order (with a canting angle θ = 10) is displayed in Fig. 4a. Compared to most of the typical RMnPn2 materials wherein the bands crossing the Fermi level EF are mainly attributed to the Bi-2 atoms within the Bi square nets21,24,27, the spectral weights of Bi-1 (Bi atoms in the MnBi4 polyhedra) p states and Mn d states are higher in SmMnBi2, resulting in a much more complicated FS structure with a large number of FS pockets and sheets constituting a diamond-like profile (Fig. 4d). The calculated FS contour (Fig. 3c) and band dispersions with this canted AFM configuration capture most of the main features observed in the ARPES experiment (Supplementary Note 4).

Fig. 4: First-principles calculated electronic structure and momentum-space Berry curvature for SmMnBi2 in the case of canted G-type AFM order.
figure 4

a The DFT-calculated band structure for SmMnBi2, considering a canted G-type AFM spin structure (Supplementary Fig. 5b). Blue and yellow arrows represent the moments of two neighboring Mn2+ ions, both have a canting of 10 towards the [100] direction. Bands shown in purple, blue, red and green colors are contributed by Sm d, Mn d, Bi-1 p and Bi-2 p orbitals, respectively. Bi-1 and Bi-2 are Bi atoms forming the MnBi4 square planar polyhedra and the square Bi nets, respectively. b Three components of AHC tensor calculated from the momentum integrals of Berry curvature on the BZ, plotted as a function of chemical potential. c Momentum-space distribution of Berry curvature Ωxy in the first BZ for the kz = 0 plane. d The 3D FS morphology of SmMnBi2 corresponding to the band structure shown in a. All the calculations include SOC.

Our DFT calculations provide solid evidence for intrinsic UAHE in SmMnBi2. The energy-dependent AHC tensor elements (\({\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\), \({\sigma }_{yz}^{{{{{{{{\rm{A}}}}}}}}}\) and \({\sigma }_{xz}^{{{{{{{{\rm{A}}}}}}}}}\)), calculated from the integrals of corresponding Berry curvature components on the first BZ (Supplementary Note 3), are shown in Fig. 4b. Large values of both \({\sigma }_{yz}^{{{{{{{{\rm{A}}}}}}}}}\) and \({\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\) are confirmed, whereas \({\sigma }_{xz}^{{{{{{{{\rm{A}}}}}}}}}\) is considerably smaller. In particular, the amplitude of \({\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\) in the vicinity of EF reached 60–70 S cm−1, in good agreement with the experimental result (Fig. 2b). This consistency unequivocally verifies the Berry-curvature origin of the AHC. In absence of the spin canting, our calculations reveal that SmMnBi2 with collinear G-type AFM order is a topological metal candidate with multiple band crossings along the Γ–X direction that are protected by the mirror symmetry; these crossings form nodal loops in the kx − ky plane (Supplementary Fig. 6). The spin canting destroys the mirror symmetry and opens gaps along the nodal loops. As has been proposed for numerous magnetic topological materials, gapped Dirac nodal lines and the remaining band touching points (e.g., Weyl nodes) serve as effective sources for abundant Berry curvature that leads to sizable AHC40,41. In addition, the Berry curvature distribution map for the kz = 0 plane (Fig. 4c) indicates that besides the gapped nodal loops, the bands around Γ (mostly associated with Mn d and Bi-1 p orbitals, Fig. 4a) contribute significantly to the nonzero Berry curvature flux; this is similar to the case in YbMnBi26,9.

Discussion

The remarkable skewed-peak profile of ρxy(H) in SmMnBi2 (which is distinct from the behavior of M) is uncommon for an intrinsic AHE. The peculiar H dependence resembles that of THE originating from real-space Berry curvature induced by large spin textures in Skyrmion lattices14. In SmMnBi2, our experimental and theoretical explorations suggest that the momentum-space Berry curvature still makes the predominant contribution. As mentioned above, the emergence of intrinsic AHE in SmMnBi2 relies on spin canting, which results in broken mirror symmetry. Thereby, the drastic variation of AHE (and AHC) under H can be naturally assigned to the field-modulated spin canting effect. We elucidate that apart from the well-known SOC-driven AHE, spin chirality also plays an essential role in contributing to the intrinsic AHE in the canted AFM state: when a spin-chirality period remains comparable to the lattice parameters, the Berry curvature derived from spin chirality can be effectively linked to band anti-crossings and/or Weyl nodes in the momentum space42,43,44. Whereas the scalar spin chirality delineated by noncoplanar spin textures44 may not be relevant in SmMnBi2, a recent proposal points out that the vector chirality produced by a small relative canting between neighboring spins can give rise to intense uncompensated momentum-space Berry curvature in both FM and AFM materials45. Our theoretical analysis presented in Supplementary Note 5 corroborates the presence of nonzero AHC in the spin-canted AFM state of SmMnBi2 even without SOC, which is presumably attributed to such vector chirality among canted spins45; the AHC is further enhanced by the inclusion of SOC (Supplementary Fig. 9), implying that both the vector spin chirality and the SOC effect contribute to the momentum space Berry curvature (and consequently the intrinsic AHE). We further note that a change of θ from 10° to 5° results in a decrease of \({\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\) at EF, underlining the effective manipulation of AHE through the field control of spin canting (Supplementary Fig. 10, more discussions in Supplementary Note 5). Indeed, both the spin-chirality term and the SOC term of AHE can be highly sensitive to the modification of spin canting38.

Aside from the spin canting, field tuning of the positions of Weyl nodes through exchange splitting of energy bands can also lead to strongly nomonotonic UAHE; this mechanism has been discussed in a few magnetic topological materials, especially those hosting magnetic rare-earth ions15,16,46. A strong effective Zeeman field stemming from the large polarized magnetic moments under H is crucial for realizing such band tuning. For SmMnBi2, however, the absence of rare-earth magnetism and the complicated band structure near EF make it difficult to verify this mechanism.

The UAHE in SmMnBi2 survives above TN (Fig. 2a, b), with no suppression in its amplitude but a deviation of the \({\rho }_{xy}-{\rho }_{xx}^{2}\) scaling (Fig. 2c). We suggest that short-ranged canted AFM order can persist in the paramagnetic state above TN46; application of H may stabilize the spin chirality related to the short-ranged order and introduce nonzero momentum-space Berry curvature. The generally unchanged shape of ρxy(H) curves for T > TN with a broadened tilted peak supports the above picture. Additionally, we can not exclude the contribution of extrinsic mechanism for the high-T signals: in a background of thermally fluctuating spins, a skew scattering process evolving chiral spin clusters is expected to induce pronounced AHE47,48.

The realization of a sizable ANE in SmMnBi2 is more fascinating since it has only been achieved in a few antiferromagnets (e.g., Mn3Sn49, Mn3Ge50, YbMnBi29,10, YMn6Sn611, and TbPtBi51). In Fig. 5a, we compared the amplitudes of anomalous Nernst signal measured in several typical FM and AFM materials9,11,49,50,52,53,54,55,56,57,58, including SmMnBi2. Conventional FM metals are characterized by a weak ANE (<1 μV K−1). Magnetic topological materials exhibit much stronger ANE due to the Berry curvature contributed by Weyl nodes and gapped Dirac band-touchings. Among the AFM compounds listed in Fig. 5a, SmMnBi2 is featured by an ANE (1.8 μV K−1 at its maximum and 1.6 μV K−1 at 300 K) larger than the noncollinear antiferromagnets Mn3X (X = Sn, Ge)49,50, comparable to that in YMn6Sn611 but smaller than the maximum ANE observed in YbMnBi29,10. Nonetheless, the giant ANE ≃ 6–15 μV K−1 in YbMnBi2 is measured with H in-plane and the voltage drop V along the c-axis, whereas for SmMnBi2, we detect the ANE with a more conventional measurement configuration, i.e., Hc and ∇T, V both in-plane. Such contrast is caused by different symmetry groups in the magnetically ordered states of the two compounds (see Supplementary Note 3).

Fig. 5: Comparison of ANE between SmMnBi2 and other typical magnetic materials.
figure 5

a A comparison of the amplitudes of ANE signals measured in 12 representative FM and AFM materials9 -- 11,49,50,52 -- 58, including SmMnBi2 in this work. Both the room-temperature (RT) values (green) and the maximum values (yellow) of ANE are shown for each compound. b The ratio of transverse thermoelectric conductivity αxy (Supplementary Fig. 3c) to the Hall conductivity σxy, plotted against T. The ratio is calculated for magnetic fields at which αxy reaches its maximum, where both αxy and σxy should be dominated by the anomalous term. The blue dashed line marks the threshold value of kB/e ≃ 86 μV K−158. The \({\alpha }_{ij}^{{{{{{{{\rm{A}}}}}}}}}/{\sigma }_{ij}^{{{{{{{{\rm{A}}}}}}}}}\) ratio for several other well-known topological magnets is also shown for comparison. Data for YbMnBi2 shown in green and blue colors were adapted from Ref.9 and Ref.10, respectively.

With compelling evidence of intrinsic UAHE in SmMnBi2, it is natural to expect that the ANE is also attributed to the momentum-space Berry curvature (i.e., an intrinsic ANE). However, the origin of ANE herein is indeed more complex. If both electric and thermoelectric anomalous transverse terms are linked to the Berry curvature of band electrons, a qualitative analysis yields \(| {\alpha }_{xy}^{{{{{{{{\rm{A}}}}}}}}}/{\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}| \simeq ({k}_{{{{{{{{\rm{B}}}}}}}}}/e)\langle {\lambda }_{{{{{{{{\rm{F}}}}}}}}}^{2}/{\Lambda }^{2}\rangle\), where kB is the Boltzmann constant, λF the Fermi wavelength, and Λ the thermal de Broglie wavelength of electrons57,58. With T increasing towards room temperature, λF and Λ become comparable, consequently the ratio \(| {\alpha }_{xy}^{{{{{{{{\rm{A}}}}}}}}}/{\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}|\) is set by a universal constant ~kB/e = 86 μV K−1. Although the derivation uses an oversimplified single-band 2D model, such universal relation is revealed to be valid for several well-studied magnetic topological materials. As illustrated in Fig. 5b, the high-T ratio of \(| {\alpha }_{ij}^{{{{{{{{\rm{A}}}}}}}}}/{\sigma }_{ij}^{{{{{{{{\rm{A}}}}}}}}}|\) for Mn3X (X = Sn, Ge)49,50, Co3Sn2S257, and Co2MnGa58 all approaches the threshold kB/e (dashed line) from the lower side; in sharp contrast, the ratio in YbMnBi29,10 and SmMnBi2 strongly exceeds the kB/e threshold. Isotherms of αxy/σxy curves plotted against H (Supplementary Fig. 3d) imply that the ratio in SmMnBi2 exceeds kB/e over broad ranges of T and H. Considering the distinct FS morphologies in YbMnBi29,24 and SmMnBi2 (Supplementary Fig. 11), the analogous notable breakdown of the empirical kB/e relation in two materials is less likely to have a band origin (multiband effect, FS anisotropy, linear band dispersion, etc). Instead, we suggest that it is more reasonable to assign this violation to extrinsic contributions (e.g., skew scattering), which can give rise to a considerable enhancement in the transverse thermoelectric response but only enter \({\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}\) as a marginal component59. As such, for both YbMnBi2 and SmMnBi2, the giant ANE is probably dominated by the extrinsic mechanism(s), while the AHE still has a predominant intrinsic origin. Further studies are required to clarify the cause of this putative dichotomy in the origins of anomalous transverse transport coefficients.

Conclusions

In summary, we give the first report on the magnetic, transport, and spectroscopic properties of the magnetic topological material candidate SmMnBi2. This compound is corroborated to be a metallic antiferromagnet exhibiting pronounced anomalous transverse responses in both electrical transport and thermoelectricity. The long-range AFM order occurs at TN = 235 K, with spins largely along the easy c-axis but having a small canting towards the ab-plane. Unconventional AHE and large ANE with unique field dependence are observed both above and below TN. Most interestingly, our results hint at an extraordinary scenario that different mechanisms are responsible for the AHE and ANE: scaling relation and DFT-calculation considering the spin-canting effect confirm the intrinsic origin of AHE described by the contribution of momentum-space Berry curvature, whereas a plausible extrinsic origin of ANE amplifies the Nernst signal significantly, as indicated by the breakdown of the empirical rule \(| {\alpha }_{xy}^{{{{{{{{\rm{A}}}}}}}}}/{\sigma }_{xy}^{{{{{{{{\rm{A}}}}}}}}}| \sim {k}_{{{{{{{{\rm{B}}}}}}}}}/e\). We mention that the anomalous transport behavior of SmMnBi2 shares similarities with that of YbMnBi2, in spite of the different crystal structures, FS geometries, and (supposedly) magnetic configurations for the two materials. Such unusual consistency singles out spin canting as an essential factor in the occurrence of large AHE and ANE in antiferromagnets. The potential presence of multiple origins of anomalous off-diagonal transport in SmMnBi2 renders it an ideal platform for investigating the interplay between the enriched underlying physical mechanisms, which can definitely benefit the future development of electronic, spintronic, and thermoelectric devices that utilize transverse responses in application purposes.

Methods

Sample growth and characterization

Single crystals of SmMnBi2 were synthesized using a bismuth self-flux method. Initially, a mixture of samarium lumps (99.5%; the surface was polished to remove the oxidation layer), manganese powder (99.9%), and bismuth lumps (99.999%) with an atomic ratio of 1:1:10 were loaded into an alumina crucible. The crucible was sealed into a quartz tube under a high vacuum. The tube was heated to 1100 °C and maintained for 12 h to homogenize the melt, then it was slowly cooled to 500 °C over a time period of 7 days. Millimeter-sized single crystals with rectangular plate shapes and shiny surfaces were obtained after the removal of the excess bismuth flux by centrifugation. The stoichiometry of the samples was confirmed using an EDX mounted on the field emission scanning electronic microscope (FESEM), Sirion200. The single-crystal XRD pattern was measured by the X-ray diffractometer (SmartLab-9, Rigaku Corp.) with Cu Kα1 radiation and a fixed graphite monochromator. Magnetic properties of SmMnBi2 were characterized using a Quantum Design MPMS3 system. The valence of Sm in our samples was determined to be +2 by electron paramagnetic resonance, thus we conclude that the Sm ions are nonmagnetic and the magnetic moments in SmMnBi2 are fully attributed to the Mn2+ ions. Specific heat measurements were performed in a Quantum Design Physical Property Measurement System (PPMS-9T).

Electrical transport measurements

Electrical transport measurements were performed in a Quantum Design Physical Property Measurement System (PPMS-9T). The longitudinal resistivity ρxx and Hall resistivity ρxy were measured simultaneously in a standard Hall bar configuration. Contact leads were made by platinum wires attached to the SmMnBi2 sample using silver paste (DuPont 4922N). Data of ρxx and ρxy were symmetrized and anti-symmetrized with respect to the magnetic field, respectively, to remove the error caused by contact misalignment. The noise level of PPMS converts to an experimental error below 0.01 μΩ cm for electrical resistivity measurements.

Thermoelectric measurements

Thermoelectric measurements were performed utilizing a home-built high-resolution thermoelectric puck which is inserted into the PPMS. An AC method based on the differential thermocouple configuration was adopted to eliminate the DC background noise. The temperature gradient was generated by a chip resistor (1 kΩ) attached to the top of the sample; a low-frequency AC heating current was applied by a Keithley 6221 current source. To ensure good thermal contact, the end of the sample was glued to a copper block by cigarette paper soaked with diluted VGE varnish. A pair of type-E thermocouples were attached to the surface sample by thermal joint compound or stycast 2850 to measure the temperature gradient. The longitudinal (Seebeck) and transverse (Nernst) voltage signals were measured by two pairs of electrical contacts made from thin silver wires. The voltage signals were collected by a Keithley 2182A nanovoltmeter. The contribution of silver wire to the total Seebeck effect was subtracted, referring to the reported thermopower curve of silver60.

Angle-resolved photoelectron spectroscopy

ARPES experiments were performed at BL03U of Shanghai Synchrotron Radiation Facility61 and the Stanford Synchrotron Radiation Lightsource (SSRL). High-quality SmMnBi2 single crystals with a typical size of 1 mm × 1 mm were cleaved in situ. The measurements were carried out at 6.5 K with a base pressure better than 5 × 10−11 Torr, using both linear horizontally and linear vertically polarized lights with varying photon energies. Data shown in Fig. 3 were collected using a photon energy of 82 eV.