Introduction

Spin-orbit coupling (SOC) is the core of many emerging phenomena, including magnetocrystalline anisotropy1, non-collinear magnetism2,3,4, anomalous Hall effect5, spin Hall effect6, spin Seebeck effect7,8, and Rashba–Edelstein effect9. These phenomena revolutionized and prospered various subfields in condensed matter such as spintronics10,11, spin-orbitronics12, and topological physics13,14,15. For instance, recently synthesized two-dimensional (2D) materials with long-range ferromagnetic order16,17,18, lifting the Mermin–Wagner restriction19, are stabilized via SOC-induced magnetocrystalline anisotropy. Similarly, SOC stabilizes the topological phases against the lattice distortion and thermal fluctuation by a nontrivial bandgap, which determines the realization temperature of the topological phases such as the quantum anomalous Hall (QAH) effect20,21,22 and the quantum spin Hall effect13,14.

However, designing materials with strong SOC in realistic materials has been quite straightforward because it simply links to the atomic mass of the constituent elements. Therefore, the candidates for spin-orbit active materials have mostly been limited to solids with heavy elements such as Bi, Sb, Te, Hg, Pt, and Pb23,24,25,26. Unfortunately, compounds containing heavy atoms usually have weaker chemical bonding; thus, they accommodate more native defects27, leading to poor stability for performing exotic functionalities. A famous example is the topological insulator Bi2Te3 with bulk conductivity owing to the heavily n-type self-doping28,29.

Therefore, it is crucial to thoroughly explore the potential of the SOC effect in materials with lighter elements. However, it is generally believed that SOC does not play an essential role in 3d transition metal materials, which are ideal for studying the interplay between symmetry, electronic occupation, and electron correlation30. On the other hand, the SOC effect was found comparable to the correlation in 4d and 5d series, leading to emergent quantum phases such as Weyl semimetal31, topological Mott insulator32,33, and quantum spin liquid34,35. The SOC effect of these systems was found to be more prominent in the presence of electron correlation, attributed to the electron localization induced by Coulomb repulsion that reduces the kinetic energy33,34. Recently, the cooperative effect between SOC and correlation was considered to explain the Fermi surface puzzle of the paramagnetic Fermi liquid Sr2RhO436, Sr2RuO437,38, as well as relatively large band splitting in other 4d, 5d, and 5f compounds39,40. These studies revealed the essential role of total angular momentum for the cooperative effects between SOC and correlation.

In this study, we aim to theoretically design materials with light elements but large effective SOC strength based on orbital symmetry, electron occupation, and the cooperative effect with correlation. The focus is on transition-metal magnetic materials, especially the 3d series, where the SOC strength is significantly smaller than the typical spin-exchange splitting. We propose that the cooperative effect of the electron correlation can significantly enhance the effective SOC through orbital polarization when there are partially occupied orbital multiplets around the Fermi level. Thereafter, we provide design principles and comprehensive databases, where we list all the Wyckoff positions and site symmetries that allow orbital multiplets in periodic crystals. The results indicate that 32 out of 80 layer groups and 125 out of 230 space groups can support large SOC effect. Therefore, for materials no matter recorded in existing databases or designed artificially, one can easily resort to our symmetry principles to predict promising candidates with strong effective SOC.

2D materials, specifically 2D magnets, have attracted significant attention because of their engineerable and integrable nature for future devices. Particularly, the high-temperature QAH effect has been investigated for the potential application of dissipationless electronics; however, it is challenging to realize41. Hence, we applied our procedure to Computational 2D materials database (C2DB)42,43 and screened out 71 2D materials (from ~1600 candidates) with an orbital multiplet near the Fermi energy, enhancing the SOC effect. As opposed to the previous case-by-case search approach, we systematically obtained nine high-temperature QAH insulators with large nontrivial band gaps of hundreds of meV. Additionally, our symmetry principles and material candidates for enhanced SOC effect are valid for searching materials with strong magnetocrystalline anisotropy, which has a significant influence on industrial ferromagnetic materials with ultrahigh coercive fields. Our study paves a new avenue for realizing light-element materials with strong effective SOC for next-generation functional materials and devices in various fields.

Enhancing SOC self-consistently by correlation

Here, we summarize the main idea of designing a large SOC effect in light 3d transition metal ions. First, we consider orbital multiplets to activate the first-order perturbation of SOC, i.e., the on-site term44. The presence of SOC splits the orbital degeneracy and slightly unquenches the orbital angular momentum, leading to orbital polarization. When the orbital multiplet is partially occupied, the strong on-site Coulomb correlation enhances the orbital polarization as well as the effective SOC. Because of the competition between correlation and hopping in 3d systems, the effective SOC is enhanced via the orbital polarization self-consistently. The mechanism is schematically shown in Fig. 1, and is presented in the following.

Fig. 1: Schematic of boosting the effective SOC by correlation.
figure 1

The density of states (DOS, the left part of the horizontal axis) and orbital polarization (OP, the right part of the horizontal axis) of the orbital doublet in single-ion limit (a, b) and periodic solids (c, d) considering the effects of SOC, inter-ion hopping and correlation. a Without SOC, the doubly degenerated orbital multiplet has quenched orbital angular momentum, \(\left|{\bar{L}}_{z}\right|=0\). b SOC splits the degenerated levels (denoted by red and blue colors) with an energy gap, ΔE, and orbital polarization \(\left|{\bar{L}}_{z}\right|=1\) at half-filling. c The hopping between ions extends the energy levels into energy bands, leaving a tiny \({\bar{L}}_{z}\) because the SOC is typically significantly weaker than the inter-ion hopping. d Competition between the delocalized hopping and the on-site correlation effect significantly enhances \({\bar{L}}_{z}\) self-consistently, as well as the SOC effect and the energy gap.

Full size image

For a single d-shell ion exposed to the crystal field, only four types of orbital multiplets are allowed by the crystallographic symmetries, including three doublets, \({E}_{1}=\{{d}_{{xz}},{d}_{{yz}}\}\), \({E}_{2}=\{{d}_{{xy}},{d}_{{x}^{2}-{y}^{2}}\}\), \({E}_{3}=\{{d}_{{z}^{2}},{d}_{{x}^{2}-{y}^{2}}\}\), and one triplet, \(T=\{{d}_{{xy}},{d}_{{yz}},{d}_{{xz}}\}\). Among them, the first-order SOC effect is absent in E3. Herein, we demonstrate the physics using the E1 doublet, whereas E2 and T multiplets can be found in Supplementary Note 1. As shown in Fig. 1a, the orbital angular momentum of the orbital doublet is quenched without SOC. Turning on the SOC, the on-site SOC Hamiltonian reads

$${\hat{H}}_{{SOC}}=\mathop{\sum}\limits_{m,{m}^{{\prime} },\sigma ,{\sigma }^{{\prime} }}\lambda \left\langle m,\sigma |{{{{{\bf{L}}}}}}\cdot {{{{{\bf{S}}}}}}|{m}^{{\prime} },{\sigma }^{{\prime} }\right\rangle {\hat{C}}_{m\sigma }^{{{\dagger}} }{\hat{C}}_{{m}^{{\prime} }{\sigma }^{{\prime} }},$$
(1)

where L and S are the angular momentum and spin operators, respectively; \({\hat{C}}_{m\sigma }^{{{\dagger}} }\) and \({\hat{C}}_{m\sigma }\) are the creation and annihilation operators on the electron state with orbital m; spin σ, and λ denotes the strength of SOC. Because spin splitting typically overwhelms the SOC effect in 3d systems, we treat SOC as a perturbation with the spin-conserved part, LzSz, only, where the matrix representation is diagonal on the basis of the projected orbital angular momentum along the z-axis, i.e., \(|l=2,\,m=\pm\! 1\rangle =(|{d}_{xz}\rangle \pm i|{d}_{yz}\rangle )/\sqrt{2}\). Under such circumstances, the SOC Hamiltonian can be rewritten as \({\hat{H}}_{{SOC}}\approx \lambda \hslash {\hat{L}}_{z}/2\), where \({\hat{L}}_{z}=\hslash \left({\hat{n}}_{+1}-{\hat{n}}_{-1}\right)\) is the operator of the orbital angular momentum and \({\hat{n}}_{\pm 1}={\hat{C}}_{\pm 1}^{{{\dagger}} }{\hat{C}}_{\pm 1}\) is the occupation operator. In the single-ion limit, \({\hat{H}}_{{SOC}}\) splits the degenerated levels with a gap

$$\Delta {E}_{0}=\lambda {{\hslash }}\left|{\bar{L}}_{z}\right|={{{\hslash }}}^{2}\lambda ,$$
(2)

where \(\left|{\bar{L}}_{z}\right|={\sum }_{m\in {occ}.}\left|\left\langle m|{\hat{L}}_{z}|m\right\rangle \right|=\hslash\) is the expectation of \({\hat{L}}_{z}\) over the occupied state, i.e., orbital polarization. As shown in Fig. 1b, the splitting energy, \(\Delta {E}_{0}\), in the 3d series is only a few dozens of meV owing to the light atomic mass.

In periodic solids, the inter-ions hopping broadens the energy levels of the atomic limit to form Bloch energy bands with a bandwidth proportional to the hopping integrals typically an order larger than λ. Consequently, for a 3d state with small SOC, the splitting bands with opposite angular momenta lead to a slightly unquenched orbital polarization (Fig. 1c). However, the SOC-induced energy gap and orbital polarization can be iteratively enhanced by considering the electron correlation effect between different orbitals, i.e., \({\hat{H}}_{C}={U}_{{{{{{\rm{eff}}}}}}}{\hat{n}}_{+1}{\hat{n}}_{-1}\), where Ueff = U − 3J for d shell; U and J are the Coulomb repulsion and Hund coupling parameters, respectively45. When the orbital doublet is half-filled, \({\hat{H}}_{C}\) modifies the effective SOC and thus the energy gap at the mean-field level

$$\Delta {E}_{{{{{{\rm{eff}}}}}}}=\Delta {E}_{0}+{U}_{{{{{{\rm{eff}}}}}}}\frac{\left|{\bar{L}}_{z}\right|}{{{\hslash }}}.$$
(3)

The derivation of Eq. (3) is provided in Supplementary Note 1. Starting from a tiny \({\bar{L}}_{z}\) and ΔE0, when considering Ueff, the separation between Lz = ±1 states enhances \(\left|{\bar{L}}_{z}\right|\), which gives rise to a larger ΔEeff. In response, an enhanced ΔEeff reduces the overlap of different orbitals, leading to an enlarged \(\left|{\bar{L}}_{z}\right|\). As a result, the final orbital polarization is iteratively enlarged and settled self-consistently (Supplementary Note 1), as shown in Fig. 1d, and so is the energy gap.

Notably, a similar correlation-enhanced SOC effect was first revealed in the paramagnetic 4d transition-metal oxide Sr2RhO4 using a mean-field approach36 when both the SOC and Coulomb terms involve the occupation difference between the total angular momentum, \(|{m}_{j}|=3/2\) and \(|{m}_{j}|=1/2\) states. In comparison, we apply the orbital polarization scheme to 3d transition systems, which typically have a ferromagnetic ground state with a weak SOC. Compared with the paramagnetic case, the enhancement of SOC in the ferromagnetic system is a function of U−3J instead of U−J. The enhancement of SOC can result in reinforcement of various emerging phenomena, as shown later.

Materials design for correlation-enhanced SOC effect

Based on the mechanism described above, we extract the crucial principles to facilitate the design of light-element materials with large effective SOC. The 3d transition-metal elements of the materials should reside at the well-chosen Wyckoff positions, of which the site symmetries should permit the existence of orbital multiplets E1, E2, or T. Doublets E1 and E2 are allowed in both tetragonal, trigonal, and hexagonal point groups, whereas the triplet T only exists in the cubic point groups, leading to 24 single point groups (Supplementary Table 1). The design principles lead to a comprehensive database with all the Wyckoff positions and site symmetries, allowing orbital multiplets in 2D and 3D periodic crystals. The results indicate that 32 out of 80 layer groups and 125 out of 230 space groups can support the correlation-enhanced SOC effect, as listed in Supplementary Tables 2 and 3, respectively. Therefore, for materials no matter recorded in existing databases such as C2DB42 and ICSD46 or artificially designed, one can easily predict candidates with potentially strong SOC effect using our database (Supplementary Tables 2 and 3): (i) if the compound belongs to the required layer/space groups, and ii) if the transition-metal ion sits on the required Wyckoff positions.

For instance, we consider a 3d compound with space group P42/mmc (No. 131), which is one of the 125 groups. Although it has 18 different types of Wyckoff positions, as shown in Supplementary Table 2, the orbital doublet is permitted for enhanced SOC effect only when the transition-metal ion is located at Wyckoff positions 2e or 2f with the same site point group −4 m2. Notably, these symmetry requirements are established under the framework of nonmagnetic groups based on the assumption of strong Hund’s exchange interaction in the 3d series. A similar methodology can be extended directly to the magnetic point and space groups.

To explicitly demonstrate the capability of our material design principle, we focus on a complete search of 2D light-element materials with strong effective SOC. The procedure is presented in Fig. 2. Starting from ~1600 2D materials in C2DB42,43, we selected 859 entries with a set of 3d and 4d elements. Among them, by comparing the layer group each compound belongs to and the Wyckoff positions the transition-metal ions are located using Supplementary Table 3, we obtained 542 candidates. Thereafter, we perform the density functional theory (DFT) calculations to narrow the material pool with potentially correlation-enhanced SOC by examining if the considered orbital multiplets are partially occupied within 1.0 eV around the Fermi level. The calculation is based on a simple SOC-free framework with a spin-polarized generalized gradient approximation (GGA) exchange-correlation functional. Because the central principle is based on symmetry requirements, our results from such an economic approach are not sensitive to details such as the lattice constant, magnetic ordering configuration, and the choice of U. Finally, we pinned down the “best of class” of 71 material candidates (Table 1 and Supplementary Table 4), for which we perform more careful GGA+U+SOC calculations. According to Eq. (3), the enhanced effective SOC is typically one order larger than the original SOC strength of a dozen meV scale in 3d/4d systems.

Fig. 2: Design procedure of the 2D materials with enhanced SOC effect.
figure 2

First, the elements of the 3d/4d series were chosen as 3d: Sc~Ni and 4d: Y~Rh, leading to 859 entries in C2DB. Second, the 3d/4d transition-metal ions need to reside at the Wyckoff positions whose site symmetries permit the existence of orbital multiplets. For instance, in a 2D Ni2O2 with layer group p4/mmm (LG 61), the Ni ion occupying 1a Wyckoff position Ni@1a (site point group 4/mmm) permits orbital doublet, whereas that in Ni2S2 Ni@2c (site point group mmm) does not. This filter obtains 542 entries remaining. Third, the considered orbital multiplets are partially occupied within 1.0 eV around the Fermi level, leading to 71 material candidates as the “best of class”.

Full size image
Table 1 Abstract of the 71 2D material candidates with 3d/4d elements at selected Wyckoff positions with enhanced SOC effect.
Full size table

As a bonus, we systematically obtain nine materials with nontrivial SOC-induced bandgaps (Table 1), rendering them large-gap QAH insulators against thermal excitation21 and local disorder47. Although some materials in Table 1 and certain artificial structures are occasionally predicted as large-gap QAH insulators48,49,50,51,52,53,54,55,56,57, a comprehensive understanding of the nontrivial gaps with temperatures above the room temperature as well as systematic material search are still lacking. Conversely, the large-gap QAH insulators in our framework can be well understood and exhaustively obtained from our material candidates (Supplementary Table 4) by performing more delicate GGA+U Wannier-representation tight-binding calculations58,59,60,61 (see Methods). In the following, we consider the monolayers of honeycomb transition-metal monochalcogenides, Fe2X2 (X = S, Se), to demonstrate our theory.

QAH insulator with huge nontrivial gap

Monolayer Fe2X2, which is shown in Fig. 3a as two stacking honeycomb FeX sublayers, has layer group p-3 m1 (No. 72) with each Fe atom of Fe2X2 at the 2c Wyckoff position (site symmetry group 3 m). According to Supplementary Table 3, three types (1a, 2b, and 2c) of the Wyckoff positions support orbital multiplets, rendering Fe2X2 a potential candidate for large effective SOC. The \(\{{d}_{{xz}},{d}_{{yz}}\}\) orbitals of Fe ions form the basis functions of the 2D irreducible representation at the K and K′ valleys at the Fermi level. Hence, there is a Dirac cone with linear dispersions at each valley owing to the constraint of the corresponding little-group62. Our spin-polarized GGA calculations show that the Dirac cones are half-occupied under the ferromagnetic ground state with a magnetic moment of 4 μB/Fe (Fig. 3b and Supplementary Note 4). Turning on SOC, uniaxial magnetic anisotropy with out-of-plane moments is preferred as discussed later.

Fig. 3: Fe2X2 (X = S, Se) monolayer.
figure 3

a Sketch of the honeycomb lattice. b, c Band structure of the Fe2S2 crystal without and with spin-orbit coupling with U = 3 eV, with color mapping the projection on \(\left\{{d}_{{xz}},{d}_{{yz}}\right\}\) orbits. Here, we consider the global coordinate with the crystal c-axis along the z-axis. d Gapless edge state inside the large bulk gap window of 400 meV. e Evolution of the SOC gap and the E-field induced gap as a function of U in DFT calculation. f Haldane’s phase diagram of transition between the normal insulator (NI) and the quantum anomalous Hall (QAH) insulator obtained from the corresponding tight-binding model.

Full size image

Through our procedure, the SOC gap is significantly enhanced by the correlation effect self-consistently in Fe2X2. Considering Fe2S2, the bandgap opened solely by the first-order SOC could be obtained by either treating SOC as a perturbation with a given U value or by setting U = 0 in a self-consistent calculation (Supplementary Note 4). Both approaches yield a typical first-order SOC gap of approximately 43 meV, which is large for Fe with SOC strength of ~15 meV63. When we consider the correlation effect self-consistently, the SOC gap is enhanced significantly, reaching 628 meV at the K valley (U = 3 eV), as shown in Fig. 3c. The enhancement ΔEeff/ΔE0 is qualitatively consistent with the analytical results, which reveals the dependence of Ueff = U − 3J (Supplementary Note 4). By checking the Chern number, C = 1, and the chiral edge modes (Fig. 3d), we efficiently predict that monolayer Fe2S2 is a potential candidate for high-temperature QAH insulators with a large nontrivial gap. Similarly, the correlation-enhanced nontrivial gap in monolayer honeycomb Fe2Se2 reaches 798 meV at the K valley, as shown in Supplementary Note 4.

Notably, the Dirac cones of Fe2X2 can also be gapped in a trivial way by an out-of-plane external electric field owing to the onsite energy splitting of different d electrons at two FeX sublayers. Because the electric field could not effectively separate the orbital degeneracy of \(\{{d}_{{xz}},{d}_{{yz}}\}\), the correlation effect can hardly enlarge the gap. Therefore, the electric-field-induced gap is more insensitive with increasing U (Fig. 3e). In addition, the competition between the SOC-induced nontrivial gap and the electric field-induced trivial gap in monolayer Fe2X2 is analogous to the Haldane phase diagram64 (Fig. 3f), which is quite rare in realistic materials. More detailed calculation results for Fe2X2 are presented in Supplementary Note 4.

Discussion

The main results of our study are based on the assumption of strong exchange splitting, for which the interaction between electrons with parallel spins dominates. In principle, the correlation-enhanced SOC effect should also be valid under more general circumstances. In moderate spin-splitting case, considering a general Kanamori-type correlation (Supplementary Note 5), we find that although spin flipping process weakens the orbital polarization of each spin channel, including the contribution of the opposite spin channels does not qualitatively change the SOC enhancement. Even for the spin-degenerate case, correlation can generally enhance the SOC effect. As reported in Liu et al.36, SOC splits the degenerated states with different total angular momentum \(|{m}_{j}|=3/2\) and \(|{m}_{j}|=1/2\). The resultant “spin-orbit polarization”, i.e., the occupation difference between \(|{m}_{j}|\) states, plays an essential role in the enhanced SOC effect along with the correlation between parallel and antiparallel spins. Therefore, our results together with previous studies36,37,38,39,40 demonstrate that correlation can in general enhance SOC via different polarization effects in various magnetic states.

In addition to the large-gap QAH insulators, the paradigm we used to design materials with large SOC effects could be applied to various scenarios. For instance, the long-range magnetism of 2D magnetic materials can only be stabilized by uniaxial anisotropy1. The single-ion magnetocrystalline anisotropy, which is one of the main components of uniaxial anisotropy, originates from the SOC effect65. With the reinforcement of the orbital polarization and SOC effect, the magnetocrystalline anisotropic energy, EMAE, for 2D magnetic materials with \(\{{d}_{{xz}},{d}_{{yz}}\}\) frontier orbital doublet can significantly increase as66

$${E}_{{MAE}}=\frac{1}{2}\left({{{\hslash }}}^{2}\lambda +{U}_{{{{{{\rm{eff}}}}}}}\frac{\left|{\bar{L}}_{z}\right|}{{{\hslash }}}\right)\left|{{\cos }}\,\,\theta \right|,$$
(4)

where θ is the angle between the magnetic moment and the vector normal to the 2D plane (Supplementary Note 16). This boosted magnetic anisotropy can stabilize the magnetic order in 2D magnetic materials with out-of-plane magnetic moments, getting rid of the Mermin–Wagner theorem for isotropic spins. The DFT calculations on the magnetocrystalline anisotropy of various materials yield qualitatively consistent results, as shown in Supplementary Note 4.

To summarize, we provide design principles for large SOC effects with the help of orbital degeneracy, electron occupation, and correlation, eliminating the need for heavy elements. To activate the correlation-enhanced SOC effect, we combined symmetry analysis of the transition-metal sites residing at specific Wyckoff positions and first-principles calculations to examine the partially occupied orbital multiplets around the Fermi level. Applying the guiding principles to the C2DB database, we found 71 2D material candidates supporting the correlation-enhanced SOC effect and nine compounds as potential candidates for high-temperature QAH insulators. The procedure can be easily extended for designing and searching 3D light-element materials with strong effective SOC in various fields of condensed matter physics, such as spintronics, spin-orbitronics, and topological phases of matter.

Methods

First-principles calculations

First-principles calculations were based on DFT with generalized gradient approximation (GGA)67,68 for exchange correlation potential. The Perdew−Burke−Ernzerholf (PBE) functional was used for the GGA as implemented in Vienna ab initio simulation package (VASP)69. The electron-ion interaction was treated using projector-augmented-wave (PAW) potentials70 with a planewave-basis cutoff of 500 eV. The entire Brillouin zone was sampled using the Monkhorst−Pack71 method. A vacuum of 15 Å was used to avoid artificial interactions caused by the periodic boundary conditions. Because of the correlation effects of 3d electrons in Fe atoms, we employed the GGA + U approach within the Dudarev scheme72 and set U varying from 0 to 3 eV. Both Fe2S2 and Fe2Se2 were fully relaxed until the force on each atom was less than 0.01 eV/Å, and the total energy minimization was performed with a tolerance of 10−5 eV. The Wannier representation was constructed by projecting the Bloch states from the first-principles calculations to Fe-3d, S-3p, and Se-4p orbitals58,59,60. The edge states and Berry curvature were calculated in the tight-binding models constructed using the Wannier representation as implemented in the WannierTools package61. The global symmetries of each material and the site symmetries of the Wyckoff positions during the screening procedure were examined via the FINDSYM module73 of the ISOTROPY software.