Introduction

Overlaying two-dimensional crystal layers with small lattice constant mismatches or interlayer twist angles creates moiré patterns. Recent experimental progress1,2,3,4,5,6 has established transition metal dichalcogenide (TMD) bilayers with moiré patterns periods on the 10 nm scale as an attractive platform7,8 to synthesize artificial one-band Hubbard model strongly-correlated electron systems. The minibands of these moiré materials are flat for a wide-range of twist angles, unlike twisted bilayer graphene which requires tuning to magic-angles9,10,11,12. Key control parameters of these artificial Hubbard models like the bandwidth, the carrier density, and the screening of long-range Coulomb interactions can be adjusted by varying twist-angle, gate voltage, and gate electrode placement13,14.

Because they are based on triangular lattice host atomic crystals15, TMD moiré materials realize either triangular lattice Hubbard models or, when the emergent low-energy moiré Hamiltonian has \({C}_{6}\) symmetry, honeycomb lattices. For example, the topmost moiré band of aligned \({WS}{e}_{2}/W{S}_{2}\) heterobilayers and twisted \({WS}{e}_{2}\) homobilayers16,17,18 mimics the triangular lattice Hubbard model, while the topmost moiré bands of \({MoS}{e}_{2}\), \({Mo}{S}_{2}\), and \(W{S}_{2}\) homobilayers have been predicted to simulate the honeycomb lattice Hubbard model19,20,21. It has been found that in moiré materials, interactions lead not only to Mott insulating states at half-filling22 (hole band filling n = 1), but also at fractional fillings23,24,25. The ground states at fractional filling factors are Wigner crystal states in which electrons occupy a subset of the available lattice sites and translational symmetry is broken, as shown for example by recent high-resolution scanning tunneling experiments for hole fillings n = 2/3, 1/2, and 1/3 in WSe2/WS226. Wigner crystal states rely on inter-site Coulomb interactions, and are likely obscured by disorder in heavily doped atomic crystals. Their prominence in experiment is a demonstration of the potential of moiré materials to reveal exotic physics. Although Wigner crystal states have charge gaps, their low-energy physics is non-trivial because of the spin degrees of freedom that remain on all occupied lattice sites. The spins are expected to order at low temperatures in most cases, but could potentially have spin-liquid ground states in some instances. The interactions between spins in Wigner crystals states are different from those in Mott insulator states because the possibilities for virtual electron hopping across the charge gap are enriched, and control the magnetic ground states that are the main subject of this paper.

To date the triangular lattice Hubbard model has been the main focus for both experimental and theoretical studies of TMD moiré materials27,28. However, \({MoS}{e}_{2}\), \({Mo}{S}_{2}\), and \(W{S}_{2}\) homobilayers with the 2H structure have Γ point29 interlayer antibonding states at the valence band maxima (VBM), unlike \({MoT}{e}_{2}\) and \({WS}{e}_{2}\) where the VBM is located at the K point. This has important implications for the moiré bands of these materials. Recent ab-initio calculations showed that a small twist angle (θ) in Γ-valley TMD homobilayers leads to the creation of moiré bands mimicking one and two orbitals honeycomb lattices, as well as kagome lattices19,20,21. The continuum model Hamiltonian was derived in19 by keeping only antibonding layer states and neglecting spin-orbit coupling because it vanishes at the Γ point. Consequently, although the vast majority of theoretical work in moiré systems have addressed triangular lattices, here we focus on honeycomb lattices and the influence of electronic interactions, anticipating that in the near future honeycomb systems will be realized experimentally.

For concreteness, we specifically discuss the example of the \({MoS}{e}_{2}\) homobilayer with twist angle θ = 1.5°. The continuum moiré Hamiltonian is \(H=-{{{\hslash }}}^{2}{k}^{2}/2{m}^{* }+\Delta ({{{{{\bf{r}}}}}})\). In this equation, Δ(r) is the moiré potential defined as \(\Delta ({{{{{\bf{r}}}}}})={\sum }_{s}{\sum }_{j=1}^{6}{V}_{s}{e}^{i({{{{{{\bf{g}}}}}}}_{j}^{s}\cdot {{{{{\bf{r}}}}}}+{\phi }_{s})}\), where \({{{{{{\bf{g}}}}}}}_{j}^{s}\) are vectors of the moiré reciprocal lattice connecting to the s-th nearest-neighbor site. The moiré Brillouin zone and the \({{{{{{\bf{g}}}}}}}_{j}^{1}\) vectors are shown in the inset of Fig. 1(b). In Fig. 1(a), the band structure for \({MoS}{e}_{2}\) is displayed. The topmost red-colored bands, highlighted in Fig. 1(b), realize a one-orbital honeycomb lattice tight-binding model. The dashed black line shows the nearest-neighbor tight-binding model result with hopping parameter t = 1/6 meV. This comparison demonstrates that for small values of the twist angle θ, the highest energy bands can be faithfully described by the nearest-neighbor honeycomb lattice. The emergence of a honeycomb lattice is understood intuitively by noticing the structure of the moiré potential, plotted in Fig. 1(c), in which the moiré potential maxima define a honeycomb lattice structure.

Fig. 1: Continuum model results.
figure 1

a This shows a nearly flat \({MoS}{e}_{2}\) band structure calculated using the continuum model in ref. 19 at twist angle \(\theta =1.{5}^{\circ }\). In b, the topmost bands are magnified. The dashed black line is the band structure of a nearest-neighbor tight binding model defined on a honeycomb lattice. c This shows the moiré potential in real space. The color bar depicts the value of the moiré potential. The model parameters \(({V}_{1},{V}_{2},{V}_{3},{\phi }_{s},{m}^{* },{a}_{0})\) = \((36.8,8.4,10.2,\pi ,1.17{m}_{e},3.295{{{{{\rm{\AA }}}}}})\) are fixed to values corresponding to \({MoS}{e}_{2}\). Here \({m}_{e}\) is the bare electron mass and \({V}_{1,2,3}\) are in \({{{{{\rm{m}}}}}}eV\) units. \({a}_{m}\) is the moiré lattice spacing.

Full size image

Once established that the Γ-valley bilayer-TMDs can effectively be described as a honeycomb lattice, it is anticipated that in their Mott insulator state (hole filling n = 1), simple collinear bipartite antiferromagnetic (AFM) order is supported. However, at the fractional fillings, the open questions arise: (i) Do we, as in the triangular lattice heterobilayer case, obtain 2D generalized Wigner crystals at special fillings in these moiré materials once the model is made realistic by including longer range electronic interactions? (ii) What kind of magnetic states we may expect in these Wigner crystals? The present work investigates the above questions by obtaining the many-body ground state for different particle fillings of the honeycomb lattice, considering the strong on-site Coulomb interaction \({U}_{0} > > t\) limit, and incorporating long-range Coulomb interactions as well. The longer-range interactions are essential to understand the phases of the fractionally filled bilayer-TMDs. However, investigating fractionally-filled Hubbard models with non-local interactions is a formidable task, even for exact Lanczos studies on small clusters or highly accurate density matrix renormalization group studies on ribbons. For this reason, static mean-field theory investigations have been extensively employed by the community to study moiré superlattices30,31,32,33,34. In this publication, we will employ an unrestricted Hartree-Fock approximation to exhaustively explore the phase diagrams of long-range Coulomb interacting Hubbard models on honeycomb lattices, with focus on fillings n = 2/3, 1/2, and 1/3. In insulating states, the unrestricted Hartree–Fock approximation allows energy to be minimized by varying the spin direction on each lattice site and is therefore equivalent to employing a classical approximation within a spin-only model for the low-energy physics. The static mean field theory approximation is accurate when charge fluctuations are suppressed, as they are in Wigner Crystal states, and ground state spin fluctuations are also weak. Importantly the Hartree–Fock completely eliminates self-interaction effects in the limit of electrons localized on lattice sites. However, the Hartree–Fock approximation cannot describe spin-liquid states.

Using the Hartree–Fock approximation we are able to address how the ground state evolves as the parameters of the model are varied. After establishing the phase diagrams and the presence of rich spin physics in the fractionally filled honeycomb moiré materials, in the last portion of our work we also discuss the magnetization evolution under external magnetic fields for the magnetic states in the strong long-range Coulomb interaction limit. These magnetization vs. external magnetic field curves are experimentally accessible, and can be used as signatures of the exotic magnetic Wigner crystals unveiled here.

Results

Lattice Hamiltonian

In this study, we extend the one-band Hubbard model on a honeycomb lattice by including the long-range Coulomb repulsion that is relevant in moiré systems. The Hamiltonian is

$$H= \mathop {\sum} \limits _{i,{i}^{{\prime} },a,b,\sigma }{t}_{{i}^{{\prime} }b}^{{ia}}{c}_{{ia}\sigma }^{{{\dagger}} }{c}_{{i}^{{\prime} }b\sigma }+{U}_{0}\mathop {\sum} \limits _{i,a}{n}_{{ia}\uparrow }{n}_{{ia}\downarrow }\\ +{U}_{1}\mathop {\sum} \limits _{i}{n}_{i0}{n}_{i1}+1/2\mathop {\sum} \limits _{i\ne {i}^{{\prime} }}{U}_{{ab}}^{i{i}^{{\prime} }}{n}_{{ia}}{n}_{{i}^{{\prime} }b},$$
(1)

where \({c}_{{ia}\sigma }^{{{\dagger}} }\) (ciaσ) is the standard fermionic creation (anhilation) operator. In the Hamiltonian above, \(\{i,{i}^{{\prime} }\}\) labels unit cell, a, b =0/1 labels sublattice, and \(\sigma \in \{\uparrow ,\downarrow \}\) denotes spin. \({n}_{{ia}\sigma }={c}_{{ia}\sigma }^{{{\dagger}} }{c}_{{ia}\sigma }\) is the fermionic density operator. The first term of Eq. (1) captures the band Hamiltonian whose hopping parameters we choose as the unit of energy by setting \({t}_{{i}^{{\prime} }b}^{{ia}}=t=-1\) for nearest neighbors and to 0 otherwise. We kept only the nearest-neighbor hopping because for Γ-valley moiré homobilayers the longer range hoppings are negligible compared with |t|, see Supplementary Note 5. Moreover, we showed specifically for \({MoS}{e}_{2}\) homobilayers that topmost bands can be well represented by just a nearest-neighbor honeycomb lattice (See Fig. 1(b)). The second (\({U}_{0}\)) and third (\({U}_{1}\)) terms represent the on-site and the nearest-neighbor Coulomb repulsion, respectively. The fourth term is the long-range tail of the Coulomb interaction, where the parameters \({U}_{{ab}}^{i{i}^{{\prime} }}\) are fixed using the screened Coulomb repulsion \(U({{{{{\bf{r}}}}}})={U}_{1}\left(\frac{1}{{{{{{\rm{|}}}}}}{{{{{\bf{r}}}}}}{{{{{\rm{|}}}}}}}-\frac{1}{\sqrt{{{{{{\rm{|}}}}}}{{{{{\bf{r}}}}}}{{{{{{\rm{|}}}}}}}^{2}+{d}^{2}}}\right)/\left(\frac{1}{{{{{{\rm{|}}}}}}{{{{{{\bf{r}}}}}}}_{1}{{{{{\rm{|}}}}}}}-\frac{1}{\sqrt{{{{{{\rm{|}}}}}}{{{{{{\bf{r}}}}}}}_{1}{{{{{{\rm{|}}}}}}}^{2}+{d}^{2}}}\right)\), where |r1| is the nearest-neighbor distance, |r| is the distance between sites \(\{i,a\}\) and \(\{{i}^{{\prime} },b\}\), and d is the screening length. We choose U1 and U0 as independent model parameters because U0 is sensitive mainly to twist angle and the strength of the moiré modulation potential, which control the size of the band Wannier function7, whereas U1 is sensitive mainly to the screening background of the moiré material. We expect the ratio of the strength of the long-range Coulomb tail to U1 to be nearly universal, as we have assumed. We solved the above model using the unrestricted Hartree-Fock approximation with details provided in the Methods section.

Results at n = 1/2, 2/3, 1/3

In the Hamiltonian used here, once the screening length d is fixed the only free parameters are U0, U1, and the band filling n. We explored the complete \({U}_{1}/{U}_{0}\) vs \({U}_{0}/t\) phase diagrams, fixing system size to \(12({L}_{x})\times 12({L}_{y})\), for the average electronic densities \(n=1/2,2/3,1/3\), where \(n={N}_{e}/(2{L}_{x}{L}_{y})\), Ne is the total number of electrons and \({L}_{x}({L}_{y})\) is number of unit cells along the x(y) direction. The model we study has particle-hole symmetry with respect to n = 1 since we employ only nearest-neighbor hopping, hence the magnetic states discussed in our paper for any fractional filling n will also be present for the case of the filling \(2-n\).

When quantum fluctuations are suppressed and we are deep in the Wigner crystal (WC) regime we obtain the configurations shown in Fig. 2 for the fillings addressed. The WC for n = 1/2 resembles a half-filled standard triangular lattice35. The other two states for n = 1/3 and n = 2/3 break rotational symmetry of the honeycomb lattice. In the following, we will calculate the \({U}_{0}/t\) vs. \({U}_{1}/{U}_{0}\) phase diagrams of Eq. (1). We will mainly discuss the phases appearing in the strong coupling regime, here defined as \({U}_{0}/t\ge 10\), because this is the regime most often explored experimentally in moiré superlattices typically and because this is the region where our method performs better. We discover a rich set of competing phases for each of the fillings, that vary as the \({U}_{0}/t\) and \({U}_{1}/{U}_{0}\) ratios are varied. The details of the phases in the weak and intermediate coupling regions are provided in Supplementary Note 1, for completeness. We also investigated the stability of the various phases found here vs. changes in screening length, by solving the above model for two screening length values \(d=2{a}_{m}\) and \(10{a}_{m}\), where \({a}_{m}\) is the moiré lattice spacing. The main message of our findings is that honeycomb lattice Wigner crystals have complex magnetic states, and that transitions between them can be tuned by varying \({U}_{0}/t\), which is very sensitive to twist angle and three-dimensional dielectric environment, or by varying \({U}_{1}/{U}_{0}\), which is sensitive to both twist angle and metallic screening backgrounds.

Fig. 2: Wigner crystal (WC) states.
figure 2

a, b, and c These show the Wigner crystals in the limit of \({U}_{0},{U}_{1} > > t\) for the fillings \(n=1/2\), \(n=2/3\), and \(n=1/3\), respectively. The parameters \({U}_{0}\), \({U}_{1},\) and \(t\) corresponds to the onsite Coulomb interaction, nearest-neighbor Coulomb interaction, and nearest-neighbor hopping, respectively.

Full size image

(a) n = 1/2 results

The main results for filling n = 1/2 are shown in Fig. 3. The calculations were performed for all the points shown as small circles in the phase diagrams in Fig. 3(a) and (b) for screening lengths \(d=2{a}_{m}\) and \(d=10{a}_{m}\), respectively. We identified multiple phases in the strong coupling limit (\({U}_{0} > > t\)) which are stable for both screening lengths, see Fig. 3(a, b).

Fig. 3: Results for filling n = 1/2.
figure 3

In a and b, the \({U}_{1}/{U}_{0}\) vs \({U}_{0}/t\) phase diagrams are shown for the screening lengths \(d=2{a}_{m}\) and \(d=10{a}_{m}\), respectively. The parameters \({U}_{0}\), \({U}_{1},t,\) and \({a}_{m}\) corresponds to the onsite Coulomb interaction, nearest-neighbor Coulomb interaction, nearest-neighbor hopping, and moiré lattice constant, respectively. c and d These illustrate the FM+WC (Ferromagnetic Wigner Crystal) and \(12{0}^{\circ }-{AFM}\) (Antiferromagnetic) states respectively. e This shows the net magnetizaion \({{{{{\rm{|}}}}}}{{{{{\bf{m}}}}}}{{{{{\rm{|}}}}}}\) and the pseudospin structure factor \({\tau }_{z}({{{{{\bf{q}}}}}})\) at momentum \({{{{{\bf{q}}}}}}=(0,0)\) for 24 × 24 system size, and the charge gap \({\Delta }_{c}\) for the 24 × 24 and 60 × 60 system sizes, for various values of \({U}_{1}/{U}_{0}\), at fixed \({U}_{0}/t=10\) and \(d=10{a}_{m}\). f and g These show the non-coplanar states and the respective polyhedra corresponding to phases \({TP}\) (Trigonal-Prism) and \({P}_{12v}\) (12-vertices polyhedron), respectively. The size of the arrows and circles in c, d, f, g is proportional to the magnitude of the local spin \({{\langle }}{{{{{{\bf{S}}}}}}}_{i}{{\rangle }}\) size and local density \({{\langle }}{n}_{i}{{\rangle }}\), respectively.

Full size image

Firstly we will discuss the region of the phase diagram where we found robust triangular WC states (shown in Fig. 2(a)) and Hartee-Fock is most trustworthy as the charge fluctuations are suppressed. The triangular WC state appears at large \({U}_{0}/t\) and large \({U}_{1}/{U}_{0}\), since the charge gap requires strong on-site and near-neighbor interactions. We find that the 120°-AFM state, (see Fig. 3(d)), is the most stable magnetic state. In this state the charge density order is identified by the pseudospin structure factor \({\tau }_{z}({{{{{\bf{q}}}}}})=1/N{\sum }_{{{{{{\bf{i}}}}}}}{\tau }_{z}({{{{{\bf{i}}}}}}){e}^{i{{{{{\bf{q}}}}}}\cdot {{{{{{\bf{r}}}}}}}_{{{{{{\bf{i}}}}}}}}\) at momentum q = (0,0), where the quantity \({\tau }_{i,z}={{\langle }}{n}_{i,b}{{\rangle }}-{{\langle }}{n}_{i,a}{{\rangle }}\) measures the electronic density difference between the two honeycomb sublattices, indicating the breaking of inversion symmetry between the two sublattices. \({\tau }_{z}(0,0)\) smoothly decreases upon decreasing \({U}_{1}/{U}_{0}\). The total magnetization \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=2/{N}_{e}{{{{{\rm{|}}}}}}{\sum }_{i}{{\langle }}{S}_{i}{{\rangle }}{{{{{\rm{|}}}}}}=0\) throughout this phase.

The 120°-AFM state is common on triangular lattices. Surprisingly we also found unexpected exotic non-coplanar AFM states: the 12-vertices polyhedron (\({P}_{12v}\)) and the Trigonal-Prism (TP). Both states have non-zero \({\tau }_{z}(0,0)\), confirming triangular lattice Wigner crystal charge order. Previous to this work, only tetrahedral non-coplanar magnetic states were reported in honeycomb36 and triangular lattice Hubbard models37,38. The \({P}_{12v}\) phase has 12 distinct spins per unit cell with the members of two sets of six spins, namely {1,2,3,4,5,6} and {7,8,9,10,11,12}, lying along two perpendicular hexagons (colored as red and green in Fig. 3(g)). Figure 3(g) also shows the positions of the non-coplanar spins in real space. The Trigonal Prism (TP) phase is shown in Fig. 3(f) where six spins are aligned towards the corner of the trigonal-prism and the real-space lattice is shown on the right side. Increasing \({U}_{1}/{U}_{0}\), inside the region named TP in the phase diagram, decreases the height (h i.e., distance between spins “1 and 5” of the prism) and on further increasing \({U}_{1}/{U}_{0}\), h suddenly drops to zero, converting the spin-configuration to that of the 120°-AFM state.

Furthermore, we checked the scalar chirality \({{{{{{\bf{S}}}}}}}_{i}\cdot ({{{{{{\bf{S}}}}}}}_{j}\times {{{{{{\bf{S}}}}}}}_{k})\) of both non-coplanar states, where \({{{{{{\bf{S}}}}}}}_{i,j,k}\) are the spins located on the emergent triangular lattice. We find that the chirality has striped patterns, with zero net chirality unlike the tetrahedron spin state with homogenous and nonzero net chirality, for details see Supplementary Note 2. Our calculations also confirmed zero Hall-conductance for both non-coplanar states, showing that they are topologically trivial.

Further decreasing \({U}_{1}/{U}_{0}\), we found a sudden decrease in \({\tau }_{z}(0,0)\) and an abrupt jump in |m| from 0 to 1, indicating a first-order AFM to FM (ferromagnetic) phase transition near \({U}_{1}/{U}_{0} \sim 0.35\) (for \(d=10{a}_{m}\)). This phase also breaks the sub-lattice inversion symmetry, hence making it a saturated FM insulator. A representative state for this FM+WC phase is shown in Fig. 3(c). In the low \({U}_{1}/{U}_{0}\) region, we found a fully polarized ferromagnetic metal with a net magnetization |m| = 1.0 i.e., a Nagaoka ferromagnetic state. This fully polarized metallic state has been reported in earlier mean-field studies of the Hubbard models on various two-dimensional lattices, including the honeycomb lattice39,40. In Fig. 3(e) we show \({\tau }_{z}(0,0)\) and the charge gap \({\Delta }_{c}\) evolution with changing \({U}_{1}/{U}_{0}\), fixing \({U}_{0}/t=10\) and \(d=10{a}_{m}\). The \({\tau }_{z}(0,0)\) and \({\Delta }_{c}\) continuously decrease to 0, suggesting a second-order phase transition in between FM insulator and FM metallic phase. Nevertheless, exact calculations are required to confirm the presence of this FM metallic state as the Hartree-Fock approximation is expected to overestimate the presence of ferromagnetic metallic states, especially for low band fillings41.

In the triangular Wigner crystal limit, as the charge fluctuations are heavily suppressed, it is natural to ponder if a low-energy spin model can explain the magnetic states we found. At large \({U}_{1}/{U}_{0}\), fourth-order processes, using the hopping as a small parameter, generate an antiferromagnetic exchange \({J}_{{AFM}}\propto {t}^{4}/{(2{U}_{1})}^{2}{U}_{0}\) between the nearly half-filled sites of the emergent triangular lattice (which are second-nearest neighbors in the original honeycomb lattice) leading to a spin-1/2 antiferromagnetic Heisenberg model on the triangular lattice. In agreement with this reasoning, we found the \(12{0}^{\circ }\)-AFM state in the large \({U}_{1}/{U}_{0}\) region of the phase diagrams, as in the ground state solution of the triangular lattice Heisenberg model in both classical and quantum limits42,43.

Theoretical studies of classical spin models on triangular lattices, when in presence of a 4-site ring exchange term, have shown the appearance of 6-sublattice and 12-sublattice non-coplanar phases resembling our Trigonal Prism and \({P}_{12v}\) phases, respectively44. In our study in the Wigner crystal region, the ring-exchange term on 4-site plaquettes as in triangular lattices can be generated by 8th-order hopping processes leading to a ring exchange \({J}_{R}\propto {t}^{8}/{(2{U}_{1})}^{2}{({U}_{0})}^{3}{(3{U}_{1})}^{2}\). Note that in this scenario \({J}_{R}/{J}_{{AFM}}\propto {t}^{4}/{({U}_{0}{U}_{1})}^{2}\) and decreasing \({U}_{1}\) increases \({J}_{R}/{J}_{A{FM}}\), showing the importance of the ring-exchange mechanism as \({U}_{1}\) is decreased. Naturally this suggests that the ring-exchange term will be dominant inside the Wigner crystal phase as both \({U}_{1}\) and \({U}_{0}\) are decreased (if the Wigner crystal remains stable), explaining the location of the non-coplanar phases at the boundary of the Wigner crystal phase and also explaining why the non-coplanar phases are destabilized as \({U}_{1}\) or \({U}_{0}\) is increased.

Quantum fluctuations can be studied using density matrix renormalization group technique45,46. Although these are limited to finite-width ribbons, they will be important to investigate the stability of the non-coplanar states and the possibility of realizing exotic quantum spin liquids. For example, it has been shown that the Trigonal Prism state has uniform vector chirality (although the net scalar chirality is zero). Including quantum fluctuations may lead to a vector chiral spin liquid state47.

(b) n = 2/3 results

The \({U}_{1}/{U}_{0}\) vs \({U}_{0}/t\) phase diagrams for filling \(n=2/3\) at \(d=2{a}_{m}\) and \(d=10{a}_{m}\) are shown in Fig. 4(a, b). Again, we unveiled a plethora of phases. In a large part of the phase diagram (the blue colored region) we found states with two distinct charge stripes with a width of 2-sites (resembling the double-striped Wigner crystal shown in Fig. 2), where one stripe follows an AFM zigzag pattern and other stripe is made up of nearest neighbor AFM dimers present at distance of \({a}_{m}\). These double-striped phases break the \({C}_{3}\) rotational invariance of the honeycomb lattice. Stripe formation in the local charge density naturally favors AFM correlations, since antiferromagnetic superexchange is expected between the nearest-neighbor nearly half-filled sites. We find three kinds of similar looking but distinct collinear antiferromagnetic double-striped states named here \({S}_{1}\) (see Fig. 4(d)), \({S}_{2}\) (see Fig. 4(e)), and \({S}_{3}\) (see Fig. 4(c)). In the \({S}_{3}\) phase, the orientation of the AFM dimers (shown by ellipses drawn around pair of opposite spins) are parallel to each other in the stripe of dimers; whereas in the \({S}_{2}\) phase we found AFM dimers are oriented opposite to each other. In the \({S}_{1}\) phase, which is stabilized for the small \({U}_{1}/{U}_{0}\) values, the AFM dimers are oriented parallel to each other, similar to \({S}_{3}\), but the spins of zigzag stripe and dimer stripe sitting at a distance of \({a}_{m}\)(next-nearest neighbout) are FM aligned, unlike in the \({S}_{3}\) phase.

Fig. 4: Results for filling n = 2/3.
figure 4

In a and b, the \({U}_{1}/{U}_{0}\) vs \({U}_{0}/t\) phase diagrams are shown for screening lengths \(d=2{a}_{m}\) and \(d=10{a}_{m}\), respectively. The parameters \({U}_{0}\), \({U}_{1},t,\) and \({a}_{m}\) corresponds to the onsite Coulomb interaction, nearest-neighbor Coulomb interaction, nearest-neighbor hopping, and moiré lattice constant, respectively. The metal-insulator transition is illustrated by the thick-dashed black line in the low \({U}_{1}/{U}_{0}\) region. FM(AFM) stands for ferromagnetic(antiferromagnetic). c, d, e, f, g display representative states of the Stripe-3 AFM (\({S}_{3}\)), Stripe-1 AFM (\({S}_{1}\)), Stripe-2 AFM (\({S}_{2}\)), Hexagonal Stripes HS, and 10-vertices polyhedron (\({P}_{10v}\)), respectively. g This also shows the 10-vertices polyhedron corresponding to the state \({P}_{10v}\). The set of spins {1,2,3,4,5,6} and {7,8,1,9,10,4} lie in the green and red planes, respectively.

Full size image

More exotic phases tend to appear at smaller values of \({U}_{0}/t\) closer to the metal-insulator transition boundaries. We observed a 10-sublattice non-coplanar phase named 10-vertices polyhedron (\({P}_{10v}\)), displayed in Fig. 4(g), where 10 distinct non-coplanar spins are placed on a double-striped Wigner crystal. The orientation of the spins is shown on the left side of Fig. 4(g). Note that the spin sets \(\{1,2,3,4,5,6\}\) and \(\{1,9,10,4,7,8\}\) lie perfectly in the green and red planes, respectively, each set making a little contorted hexagon (i.e., the angle between spins “5 and 6”, “5 and 4”, etc. is nearly \(6{0}^{\circ }\); the precise angle between the red and green plane depends on the parameter values). The stripe of the spins “4 and 1” (oriented opposite to each other) is an AFM collinear stripe and the other stripe is non-collinear with either spins “7, 8, 9, and 10” or “2, 3, 5, and 6” still having nearest-neighbor AFM dimers similar to the \({S}_{(1,2,3)}\) phases.

We noticed that for the larger screening length, namely with robust long-range interactions, the non-coplanar phases are stabilized in a larger region of the phase diagram for both \(n=1/2\) and \(2/3\) fillings. For \(n=2/3\), we also observed, near the \({U}_{0}/t=10\) region and at large \({U}_{1}/{U}_{0}\), a state with AFM hexagonal stripes made up of nearest-neighbor sites which are separated by the nearest-neighbor AFM dimers, named Hexagonal Stripes (HS). The HS state respects the rotational symmetry of the honeycomb lattice unlike the double-striped states. We believe the double-striped Wigner crystal states will be most relevant for experiments at filling \(n=2/3\) for honeycomb moiré superlattices. Its existence can be indirectly confirmed by optical anisotropy measurements25.

Similarly to the \(n=1/2\) case, we found a fully polarized FM metallic phase for \({U}_{1}/{U}_{0}=0\), but now a larger \({U}_{0}/t\) is required. This \(n=2/3\) fully polarized FM phase is rapidly suppressed when including long-range interactions, contrary to the \(n=1/2\) case. A large portion of the small \({U}_{1}/{U}_{0}\) phase diagram (for \({U}_{0}/t\ge 10\)) is covered by the non-collinear Vortices phase, details are shown in Supplementary Note 1.

We also estimated the metal-insulator transition line by calculating the single particle density of states (shown in Supplementary Note 3). We found that the low \({U}_{1}/{U}_{0}\) phases, namely the FM and Vortices phases are mainly metallic. On increasing the \({U}_{1}/{U}_{0}\), the FM and Vortices phases gradually develops a weak charge density wave (CDW) and a small gap at the chemical potential and eventually system transits into the insulating Stripe phases with much stronger double-striped Wigner crystallization. The thick-dashed black line in the phase diagrams (in low \({U}_{1}/{U}_{0}\) region) depicts the metal-insulator transition.

(c) n = 1/3 results

Figure 5(a, b) shows the \({U}_{1}/{U}_{0}\) vs \({U}_{0}/t\) phase diagrams for \(n=1/3\), again for both \(d=2{a}_{m}\) and \(d=10{a}_{m}\). As for other fillings, we will discuss the dominant phases in the \({U}_{0}/t\ge 10\) region. In the large \({U}_{1}/{U}_{0}\) we found a Zigzag Wigner crystal state with collinear AFM ordering \(({AFM}+{WC})\) present in a robust portion of the phase diagram, see Fig. 5(c). The representative state for the \({AFM}+{WC}\) phase, see Fig. 5(b), has Bravais lattice vectors \((3x,0)\) and \((0,\sqrt{3}y)\). Similar zigzag pattern states for filling \(n=1/3\) in a honeycomb lattice, forming a charge density, were also found by minimizing the energy of only the interaction part of the Hamiltonian48. Intuitively, the robustness of the \({AFM}+{WC}\) phase suggests it may survive adding quantum fluctuations because it dominates a large portion of the phase diagram, and hence it may be present for real honeycomb moiré materials at filling \(n=1/3\). In the opposite limit of \({U}_{1}=0\), we again notice a fully polarized metallic FM phase at large \({U}_{0}/t\), as at \(n=1/2\) and \(2/3\) (near \({U}_{0}/t=10\) we also found a spiral phase in a small region). Turning on the long-range interactions, the FM phase now manifests itself via various CDW arrangements, in order of increasing \({U}_{1}/{U}_{0}\), for details see Supplementary Note 1.

Fig. 5: Results for filling n = 1/3.
figure 5

In a and b, the \({U}_{1}/{U}_{0}\) vs \({U}_{0}/t\) phase diagrams for screening lengths \(d=2{a}_{m}\) and \(d=10{a}_{m}\) are shown, respectively. The parameters \({U}_{0}\), \({U}_{1},t,\) and \({a}_{m}\) corresponds to the onsite Coulomb interaction, nearest-neighbor Coulomb interaction, nearest-neighbor hopping, and moiré lattice constant, respectively. FM, AFM, M, WC, TS, and CDW stands for ferromagnetic, antiferromagnetic, metal, Wigner crystal, twin stripes, and charge density wave, respectively. c, d, e These show the representative states of Zigzag Wigner crystal with collinear AFM ordering (AFM+WC), Zigzag 120°-Twin Stripes (\(12{0}^{\circ }\)-TS), and Long Zigzag collinear phase, respectively.

Full size image

For intermediate \({U}_{1}/{U}_{0}\), we found exotic magnetic patterns. For example, the coplanar state with, again, strong zigzag charge density wave stripes but with spins pointing at \(12{0}^{\circ }\) with respect to each other inside the zigzag stripe. However, the nearby twin \(12{0}^{\circ }\) stripe is oriented along some arbitrary angle, see Fig. 5(d), hence we name this state zigzag \(12{0}^{\circ }\)-twin stripes (\(12{0}^{\circ }\)-TS). Moreover, we also found a collinear state with long zigzag spin stripes aligned opposite to each other, see Fig. 5(e). We believe that these complex states emerge in the large \({U}_{0}/t\) and intermediate \({U}_{1}/{U}_{0}\) region as a compromise between strong tendencies towards an AFM Wigner crystal at larger \({U}_{1}/{U}_{0}\) and ferromagnetism at smaller \({U}_{1}/{U}_{0}\). We noticed that for \(n=1/3\) we did not find any non-coplanar states, in contrast to the \(n=1/2\) and \(n=2/3\) phase diagrams. This might be due to smaller higher order exchanges between localized spins (like the ring-exchange discussed for the \(n=1/2\) case) because of relatively larger distances between the half-filled sites of Zigzag Wigner crystal states.

Our discovery of a variety of exotic spin arrangements close to the dominant AFM and FM states, including the \(n=1/2\) and \(n=2/3\) exotic phases, is common occurrence in other families of materials where there is strong phase competition, such as in manganites49,50,51,52,53, ruthenates54, and ladder iron superconductors55,56. These unusual states all arise because near a possible first-order AFM-FM transition, there are spin arrangements mixing both characteristics of the dominant states that can reduce even further the energy.

Magnetization evolution as signature of exotic states

Not all tools available to observe the magnetic structure of bulk materials are available for two-dimensional moiré materials. In particular, neutron diffraction methods cannot be applied because of small sample sizes and magnetizations that are very small when measured per atom. Fortunately, it turns out that the total spin magnetization of moiré materials can be measured optically3, at least in the \(K\)-valley case. With the primary purpose of assessing how optical studies might see the magnetization of moiré materials, in this section we will discuss the evolution of the net magnetization \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=2/{N}_{e}{{{{{\rm{|}}}}}}{\sum }_{i}{{\langle }}{S}_{i}{{\rangle }}{{{{{\rm{|}}}}}}\) with external magnetic field \(h\) along the \(z\) direction, which adds a Zeeman splitting term \(-1/2{\sum }_{i}({n}_{i\uparrow }-{n}_{i\downarrow })h\) to the Hamiltonian.

Firstly, we discuss the \(n=1/2\) case (for \(d=10{a}_{m}\)), with main focus on the phases at large \({U}_{1}/{U}_{0}\), namely the \(120\)-\({AFM}\), Trigonal Prism (TP), and 12-vertices polyhedron (\({P}_{12v}\)) phases. We use two representative parameter points [\(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.5)\) and \(({U}_{0}=16t,{U}_{1}/{U}_{0}=0.45)\)] for the \(120\)-\({AFM}\) state [see phase diagram in Fig. 3(b)]. The magnetization evolution for the \(120\)-\({AFM}\) state is shown in Fig. 6(a). As \(h/t\to 0\), \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) increases linearly, \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\approx {\chi }_{s}h/t\). We find that the spin susceptibility defined as \({\chi }_{s}={{{{{{\rm{lim}}}}}}}_{h\to 0}d{{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}/{dh}\) is close to 8 and 16 for \(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.5)\) and \(({U}_{0}=16t,{U}_{1}/{U}_{0}=0.45)\), respectively.

Fig. 6: Magnetization in an external magnetic field.
figure 6

Evolution of the net magnetization \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) with an external magnetic field \(h\) (along \(z\)), at filling \(n=1/2\) and \(d=10{a}_{m}\). We selected representative parameter points for the respective phases using the phase diagram Fig. 3(b). a This contains results for the points \(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.5)\) and \(({U}_{0}=16t,{U}_{1}/{U}_{0}=0.45)\), choosen from the \(12{0}^{\circ }\)-\({AFM}\) (antiferromagnetic) phase region. b This displays results for the Trigonal Prism (TP) state at \(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.4)\) and \(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.45)\). Results for the 12-vertices polyhedron state (\({P}_{12v}\)) are shown in c at \(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.35)\) and \(({U}_{0}=12t,{U}_{1}/{U}_{0}=0.35)\). The insets of a and c show the plateau states \({uud}\) (\({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1/3\)) and \({uuud}\) (\({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1/2\)), respectively. The parameters \({U}_{0}\), \({U}_{1},t,\) and \({a}_{m}\) corresponds to the onsite Coulomb interaction, nearest-neighbor Coulomb interaction, nearest-neighbor hopping, and moiré lattice constant, respectively.

Full size image

At \(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.5)\) a very clear \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1/3\) plateau was unveiled, while at \(({U}_{0}=16t,{U}_{1}/{U}_{0}=0.45)\) this was replaced by a small, but still visible, kink. The plateau in magnetization signals the stability of a three-sublattice uud state, where \(u(d)\) denotes up(down)-spin. This uud state is shown in the inset of Fig. 6(a). The presence of a \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1/3\) plateau in \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) vs \(h\) curves was reported before for a quantum spin \(S=1/2\) nearest-neighbor antiferromagnetic Heisenberg model on a triangular lattice, employing exact calculations at zero temperature57; the classical spin Heisenberg model does not show this plateau58 at least for the nearest-neighbor interactions. Thus, it is fascinating that our HF calculations can capture this \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1/3\) plateau in the emergent triangular lattice Wigner crystal, especially at \(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.5)\) which is close to the region of competition with non-coplanar phases. As already discussed, ring-exchange is expected to play an important role in non-coplanar phases. Thus, it is reasonable to invoke such terms in the effective spin model to explain the plateau’s formation in a mean-field calculation. This conclusion is in agreement with calculations for the classical spin models including ring-exchange which also shows plateau formation in \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) vs \(h\) curves59,60.

We also investigated the evolution of \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) with \(h\) in non-coplanar phases [see Fig. 6(b, c)]. We found that the Trigonal Prism phase shows a very similar \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) vs \(h\) curve as displayed before, i.e., with a linearly increasing \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) for small magnetic field \(h\) and the presence of a \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1/3\) plateau. The spin order in the Trigonal Prism state breaks the rotational invariance of the lattice unlike in the 120-AFM state. This symmetry breaking eventually can be observed in transport measurements similarly as when rotational symmetry broken by nematic order was recently observed in experiments61,62. Thus, the combination of the \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1/3\) plateau, and also optical anisotropy measurements25, can be used as the fingerprints of the TP phase and to distinguish this phase from the 120-AFM state. The 12-vertices polyhedron phase shows distinctively different features in the \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) evolution [see Fig. 6(c)], with \({{{{{{\rm{lim}}}}}}}_{h\to 0}{{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\approx {\chi }_{s}h+a{h}^{2}\) where \(a/{\chi }_{s} > > 1\). For example, at \(({U}_{0}=10t,{U}_{1}/{U}_{0}=0.35)\) we found \({\chi }_{s}\approx 0.82\) and \(a\approx 5\times 1{0}^{4}\). We believe that the small susceptibility and robust second-order term arises from the presence of ferromagnetic exchange in the \({P}_{12v}\) phase: as discussed in ref. 44 the \({P}_{12v}\) phase can be obtained from the ferromagnetic classical spin model with ring-exchange on a triangular lattice. Moreover, we found the presence of a \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\approx 1/2\) plateau with a slightly canted uuud state (see inset of Fig. 6(c)) at location \(({U}_{0}=12t,{U}_{1}/{U}_{0}=0.35)\). We noticed that having a robust \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1/2\) plateau depends on the parameter values we select, and was only present at large \({U}_{0}/t\) values. The magnetization evolution for the \({P}_{12v}\) phase always shows a first-order transition to the saturated ferromagnetic state, with a robust jump of \({{{{{\rm{|}}}}}}\Delta m{{{{{\rm{|}}}}}}\ge 1/2\) unlike the case of the \(120\)-\({AFM}\) and Trigonal Prism phases, where \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}\) smoothly increases up to its saturation value with a gradual spin canting in the states.

The saturation magnetic field values can be converted in Teslas using \(H=h/2{\mu }_{B}\) where \({\mu }_{B}\) is the Bohr magneton. Note that the hopping parameter \(t\) increases with the twist angle (\(\theta\)) between the layers and varies in the approximate range \(\{0.16,3\}\) meV for \(\theta \in \{1.{5}^{\circ },2.{5}^{\circ }\}\) (for details, see Supplementary Note 5). Using these hopping values, for a given \({U}_{0}/t\) we can crudely estimate the range of saturation magnetic fields. For example, we checked that in the \(12{0}^{\circ }\) phase at \(({U}_{1}/{U}_{0}=0.5,{U}_{0}=10t)\) (the point with the largest saturation field \(h/t=0.24\) in the phase diagram of Fig. 3(b)) the saturation is reached at 0.35 T for \(t=1/6\) meV and at 6.2 T for \(t=3\) meV. This analysis suggests that saturation magnetic fields can easily be reached experimentally. We also studied the magnetization evolution of Wigner crystal phases for the fillings \(n=2/3\) and \(1/3\) but did not find any fractional magnetization plateaus (see Supplementary Note 4).

Influence of direct exchange interaction, quantum fluctuations, and thermal fluctuations

In the present work, we did not incorporate the direct exchange term (\({S}_{i}\cdot {S}_{j}\)) in the Hamiltonian, as shown in Eq. (1), thus the magnetic properties are only generated via kinetic exchange. In most theoretical studies of conventional materials the direct exchange term is ignored because of its small magnitude. More specifically, in theoretical studies of the TMD triangular moiré superlattices the direct exchange has been also ignored30,31,32,33,34. Consequently, as we investigated the new honeycomb moiré lattice we kept our model simple and consistent with similar models studied before in this field, and we did not incorporate the direct exchange as well. Only recently, the relevance of direct exchange was discussed in TMD moiré lattices for half-filled bands63. In the present paper, however, our focus was to analyze the magnetism in Wigner crystals at fractional fillings. As shown in Fig. 2, in this limit the half-filled sites are separated by almost empty sites (specially at \(n=1/2\) and \(n=1/3\)) which means only direct exchange terms beyond nearest-neighbors can play an important role, and those are expected to be even much smaller than nearest neighbors. Moreover, at large values of the dielectric constant and large twist angles the ratio (kinetic exchange)/(direct exchange) increases and hence the kinetic exchange is expected to dominate magnetic properties, as also shown in ref. 63. Thus, we believe that for the moiré materials with large dielectric constant and large twist angles (\(\approx\) \({2}^{\circ }\) to \({3}^{\circ }\)) our results are stable against introducing the small direct exchange of real moiré materials. Moreover, the dielectric constant will be enhanced in moiré materials by charge-fluctuations between the valence moiré band and the lower energy moiré bands, and also from the screening by nearby conducting gate layers.

Incorporating the FM direct exchange will increase the frustration as it will compete with the AFM superexchange. Thus, at prima facie we believe adding direct exchange will further increase the size of the region with the exotic non-coplanar and non-collinear phases, because they are “sandwiched” between the small \({U}_{1}/{U}_{0}\) FM state and large \({U}_{1}/{U}_{0}\) AFM state. Thus, we anticipate that direct exchange can have interesting consequences, specially in the presence of small twist angle and weakly screened interactions, and those consequences will reinforce our main conclusions. The influence of direct exchange can certainly be studied in future.

Quantum fluctuations is another aspect that we have not incorporated in the present work because we used a static mean-field theory. Employing more sophisticated many-body techniques applied to this complex problem are highly difficult. For example, the use of density matrix renormalization group would be problematic because of the high entanglement brought up by the long-range interactions that will drastically increase the number of states \(m\) needed, as compared with canonical short-range models. Nonetheless, we can qualitatively anticipate what the effect of quantum fluctuations will be. For example, as discussed earlier, (1) the 120-AFM state we found in the \(n=1/2\) Wigner crystal will likely survive including quantum fluctuations, because even the more fluctuating just nearest-neighbor spin = 1/2 Heisenberg model has also a 120-AFM state as the ground state, both in mean field and with quantum effects included. (2) On the other hand, the non-coplanar states we found for \(n=1/2\) and \(n=2/3\) can become quantum spin liquids after incorporating quantum fluctuations, because these states already have inbuilt magnetic frustration. (3) The double-striped WC (\(n=2/3\)) and zigzag WC (\(n=1/3\)) appear very stable because they survive up to the largest values of \({U}_{1}/{U}_{0}\) and \({U}_{0}\) studied in our phase diagrams. However, embedded in these states are quasi-1D spin chains and quasi-0D spin dimers, and it is very likely that after adding quantum fluctuations the dimers in double-striped WC (\(n=2/3\)) will become spin singlets while the quasi-1D spin chains in the zigzag WC (\(n=1/3\)) and double-striped WC (\(n=2/3\)) will behave as the 1D spin=1/2 Heisenbeg chains leading to spin liquids. Similarly, the isolated hexagons and the spin dimers of Fig. 4(f) will also have a strongly quantum nature. Certainly, more sophisticated calculations are required to investigate if the long-range kinetic exchange stabilizes the long-range magnetic structures we found over the possibility of quantum spin liquids. We anticipate in many cases quantum fluctuations can drive these ordered states to quantum spin liquids, as discussed above, rendering the search for experimental realizations of these honeycomb systems even more pressing.

All our Hartree–Fock calculations were performed at zero temperature. At finite temperature, a priori one anticipates that the Mermin-Wagner (M-W) theorem forbids true long-range order in any perfectly 2D moiré materials at any nonzero temperature (not just our honeycomb system studied here but any moiré material including those with triangular arrangements). However, note that it is widely believed that 2D spin-spin correlation lengths (\(\xi\)) behave with temperature \(T\) as \(\xi \propto {e}^{(A/T)}\) where \(A\) is a positive constant. Thus, at low temperatures these correlation lengths can become very large, due to the essential singularity growth as \(T\) is reduced. Then, \(\xi\) probably becomes of the order of the inter defect distances, or larger, and the influence of the M-W theorem will be masked by other effects. Moreover, in our case the Coulombic interactions are of a long-range nature, so the M-W theorem influence may be smaller than expected and correlation lengths could be as large as several hundred lattice spacings below a characteristic temperature, thus mimicking long-range order.

Discussion

In this work, we comprehensively studied the \({U}_{1}/{U}_{0}\) vs \({U}_{0}/t\) phase diagrams of a moiré honeycomb lattice model with long-range Coulomb interactions, for fillings \(n=1/2,1/3\), and \(2/3\), employing the Hartree Fock approximation. We believe this study, specially the larger \({U}_{1}/{U}_{0}\) regions of the phase diagrams with Wigner crystal ground states, is directly relevant for potential future experiments on \(\Gamma\)-valley homobilayer-TMD’s, such as \({MoS}{e}_{2},{Mo}{S}_{2}\), and \(W{S}_{2}\)19,20,21. Specifically, we have found emergent patterns involving triangular lattices, double-striped, and zig-zag Wigner crystals for the fillings \(n=1/2,2/3,\) and \(1/3\), respectively, in the region of robust long-range correlation strengths. These Wigner crystals can be directly observed using recently developed high-resolution sensitive, but non-invasive, scanning tunneling microscopy which has already been employed to image Wigner crystals in fractionally filled triangular \({WS}{e}_{2}/W{S}_{2}\) moiré superlattices26. Moreover, indications for stripe and zig-zag charge ordering in Wigner crystals at \(n=2/3\) and \(1/3\), respectively – states that break rotational symmetry – can be indirectly observed via anisotropies in optical conductivity measurements25.

We extensively studied the magnetism of Wigner crystal states and found that the long-range interactions can lead to an unexpected abundance of exotic magnetic phases. As intuitive guidance to the vast complexity observed, we noticed common features at the three densities \(n=1/2,1/3\), and \(2/3\). For instance, near the metal-insulator transition, the system always transits from a FM-metal to an AFM Wigner crystal insulator (at robust long-range electronic repulsion), often via an intermediate FM charge density wave and other more complex states (with \({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=0\)). These exotic states emerge from the competition between the robust FM vs AFM Wigner system tendencies in the small and large \({U}_{1}/{U}_{0}\) extremes of the phase diagram. We noticed for all three fillings (\(1/2,2/3,\) and \(1/3\)) the fully polarized FM (\({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=1\)) to AFM (\({{{{{\rm{|}}}}}}m{{{{{\rm{|}}}}}}=0\)) transition is of first order. In other words the magnetization drops from 1 to 0 suddenly on increasing the strength of \({U}_{1}/{U}_{0}\), at fixed \({U}_{0}/t\). As recently shown, the magnetic field-induced Zeeman splitting in moiré excitons can be used to measure the magnetic susceptibility and the related Weiss constant, directly indicating whether the system is in any of the above discussed FM to AFM regions3,64.

In real experiments, tuning the distance between the gate and the device (\(D\))23 directly affects the screening length \(d\) as \(d=2D\), which primarily corresponds to varying the \({U}_{1}/{U}_{0}\) strength of the long-range interaction. Similarly, changing the dielectric environment to tune the dielectric constant \(\varepsilon\) corresponds to moving along the horizontal axis of our phase diagrams i.e., tuning \({U}_{0}/t\) while keeping constant \({U}_{1}/{U}_{0}\) because \({U}_{i}\propto {\varepsilon }^{-1}\) for any \(i\ge 0\). Thus, we believe that changing the screening length and dieletric enviroment can help to reach the most stable exotic non-collinear and non-coplanar phases discussed in this work.

We also discussed the magnetization evolution with an external magnetic field, for the various exotic states, which can be used experimentally as indirect evidence for the states found here. Our study shows that honeycomb moiré materials can harbor very rich spin physics in two dimensions, as much as in the extensively studied triangular moiré materials. Recent high resolution angle-resolved photoemission spectroscopy and scanning tunneling microscopy experiments have shown the presence of \(\Gamma\)-valley moiré bands and its associated real-space honeycomb lattice, respectively, in the twisted \({WS}{e}_{2}\) homobilayer65. Moreover, ab-initio calculations showed that the sufficient pressure on twisted \({WS}{e}_{2}\) homobilayer can push \(\Gamma\)-valley moiré bands to higher energy, mainly above the \(K\)-valley bands, making them the valence bands. Above discovery broadens the horizon of our work as the results discussed in this paper can also be realized in the already synthesized twisted \({WS}{e}_{2}\) homobilayer materials. We hope our study will also encourage experimentalists to synthesize the \(\Gamma\)-valley homobilayer TMDs because exotic magnetic states and Wigner crystals can potentially be achieved in these materials, as our work indicates.

Methods

Hartree–Fock decomposition

We will now describe the mean field method used in this work. We can imagine the \({L}_{x}\times {L}_{y}\) lattice tessellated by an \({n}_{x}\times {n}_{y}\) number of smaller cells of \({l}_{x}\times {l}_{y}\) size, where \({l}_{x(y)}={L}_{x(y)}/{n}_{x(y)}\). In the basis of the smaller cells, we can write the Hamiltonian interaction terms of Eq. (1) in the following manner:

$${H}_{{int}}= {U}_{0}\mathop{\sum} \limits_{j,\alpha ,a}{n}_{j\alpha a\uparrow }{n}_{j\alpha a\downarrow }+{U}_{1}\mathop{\sum} \limits_{j\alpha }{n}_{j\alpha 0}{n}_{j\alpha 1}\\ +1/2\mathop{\sum} \limits_{j,\alpha \ne {\alpha }^{{\prime} },a,b}{U}_{{ab}}^{\alpha {\alpha }^{{\prime} }}{n}_{j\alpha a}{n}_{j{\alpha }^{{\prime} }b}\\ +1/2\mathop{\sum} \limits _{j\ne {j}^{{\prime} },\alpha ,{\alpha }^{{\prime} },a,b}{U}_{{ab}}^{j{j}^{{\prime} }\alpha {\alpha }^{{\prime} }}{n}_{i\alpha a}{n}_{j{\alpha }^{{\prime} }b}.$$
(2)

Now the site index \(i\) of Eq. (1) is replaced by \(\{j,\alpha \}\), where \(j\) is the cell index and \(\alpha\) is the site index with respect to the \(j\)th cell. All four fermionic terms in the above Hamiltonian are treated under the Hartree-Fock approximation. The (many) order parameters to be used to minimize the energy are the elements of the single-particle density matrix, namely \({{\langle }}{c}_{j\alpha a\sigma }^{{{\dagger}} }{c}_{{j}^{{\prime} }{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}{{\rangle }}\). We assume the mean-field solution can have broken translational symmetry with a new emergent unit cell \({l}_{x}\times {l}_{y}\). Under the above assumption we can write that any observable satisfies \(O(j,\alpha ,{j}^{{\prime} },{\alpha }^{{\prime} })=O({{{{{{\bf{r}}}}}}}_{{j}^{{\prime} }}-{{{{{{\bf{r}}}}}}}_{j}, {\alpha }^{{\prime} }, \alpha )\). After imposing the above assumption on the order parameters, using \({c}_{j\alpha a\sigma }^{{{\dagger}} }=\frac{1}{\sqrt{{N}_{{cells}}}}{\sum }_{{{{{{\bf{k}}}}}}}{e}^{\iota {{{{{\bf{k}}}}}}.{{{{{{\bf{r}}}}}}}_{j}}{c}_{{{{{{\bf{k}}}}}}\alpha a\sigma }^{{{\dagger}} }\) we can write the Hamiltonian in momentum (\({{{{{\bf{k}}}}}}\)) space as follows,

$${H}_{{int}}^{{HF}}({{{{{\bf{k}}}}}})= \, {U}_{0}\mathop{\sum} \limits _{\alpha ,a,\sigma }{{\langle }}{n}_{\alpha a\bar{\sigma }}{{\rangle }}{n}_{{{{{{\bf{k}}}}}}\alpha a\sigma }-({{\langle }}{S}_{\alpha a}^{+}{{\rangle }}{S}_{{{{{{\bf{k}}}}}}\alpha a}^{-}+h.c.)\\ +\,{U}_{1}\mathop{\sum} \limits_{\alpha ,\sigma ,{\sigma }^{{\prime} },a}{{\langle }}{n}_{\alpha \bar{a}\sigma }{{\rangle }}{n}_{{{{{{\bf{k}}}}}}\alpha a{\sigma }^{{\prime} }}-{{\langle }}{c}_{\alpha \bar{a}\sigma }^{{{\dagger}} }{c}_{\alpha a{\sigma }^{{\prime} }}{{\rangle }}{c}_{{{{{{\bf{k}}}}}}\alpha a{\sigma }^{{\prime} }}^{{{\dagger}} }{c}_{{{{{{\bf{k}}}}}}\alpha \bar{a}\sigma }\\ +\,\mathop{\sum} \limits_{\begin{array}{c}\alpha \ne {\alpha }^{{\prime} },\\ a,b,\sigma ,{\sigma }^{{\prime} }\end{array}}{U}_{{ab}}^{\alpha {\alpha }^{{\prime} }}({{\langle }}{n}_{\alpha a\sigma }{{\rangle }}{n}_{{{{{{\bf{k}}}}}}{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}-{{\langle }}{c}_{\alpha a\sigma }^{{{\dagger}} }{c}_{{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}{{\rangle }}{c}_{{{{{{\bf{k}}}}}}{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}^{{{\dagger}} }{c}_{{{{{{\bf{k}}}}}}\alpha a\sigma })\\ +\,\mathop{\sum} \limits _{\begin{array}{c}\alpha ,{\alpha }^{{\prime} },\\ a,b,\sigma ,{\sigma }^{{\prime} }\end{array}}{\bar{U}}_{{ab}}^{\alpha {\alpha }^{{\prime} }}{{\langle }}{n}_{{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}{{\rangle }}{n}_{{{{{{\bf{k}}}}}}\alpha a\sigma }-{V}_{\alpha a\sigma }^{{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}({{{{{\bf{k}}}}}}){c}_{{{{{{\bf{k}}}}}}{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}^{{{\dagger}} }{c}_{{{{{{\bf{k}}}}}}\alpha a\sigma }$$
(3)

where \({\bar{U}}_{{ab}}^{\alpha {\alpha }^{{\prime} }}={\sum }_{j\ne 0}{U}_{{ab}}^{0\alpha ,j{\alpha }^{{\prime} }}\), and \({V}_{\alpha a\sigma }^{{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}({{{{{\bf{k}}}}}})={\sum }_{j\ne 0}{U}_{{ab}}^{0\alpha ,j{\alpha }^{{\prime} }}{e}^{\iota {{{{{\bf{k}}}}}}j}{{\langle }}{c}_{0\alpha a\sigma }^{{{\dagger}} }{c}_{j{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}{{\rangle }}\). The kinetic energy term in \({{{{{\bf{k}}}}}}\)-space becomes

$${H}_{{KE}}({{{{{\bf{k}}}}}})=\mathop{\sum} \limits_{\alpha ,{\alpha }^{{\prime} },a,b,\sigma ,{\sigma }^{{\prime} }}{\varepsilon }_{{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}^{\alpha a\sigma }({{{{{\bf{k}}}}}}){c}_{{{{{{\bf{k}}}}}}\alpha a\sigma }^{{{\dagger}} }{c}_{k{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}$$
(4)

where \({\varepsilon }_{{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}^{\alpha a\sigma }({{{{{\bf{k}}}}}})={\sum }_{j}{t}_{0{\alpha }^{{\prime} }b{\sigma }^{{\prime} }}^{j\alpha a\sigma }{e}^{\iota {{{{{\bf{k}}}}}}\cdot {{{{{{\bf{r}}}}}}}_{j}}\). The total Hartree–Fock Hamiltonian is then written as

$${H}^{{HF}}=\mathop{\sum} \limits_{{{{{{\bf{k}}}}}}}{H}_{{KE}}({{{{{\bf{k}}}}}})+{H}_{{int}}^{{HF}}({{{{{\bf{k}}}}}})-1/2{{\langle }}{H}_{{int}}^{{HF}}({{{{{\bf{k}}}}}}){{\rangle }}.$$
(5)

The main advantages of performing Hartree–Fock in this manner, for a given \({l}_{x(y)} < {L}_{x(y)}\), is two folded: (i) as \({H}^{{HF}}\) is block-diagonal in \({{{{{\bf{k}}}}}}\)-space only the diagonalizations of smaller block matrices corresponding to different \({{{{{\bf{k}}}}}}\)’s is needed, rendering the calculation faster; (ii) unrestricted Hartree-Fock calculations starting from random values tend to converge to inhomogenous states with higher energy than the actual mean-field ground states found by our methodology. Here, by performing calculations for various clusters \({l}_{x}\times {l}_{y}\) and comparing energies, increases our confidence of convergence towards the true Hartree-Fock ground state. Obviously for \({l}_{x(y)}={L}_{x(y)}\) the above technique becomes the canonical real-space unrestricted Hartree–Fock, which is also performed in the present work.

Convergence methodology

The Hartree–Fock ground state calculations were performed for system sizes up to 12 × 12. Self-consistent solutions with a maximum error of \(1{0}^{-5}\) were achieved starting from ten different random initial configurations for the order parameters (while assuming the solution fits in 4 × 4, 6 × 6, and 12 × 12 emergent magnetic unit cells), which makes for a total of 30 runs for each parameter point. The converged solution with the lowest energy was used as the ground state. The number of different order parameters for the \({l}_{x}={l}_{y}=12\) case is nearly 1.6 × 105, giving an idea of how challenging this effort becomes when searching for unbiased results within real-space Hartree Fock. To accelerate the convergence we used the Anderson mixing approach66.