INTRODUCTION

After the discovery of Mott insulating states [1, 2] and superconductivity [2, 3] in twisted bilayer graphene (tBLG), this material is intensively studied. In tBLG one graphene layer is rotated with respect to another one by a twist angle \(\theta \). As a result, the system has a superstructure for certain commensurate twist angles [4]. The low-energy electronic properties substantially depend on θ. At the so-called first magic angle \({{\theta }_{c}} \cong 1^\circ \) the energy spectrum is characterized by four (eight if spin is included) almost flat bands close to the Fermi level. This flatness makes the system very susceptible to electron–electron interactions. Interactions lead to the spontaneous violation of certain symmetries. The nature of non-superconducting many-body states in tBLG is not yet definitively known. Different candidates were proposed for the observed many-body insulating states in the tBLG [515]. The spin density wave (SDW) is among them [7, 8, 10, 11].

In [10, 11], we assumed the planar SDW order to be the ground state of the system in the doping range \( - 4 < x < 4\) electrons per supercell. This choice was not arbitrary. It is known that in the tBLG at small twist angle the electrons at the Fermi level are located in the AA regions of the superlattice cell [4, 16]. The ground state of the AA stacked bilayer graphene should be antiferromagnetic (AFM) [17, 18]. Based on the assumption of the SDW order, our simulations reproduced qualitatively well the dependence of the conductivity on the doping observed experimentally. We also showed [11] that doping the system away from the charge neutrality point leads to the formation of the nematic state, which is also detected by the STM [1922].

In the present paper we extend the variety of possible order parameters, considering the non-coplanar SDW order on a backdrop of the inhomogeneous charge distribution. We show that the spin texture changes from collinear to almost coplanar with doping. It is demonstrated that the symmetry of the SDW order is reduced with doping. This leads to the formation of the nematic spin state. At the same time, the rotational symmetry of the charge density is almost unaffected by doping. The nematic state can manifests itself in the non-symmetric spatial distribution of the local density of states, which was observed experimentally [1922]. Finally, we show that in energy units the local spin density is much stronger than the charge density variation unless the doping exceeds 3 extra electrons or holes per supercell.

MODEL HAMILTONIAN

The periodic superstructure exists in the tBLG when \(\theta \) satisfies the condition

$$\cos \theta = (3m_{0}^{2} + 3{{m}_{0}}r + {{r}^{2}}{\text{/}}2){\text{/}}(3m_{0}^{2} + 3{{m}_{0}}r + {{r}^{2}}),$$
(1)

with \({{m}_{0}}\) and r being mutually coprime positive integers [4]. The superlattice cell has a form of a right rhombus. When \(r = 1\) the superlattice cell coincides with the moiré cell. In our study we consider only such superstructures. When the twist angle is small, the supercell can be considered as consisting of regions with AA, AB, and BA stacking [4, 23].

We use the following model Hamiltonian of tBLG:

$$\begin{gathered} H = \sum\limits_{\begin{subarray}{l} {\mathbf{nm}}ij \\ \alpha \beta \sigma \end{subarray}} t({\mathbf{r}}_{{\mathbf{n}}}^{{i\alpha }};{\mathbf{r}}_{{\mathbf{m}}}^{{j\beta }})d_{{{\mathbf{n}}i\alpha \sigma }}^{\dag }{{d}_{{{\mathbf{m}}j\beta \sigma }}} + U\sum\limits_{{\mathbf{n}}i\alpha \sigma } {{n}_{{{\mathbf{n}}i\alpha \uparrow }}}{{n}_{{{\mathbf{n}}i\alpha \downarrow }}} \\ + \;\frac{1}{2}\mathop {\sum '}\limits_{\begin{subarray}{l} {\mathbf{nm}}ij \\ \alpha \beta \sigma \sigma ' \end{subarray}} V({\mathbf{r}}_{{\mathbf{n}}}^{{i\alpha }} - {\mathbf{r}}_{{\mathbf{m}}}^{{j\beta }}){{n}_{{{\mathbf{n}}i\alpha \sigma }}}{{n}_{{{\mathbf{m}}j\beta \sigma '}}}. \\ \end{gathered} $$
(2)

Here, \(d_{{{\mathbf{n}}i\alpha \sigma }}^{\dag }\) (\(d_{{{\mathbf{n}}i\alpha \sigma }}^{\quad }\)) are the creation (annihilation) operators of the electron with spin σ (\( = \uparrow \), \( \downarrow \)) at the unit cell n in the layer i (\( = 1,2\)) in the sublattice α (\( = \mathcal{A},\mathcal{B}\)), while \({{n}_{{{\mathbf{n}}i\alpha \sigma }}} = d_{{{\mathbf{n}}i\alpha \sigma }}^{\dag }d_{{{\mathbf{n}}i\alpha \sigma }}^{\quad }\). The first term in Eq. (2) is the single-particle tight-binding Hamiltonian with \(t({\mathbf{r}}_{{\mathbf{n}}}^{{i\alpha }};{\mathbf{r}}_{{\mathbf{m}}}^{{j\alpha }})\) being the amplitude of the electron hopping from site in the position \({\mathbf{r}}_{{\mathbf{m}}}^{{j\beta }}\) to the site \({\mathbf{r}}_{{\mathbf{n}}}^{{i\alpha }}\). The second term in Eq. (2) describes the on-site (Hubbard) interaction of electrons, while the last term corresponds to the inter-site Coulomb interaction (the prime means that the elements with \({\mathbf{r}}_{{\mathbf{n}}}^{{i\alpha }} = {\mathbf{r}}_{{\mathbf{m}}}^{{j\beta }}\) should be excluded).

Now we have to choose a parametrization of the hopping amplitudes. We keep only nearest-neighbor terms for the intralayer hopping with \(t = - 2.57\) eV. The inter-plane hopping amplitudes are parameterized by the following Slater–Koster formula for \({{p}_{z}}\) electrons

$$t({\mathbf{r}};{\mathbf{r}}') = \frac{{{{{\left[ {({\mathbf{r}} - {\mathbf{r}}'){{{\mathbf{e}}}_{z}}} \right]}}^{2}}}}{{{\text{|}}{\mathbf{r}} - {\mathbf{r}}'{{{\text{|}}}^{2}}}}{{V}_{\sigma }}({\mathbf{r}} - {\mathbf{r}}'),$$
(3)

where \({{{\mathbf{e}}}_{z}}\) is the unit vector transverse to the layers,

$${{V}_{\sigma }}({\mathbf{r}}) = {{t}_{0}}{{e}^{{ - (|{\mathbf{r}}| - d)/{{r}_{0}}}}}{{F}_{c}}({\text{|}}{\mathbf{r}}{\text{|}}),\quad {{F}_{c}}(r) = \frac{1}{{1 + {{e}^{{(r - {{r}_{c}})/{{l}_{c}}}}}}},$$
(4)

and the cutoff function \({{F}_{c}}(r)\) is introduced to nullify the hopping amplitudes at distances larger than \({{r}_{c}}\). We use \({{r}_{c}} = 4.92\) Å, \({{l}_{c}} = 0.2\) Å. The parameter \({{t}_{0}}\) defines the largest interlayer hopping amplitude. We choose \({{t}_{0}} = 0.37\) eV (this value was used to describe the AB bilayer graphene [4]). The parameter \({{r}_{0}}\) describes how fast the hopping amplitudes decay inside the region \(r < {{r}_{c}}\). We choose \({{r}_{0}} = 0.34\) Å.

At the first magic angle \(\theta = {{\theta }_{c}}\) the low-energy band structure consists of 4 almost flat almost degenerate bands separated by energy gaps from lower and higher dispersive bands. The bandwidth \(W\) of these flat bands has a minimum at \(\theta = {{\theta }_{c}}\). For model parameters used here we numerically determine that \(W = \) 1.8 meV, the gaps between flat and dispersive bands are ≈2.5 meV, and \({{\theta }_{c}} = 1.08^\circ \) (which is close to the experimentally observed and corresponds to the superstructure with \({{m}_{0}} = 30\) and \(r = 1\)).

SPIN AND CHARGE DENSITIES

When the system has flat bands crossing the Fermi level, the interactions become very important. The interactions break the symmetry of the single particle Hamiltonian inducing some order parameter. We start with the SDW-like ordering, with multicomponent order parameter. First, we introduce on-site order parameters

$$\begin{gathered} {{\Delta }_{{{\mathbf{n}}i\alpha }}} = U\langle d_{{{\mathbf{n}}i\alpha \uparrow }}^{\dag }d_{{{\mathbf{n}}i\alpha \downarrow }}^{{}}\rangle , \\ \Delta _{{{\mathbf{n}}i\alpha }}^{z} = U\left[ {\langle d_{{{\mathbf{n}}i\alpha \uparrow }}^{\dag }d_{{{\mathbf{n}}i\alpha \uparrow }}^{{}}\rangle - \langle d_{{{\mathbf{n}}i\alpha \downarrow }}^{\dag }d_{{{\mathbf{n}}i\alpha \downarrow }}^{{}}\rangle } \right]{\text{/}}2. \\ \end{gathered} $$
(5)

The quantity \({{\Delta }_{{{\mathbf{n}}i\alpha }}}\) is complex, while \(\Delta _{{{\mathbf{n}}i\alpha }}^{z}\) is real. They describe on-site magnetization

$${{{\mathbf{S}}}_{{{\mathbf{n}}i\alpha }}} = \left[ {\operatorname{Re} ({{\Delta }_{{{\mathbf{n}}i\alpha }}}),{\kern 1pt} \;\operatorname{Im} ({{\Delta }_{{{\mathbf{n}}i\alpha }}}),\;\Delta _{{{\mathbf{n}}i\alpha }}^{z}} \right]{\text{/}}U.$$
(6)

Parameters \({{\Delta }_{{{\mathbf{n}}i\alpha }}}\) and \(\Delta _{{{\mathbf{n}}i\alpha }}^{z}\) are controlled by the Hubbard interaction. We take \(U = 2t\). This value is somewhat smaller than the critical value for a single-layer graphene transition into a mean-field AFM state [24], \({{U}_{c}} = 2.23t\). Thus, our Hubbard interaction is rather strong, but not too strong to open a gap in single layer graphene.

In a graphene layer, each atom in one sublattice has three nearest neighbors belonging to another sublattice. For this reason, we consider three types of in-plane nearest-neighbor order parameters of the SDW type, \(A_{{{\mathbf{n}}i\sigma }}^{{(\ell )}}\) and \(A_{{{\mathbf{n}}i}}^{{z(\ell )}}\) (\(\ell = 1,{\kern 1pt} 2,{\kern 1pt} 3\)), corresponding to three different links connecting the nearest-neighbor sites. These order parameters are defined as follows

$$\begin{gathered} A_{{{\mathbf{n}}i\sigma }}^{{(\ell )}} = {{V}_{{{\text{nn}}}}}\langle d_{{{\mathbf{n}} + {{{\mathbf{n}}}_{\ell }}i\mathcal{A}\sigma }}^{\dag }d_{{{\mathbf{n}}i\mathcal{B}\bar {\sigma }}}^{{}}\rangle , \\ A_{{{\mathbf{n}}i}}^{{z(\ell )}} = \frac{{{{V}_{{{\text{nn}}}}}}}{2}\left( {\langle d_{{{\mathbf{n}} + {{{\mathbf{n}}}_{\ell }}i\mathcal{A} \uparrow }}^{\dag }d_{{{\mathbf{n}}i\mathcal{B} \uparrow }}^{{}}\rangle - \langle d_{{{\mathbf{n}} + {{{\mathbf{n}}}_{\ell }}i\mathcal{A} \downarrow }}^{\dag }d_{{{\mathbf{n}}i\mathcal{B} \downarrow }}^{{}}\rangle } \right), \\ \end{gathered} $$
(7)

where \({{{\mathbf{n}}}_{1}} = (0,{\kern 1pt} 0)\), \({{{\mathbf{n}}}_{2}} = (1,{\kern 1pt} 0)\), \({{{\mathbf{n}}}_{3}} = (0,{\kern 1pt} 1)\), \(\bar {\sigma } = - \sigma \), and \({{V}_{{{\text{nn}}}}} = V({\text{|}}\boldsymbol{\delta} {\text{|}})\) is the in-plane nearest-neighbor Coulomb repulsion energy (δ is the vector connecting \(\mathcal{A}\) and \(\mathcal{B}\) sites in the graphene unit cell). We take \({{V}_{{{\text{nn}}}}}{\text{/}}U = 0.59\), in agreement with [25]. The order parameters (7) define spins on links connecting in-plane nearest neighbor sites according to

$$\begin{gathered} {\mathbf{S}}_{{{\mathbf{n}}i}}^{{(\ell )}} = \frac{1}{2}\sum\limits_{\sigma \sigma '} {\boldsymbol{\sigma }_{{\sigma \sigma '}}}\langle d_{{{\mathbf{n}} + {{{\mathbf{n}}}_{\ell }}i\mathcal{A}\sigma }}^{\dag }d_{{{\mathbf{n}}i\mathcal{B}\sigma '}}^{{}}\rangle + {\text{c}}.{\text{c}}{\text{.}} \\ = \frac{1}{{{{V}_{{{\text{nn}}}}}}}\left( {\frac{1}{2}[A_{{{\mathbf{n}}i \uparrow }}^{{(\ell )}} + A_{{{\mathbf{n}}i \downarrow }}^{{(\ell )}}],\frac{1}{2}\operatorname{Im} [A_{{{\mathbf{n}}i \uparrow }}^{{(\ell )}} - A_{{{\mathbf{n}}i \downarrow }}^{{(\ell )}}],A_{{{\mathbf{n}}i}}^{{z(\ell )}}} \right), \\ \end{gathered} $$
(8)

where \(\boldsymbol{\sigma } \) is a vector of the Pauli matrices. In contrast to [11], the spins \({{{\mathbf{S}}}_{{{\mathbf{n}}i\alpha }}}\) and \({\mathbf{S}}_{{{\mathbf{n}}i}}^{{(\ell )}}\) are allowed to have the z components. In [11] we also considered inter-layer nearest-neighbor SDW order parameters. The calculations showed, however, that these components are by order of magnitude smaller than \(A_{{{\mathbf{n}}i\sigma }}^{{(\ell )}}\) (which are smaller than \({{\Delta }_{{{\mathbf{n}}i\alpha }}}\)). In present paper we neglect such order parameters.

Besides the SDW order parameters, we consider here the charge-density-wave-like contributions. First, we take into account that the charges are not uniformly distributed inside the superlattice cell and introduce the quantity

$$\Delta _{{{\mathbf{n}}i\alpha }}^{c} = \frac{U}{2}\left[ {\sum\limits_\sigma \langle d_{{{\mathbf{n}}i\alpha \sigma }}^{\dag }d_{{{\mathbf{n}}i\alpha \sigma }}^{{}}\rangle - 1} \right].$$
(9)

This value can be considered as the on-site potential due to charge inhomogeneity. Note that \(\Delta _{{{\mathbf{n}}i\alpha }}^{c}\) is finite even in the absence of any symmetry breaking since the sites inside a supercell are non-identical. Thus, it cannot be considered as an order parameter. In our simulations \(\Delta _{{{\mathbf{n}}i\alpha }}^{c}\) is normalized according to

$$\frac{2}{{U{{\mathcal{N}}_{{sc}}}}}\sum\limits_{{\mathbf{n}}i\alpha } \Delta _{{{\mathbf{n}}i\alpha }}^{c} = x,$$
(10)

where \({{\mathcal{N}}_{{sc}}}\) is the number of supercells in the system, and x is the doping level, that is, the number of extra electrons or holes per one supercell. Besides \(\Delta _{{{\mathbf{n}}i\alpha }}^{c}\), we introduce the in-plane nearest-neighbor (inter-site) potentials

$$A_{{{\mathbf{n}}i}}^{{c(\ell )}} = \frac{{{{V}_{{{\text{nn}}}}}}}{2}\sum\limits_\sigma \langle d_{{{\mathbf{n}} + {{{\mathbf{n}}}_{\ell }}i\mathcal{A}\sigma }}^{\dag }d_{{{\mathbf{n}}i\mathcal{B}\sigma }}^{{}}\rangle .$$
(11)

For all these charge and spin distributions we assume the same periodicity as the supercell periodicity.

MEAN-FIELD ITERATION SCHEME

To calculate the order parameters and potentials, we use a mean-field approach. It is based on the decoupling of the quadric terms in the Hamiltonian (2):

$$n_{{{\mathbf{n}}i\alpha \sigma }}^{{}}n_{{{\mathbf{m}}j\beta \sigma '}}^{{}} \to n_{{{\mathbf{n}}i\alpha \sigma }}^{{}}\langle n_{{{\mathbf{m}}j\beta \sigma '}}^{{}}\rangle + n_{{{\mathbf{m}}j\beta \sigma '}}^{{}}\langle n_{{{\mathbf{n}}i\alpha \sigma }}^{{}}\rangle $$
$$\begin{gathered} - \;\langle n_{{{\mathbf{n}}i\alpha \sigma }}^{{}}\rangle \langle n_{{{\mathbf{m}}j\beta \sigma '}}^{{}}\rangle - d_{{{\mathbf{n}}i\alpha \sigma }}^{\dag }d_{{{\mathbf{m}}j\beta \sigma '}}^{{}}\langle d_{{{\mathbf{m}}j\beta \sigma '}}^{\dag }d_{{{\mathbf{n}}i\alpha \sigma }}^{{}}\rangle \\ - \;d_{{{\mathbf{m}}j\beta \sigma '}}^{\dag }d_{{{\mathbf{n}}i\alpha \sigma }}^{{}}\langle d_{{{\mathbf{n}}i\alpha \sigma }}^{\dag }d_{{{\mathbf{m}}j\beta \sigma '}}^{{}}\rangle \\ \end{gathered} $$
(12)
$$ + \;\langle d_{{{\mathbf{n}}i\alpha \sigma }}^{\dag }d_{{{\mathbf{m}}j\beta \sigma '}}^{{}}\rangle \langle d_{{{\mathbf{m}}j\beta \sigma '}}^{\dag }d_{{{\mathbf{n}}i\alpha \sigma }}^{{}}\rangle .$$

As a result, we obtain a mean-field Hamiltonian \({{H}^{{{\text{MF}}}}}\). Numerical algorithm for finding the order parameters and the potentials is similar to that used in [11]. First, we write down the electronic operators in the momentum representation [26]

$$d_{{{\mathbf{pG}}i\alpha \sigma }}\frac{1}{{\sqrt {\mathcal{N}} }}\sum\limits_{\mathbf{n}} \exp [ - i({\mathbf{p}} + {\mathbf{G}}){\mathbf{r}}_{{\mathbf{n}}}^{i}]{{d}_{{{\mathbf{n}}i\alpha \sigma }}},$$
(13)

where \(\mathcal{N}\) is the number of the graphene unit cells in one layer of the sample, \({\mathbf{r}}_{{\mathbf{n}}}^{i}\) is the position of the nth unit cell of the ith layer, the momentum p lies in the first Brillouin zone of the superlattice, and G are the reciprocal vectors of the superlattice lying in the first Brillouin zone. The number of vectors G is equal to \({{N}_{{{\text{sc}}}}} = \mathcal{N}{\text{/}}{{\mathcal{N}}_{{{\text{sc}}}}}\) for each graphene layer.

The mean-field Hamiltonian \({{\hat {H}}_{{\mathbf{p}}}}\) is \({{N}_{R}} \times {{N}_{R}}\) matrix, where \({{N}_{R}} = 8{{N}_{{{\text{sc}}}}}\) (factor 8 is due to the spin, layer, and sublattice indices). The rank of this matrix is too large to perform numerical integration over the quasimomentum p for a realistic time. Indeed, for the first magic angle \({{\theta }_{c}} = 1.08^\circ \) we have \({{N}_{R}} = \) 22 328. For this reason, we use simplifications. The main contribution to the order parameters comes from the low-energy states. Consequently, the contributions from other states can be approximated. In the limit of uncoupled (\({{t}_{0}} = 0\)) graphene layers and zero order parameters, the matrix \({{\hat {H}}_{{\mathbf{p}}}}\) is block-diagonal with the 2 × 2 matrices on its diagonal

$$ - t\left( {\begin{array}{*{20}{c}} 0&{f_{{{\mathbf{p}} + {\mathbf{G}}}}^{i}} \\ {f_{{{\mathbf{p}} + {\mathbf{G}}}}^{{i*}}}&0 \end{array}} \right),$$
(14)

where \(f_{{\mathbf{p}}}^{i} = 1 + {{e}^{{ - i{\mathbf{pa}}_{1}^{i}}}} + {{e}^{{ - i{\mathbf{pa}}_{2}^{i}}}}\), \({\mathbf{a}}_{{1,2}}^{i}\) are the unit vectors of the ith layer. The eigenenergies of such a matrix are \( \pm t{\kern 1pt} {\text{|}}{\kern 1pt} f_{{{\mathbf{p}} + {\mathbf{G}}}}^{i}{\text{|}}\). The interlayer hopping amplitudes and order parameters are much smaller than t. As long as we are interested in low-energy features, we can use the truncated matrix \(\hat {H}{}_{{\mathbf{p}}}^{'}\) excluding the rows and columns in \({{\hat {H}}_{{\mathbf{p}}}}\) containing elements with \(t{\text{|}}f_{{{\mathbf{G}} + {\mathbf{p}}}}^{i}{\text{|}} > {{E}_{R}}\), where \({{E}_{R}}\) is the cutoff energy. The rank of the truncated matrix is \(N_{R}^{'} < {{N}_{R}}\). The eigenenergies \(E_{{\mathbf{p}}}^{{(S)}}\) of \(\hat {H}{}_{{\mathbf{p}}}^{'}\), which lie close to \( \pm {{E}_{R}}\), are calculated with significant errors. To fix this problem, we take into account only bands with \({\text{|}}E_{0}^{{(S)}}{\text{|}} < {{E}_{0}}\), where \({{E}_{0}} < {{E}_{R}}\). The number of such bands is \({{N}_{0}} < N_{R}^{'}\). We use \({{E}_{0}} = 0.2t\), \({{E}_{R}} = 0.4t\) (\({{N}_{0}} = 480\), \(N_{R}^{'} = 720\)). Calculations with smaller and larger \({{E}_{R}}\) and \({{E}_{0}}\) show that the results are almost independent of these quantities.

The contribution to the total energy from the discarded states \(E_{{\mathbf{p}}}^{{(S)}} < - {{E}_{0}}\) must be accounted for separately. Since \({{E}_{0}}\) is much larger than order parameters, this can be done perturbatively. The leading corrections to the total energy are quadratic in SDW order parameters \({{\Delta }_{{{\mathbf{n}}i\alpha }}}\), \(\Delta _{{{\mathbf{n}}i\alpha }}^{z}\), \(A_{{{\mathbf{n}}i\sigma }}^{{(\ell )}}\), and \(A_{{{\mathbf{n}}i}}^{{z(\ell )}}\). We assume that the proportionality coefficients are identical for all order parameters and are equal to

$$ - \frac{1}{{{{V}_{c}}({{E}_{0}})}} = - \frac{1}{2}\int\limits_{{{E}_{0}}}^{3t} {dE} \frac{{{{\rho }_{0}}(E)}}{E},$$
(15)

where \({{\rho }_{0}}(E)\) is the single-layer graphene density of states. Such a correction can be taken into account by the replacement in the mean-field Hamiltonian

$$\begin{gathered} {{H}^{{{\text{MF}}}}} \to H{\kern 1pt} {{'}^{{{\text{MF}}}}} - \sum\limits_{{\mathbf{n}}i\alpha } \frac{{{\text{|}}{{\Delta }_{{{\mathbf{n}}i\alpha }}}{{{\text{|}}}^{2}}}}{{{{V}_{c}}({{E}_{0}})}} - \sum\limits_{{\mathbf{n}}i\alpha } \frac{{{\text{|}}\Delta _{{{\mathbf{n}}i\alpha }}^{z}{{{\text{|}}}^{2}}}}{{{{V}_{c}}({{E}_{0}})}} \\ - \;\sum\limits_{{\mathbf{n}}i\ell \sigma } \frac{{{\text{|}}A_{{{\mathbf{n}}i\sigma }}^{{(\ell )}}{{{\text{|}}}^{2}}}}{{{{V}_{c}}({{E}_{0}})}} - \sum\limits_{{\mathbf{n}}iz\ell \sigma } \frac{{{\text{|}}A_{{{\mathbf{n}}i}}^{{z(\ell )}}{{{\text{|}}}^{2}}}}{{{{V}_{c}}({{E}_{0}})}}, \\ \end{gathered} $$
(16)

where \(H{\kern 1pt} {{'}^{{{\text{MF}}}}}\) is the Hamiltonian in the truncated basis. The contributions to \({{\Delta }^{c}}\) and \({{A}^{{c(\ell )}}}\) from the bands with \(E_{{\mathbf{p}}}^{{(S)}} < - {{E}_{0}}\) are almost independent of the interactions and ordering. In the limit of uncoupled layers and zero order parameters, the contribution to \(\Delta _{{{\mathbf{n}}i\alpha }}^{c}\) from each excluded band equals \(U{\text{/}}(4{{N}_{{sc}}})\). As a result, we obtain the self-consistency condition for \(\Delta _{{{\mathbf{n}}i\alpha }}^{c}\)

$$\begin{gathered} \frac{2}{U}\Delta _{{{\mathbf{n}}i\alpha }}^{c} = \frac{1}{{{{N}_{{sc}}}}}\mathop {\sum '}\limits_S \sum\limits_{{\mathbf{GG}}'\sigma } \int \frac{{{{d}^{2}}{\mathbf{p}}}}{{{{v}_{{RBZ}}}}}\Phi _{{{\mathbf{pG}}i\alpha \sigma }}^{{(S)*}}\Phi _{{{\mathbf{pG}}'i\alpha \sigma }}^{{(S)}} \\ \times \;{{e}^{{ - i({\mathbf{G}} - {\mathbf{G}}'){\mathbf{r}}_{{\mathbf{n}}}^{i}}}}\Theta \left( {\mu - E_{{\mathbf{p}}}^{{(S)}}} \right) - {{N}_{0}}{\text{/}}8{{N}_{{sc}}}, \\ \end{gathered} $$
(17)

where \(\Phi _{{{\mathbf{pG}}i\alpha \sigma }}^{{(S)}}\) are eigenfunctions of \(\hat {H}{}_{{\mathbf{p}}}^{'}\), \(\Theta (x)\) is the step function, \({{v}_{{RBZ}}}\) is the area of the supercell’s Brillouin zone, and \(\mu \) is the chemical potential. The summation in Eq. (17) is performed over \({{N}_{0}}\) bands and truncated basis, and the equation for chemical potential is

$$\mathop {\sum '}\limits_S \int \frac{{{{d}^{2}}{\mathbf{p}}}}{{{{v}_{{RBZ}}}}}\Theta \left( {\mu - E_{{\mathbf{p}}}^{{(S)}}} \right) - \frac{{{{N}_{0}}}}{2} = x.$$
(18)

The contribution to \(A_{{{\mathbf{n}}i}}^{{c(\ell )}}\) from excluded bands is nonzero (moreover, one can check that it is nonzero even for single-layer graphene). In the limit of uncoupled layers this contribution is independent of n and i. Thus, one can write \(A_{{{\mathbf{n}}i}}^{{c(\ell )}} = \bar {A} + \delta A_{{{\mathbf{n}}i}}^{{c(\ell )}}\), where \(\bar {A}\) can be estimated as

$$\bar {A} = \frac{{{{V}_{{{\text{nn}}}}}}}{2}\int \frac{{{{d}^{2}}{\mathbf{k}}}}{{{{v}_{{BZ}}}}}{{e}^{{ - i\varphi _{{\mathbf{k}}}^{i}}}}\Theta \left( {t{\text{|}}f_{{\mathbf{k}}}^{i}{\text{|}} - {{E}_{0}}} \right),$$
(19)

where \(\varphi _{{\mathbf{k}}}^{i} = \arg (f_{{\mathbf{k}}}^{i})\), and \(\bar {A}\) is independent of i. For parameters chosen, we have \(\bar {A}{\text{/}}{{V}_{{{\text{nn}}}}} = 0.26\). The parameter \(\bar {A}\) renormalizes the in-plane nearest-neighbor hopping amplitude t. We assume that this renormalization is already absorbed into the value \(t = - 2.57\) eV, and thus can be ignored. As for \(\delta A_{{{\mathbf{n}}i}}^{{c(\ell )}}\), we add quadratic term to the effective Hamiltonian, similar to that in Eq. (16).

To calculate the order parameters and the potentials we minimize the total energy \(\mathcal{E}\) iteratively using the method of successive approximations. In each iteration, we calculate \(E_{{\mathbf{p}}}^{{(S)}}\) and \(\Phi _{{{\mathbf{pG}}i\alpha \sigma }}^{{(S)}}\) of the matrix \(\hat {H}{}_{{\mathbf{p}}}^{'}\), and the gradients \(\partial \mathcal{E}{\text{/}}\partial \lambda = \langle \partial {{H}^{{{\text{MF}}}}}{\text{/}}\partial \lambda \rangle \), where \(\lambda = {{\Delta }_{{{\mathbf{n}}i\alpha }}}\), \(\Delta _{{{\mathbf{n}}i\alpha }}^{z}\), \(A_{{{\mathbf{n}}i\sigma }}^{{(\ell )}}\), \(A_{{{\mathbf{n}}i}}^{{z(\ell )}}\), or \(\delta A_{{{\mathbf{n}}i}}^{{c(\ell )}}\). These gradients are used to calculate new values of the aforementioned quantities. New value of \(\Delta _{{{\mathbf{n}}i\alpha }}^{c}\) is found using Eq. (17) with μ from Eq. (18).

RESULTS

Our numerical calculations reveal that the SDW order exists in the system at any doping level. However, the spatial distributions of the SDW magnetization are qualitatively different for different \(x\).

Let us start from \(x = 0\). Figures 1a–1d show the spatial distributions of \({{\mathcal{D}}_{{{\mathbf{n}}i\alpha }}} = U{\text{|}}{{{\mathbf{S}}}_{{{\mathbf{n}}i\alpha }}}{\text{|}}\) and \(\mathcal{A}_{{{\mathbf{n}}i}}^{{(\ell )}} = \) \({{V}_{{{\text{nn}}}}}{\text{|}}{\mathbf{S}}_{{{\mathbf{n}}i}}^{{(\ell )}}{\text{|}}\) inside the superlattice cell for \(i = 1\). These quantities describe the absolute values of the on-site spins and spins on the links, correspondingly. We see that the order parameters are nonzero only in the AA region. At zero doping, the order parameters spatial distributions have rather symmetric form. The area of nonzero \({{\mathcal{D}}_{{{\mathbf{n}}1\alpha }}}\) has a shape of a “rounded” hexagon. This hexagon is invariant under rotation on 60° around the center of the AA region \({{{\mathbf{R}}}_{0}} = ({{{\mathbf{R}}}_{1}} + {{{\mathbf{R}}}_{2}}){\text{/}}3\), with \({{{\mathbf{R}}}_{{1,2}}}\) being the elementary vectors of the superlattice. Figures 1b–1d demonstrate that spin magnetization on the links remain finite within the areas shaped like a dumbbell. The rotation on 180° and also on 120° around \({{{\mathbf{R}}}_{0}}\) preserves \(\mathcal{A}_{{{\mathbf{n}}1}}^{{(\ell )}}\).

Fig. 1.
figure 1

(Color online) Spatial distribution of the order parameters (a, e) \({{\mathcal{D}}_{{{\mathbf{n}}1\alpha }}}\) and (b–d, f– h) \(\mathcal{A}_{{{\mathbf{n}}1}}^{{(\ell )}}\) (for definition, see the text) calculated at x = (a–d) 0 and (e–h) –2. Doping reduces the symmetry of the order parameters from \({{C}_{6}}\) down to \({{C}_{2}}\).

When \(x = 0\) all spins, \({{{\mathbf{S}}}_{{{\mathbf{n}}i\alpha }}}\) and \({\mathbf{S}}_{{{\mathbf{n}}i}}^{{(\ell )}}\), are collinear. For layer 1, all on-site spins \({{{\mathbf{S}}}_{{{\mathbf{n}}1A}}}\) of the sublattice \(\mathcal{A}\) point along x axis, while in the sublattice \(\mathcal{B}\) they have the opposite direction. The same is true for layer \(2\). Therefore, the on-site spin texture has the collinear AFM arrangement. The spins on the links \({\mathbf{S}}_{{{\mathbf{n}}i}}^{{(\ell )}}\) also form a kind of AFM structure: in one part of the dumbbell \({\mathbf{S}}_{{{\mathbf{n}}i}}^{{(\ell )}}\) directed along x axis, while in another part of the dumbbell they have the opposite direction.

The order parameters distributions change with doping. In Figs. 1e–1h we plot the spatial distribution of \({{\mathcal{D}}_{{{\mathbf{n}}1\alpha }}}\) and \(\mathcal{A}_{{{\mathbf{n}}1}}^{{(\ell )}}\) at hole doping \(x = - 2\). We see that now no hexagonal symmetry exists. The profile of \({{\mathcal{D}}_{{{\mathbf{n}}1\alpha }}}\) is stretched unidirectionally, and 120° rotation is no longer a symmetry of \(\mathcal{A}_{{{\mathbf{n}}1}}^{{(\ell )}}\). However, all these order parameters are symmetric under rotation on 180°. In doped systems the spin textures are no longer collinear. The on-site spins turn out to be coplanar. We believe that this is not an artifact since our numerical procedure accounts for non-coplanar textures. In our simulations the on-site spins \({{{\mathbf{S}}}_{{{\mathbf{n}}i\alpha }}}\) lie in the \(xz\) plane (see Fig. 2), they form a helical AFM structure. The spins on links, \({\mathbf{S}}_{{{\mathbf{n}}i}}^{{(\ell )}}\), form a non-coplanar structures. However, almost all of these spins lie in the \(xz\) plane, and only small fraction of them have y components. Figures 3a–3c show the directions of spins \({\mathbf{S}}_{{{\mathbf{n}}i}}^{{(\ell )}}\) projected to the \(xy\) plane, while Figs. 3d–3f show the directions of spins projected to the \(xz\) plane (with subsequent rotation to the \(xy\) plane). We see that almost all spins lie in the \(xz\) plane. The spins which violate the coplanarity lie along three lines passing through the center of the AA region. These lines are visible in Figs. 3a–3c.

Fig. 2.
figure 2

(Color online) On-site spins \({{{\mathbf{S}}}_{{{\mathbf{n}}1\alpha }}}\) in sublattices (a) \(\mathcal{A}\) and (b) \(\mathcal{B}\) at \(x = - 2\). Spins lie in the \(xz\) plane. For the visual representation, the spins are rotated from the \(xz\) plane to the \(xy\) plane. Only the AA region of the supercell is shown.

Fig. 3.
figure 3

(Color online) Textures of spins on the link, \({\mathbf{S}}_{{{\mathbf{n}}1}}^{{(\ell )}}\) calculated at \(x = - 2\). (a–c) \(xy\) projections of the spins for different link parameters \(\ell = 1,2,3\), and (d–f) the \(xz\) projections (after rotation to the \(xy\) plane). Only the AA region of the supercell is shown.

In Fig. 4 we plot the spatial distributions of the on-site potential \(\Delta _{{{\mathbf{n}}i\alpha }}^{c}\) calculated at zero doping (panels (a–d)) and at \(x = - 2\) (panels (e–h)). The distributions are shown for each layer and sublattice separately. At \(x = 0\) there is an excess of electrons in the AA region and lack of electrons in the AB (corners of the supercell) and BA (area centered at \({{{\mathbf{R}}}_{{BA}}} = 2({{{\mathbf{R}}}_{1}} + {{{\mathbf{R}}}_{2}}){\text{/}}3\)) regions of the superlattice cell. At \(x = - 2\), the area, where excess of electrons is observed, expands; extra electrons also appear in the AB (for \(\Delta _{{{\mathbf{n}}1\mathcal{B}}}^{c}\) and \(\Delta _{{{\mathbf{n}}2\mathcal{A}}}^{c}\)) and BA (for \(\Delta _{{{\mathbf{n}}1\mathcal{A}}}^{c}\) and \(\Delta _{{{\mathbf{n}}2\mathcal{B}}}^{c}\)) regions. At zero doping the charge distributions remain invariant under rotation on 120° around the point \({{{\mathbf{R}}}_{0}}\). The potentials \(\Delta _{{{\mathbf{n}}i\mathcal{A}}}^{c}\) and \(\Delta _{{{\mathbf{n}}i\mathcal{B}}}^{c}\) are transformed into each other under the rotation on 60° around the point \({{{\mathbf{R}}}_{0}}\). The same is true approximately at \(x = - 2\). Thus, the nematicity does not manifest itself through the charge distribution.

Fig. 4.
figure 4

(Color online) Spatial distribution of the on-site potential (a, e) \(\Delta _{{{\mathbf{n}}1\mathcal{A}}}^{c}\), (b, f) \(\Delta _{{{\mathbf{n}}1\mathcal{B}}}^{c}\), (c, g) \(\Delta _{{{\mathbf{n}}2\mathcal{A}}}^{c}\), and (d, h) \(\Delta _{{{\mathbf{n}}2\mathcal{B}}}^{c}\) calculated at x = (a–d) 0 and (e–h) –2.

Figure 5 shows the spatial distributions of the inter-site potentials \(\delta A_{{{\mathbf{n}}1}}^{{c(\ell )}}\), calculated at \(x = 0\) and \(x = - 2\). The profiles of \(\delta A_{{{\mathbf{n}}1}}^{{c(\ell )}}\) are approximately the same for \(x = 0\) and \(x = - 2\), and only absolute values of \(\delta A_{{{\mathbf{n}}1}}^{{c(\ell )}}\) change. These spatial distributions are approximately symmetric under rotation on 180° around \({{{\mathbf{R}}}_{0}}\). The inter-site potentials are transformed into each other under the rotation on 120° around the point \({{{\mathbf{R}}}_{0}}\).

Fig. 5.
figure 5

(Color online) Spatial distribution of the inter-site potentials \(\delta A_{{{\mathbf{n}}1}}^{{c(\ell )}}\) for three different \(\ell \), calculated at (a–c) x = 0 and (d‒f) x = –2.

Thus, the doping reduces the symmetry of the SDW order from \({{C}_{6}}\), which is the symmetry of the lattice, to \({{C}_{2}}\) indicating the appearance of the nematic state [11]. At the same time the symmetry of the charge-related quantities \({{\Delta }^{c}}\) and \({{A}^{{c(\ell )}}}\) is virtually unaffected by doping. These parameters are integrated quantities that are influenced by contributions of high-energy states, the latter being insensitive to the low-energy symmetry breaking. At the same time, the electron nematic state can manifest itself as a symmetry reduction of the local density of states, which can be detected in an STM experiment. This is confirmed by our simulations. Figure 6 shows the spatial distribution of the local density of states calculated close to half filling (at \(x = - 2\) the density of states is almost zero). The spatial profile is stretched indicating the appearance of the nematic state. Such a feature was observed in the experiments [1922].

Fig. 6.
figure 6

(Color online) Local density of states calculated close to half filling (\(x = - 1.75\)). The spatial profile is stretched indicating the appearance of the electron nematic state.

Finally, we compare the values of the spin and charge density variations. At zero doping, the on-site spin order parameter \({{\mathcal{D}}_{{{\mathbf{n}}i\alpha }}}\) reaches the maximum about 34 meV. At the same time, the on-site charge density variation \(\Delta _{{{\mathbf{n}}i\alpha }}^{c}\) is about 6 times smaller. The SDW order parameters decrease down to zero, when the doping changes from \(x = 0\) to \(x = \pm 4\). At the same time, the values of \({{\Delta }^{c}}\) do not change much with x. However, even at \(x = \pm 2\) the maximum value of \({{\mathcal{D}}_{{{\mathbf{n}}i\alpha }}}\) is about 3 times larger than that of \({\text{|}}\Delta _{{{\mathbf{n}}i\alpha }}^{c}{\text{|}}\), and only at \({\text{|}}x{\text{|}} \approx 3\), the on site SDW order parameter becomes comparable to \({\text{|}}\Delta _{{{\mathbf{n}}i\alpha }}^{c}{\text{|}}\).

CONCLUSIONS

We study numerically the coexisting spin and charge density waves in the magic-angle tBLG in the doping range \(x = \pm 4\) extra electrons per supercell using the mean-field approach. The single-electron spectrum of the material has eight almost flat almost degenerate bands. The electron–electron interaction breaks down the symmetries of the single-particle state forming a set of order parameters. We calculate self-consistently the charge distribution in the supercell and spin structure of the SDW order parameters. We found that the SDW order is stable in the whole doping range. The spin texture of the SDW order parameters depends crucially on x changing from collinear at \(x = 0\) to almost coplanar at finite doping. The ground state of the doped system has the nematic symmetry.