Vestigial nematic order and superconductivity in the doped topological insulator Cu _x Bi₂Se₃

Abstract

If the topological insulator Bi₂Se₃ is doped with electrons, superconductivity with T_c = 3–4 K emerges for a low density of carriers (n = 10²⁰ cm⁻³) and with a small ratio of the superconducting coherence length and Fermi wave length: ξ/λ _F  = 2…4. These values make fluctuations of the superconducting order parameter increasingly important, to the extend that the T_c-value is surprisingly large. Strong spin–orbit interaction led to the proposal of an odd-parity pairing state. This begs the question of the nature of the transition in an unconventional superconductor with strong pairing fluctuations. We show that for a multi-component order parameter, these fluctuations give rise to a nematic phase at T_nem > T_c. Below T_c several experiments demonstrated a rotational symmetry breaking where the Cooper pair wave function is locked to the lattice. Our theory shows that this rotational symmetry breaking, as vestige of the superconducting state, already occurs above T_c. The nematic phase is characterized by vanishing off-diagonal long range order, yet with anisotropic superconducting fluctuations. It can be identified through direction-dependent para-conductivity, lattice softening, and an enhanced Raman response in the E_g symmetry channel. In addition, nematic order partially avoids the usual fluctuation suppression of T_c.

Introduction

The electron-doped topological insulator Bi₂Se₃ has been reported to exhibit a low carrier density (n = 10²⁰ cm⁻³), together with a small ratio ξ/λ _F  = 2…4 of the Fermi wave length and coherence length.^1,2,3 Recently, NMR Knight-shift measurements⁴ and measurements of the angular-dependent specific heat in the magnetic field⁵ revealed spontaneous symmetry breaking of the superconducting state in addition to the global U(1)-symmetry. The three-fold symmetry of the underlying lattice is broken. Similar nematic superconductivity was observed in Sr _x Bi₂Se₃^6,7 and in the closely related Nb-doped Bi₂Se₃.^8,9 Early on, Fu and Berg made the proposal that Cu _x Bi₂Se₃ may have an odd-parity two-component superconducting order parameter.¹⁰ Rotational symmetry breaking below T_c is then a possible consequence of this pairing state which is, by symmetry, in the E_u representation of the point group D_3d, see refs.^10,11,12,13. This is the representation that transforms like the in-plane coordinates x = (x, y).

In this paper, we show that superconducting fluctuations induce a phase transition to a nematic state. We find that these fluctuations either give rise to a nematic phase transition at T_nem > T_c or drive the superconducting transition weakly first order. Our quantitative analysis prefers the former scenario, where nematicity is a vestigial precursor phase of superconductivity. This is due to the pronounced two-dimensional electronic structure seen in ARPES measurements¹⁴ that is induced by Cu-intercalation. In distinction to the usual expectation where fluctuations suppress T_c, we find that nematic order largely off-sets this suppression, i.e., strengthens pairing compared to the case without nematic order. In the nematic state, the overall superconducting phase averages out to zero, yet the relative orientation of the two components of the Cooper pair field condenses in a long ranged ordered state with broken Z₃ or three-states Potts model symmetry at T_nem. Superconductivity sets in at a temperature slightly below T_nem. The resulting phase diagram for doped Bi₂Se₃ is sketched in Fig. 1. Because of the locking of the Cooper pair wave function to the lattice, the elastic constant $c_{E_{\mathrm{g}}}$ together with the sound velocity along certain high-symmetry directions are reduced at the upper temperature T_nem. As the nematic transition of a clean system turns out to be weakly first order, the elastic constant will however not completely vanish. Weak disorder changes the transition to become second order giving rise to a vanishing elastic constant. The nematic state above T_c can also be identified through anisotropic paraconductvity T_c < T < T_nem. We determine this anisotropy from the fluctuation spectrum of the Cooper pair field. The nematic order discussed here has several parallels to spin-induced Ising nematic order above a striped magnetic state of the iron-based superconductors^15,16,17,18 or to time-reversal symmetry breaking proposed for chiral superconductors in the context of SrRuO₄,¹⁹, revealing the universality of the underlying principle of composite or intertwined order.²⁰

Fig. 1

Schematic phase diagram of Cu _x Bi₂Se₃ with data points (blue bullets) taken from the ref. ³ We predict a purely nematic phase above the superconducting phase (indicated in red), where superconducting fluctuations create an ordered state that breaks the threefold rotational symmetry. Following ref.¹⁵, we expect for low temperatures tricritical points (red bullets) below which the transitions should be joint first order

Before we discuss the details of our analysis we summarize the key idea of this paper. The low energy Hamiltonian that describes the superconducting state of doped Bi₂Se₃ in the band basis is of the form

$$H = \mathop {\sum}\limits_{{\bf{k}}s} {\kern 1pt} \varepsilon _{\bf{k}}\psi _{{\bf{k}},s}^\dagger \psi _{{\bf{k}},s} + \mathop {\sum}\limits_{\bf{k}} \left[ {\psi _{\bf{k}}^\dagger \left( {{\bf{d}}_{\bf{k}} \cdot \widetilde {\boldsymbol{\sigma }}} \right)i\tilde \sigma ^y\psi _{ - {\bf{k}}}^\dagger + h.c.} \right],$$

(1)

with the pseudo spin index s = {1, 2} and $\psi _{\bf{k}}^\dagger$ = $\left( {\psi _{{\bf{k}},1}^\dagger ,\psi _{{\bf{k}},2}^\dagger } \right)$ denoting fermionic creation operators in the conduction band (cf. Methods section for details). The pairing is given by the d _k -vector as

$${\bf{d}}_{\bf{k}} = {\mathrm{\Delta }}^{\left( x \right)}\left( {k_z{\bf{e}}_x - \frac{{v_0}}{{v_z}}k_x{\bf{e}}_z} \right) + {\mathrm{\Delta }}^{\left( y \right)}\left( {k_z{\bf{e}}_y - \frac{{v_0}}{{v_z}}k_y{\bf{e}}_z} \right),$$

(2)

where Δ = (Δ^(x), Δ^(y))^T form a two-component order parameter in the E_u representation. The broken rotation symmetry below T_c naturally implies that this is the appropriate pairing state. Within the D_3d point group, the only alternative would be pairing in the even-parity state E_g. This corresponds to $\left( {d_{x^2 - y^2} + d_{yz},d_{xy} + d_{xz}} \right)$ superconductivity. Most of our analysis would proceed without changes if this were the case. Nematic order can generally be characterized in terms of a symmetric trace-less second rank tensor. The nematic tensor in our problem is

$$\widehat{q}_{\alpha \beta } = - \mathop {\sum}\limits_{\mu ,\nu } {\kern 1pt} \widehat {\boldsymbol{\tau }}_{\mu \nu } \cdot \widehat {\boldsymbol{\tau }}_{\alpha \beta }{\mathrm{\Delta }}^{\left( \mu \right) \ast }{\mathrm{\Delta }}^{\left( \nu \right)},$$

(3)

where we use the Pauli matrices $\widehat {\boldsymbol{\tau }}$ = $\left( {\hat \tau ^z,\hat \tau ^x} \right)$. Throughout this paper, we use hat symbols for 2 × 2 matrices and bold symbols for vectors. The expectation value $\left\langle {\Delta ^{\left( \mu \right)^\ast }\Delta ^{\left( \nu \right)}} \right\rangle _{}^{}$ measures Cooper pair correlations. While such an expectation value does not break an additional symmetry in the case of a single-component pairing state, we show below that $\left\langle {\widehat q_{\alpha \beta }} \right\rangle \ne 0$ breaks another symmetry. For example, $\left\langle {\widehat q_{xx}} \right\rangle \ne 0$ implies $\left\langle {{\mathrm{\Delta }}^{\left( x \right) \ast }{\mathrm{\Delta }}^{\left( x \right)}} \right\rangle$ ≠ $\left\langle {{\mathrm{\Delta }}^{\left( y \right) \ast }{\mathrm{\Delta }}^{\left( y \right)}} \right\rangle$, while in the high temperature phase both expectation values are equal. We show that the additional symmetry is separately broken at a distinct temperature. The emerging nematic phase is not superconducting but induced by superconducting fluctuations. It is a vestige of the superconducting phase. This is only possible because Δ is a two-component order parameter, i.e., the irreducible representation E_u has dimensionality two. To be precise, $\left\langle {\widehat q_{\alpha \beta }} \right\rangle$ transforms according to the representation E_g of the point group. Since E_u ⊗ E_u = A_1g ⊕ A_2g ⊕ E_g such a composite order, made up of a bilinear combination, is indeed allowed. Since E_u ⊗ E_u = E_g ⊗ E_g the analysis of nematic order does not change for even-parity multi-component superconductivity. The other non-trivial bilinear form is $q^y \equiv \mathop {\sum}\nolimits_{\mu \nu } {\kern 1pt} {\mathrm{\Delta }}^{\left( \mu \right) \ast }\hat \tau _{\mu \nu }^y{\mathrm{\Delta }}^{\left( \nu \right)}$, which transforms under A_2g and breaks time reversal symmetry. In what follows we focus on the nematic order parameter $\left\langle {\widehat q_{\alpha \beta }} \right\rangle$. It takes the general form

$$\left\langle {\widehat q_{\alpha \beta }} \right\rangle = - 2q_0\left( {n_\alpha n_\beta - \frac{1}{2}\delta _{\alpha \beta }} \right).$$

(4)

The amplitude q₀ sets in at the nematic transition temperature T_nem. The unit vector n = (cos θ, sin θ)^T is the director of the nematic state that determines the eventual orientation of the superconducting order parameter

$${\bf{\Delta }} = {\mathrm{\Delta }}_0\left( {{\mathrm{cos}}{\kern 1pt} \theta ,\,{\mathrm{sin}}{\kern 1pt} \theta } \right)^T.$$

(5)

Thus, the superconducting Cooper pair field acts as nematogen that enables a rotational symmetry breaking, even without superconducting long-range order. Finally, the lattice symmetry of Cu _x Bi₂Se₃ allows for three distinct values of the angle θ = $\left\{ {0,\frac{\pi }{3},\frac{{2\pi }}{3}} \right\}$ (cf. Fig. 3). The statistical mechanics of the nematic state then corresponds to a three state Potts model.

Results

Collective nematic fluctuations

Our starting point is the well established microscopic Hamiltonian (23) for Bi₂Se₃, yet, for the sake of clarity we refer to the Methods section for a discussion. It leads to the Ginzburg-Landau expansion valid in the vicinity of the superconducting phase transition. In terms of the two-component order parameter Δ = (Δ^(x), Δ^(y))^T the action reads:

$$\begin{array}{*{20}{l}} {S} \hfill & = \hfill & {r_0{\int}_x {\kern 1pt} {\bf{\Delta }}^\dagger {\bf{\Delta }} + {S}^{{\mathrm{grad}}}} \,{ + {\int}_x \left( {u\left( {{\bf{\Delta }}^\dagger {\bf{\Delta }}} \right)^2 + v\left( {{\bf{\Delta }}^\dagger \hat \tau _y{\bf{\Delta }}} \right)^2} \right),} \hfill \end{array}$$

(6)

with $r_0 = \frac{1}{g} - \rho _F$ $\log \frac{{\omega _0}}{T}$, where g and ω₀ are the strength and characteristic energy of the pairing interaction, and ρ _F the density of states at the Fermi level, respectively. The gradient term is in momentum space given as

$${\cal S}^{{\mathrm{grad}}} = {\int}_p {\kern 1pt} {\bf{\Delta }}_{\bf{p}}^\dagger \left( {m_0\left( {\bf{p}} \right)\hat \tau _0 + {\bf{m}}\left( {\mathbf{p}} \right) \cdot \widehat {\boldsymbol{\tau }}} \right){\bf{\Delta }}_{\mathbf{p}}$$

(7)

with $m_0\left( {\bf{p}} \right)$ = $d_\parallel \left( {p_x^2 + p_y^2} \right)$ + $d_zp_z^2$, $m_1\left( {\bf{p}} \right)$ = $d^{\prime}\left( {p_x^2 - p_y^2} \right)$ +$\bar d{\kern 1pt} p_yp_z$, and $m_2\left( {\bf{p}} \right)$ = $2d^{\prime}p_xp_y$ + $\bar d{\kern 1pt} p_xp_z$, and characterized by four parameters.¹³ We use the shorthand notation ${\int}_p \cdots$ = $T{\int} \frac{{d^3{\bf{p}}}}{{\left( {2\pi } \right)^3}} \cdots$. From the microscopic Hamiltonian one can determine the coefficients of the Ginzburg-Landau expansion at weak coupling λ = gρ _F  ≪ 1. This analysis yields u > 0 and v > 0, in full agreement with earlier calculations.¹² Crucial for our analysis is however not the applicability of this expansion, but only the signs of u and v. The key implication of the positive sign of v is that the superconducting order parameter is time reversal symmetric and can be written in the form of Eq. (5). If v < 0 we would find time-reversal symmetry breaking and an analyis analogous to ours leads to a vestigial order parameter $\left\langle {q^y} \right\rangle$. The regime v < 0 was predicted in the ref. ²¹ for thin layers of doped Bi₂Se₃. Thus, no matter what the sign of v, one always has an accompanying symmetry breaking. This is true for all crystalline symmetries that allow for multi-component Cooper pair fields. For a superconducting order parameter, transforming according to a higher dimensional irreducible representation Γ, the product representation Γ^* ⊗ Γ contains non-trivial irreducible represenations. For all point groups relevant to periodic systems these give rise to vestigial order parameters. Thus, there is no direct second-order superconducting phase transition for a multi-component Cooper pair field. Either there is a vestigial phase above T_c, or the transition is of first order due to the coupling to the vestigial order parameter.

For an analysis of fluctuation effects and the description of the nematic ordering, it is efficient to express the interaction in terms of the quadrupolar tensor $\widehat q_{\alpha \beta }$ of Eq. (3):

$${\cal S}^{\left( 4 \right)} = \frac{{u^{\prime}}}{2}{\int}_x {\kern 1pt} {\mathrm{tr}}\left( {\widehat r\widehat r} \right) - \frac{v}{2}{\int}_x {\kern 1pt} {\mathrm{tr}}\left( {\widehat q\widehat q} \right),$$

(8)

where we introduced u′ = u + v and $\widehat r$ = $\left( {{\bf{\Delta }}^\dagger {\bf{\Delta }}} \right)\widehat \tau _0$. We decouple the two terms in ${\cal S}^{\left( 4 \right)}$ via Hubbard-Stratonovich transformations, e.g., ${\int} {D\widehat Q{\kern 1pt} e^{ - \frac{1}{{8v}}{\mathrm{tr}}\left( {\widehat Q\widehat Q} \right) - \frac{1}{2}{\mathrm{tr}}\left( {\widehat Q\widehat q} \right)}}$ ∝ $e^{\frac{v}{2}{\mathrm{tr}}\left( {\widehat q\widehat q} \right)}$, and obtain

$${\cal S} = \frac{1}{4}{\int}_x \left( {\frac{1}{v}{\bf{Q}} \cdot {\bf{Q}} - \frac{1}{{u^{\prime}}}R^2} \right) + {\int}_p {\bf{\Delta }}_{\bf{p}}^\dagger {\kern 1pt} \chi _{\bf{p}}^{ - 1}{\bf{\Delta }}_{\bf{p}},$$

(9)

with the pairing susceptibility

$$\widehat \chi _{\bf{p}}^{ - 1} =\left( {r_0 + R + m_0\left( {\bf{p}} \right)} \right)\widehat \tau _0 + \left( {{\bf{Q}} + {\bf{m}}\left( {\bf{p}} \right)} \right) \cdot \widehat {\boldsymbol{\tau }}.$$

(10)

Here, we have expanded the matrices $\widehat R = R{\kern 1pt} \widehat \tau _0$ and $\widehat Q = {\bf{Q}} \cdot \widehat {\boldsymbol{\tau }}$ in terms of the Pauli matrices with Q = (Q₁, Q₂)^T. Next, the superconducting order parameter fluctuations are integrated out in both regimes, T < T_c and T > T_c. To this end, we include Gaussian fluctuations of the pairing field, treated formally within a large-N expansion of the vector field Δ. We also allow for superconducting symmetry breaking with the condensed pairing field ${\bf{\Delta }}_0 = \left( {{\mathrm{\Delta }}_0^{\left( x \right)},{\mathrm{\Delta }}_0^{\left( y \right)}} \right)$ where ${\mathrm{\Delta }}_0^{\left( {x,y} \right)} \in {\Bbb R}$:

$$\begin{array}{*{20}{l}} {\cal S} \hfill & = \hfill & {\frac{1}{4}{\int}_x \left( {\frac{1}{v}{\bf{Q}} \cdot {\bf{Q}} - \frac{1}{{u^{\prime}}}R^2} \right) + {\bf{\Delta }}_0^T\hat\chi _0^{ - 1}{\bf{\Delta }}_0} { + {\int}_p {\kern 1pt} {\mathrm{tr}}{\kern 1pt} {\mathrm{log}}{\kern 1pt}\hat\chi _{\bf{p}}^{ - 1}.} \hfill \end{array}$$

(11)

Using the saddle point approximation we finally obtain the five coupled equations of state

$$R = 2u^{\prime}{\bf{\Delta }}_0^2 + 2u^{\prime}{\int}_p {\kern 1pt} {\mathrm{tr}}\left( {\widehat\chi _{\bf{p}}\widehat \tau _0} \right)$$

(12)

$${\bf{Q}} = - 2v{\bf{\Delta }}_0^T\widehat {\boldsymbol{\tau }}{\bf{\Delta }}_0 - 2v{\int}_p {\kern 1pt} {\mathrm{tr}}\left( {\widehat\chi _{\bf{p}}\widehat {\boldsymbol{\tau }}} \right)$$

(13)

$$0 = 2\hat\chi _0^{ - 1}{\bf{\Delta }}_0.$$

(14)

The saddle point value of the collective variable $\widehat Q_{\alpha \beta }$ equals the desired order parameter $\left\langle {\widehat q_{\alpha \beta }} \right\rangle$. Vestigial nematicity is fluctuation induced. Without such superconducting fluctuations, Eq. (13) does not allow for a finite nematic order, q₀ ≠ 0, above T_c.

Transition temperatures

The result of the numerical solution of the coupled set of Eqs. 12, 13, and 14 are shown in Fig. 2. Here, we plot the order parameters as function of $\delta r$ = $r_0 - r_0^ \ast \propto \left( {T - T^ \ast } \right)$, where T^* denotes the transition temperature without nematic order present, i.e., $r_0^ \ast$ = $- 2u^{\prime}{\int}_p {\kern 1pt} {\mathrm{tr}}\left.{\left( {\widehat\chi _p\widehat \tau _0} \right)} \right|_{r = {\bf{Q}} = 0}$. We find that the nematic order parameter sets in above the superconducting transition temperature T _c . The superconducting transition is of second order and T_c can also be obtained from the divergence of the uniform pairing susceptibility $\chi _{{\mathrm{pair}}}^{ - 1}$ = $\left( {r^2 - {\bf{Q}}^2} \right){\mathrm{/}}r$ with r = r₀ + R, see Fig. 2. Note that χ_pair denotes the largest eigenvalue of the matrix $\widehat \chi _{\bf{p}}$ for p = 0. The nematic transition at T_nem is weakly first order. The origin of this behavior is the trigonal symmetry which allows for a cubic invariant. Up to fourth order terms, the action for the real order parameter Q reads:

$$\begin{array}{*{20}{l}} {{\cal S}_Q} \hfill & = \hfill & {\frac{{r_Q}}{2}{\int}_x \left( {Q_1^2 + Q_2^2} \right) + \frac{w}{3}{\int}_x {\kern 1pt} Q_1\left( {Q_1^2 - 3Q_2^2} \right)} { + \frac{{u_Q}}{4}{\int}_x \left( {Q_1^2 + Q_2^2} \right)^2.} \hfill \end{array}$$

(15)

Fig. 2

The calculated temperature dependence of the nematic (q₀) and the superconducting (Δ₀) order parameters. The temperature axis is given by $\delta r = r_0 - r_0^ \ast \propto T - T^ \ast$, where T* is the transition temperature without nematic order. The positive value of T_c demonstrates its enhancement due to nematicity as compared to the case with fluctuations but without nematic order. We find a first order nematic phase transition at T = T_nem, followed by a second order superconducting phase transition at T = T_c. We also depict the inverse uniform pairing susceptibility $\chi _{{\mathrm{pair}}}^{ - 1}$, which experiences a sudden drop at T_nem before it vanishes at T_c. Finally, we show the inverse nematic susceptibility $\chi _{{\mathrm{nem}}}^{ - 1}$ that reflects nematic fluctuations above T_nem. Plotted are the dimensionless quantities ${\mathrm{\Delta }}_0\sqrt {d_\parallel V_0^2{\mathrm{/}}\left( {T_cV\tilde v} \right)}$, $q_0{\mathrm{/}}\left( {2d_\parallel \tilde v} \right)$, $\chi _{{\mathrm{pair}}}^{ - 1}{\mathrm{/}}\left( {d_\parallel \tilde v} \right)$, and $\delta r{\mathrm{/}}\left( {d_\parallel \tilde v} \right)$ with definitions and parameter values given in the Methods section

This is the well-known Landau expansion of a three-states Potts model.²² Expanding the coupled set of Eqs. 12, 13, and 14 for small Q _i yields exactly this term with $w \propto \bar d^2d^{\prime} > 0$. Thus, overall the first-order transition is expected to be weak, where it holds for the jump of the order parameter δq₀ ∝ w. Given the uncertainty in several parameters we cannot reliably predict T_nem − T_c in Kelvin. Our numerical analysis suggests however that it can be up to 10% of T_c. In Fig. 2 we see also the positive effect of nematic order on superconductivity. Fluctuations without nematic order suppress the transition temperature to $T^ \ast \ll T_c^0 \approx \omega _0e^{ - 1/\lambda }$ as the coupling constant is reduced λ⁻¹ → λ⁻¹ + R/ρ _F . With nematic order this effect is significantly weakened as now $\lambda ^{ - 1} \to \lambda ^{ - 1}$ + $\left( {R - \left| {\bf{Q}} \right|} \right){\mathrm{/}}\rho _F$. If we assume a more isotropic electronic structure we obtain instead two joint first order transitions, a trend that also occurs in other problems with vestigial precursor order.¹⁵ Photoemission experiments¹⁴ for Cu _x Bi₂Se₃ strongly support a very anisotropic Fermi surface, i.e., split transitions.

Degeneracy of superconducting and nematic ground states

Let us analyze the allowed orientation of the nematic director n, i.e., the allowed values for the angle θ. As shown in Fig. 3, the fluctuation induced term Eq. (15) picks three distinct values of the angle θ = $\left\{ {0,\frac{\pi }{3},\frac{{2\pi }}{3}} \right\}$, i.e., we have

$${\bf{Q}} = q_0\left\{ {\left( {\begin{array}{*{20}{c}} { - 1} \\ 0 \end{array}} \right),\left( {\begin{array}{*{20}{c}} {\frac{1}{2}} \\ {\frac{{\sqrt 3 }}{2}} \end{array}} \right),\left( {\begin{array}{*{20}{c}} {\frac{1}{2}} \\ { - \frac{{\sqrt 3 }}{2}} \end{array}} \right)} \right\}.$$

(16)

Fig. 3

Illustration of the correspondence between the nematic and the superconducting order parameters. The green triangles denote the threefold degenerate nematic ground state (16). Each nematic ground state corresponds to two superconducting ground states (blue hexagons) which differ by an overall phase of π from one another, indicated by the purple arrows. The three insets visualize the respective superconducting ground state and the entailed threefold rotational symmetry breaking. The d _z (x, y)-component of the order parameter (see Eq. (2)) is plotted color coded on the ground with the hexagonal atomic structure of the unit cell on top. The gap affects the electronic bonds differently, leading to the aforementioned symmetry breaking

The presence of a finite nematic order parameter predetermines which of the degenerate superconducting ground states of Eq. (5) will be realized. At T_c no additional rotational symmetry breaking takes place and only the global U(1) symmetry of the superconductor and parity are broken. The one-to-one correspondence between the superconducting and the nematic order parameters follows from Eq. (14), or equivalently, by determination of the angle θ in (4) and (5). We find that the first solution of (16) leads to the superconducting ground states with θ _n  = {0,π}, while the second and third solutions lead to θ _n  = $\left\{ {\frac{{2\pi }}{3},\frac{{5\pi }}{3}} \right\}$ and θ _n  = $\left\{ {\frac{\pi }{3},\frac{{4\pi }}{3}} \right\}$, respectively (see Fig. 3). To visualize the origin of the in-plane anisotropy in real space, the three insets of Fig. 3 show the components of the triplet vector d _z (x, y) (see Eq. (2)) of the respective superconducting ground states in real space. We also show the Bi and Se atoms in the respective layers of the crystalline unit cell to demonstrate how the bonds are affected by the anisotropic superconducting gap. For the remainder of the work, we choose, without restriction, the first of the three degenerate nematic solutions, where θ _n  = 0.

Experimental implications

In the following we study the experimental implication in the nematic phase above T_c and in the high temperature phase above T_nem. Above T_nem the onset of nematicity can be probed via renormalizations of the elastic moduli of the system. The elastic energy relevant for the transition is

$$\begin{array}{*{20}{l}} {{\cal S}_{{\mathrm{el}}}} \hfill & = \hfill & {\frac{1}{4}{\int}_x {\kern 1pt} c_{A_{1g}}\left( {\widehat \varepsilon _{xx} + \widehat \varepsilon _{yy}} \right)^2} { + \frac{1}{4}{\int}_x {\kern 1pt} c_{E_g}{\kern 1pt} \left[ {\left( {\widehat \varepsilon _{xx} - \widehat \varepsilon _{yy}} \right)^2 + 4\widehat \varepsilon _{xy}} \right],} \hfill \end{array}$$

(17)

where we focus on in-plane distortions. The symmetry-allowed coupling between the Cooper pair field and the elastic strain $\widehat \epsilon _{\alpha \beta }$ is

$${\cal S}_{{\mathrm{nem}} - {\mathrm{el}}} = - \kappa {\int}_x {\kern 1pt} {\mathrm{tr}}\left( {\widehat \varepsilon \widehat q} \right).$$

(18)

We can now add an external stress to the energy and determine the renormalized elastic constants. Alternatively, we can add a conjugate field to the nematic degrees of freedom and obtain the nematic susceptibility

$$\widehat \chi _{{\mathrm{nem,}}ij}\left( {\bf{p}} \right) = \left\langle {Q_i\left( {\bf{p}} \right)Q_j\left( { - {\bf{p}}} \right)} \right\rangle ,$$

(19)

where the Q _i are again the expansion parameters of the nematic tensor in the Pauli basis $\widehat {\boldsymbol{\tau }} = \left( {\hat \tau ^z,\hat \tau ^x} \right)$ that we have been using. As long as the lattice is purely harmonic we obtain the following relation between the renormalized elastic modulus $c_{E_{\mathrm{g}}}^ \ast$ and its bare value $c_{E_{\mathrm{g}}}$:

$$\left( {c_{E_{\mathrm{g}}}^ \ast } \right)^{ - 1} = \left( {c_{E_{\mathrm{g}}}} \right)^{ - 1} + \frac{{\kappa ^2}}{{2c_{E_{\mathrm{g}}}}}{\mathrm{tr}}\widehat \chi _{{\mathrm{nem}}},$$

(20)

where $\widehat \chi _{{\mathrm{nem}}}$ = $\widehat \chi _{{\mathrm{nem}}}\left( {{\bf{p}} \to 0} \right)$. A similar result for spin-induced nematicity was previously derived in the ref. ¹⁸ As T → T_nem from above the nematic susceptibility rises, leading to a suppression of elastic constants. Within the Gaussian fluctuation regime the nematic susceptibility can be obtained explicitly and is given by:

$$\widehat \chi _{{\mathrm{nem}}} = 2v\widehat \chi _{{\mathrm{nem}}}^{\left( {\mathrm{0}} \right)}\left( {1 - 2v\widehat \chi _{{\mathrm{nem}}}^{\left( {\mathrm{0}} \right)}} \right)^{ - 1},$$

(21)

where $\widehat \chi _{{\mathrm{nem}},ij}^{\left( 0 \right)}$ = ${\int}_p {\kern 1pt} {\mathrm{tr}}\left( {\widehat \tau _i\widehat \chi _{\bf{p}}\widehat \tau _j\widehat \chi _{\bf{p}}} \right)$. In Fig. 2 we also show the temperature dependence of $\widehat \chi _{{\mathrm{nem}},ii}^{ - 1}$ which displays a Curie-Weiss dependence. Since the transition is first order, $\widehat \chi _{{\mathrm{nem}}}$ will not diverge, and for the offset holds $\widehat \chi _{{\mathrm{nem}}}^{ - 1}$ $\left( {T = T_{{\mathrm{nem}}}} \right) \propto w^2$. However, as the first order transition is weak, the nematic susceptibility is significantly enhanced. Note that $\widehat \chi _{{\mathrm{nem}},ii}^{ - 1} \propto r_Q$ with r _Q occuring in (15) and $\widehat \chi _{{\mathrm{nem}},ij}^{ - 1}$ = 0 for i ≠ j and T > T_nem. To determine κ and the actual lattice softening one would need to know the change in lattice parameters deep in the superconducting state. $\widehat \chi _{{\mathrm{nem}}}$ is however directly observable via electronic Raman scattering^23,24 in the E_g-channel.

Next, we study observables in the nematic phase, i.e. for T_c < T < T_nem, where the threefold symmetry is broken. As the nematic state is fluctuation-induced, the most natural quantity to reflect this anisotropy is the paraconductivity of the system. Our calculation of the fluctuation contribution to the resistivity is a natural generalization of the classical works by Aslamasov and Larkin.^25,26 We obtain the conductivity:

(22)

with the lattice constants a _j , the velocity matrix $\widehat V_{j,{\bf{p}}} = \partial \widehat \chi _{\bf{p}}^{ - 1}{\mathrm{/}}\partial p_j$ and the matrix of the pairing field $\widehat \chi _{\bf{p}}$. The calculated temperature dependence of the resistivity $\widehat \rho _{\alpha \alpha }$ = $\left( {\widehat \sigma ^{ - 1}} \right)_{\alpha \alpha }$ is plotted in Fig. 4. As expected, we find an anisotropy between the two in-plane components. Moreover, the sudden drop at T_nem once again indicates the first order nature of the transition and evidences that fluctuation effects are more pronounced inside the nematic phase. For the chosen ground state, i.e., θ _n  = 0, the resistivity in y-direction $\widehat \rho _{yy}$ is larger than $\widehat \rho _{xx}$, since the fluctuating pairing amplitude along the x-direction is much larger than in the orthogonal direction, see Fig. 4.

Fig. 4

The calculated temperature behavior of the in-plane dc-resistivities, $\widehat \rho _{xx}$ (solid green) and $\widehat \rho _{yy}$ (dashed blue). The calculation only takes into account contributions from superconducting order parameter fluctuations. As expected, we find an anisotropy in the nematic phase where the sudden drop indicates that fluctuation contributions become increasingly important in the nematic phase. For the chosen ground state (depicted in the inset), the resistivity in y-direction is larger, i.e., $\widehat \rho _{yy} > \widehat \rho _{xx}$

Effect of disorder on the nematic phase

Apart from the usual pair-breaking effects, disorder has a profound impact on states with vestigial order (see also²⁷). A disorder configuration that locally changes a certain crystalline orientation will naturally nucleate a specific value of the nematic order parameter in its vicinity. Thus, ordinary potential scatters act as random-field disorder for the vestigial order parameter, which is according to (15) a three-states Potts variable. The random-field three-states Potts model was analyzed in the ref. ^28,29 Using the results of these papers the implication for our problem is that disorder changes the first-order transition to become second order. Thus, the lattice softening should become more pronounced. Most importantly, weak disorder is not expected to destroy the nematic state.

Discussion

We showed that superconductivity with odd-parity pairing in the doped topological insulator Bi₂Se₃ is dramatically affected by fluctuations. These fluctuations are important, given the low carrier concentration. As a result, the U(1) gauge symmetry and the rotational symmetry are separately broken at the temperatures T_c and T_nem > T_c, respectively. The intermediate nematic state is characterized by strong anisotropic superconducting fluctuations that give rise to an anisotropic para-conductivity. The symmetry breaking will certainly be inherited by the helical surface states of Bi₂Se₃. The three-states Potts universality class of the vestigial order parameter implies that there should be three distinct domains of vestigial order that can be aligned by applying external stress in the E_g symmetry, i.e., for finite $\widehat \epsilon _{xx} - \widehat \epsilon _{yy}$ or $\widehat \epsilon _{xy}$. In addition, a lattice softening and enhanced Raman response are expected above T_nem. This mechanism of composite order gives rise to an enhancement of the superconductivity if compared to the usual fluctuation suppression of T_c. While the transition temperature should still be smaller than the mean-field temperature, it offers an explanation for the comparatively large transition temperature of Cu _x Bi₂Se₃, given the low carrier concentration. This observation further suggests to search for similar states of composite order in other low-carrier superconductors with strong spin–orbit interaction.

Methods

Derivation of Ginzburg-Landau expansion

We start from the established electronic structure of Bi₂Se₃ near the center of the Brillouin zone with the Hamiltonian³⁰ H₀ = $\mathop {\sum}\nolimits_{\bf{k}} {\kern 1pt} c_{\bf{k}}^\dagger \left( {h_{\bf{k}} - \mu _0} \right)c_{\bf{k}}$, where

$$h_{\bf{k}} = v_0\hat \tau ^z\left( {k_x\hat \sigma ^y - k_y\hat \sigma ^x} \right) + \left( {v_zk_z\hat \tau ^y + m\hat \tau ^x} \right)\hat \sigma ^0 + h_{\bf{k}}^\prime {\kern 1pt} .$$

(23)

Here, c _k  = $\left( {c_{{\bf{k}}, + \uparrow },c_{{\bf{k}}, + \downarrow },c_{{\bf{k}}, - \uparrow },c_{{\mathbf{k}}, - \downarrow }} \right)$ refers to the electron annihilation operators for momentum k, located in the two relevant p _z -orbitals in the unit cell (±), and with spin (↑↓).³⁰ The Pauli matrices $\hat \tau ^i$ and $\hat \sigma ^j$ act in orbital and spin space, respectively. The last term $h_{\bf{k}}^\prime$ = $- \lambda \left( {k_ + ^3 + k_ - ^3} \right)\hat \tau ^z\hat \sigma ^z$ with k_± = k _x  ± ik _y takes into account the point symmetry of the hexagonal lattice. The origin of the Rashba-type spin–orbit interaction in Eq. (23) is caused by the lack of local inversion symmetry of the Bi-Se layers. Globally, the system is inversion symmetric, hence the coupling to $\hat \tau ^z$, that is odd under parity, in orbital space.

The superconducting pairing states of doped Bi₂Se₃ were classified in the ref.¹⁰ The state that is compatible with a rotational symmetry breaking has odd-parity and gives rise to the expectation value $\left\langle {c_{\bf{k}}^\dagger \hat \tau ^y\left( {i\hat \sigma ^\mu \hat \sigma ^y} \right)c_{ - {\bf{k}}}^\dagger } \right\rangle$, where μ = {x, y}. Thus, we consider linear combinations of equal spin pairing between distinct orbitals $\left\langle {c_{{\bf{k}}, + \uparrow }^\dagger c_{ - {\bf{k}}, - \uparrow }^\dagger } \right\rangle$ and $\left\langle {c_{{\bf{k}}, + \downarrow }^\dagger c_{ - {\bf{k}}, - \downarrow }^\dagger } \right\rangle$. Such a state is generated by the Hamiltonian H = H₀ + H_pair with pairing interaction

$$H_{{\mathrm{pair}}} = - g\mathop {\sum}\limits_{{\bf{k}},{\bf{k}}\prime ,{\bf{p}},\mu = \left\{ {x,y} \right\}} b_{{\bf{k}},{\bf{p}}}^{\dagger \left( \mu \right)}b_{{\bf{k}}\prime ,{\bf{p}}}^{\left( \mu \right)}$$

(24)

in the odd-pairity symmetry channel: $b_{{\bf{k}},{\bf{p}}}^{\dagger \left( \mu \right)}$ = $c_{\bf{k}}^\dagger \hat \tau ^y\left( { - i\hat \sigma ^\mu \hat \sigma ^y} \right)c_{ - {\bf{k}} + {\bf{p}}}^\dagger$. The band structure of H₀ gives rise to four bands. In bulk, two pairs of Kramers degenerate bands are separated by a gap. In electron doped Bi₂Se₃, such as Cu _x Bi₂Se₃, the Fermi energy is shifted to the upper two bands. Thus, we follow ref. ¹² and project into the conduction bands. Specifically, we use the manifestly covariant Bloch basis ψ _{k s} , see ref. ³¹, that respects the transformation behavior in coordinate and spin space. The index s = {1, 2} refers to the pseudo-spin that labels Kramers degeneracy. It follows H₀ = $\mathop {\sum}\nolimits_{{\bf{k}}s} {\kern 1pt} \psi _{{\bf{k}},s}^\dagger \left( {\varepsilon _{\bf{k}} - \mu _0} \right)\psi _{{\bf{k}},s}$ with $\varepsilon _{\bf{k}}$ = $\sqrt {m^2 + v_0^2\left( {k_x^2 + k_y^2} \right) + v_z^2k_z^2}$. The pair creation operator in this basis is given by

$$b_{{\bf{k}},{\bf{p}}}^{\dagger \left( \mu \right)} = - \psi _{\bf{k}}^\dagger \left( {\varphi _{{\bf{k}},{\bf{p}}}^{\left( \mu \right)}i\tilde \sigma ^y} \right)\psi _{ - {\bf{k}} + {\bf{p}}}^\dagger {\kern 1pt} ,$$

(25)

where $\psi _{\bf{k}}^\dagger$ = $\left( {\psi _{{\bf{k}},1}^\dagger ,\psi _{{\bf{k}},2}^\dagger } \right)$ and the $\tilde \sigma ^l$ are Pauli matrices in pseudo-spin space. The full expressions for the form factors $\varphi _{{\bf{k}},{\bf{p}}}^{\left( \mu \right)}$ = $d_{{\bf{k}},{\bf{p}}}^{s\left( \mu \right)}\tilde \sigma ^0 + {\bf{d}}_{{\bf{k}},{\bf{p}}}^{\left( \mu \right)} \cdot \widetilde {\boldsymbol{\sigma }}$ are given by $d_{{\bf{k}},{\bf{p}}}^{s\left( x \right)}$ = $- if_{{\bf{k}},{\bf{p}}}^{y, - }$, $d_{{\bf{k}},{\bf{p}}}^{s\left( y \right)}$ = $if_{{\bf{k}},{\bf{p}}}^{x, - }$, ${\bf{d}}_{{\bf{k}},{\bf{p}}}^{\left( x \right)}$ = $\left( {f_{{\bf{k}},{\bf{p}}}^{z, + },0, - f_{{\bf{k}},{\bf{p}}}^{x, + }} \right)$ and ${\bf{d}}_{{\bf{k}},{\bf{p}}}^{\left( y \right)}$ = $\left( {0,f_{{\bf{k}},{\bf{p}}}^{z, + }, - f_{{\bf{k}},{\bf{p}}}^{y, + }} \right)$, where

$$f_{{\bf{k}},{\bf{p}}}^{j, \pm } = \frac{{v_j}}{2}\frac{{\left( {m + \epsilon _{\bf{k}}} \right)\left( {k_j - p_j} \right) \pm \left( {m + \epsilon _{{\bf{k}} - {\bf{p}}}} \right)k_j}}{{\sqrt {\epsilon _{\bf{k}}\epsilon _{{\bf{k}} - {\bf{p}}}\left( {m + \epsilon _{\bf{k}}} \right)\left( {m + \epsilon _{{\bf{k}} - {\bf{p}}}} \right)} }},$$

(26)

j = {x, y, z}, and v _x  = v _y  = v₀. For p → 0 holds that $d_{{\bf{k}},{\bf{0}}}^{s\left( \mu \right)}$ = 0 and ${\bf{d}}_{{\bf{k}},0}^{\left( x \right)}$ = $\frac{1}{{\varepsilon _{\bf{k}}}}\left( {v_zk_z,0, - v_0k_x} \right)$ as well as ${\bf{d}}_{{\bf{k}},0}^{\left( y \right)}$ = $\frac{1}{{\varepsilon _{\bf{k}}}}\left( {0,v_zk_z, - v_0k_y} \right)$, see also ref. ¹² This fully defines the low-energy Hamiltonian of doped Bi₂Se₃. For the evaluation of the Ginzburg-Landau parameters in Eqs. (6) and (7) we use the parameters v₀ = 3.3 eV Å, m = 0.28 eV as in,³² μ = 0.50 eV as in,³³ a ≡ a _x  = a _y  = 4.1 Å as in³⁴ and T_c = 3.8 K. We chose g to reproduce the experimental transition temperature. As there is experimental evidence that the velocity v _z depends on the amount of Cu substitution, we kept v _z as a tuning parameter. Depending on the choice of v _z we found either a joint first order transition, or the scenario depicted e.g., in Fig. 2 where we used (v _z /a _z )/(v₀/a) = 1/20. The plotted dimensionless quantities from Fig. 2 read ${\mathrm{\Delta }}_0\sqrt {d_\parallel V_0^2{\mathrm{/}}\left( {T_cV\tilde v} \right)}$, $q_0{\mathrm{/}}\left( {2d_\parallel \tilde v} \right)$, $\chi _{{\mathrm{pair}}}^{ - 1}{\mathrm{/}}\left( {d_\parallel \tilde v} \right)$ and $\delta r{\mathrm{/}}\left( {d_\parallel \tilde v} \right)$, where we defined $\tilde v$ = $\left( {vT_c} \right){\mathrm{/}}\left( {d_\parallel ^2V_0} \right)$ and V₀ = a _x a _y a _z .

Data availability

The data that support the findings of this study are available from the authors on request.