Nematic fluctuations in an orbital selective superconductor Fe_1+yTe_1−xSe_x

Abstract

Fe_1+yTe_1−xSe_x is characterized by its complex magnetic phase diagram and highly orbital-dependent band renormalization. Despite this, the behavior of nematicity and nematic fluctuations, especially for high tellurium concentrations, remains largely unknown. Here we present a study of both B_1g and B_2g nematic fluctuations in Fe_1+yTe_1−xSe_x (0 ≤ x ≤ 0.53) using the technique of elastoresistivity measurement. We discovered that the nematic fluctuations in two symmetry channels are closely linked to the corresponding spin fluctuations, confirming the intertwined nature of these two degrees of freedom. We also revealed an unusual temperature dependence of the nematic susceptibility, which we attributed to a loss of coherence of the d_xy orbital. Our results highlight the importance of orbital differentiation on the nematic properties of iron-based materials.

Introduction

The intricate interplay between magnetism and nematicity in different families of iron-based superconductors has attracted great interest in the past few years^1,2,3,4. In iron pnictides, magnetism, and nematicity are tightly coupled; the antiferromagnetic (AFM) transition is always coincidental with, or closely preceded by, a tetragonal-to-orthorhombic structural transition. The proximity of the two transitions can be naturally explained within the spin-nematic scenario, where the structural transition is driven by a vestigial nematic order arising from fluctuations associated with the antiferromagnetic stripe transition (see Fig. 1a)^5,6,7. In iron chalcogenides, the coupling between magnetism and nematicity is less obvious. FeSe undergoes a nematic phase transition without any long-range magnetic order^8,9, which has been interpreted as evidence that the nematic order in FeSe is of orbital origin¹⁰. Nevertheless, spin stripe fluctuations do develop below the nematic transition^10,11, and static stripe order can be induced by hydrostatic pressure^12,13.

Fig. 1: Spin configurations and elastoresistivity configurations.

a, b Schematic spin configurations of the (a) single-stripe phase, with Q = (π, π), and (b) double-stripe phase, with wave-vector Q = (π, 0). c, d Schematic diagrams of the Montgomery method for the elastoresistivity measurement in c B_2g and d B_1g configuration. e, f The anisotropic resistivity (ρ_xx−ρ_yy) as a function of anisotropic strain (ϵ_xx−ϵ_yy) for annealed Fe_1+yTe_1−xSe_x (0 ≤ x ≤ 0.53) at T = 20 K in e B_2g and f B_1g channels. The B_1g elastoresistivity coefficient m₁₁−m₁₂ and B_2g elastoresistivity coefficient 2m₆₆ can be obtained by fitting the linear slope of resistivity versus strain. The samples with high doping concentrations (x = 0.38, 0.45, 0.53) show predominantly a B_2g response while the low doping ones (x = 0, 0.12) show comparable B_1g and B_2g responses.

While there are ongoing debates on the mechanism by which nematicity forms without static magnetism in FeSe^{14,15,16,17,18}, Fe_1+yTe_1−xSe_x provides another platform to approach this problem. As selenium is replaced by tellurium (i.e., x is changed from 1 to 0), the nematic phase transition is suppressed and the superconducting critical temperature reaches optimal near the putative nematic quantum critical point (x ~ 0.5)^19,20. In the high tellurium concentration regime (x < 0.5), inelastic neutron scattering experiments revealed a complex evolution of the spin correlations associated with different magnetic patterns^21,22,23,24. Close to optimal doping, the wave-vector of spin fluctuations at low temperatures is (π, π) [in the crystallographic Brillouin zone], identical to the AFM order in the iron pnictides. As the tellurium concentration increases, both superconductivity and the (π, π) spin fluctuations disappear. The latter are replaced by short-range magnetic correlations with checkerboard character near (0, π), and eventually at low temperatures the competing double-stripe phase forms in Fe_1+yTe (Fig. 1b) by virtue of the ferro-orbital ordering that leads to the formation of ferromagnetic Zigzag chain^25,26,27. Previous elastoresistivity measurements revealed a diverging B_2g nematic susceptibility in optimally doped Fe_1+yTe_1−xSe_x^20,28. While the diverging nematic susceptibility is naturally expected as a consequence of the nematic quantum critical point, it is also consistent with the existence of (π, π) spin fluctuations found in the same doping. This finding suggests that nematic and magnetic fluctuations remain strongly intertwined even in the absence of static nematic and magnetic orders, consistent with previous inelastic neutron scattering study²³. Nevertheless, in contrast to the magnetic sector, the behavior of nematic fluctuations for high tellurium concentrations (i.e., x < 0.5) is still poorly characterized. The compositional dependence of the nematic susceptibility in Fe_1+yTe_1−xSe_x would therefore constitute an important step in the effort to elucidate the relationship between nematicity and magnetism.

Another motivation to study Fe_1+yTe_1−xSe_x is to understand the influence of orbital selectivity on nematic instability. Orbital selectivity (or orbital differentiation) refers to the fact that different orbitals are renormalized differently by electronic correlations, a characteristic property of Hund’s metals that appears to be much more prominent in the iron chalcogenides in comparison with the pnictides^29,30,31. Experimentally, recent scanning tunneling microscopy (STM) measurements revealed the impact of orbital differentiation on the superconducting state³². Theoretically, it has been suggested that orbitally selective spin fluctuations may be the origin of nematicity without magnetism in FeSe³³. The relation between orbital hybridization, spin fluctuations, and nematicity, was also suggested by an earlier inelastic neutron scattering experiment²³. Nematic order was also proposed to enhance orbital selectivity by breaking the orbital degeneracy, leading to asymmetric effective masses in different d-orbitals³⁴. The effect of orbital differentiation becomes even more extreme as selenium is replaced by tellurium. In Fe_1+yTe_1−xSe_x, angle-resolved photoemission spectroscopy (ARPES) revealed a strong loss of spectral weight of the d_xy orbital at high temperatures, which was interpreted in terms of proximity to an orbital-selective Mott transition³⁵. Similar drastic changes were also observed as a function of doping^36,37, mimicking the evolution of spin fluctuations. Nevertheless, the impact of orbital incoherence on nematicity remains little explored³⁸.

In this report, we present systematic measurements of both the B_1g and B_2g nematic susceptibilities of Fe_1+yTe_1−xSe_x (0 ≤ x ≤ 0.53) using the elastoresistivity technique. We demonstrate that the doping dependence of the two nematic susceptibilities closely track the evolution of the corresponding spin fluctuations. In particular, a diverging B_1g nematic susceptibility is observed in the parent compound Fe_1+yTe, suggesting that the spin-nematic paradigm also applies to the double-stripe AFM order^39,40,41. A diverging B_2g nematic susceptibility is observed over a wide range of doping (0.17 ≤ x ≤ 0.53), and its magnitude is strongly enhanced by both Se doping and annealing. In addition, the temperature dependence of the B_2g nematic susceptibility shows significant deviation from Curie–Weiss behavior above 50 K. This is in sharp contrast to the iron pnictides, where the Curie–Weiss temperature dependence extends all the way to 200 K. This unusual temperature dependence is captured by a theoretical calculation that includes the loss of spectral weight of the d_xy orbital, revealing its importance for B_2g nematic instability.

Results and discussion

Doping dependence and temperature dependence of elastoresistivity

By symmetry, the B_1g and B_2g nematic susceptibilities (${\chi }_{{B}_{1g}}$ and ${\chi }_{{B}_{2g}}$) are proportional to the elastoresistivity coefficients m₁₁ − m₁₂ and 2m₆₆, respectively. We performed the elastoresistivity measurements in the Montgomery geometry, which enables simultaneous determination of the full resistivity tensor, hence the precise decomposition into different symmetry channels, as illustrated in Fig. 1c, d. Representative data of anisotropic resistivity as a function of anisotropic strain at 20 K in B_2g and B_1g channels are shown in Fig. 1e, f. The B_1g elastoresistivity coefficient m₁₁ − m₁₂ and the B_2g one, 2m₆₆, can be obtained by fitting the linear slope of resistivity versus strain. Samples with high doping concentrations (x = 0.38, 0.45, 0.53) show predominantly a B_2g response while the low doping ones (x = 0, 0.12) show comparable B_1g and B_2g responses.

Figure 2a, b shows the temperature dependence of m₁₁ − m₁₂ and 2m₆₆ of annealed Fe_1+yTe_1−xSe_x for 0 ≤ x ≤ 0.53. For 0.28 ≤ x ≤ 0.53, 2m₆₆ shows a strong temperature dependence that grows continuously as temperature decreases. For x = 0.45, 2m₆₆ reaches a value of ~100, comparable to optimally doped pnictides. While preserving a similar diverging temperature dependence, the maximum value of 2m₆₆ decreases rapidly as selenium concentration decreases, from 100 for x = 0.45 to 15 for x = 0.28. The magnitude of 2m₆₆ continues to decrease but changes sign for x = 0.17. On the other hand, m₁₁ − m₁₂ shows a diverging response when x is below 0.17, which is in the vicinity of the double-stripe AFM order. As selenium concentration increases, m₁₁ − m₁₂ evolves to a temperature-independent response, with small kinks at low temperatures likely coming from contamination of 2m₆₆ due to misalignment. Overall, our observation of the doping dependence of 2m₆₆ and m₁₁ − m₁₂ is consistent with the evolution of low-temperature spin fluctuations from predominantly (π, 0) at small x to predominantly (π, π) at optimal doping x ~ 0.5^21,22,23,24.

Fig. 2: Temperature and doping dependence of nematic fluctuations of annealed Fe_1+yTe_1−xSe_x.

a Temperature and doping dependence of nematic fluctuations in the B_1g channel, with elastoresistivity coefficient m₁₁ − m₁₂, and b B_2g channel, with 2m₆₆. For clarity, the elastoresistivity data for each doping are offset by 15 and 20 in a, b, respectively. c–e Temperature dependence of c m₁₁ − m₁₂ for x = 0, d 2m₆₆ for x = 0.45 and e −2m₆₆ for Ba(Fe_0.93Co_0.07)₂As₂. Lower panels show the inverse. Solid black curves are Curie–Weiss fits. The optimal fitting range is determined by the greatest corresponding adjusted R-square value (see Supplementary Note 4). Shaded gray regions indicate the range of temperatures where the elastoresistivity coefficients follow a Curie–Weiss law.

To gain more insight, we fit the 2m₆₆ and m₁₁ − m₁₂ to a Curie–Weiss temperature dependence:

$$m={m}^{0}+\frac{\lambda }{a(T-{T}^{* })}$$

(1)

For the parent compound Fe_1+yTe, m₁₁ − m₁₂ can be well fitted to a Curie–Weiss behavior in the temperature range just above the double stripe AFM ordering temperature T_mag = 71.5 K (Fig. 2c). The fitted Curie–Weiss temperature T^* as listed in Supplementary Table III is slightly smaller than T_mag. Despite the smaller absolute value (~10 at maximum), the behavior of m₁₁ − m₁₂ is reminiscent of the 2m₆₆ in the parent phase of iron pnictides, suggesting that the spin-nematic mechanism is still at play here, in agreement with theoretical expectations^39,40,41.

Figure 2d shows the Curie–Weiss fitting of 2m₆₆ for the x = 0.45 sample. The fitting of 2m₆₆ only works at low temperatures, as can be seen in the linear temperature dependence of $| 2{m}_{66}-2{m}_{66}^{0}{| }^{-1}$ below 50K. It shows a significant deviation for temperature >50 K. The T^* obtained from the low-temperature fitting of 2m₆₆ is close to 0K. Intriguingly, the T^* extracted from the Curie–Weiss fitting is approximately zero for all 0.17 ≤ x ≤ 0.53, while the Curie constant λ/a decreases as x decreases (Supplementary Table II). While the number of doping concentrations studied in the current work is insufficient to support a power law analysis, 2m₆₆ at constant T = 16 K appears to be diverging as x increases from 0.17 to 0.45 (Fig. 3a). Both the doping dependence and the near zero T^* are consistent with the existence of a putative nematic quantum critical point at x ~ 0.5 discovered recently²⁰.

Fig. 3: Comparison between the elastoresistivity and the Hall coefficient R_H.

a Doping dependence of the B_2g elastoresistivity coefficient 2m₆₆ (red squares) and of R_H (black diamonds) at 16K. Dashed lines are guide to the eyes. b Colormap of the negative Hall coefficient -R_H (raw data see Supplementary Note 5), c of m₁₁ − m₁₂ d and of 2m₆₆ as a function of temperature and doping. The double-spin stripe and the superconducting transition temperatures are denoted as blue squares and yellow triangles, respectively.

This deviation from Curie–Weiss at high temperatures is very unusual. In the iron pnicitides, such a deviation was only observed at low temperatures in transition-metal doped BaFe₂As₂ (Fig. 2e) and LaFeAsO⁴². This unusual temperature dependence of 2m₆₆ is consistent with the thermal evolution of spin correlations observed by neutron scattering⁴³ and appears to echo the coherent-incoherent crossover observed by ARPES³⁵, where the spectral weight of the d_xy orbital is strongly suppressed as the temperature increases or as the selenium concentration decreases. To further confirm this correlation, we measured the Hall coefficient R_H, which has been demonstrated to be a good indicator of this incoherent-to-coherent crossover^36,44,45. The recovery of the d_xy spectral weight is generally correlated with a sign-change of R_H⁴⁵ from positive to negative. Figure 3a shows the low-temperature R_H and 2m₆₆ as a function of doping, whereas Fig. 3b–d contain the full temperature and doping dependence of R_H, m₁₁ − m₁₂, and 2m₆₆, respectively. These plots reveal the strong correlation between a negative R_H and an enhancement of 2m₆₆.

Effect of annealing

The properties of Fe_1+yTe_1−xSe_x also depend on the amount of excess iron, which can only be removed by annealing⁴⁶. Taking x = 0.45 as an example, the resistance of the annealed sample is metallic for temperatures below 150 K (Fig. 4a). As Fig. 4b shows, at around 40 K the Hall coefficient of the annealed sample turns from positive to negative, which is a signature of incoherent to coherent crossover^36,44,45,47. In contrast, the resistance of the as-grown sample shows a weakly insulating upturn at low temperatures (Fig. 4a black dashed curve), and the Hall coefficient remains positive at all temperatures (Fig. 4b black circles), indicating that the d_xy orbital is still incoherent at low temperatures. Interestingly, at the same temperature where the resistance and the Hall coefficient of the as-grown and annealed samples depart from each other, the elastoresistivity coefficient 2m₆₆ shows a pronounced enhancement for the annealed sample (Fig. 4c). Such an enhancement was observed in all annealed samples (Supplementary Note 2), providing further evidence of the correlation between the enhancement of the nematic susceptibility and the coherence of the d_xy orbital.

Fig. 4: The effect of annealing on the nematic susceptibility of Fe_1+yTe_1−xSe_x.

a–c Temperature dependence of (a) normalized in-plane resistivity (R/R_300K), b Hall coefficient R_H and c elastoresistivity coefficient 2m₆₆ of as-grown (black) and annealed (red) samples for x = 0.45. The vertical gray line marks the temperature below which the behavior of annealed and as-grown samples starts to deviate from each other. Inset of a shows the temperature dependence of the zero field cooling (ZFC) magnetic susceptibility measured at 100Oe (H∥ab). The superconducting volume fraction is significantly enhanced for the annealed sample.

Effect of orbital selectivity on nematic fluctuations

The doping and annealing dependences of 2m₆₆ presented above indicate that the B_2g nematic susceptibility also have an orbitally selective character. This is in agreement with a recent theoretical calculation that reveals a strongly orbitally dependent nematic susceptibility. In particular, the d_xy orbital contributes most to the overall nematic instability⁴⁸. To gain more insight, we calculated the nematic susceptibility with and without a reduced spectral weight in the d_xy orbital to simulate the orbital correlation effect in Fe_1+yTe_1−xSe_x and BaFe₂(As_1−xP_x)₂, respectively. The calculated B_2g nematic susceptibility and its inverse are plotted in Fig. 5a, b. The excellent agreement with the experimentally measured 2m₆₆ and $| 2{m}_{66}-2{m}_{66}^{0}{| }^{-1}$ (Fig. 5c, d) confirms the orbitally selective nature of the nematic susceptibility.

Fig. 5: Effect of orbital selectivity on nematic fluctuations.

a, b Theoretical calculation of the normalized nematic susceptibility χ_nem (a) and its inverse (b), plotted as a function of the relative temperature with respect to the theoretical nematic transition temperature (T − T_nem) for different spectral weight 0.7 ≤ Z_xy ≤ 1. c, d Temperature dependence of c ∣2m₆₆∣ and d $| 2{m}_{66}-2{m}_{66}^{0}{| }^{-1}$ of optimally doped Fe_1+yTe_0.55Se_0.45 (red circles) and BaFe₂(As_0.66P_0.34)₂ (black squares). The red and black lines show Curie–Weiss fittings. The data for BaFe₂(As_0.66P_0.34)₂ follows a Curie–Weiss behavior all the way up to 200K, whereas for Fe_1+yTe_0.55Se_0.45, it deviates from Curie–Weiss behavior above ~50 K.

Indeed, previous theoretical works have highlighted the impact of orbital degrees of freedom on spin-driven nematicity^{17,33,34,49,50,51}. Using a slave-spin approach, ref. ³⁸ found a suppression of the orbital-nematic susceptibility due to orbital incoherence. To model our data, we employ the generalized random phase approximation (RPA) of ref. ⁴⁸ to compute the spin-driven nematic susceptibility for the five-orbital Hubbard-Kanamori model (details in Supplementary Note 6). For fully coherent orbitals, it was found that the largest contribution to the nematic susceptibility χ_nem comes from the d_xy orbital. Thus, one expects that χ_nem would be suppressed if the d_xy orbital were to become less coherent.

To verify this scenario, we calculated how χ_nem changes upon suppressing the spectral weight Z_xy of the d_xy orbital. For our purposes, the reduction in Z_xy acts phenomenologically as a proxy of the incoherence of this orbital, similarly to ref. ³², but its microscopic origin is not important. Fig. 5a, b contrasts the nematic susceptibility for 0.7 ≤ Z_xy ≤ 1. We note two main trends arising from the suppression of d_xy spectral weight: first, as anticipated, the nematic susceptibility (and the underlying nematic transition temperature, which is non-zero in the model) is reduced (Fig. 5a). Second, its temperature dependence changes from a Curie–Weiss-like behavior over an extended temperature range to a behavior in which the inverse nematic susceptibility quickly saturates and strongly deviates from a linear-in-T dependence already quite close to the nematic transition (Fig. 5b). These behaviors are similar to those displayed by the elastoresistance data shown in Fig. 5c, d, with Z_xy = 1 mimicking the behavior of optimally P-doped BaFe₂As₂ and Z_xy < 1, of optimally doped Fe_1+yTe_1−xSe_x. Interestingly, the susceptibility associated with (π, π) fluctuations is also suppressed by the decrease in Z_xy, in qualitative agreement with the neutron scattering experiments⁵² (for a more detailed discussion, see Supplementary Note 6). Of course, since Z_xy in our model is an input, and not calculated microscopically, our model is useful to capture tendencies, but not to extract the experimental value of Z_xy. Furthermore, note that in our calculation Z_xy is temperature-independent, while in the experiment it changes with temperature.

Conclusions

In summary, our results reveal the close connection between nematic fluctuations and spin fluctuations in Fe_1+yTe_1−xSe_x for both B_1g and B_2g channels. Additionally, the unusual temperature dependence of the B_2g nematic susceptibility can be attributed to the coherent-to-incoherent crossover experienced by the d_xy orbital, providing direct evidence for the orbital selectivity of the nematic instability. Our work presents Fe_1+yTe_1−xSe_x as an ideal platform to study the physics of intertwined orders in a strongly correlated Hund’s metal.

Methods

Crystal growth

Single crystals of Fe_1+yTe_1−xSe_x were grown by the modified Bridgeman method. The electrical, magnetic, and superconducting properties of Fe_1+yTe_1−xSe_x are known to sensitively depend on y, the amount of excess iron. To study these effects, crystals were annealed in selenium vapor to reduce the amount of excess iron. Crystals are cleaved into thin slices (~1 mm), loaded in a crucible with another crucible of an appropriate amount (~the amount of excess iron in atomic weight) of selenium powder beneath it, sealed in quartz tubes, and annealed at 500 ^∘C for a week.

Elastoresistivity measurements

The elastoresistivity measurements were performed in the Montgomery geometry, which enables simultaneous determination of the full resistivity tensor²⁸. Crystals are prepared into thin square plates with edges along the Fe-Fe (B_2g) and Fe-Chalcogen (B_1g) bonding directions. The crystal orientation is determined by polarization-resolved Raman spectroscopy, as shown in Supplementary Note 1. Samples were glued on the sidewall of  a piezoelectric stack, which generates a linear combination of anisotropic and isotropic strain. Electrical contacts made at the four corners enable the simultaneous determinations of the resistivities along two perpendicular directions of the same piece of sample. This setup allows the precise decomposition into the anisotropic resistivity change (B_1g and B_2g) Δρ_xx − Δρ_yy and isotropic resistivity change (A_1g) Δρ_xx + Δρ_yy in response to different symmetries of strain (see details in Supplementary Note 3).