Introduction

The three-dimensional (3D) topological insulator (TI) is a new state of matter that possesses metallic surface states with linear E-K relationship and gapped bulk states. The unique surface states with spin-momentum-lock that caused by strong spin-orbit coupling, hosts a variety of interesting quantum phenomena1,2,3,4 and attracts a ton of research interests. While the progress in 3D TIs have moved very fast, most of the binary 3D TIs, Bi2Se35, Bi2Te36 and Sb2Te37, due to their material imperfection, have their Fermi level lay across the bulk conduction or valence band, shadowing the surface conduction. To date, to achieve bulk insulating TI materials, alloys of those binary TIs, such as Bi2Te2Se8, Bi2TeSe29, (BixSb1-x)2Te310,11, (BixSb1-x)2Se312, have been widely studied and demonstrated suppressed bulk carrier density and the enhancement of the surface conductance. Following the same line of thought, the quaternary alloy of Bi2-xSbxTe3-ySey13 may further improve the bulk insulating properties. Indeed, people have reported the optimization of the compound material and achieved very high resistive material of a few Ω.cm at the composition of Bi1.5Sb0.5Te1.7Se1.313,14.

In the present paper, we perform a systematic study of the crystal structure and the magneto-transport measurement at around the optimized compound concentration suggested by Ando et al.13 We have demonstrated that the Bi1.5Sb0.5Te1.8Se1.2 does show a pronounce peak in all the aspects. At the same time, our structural analyses indicate that the epitaxial constraints may be responsible for controlling the observed property peaks. The findings may guide us more in optimizing the properties of the quaternary alloy of Bi2-xSbxTe3-ySey.

Results

Crystal Structure of Bi1.5Sb0.5Te3-xSex

The single crystals of Bi1.5Sb0.5Te3-xSex were grown by modified Bridgman method, details can be found in the method section. The resulting crystals can be easily cleaved along the (111) plane with a shiny surface, as shown in Figure 1a. Transmission electron microscopy (TEM) was performed to determine the structural characteristics. Flakes of the Bi1.5Sb0.5Te3-xSex (BSTS) were obtained by mechanical exfoliation of cleaved crystals. Selected-area electron diffraction (SAED) patterns for x = 1.2 to 1.5 samples indicate the perfect single crystalline rhombohedral phase13 with a six-fold symmetry, as shown in Figure 1b right. On the other hand, for the x = 1.1 sample, Bi1.5Sb0.5Te1.9Se1.1, the relatively even peak intensity suggested a random mixed hexagonal structure. This demonstrates a transition taken in place as Se concentration decreased from x = 1.2 to 1.1.

Figure 1
figure 1

(a) The image of the as-grown bulk crystal, sample size is about 10 mm × 7 mm. Inset shows the optical image of the Hall bar device. (b) Selected area electron diffraction patterns of Bi1.5Sb0.5Te3-xSex samples. Six-fold symmetry is present for x = 1.3 sample. (c) XRD patterns for a series of Bi1.5Sb0.5Te3-xSex samples. The data have been shifted by one order of magnitude each for clarity. All major diffraction peaks can be indexed as the (003) family. A systematic shift to the left can be observed as Se concentration x decreases. Additional small peaks, indicated by arrows, exist for x below 1.2 samples. Those peaks can be identified as diffraction peaks coming from Sb2Te2Se and Sb2Te3, demonstrating the presence of phase segregation. (d) (red line) The out-of-plane lattice constant c, calculated from the majority peaks, demonstrate a monotonic increases as Se concentration decreases. (blue line) The height of Sb2Te3 (0 1 11) peak vs Se concentration x clearly shows that it emerges below x = 1.2.

Full size image

The transition can also been seen in the crystal structure analysis investigated by X-ray diffraction (XRD) with Cu Ka emission. Figure 1c shows the XRD patterns for a series of Bi1.5Sb0.5Te3-xSex with x = 0.9 to 1.5. The majority peaks are almost identical for all the compositions and can be indexed by (003) family, suggesting the cleave surface is indeed (111). A systematic shifting to higher angles can be observed as Se concentration increases. This can be seen more clearly in Figure 1d. The calculated out-of-plane lattice constant c demonstrates monotonic increase from 1.5 to 0.9. This is easy to understand as the atomic size of Se is much smaller than that of Te. The linear dependence of lattice constant suggests an elastic change of the lattice as more Se replaced by Te within this range.

Expect the major (003) family peaks, additional small peaks can be resolved for samples with x below 1.2, as indicated by the arrows in Figure 1c. These peaks have been identified as diffraction peaks coming from Sb2Te2Se and Sb2Te3, respectively, suggesting the presence of phase segregation at those concentrations. The peak height of Sb2Te3 (blue line in Figure 1d) exhibits the appearance of Sb2Te3 phase as more Te atoms mixed into the compound, starting at x = 1.1.

Transport properties of Bi1.5Sb0.5Te3-xSex

We have studied the electronic properties of samples, as shown in Figure 1a inset, for all the concentration x from 1.0 to 1.5. The longitudinal resistivity (ρxx) as a function of temperature (T) is displayed in Figure 2a. For all the samples, except x = 1.5 and 1.0, the R-T curves demonstrate an increase as temperatures decrease at high temperatures, a typical behavior of semiconductors. The activation energy can be estimated from the Arrhenius plot of ln(ρxx) vs 1/T, as shown in Figure 2b at high temperatures. Figure 2c exhibits the activation energy extracted from the linear fit. A peak at composition x = 1.2 is evidenced.

Figure 2
figure 2

(a) Temperature dependences of ρxx for varies Se concentration x.(b) The Arrhenius plot (ln(ρxx) vs. 1/T) at high temperature. Dashed black lines represent the linear fittings, which give estimations of the activation energy. (c) The concentration dependence of the activation energy and the highest resistivity at low temperature. Both peak at x = 1.2.

Full size image

As shown in Figure 2a, below 10 K, the resistivity of all the samples approaches to some constant value unlike those conventional semiconductors whose resistance diverges at low temperatures. This suggests a temperature independent conduction channel, which we believe is associated with the surface states, that has been reported a lot5,8,13,15. It is interesting to notice that the composition dependent low temperature resistivity presented in Figure 2c also shows a peak at x = 1.2.

We have noticed that our resistivity data have weaker temperature dependence compared with previous work8,13. And we have attributed this discrepancy to the device dimensions. In those work, the sample thickness is 100 μm, while our sample is only 100 nm. This increases the surface to value ratio 1000 times, thus our R-T is more affected by the temperature independent surface conduction, which will be discuss later. Other than this, our lattice constant and low temperature resistivity are in good agreement with previous work13.

To further understand the transport properties, we have performed the Hall measurement. Figure 3a shows the temperature dependence of the low-field (1T) Hall coefficient RH, for all the samples. The RH in Bi1.5Sb0.5Te1.8Se1.2, shows a crossing from positive to negative at 100 K. Considering the activation behavior at high temperature, this can be understood as the electrons from bulk valence band excited to the impurity band leaving the bulk holes dominate conducting8,13,16, at high temperatures. At low temperatures, the excited bulk electrons freeze back to the bulk valence band, leaving electrons in the impurity band and surface states dominate the conductance. Thus the majority carrier type changes back to electrons, although both electrons and holes co-exist in the system, the observed carrier type change is due to the competition between them, which have been observed many times in similar highly compensated systems8,12.

Figure 3
figure 3

(a) Temperature dependences of the low field (1T) Hall coefficient RH for varies Se concentration x.Positive RH has been observed above 100 K for x = 1.2. (b) The value of RH at 1.9 K as a function of Se concentration x. The largest |RH| can be observed at x = 1.2, corresponding a 2D carries density of 1.08 × 10−13 cm−2. c) Magnetic field dependent Hall conductance Gxy for Bi1.5Sb0.5Te1.8Se1.2 (x = 1.2) at 100 K. The black line is a three band model fitting. The brown, green and blue dashed lines represent bulk, surface and impurity channels, respectively.

Full size image

On the contrary, all other samples demonstrate negative RH at all the temperatures, suggesting the electrons in the impurity band and surface states overwhelmed the bulk holes in all the temperature range. Considering the activation energy is smaller for these samples (Figure 2c), the impurity band is closer to the bulk valence band, make it more like a normal degenerate doped n-type semiconductor.

Figure 3b shows the variation in terms of the Hall slope at 1.9 K plotted against the Se concentration. The largest |RH| was again observed at x = 1.2, where RH = −58 Ω/T, corresponding a 2D carrier density of 1.08 × 1013 cm−2.

Three conducting channels

Because the Hall resistance Rxy(B) were found to be nonlinear at most of the temperatures, as we discussed before, there must be multiple conducting channels co-exist in the system. Here we have adopted the model of three conducting channels: the bulk, the impurity band and the surface state15,17. As an example, we present the analysis results of the Hall conductance for Bi1.5Sb0.5Te1.8Se1.2 sample at 100 K. One can make a reasonable fitting to this experimental data [solid black line in Figure 3c], which yields the carrier density of the bulk, impurity band and surface states are 3.3 × 1014 cm−2, 1.11 × 1013 cm−2 and 1.73 × 1011 cm−2, respectively. And the mobilities of the three channels are 785 cm2/Vs, 1100 cm2/Vs and 4310 cm2/Vs, respectively.

The temperature dependent analysis results are plotted in Figure 4. As we discussed before, at high temperature, the dominated carrier are holes in the bulk valence band. When temperature decreases from the room temperature, the bulk carrier densities (blue squares in Figure 4a) decrease exponentially as carriers are frozen from the impurity band. Their temperature dependent density can be described as a classic Fermi-Dirac distribution of (solid purple line in Figure 4a), where n0 is the bulk carrier density of 5.3 × 1014 cm−2 and Ea of 18 meV is the energy gap between the impurity band and the top of the bulk valence band. This activation energy is very close to our previous estimation of 17 meV using Arrhenius plot (Figure 2b), assuring the consistency of our analysis. On the other hand, the carrier density of the impurity band (green triangles) and surface states (red circles) remain approximately constants of 1.3 × 1013 cm−2 and 1.2 × 1011 cm−2, respectively.

Figure 4
figure 4

(a), (b) and (c) 2D carrier density (n2D), mobility (μ) and conductivity (G) of three conducting channels: Surface, Impurity band and Bulk.

The solid purple line represents the Fermi-Dirac distribution of the bulk carrier density equivalent to 2D. The dashed red line represents the power law dependence of μ ∝ T−1, which suggests the mechanism of phonon scattering in these temperatures.

Full size image

Figure 4b shows the corresponding mobilities at different temperatures. The mobility of the bulk holes exhibits a power law dependence of the temperature, μ ∝ T−1, as indicated by the red dashed line in Figure 4b. This suggests the dominant phonon scattering at high temperature ranges. On the contrary, the mobilities of both the impurity band and the surface states stay almost constant for the whole temperature range at 1100 and 8000 cm2/Vs, respectively. This indicates the phonon scattering has very little effects on the surface states and impurity band as expected for the nature of TI materials18.

On the basis of the above analysis, we can subsequently estimate the longitudinal conductance Gxx ( = neμ) for each conducting channels at varies temperature, as shown in Figure 4c. As we expected, at high temperature (>200 K), the bulk holes with a conduction of 300 e2/h, dominates the total conduction. As temperature decreases, thermally excited electrons freeze back from the impurity band, bulk hole density decreases, the conductance also decreases although the mobility increases. Below 70 K, the impurity band conductance starts to overwhelm and the bulk contribution can be neglected as the temperature decreases even more. At very low temperatures, the conductance of impurity band and surface states stay relatively constant of 45 and 4 e2/h, respectively, as shown in Figure 4c. Thus the surface state's contribution to the total conductance, Gs/Gtot, is about 8%.

Discussion

In summary, we have studied the structural and magneto transport properties of Bi1.5Sb0.5Te3-xSex with the concentration x varies from 0.9 to 1.5. By tuning the Se and Te concentration, effectively the acceptors and donors due to various defects compensated with each others, we have realized a maximum bulk-insulating state at Bi1.5Sb0.5Te1.8Se1.2. At this optimized concentration, we found there exists an impurity band at 18 meV above the bulk valence band and the surface contributes 8% of the total conductance at low temperature.

We have also found a direct correlation between the crystal structure and the physical properties. The physical properties of the BSTS samples consistently improve as x decrease from 1.5 to 1.2, while the crystal structure keeps the same. Below 1.2, the structural instability being reached, the crystal structure experienced a phase segregation, evidenced by the emerging of new diffraction peaks and the even peaks in the SAED patterns. This is accompanied by a dramatic change in the physical properties, such as decreasing bulk resistivity and increasing carrier density. This finding suggests a possible route to further improve the crystal properties by suppressing the phase change while decreasing the Se concentration.

Methods

Growth of Bi1.5Sb0.5Te3-xSex bulk crystals

The high-quality single crystalline Bi1.5Sb0.5Te3-xSex (BSTS) were synthesized by the modified Bridgman technique. The high purity metals of Bismuth(5N), Antimony(5N), selenium(5N) and tellurium(5N) were sealed in an evacuated quartz tube and melted at 900°C for two days. To ensure the homogeneity, the quartz tube was kept on rotation in the two days. It is then followed by cooling slowly to 550°C in 7 days and then annealing at this temperature for another 7 days before quenched in the liquid nitrogen. The resulting crystals can be easily cleaved and expose the mirror-like surface.

Device fabrication

The BSTS microflakes were exfoliated by Scotch tapes and then transferred onto Si substrates capped with 300 nm SiO2 layer. The Au electrodes were sputtered onto the flakes, using a standard photolithograph technique.

Electrical measurements

The longitudinal resistance Rxx and the transverse resistance Rxy were measured by a standard six-probe method in a Quantum Design physical properties measurement system (PPMS-9T). The electrical characteristics were measured using resistivity option with a current of 50 nA.

X-ray Diffraction Characterization

The crystal structures were analyzed by the X-ray diffraction method (D/MAX-RB, Rigaku, Japan) with Cu Kα radiation ( λ = 1.54056 nm). The data were collected at 25°C with a step scan procedure in the range of 20° to 60°. The step interval was 0.01° and the scan speed was 1°/min.