An intelligent threshold selection method to improve orbital angular momentum-encoded quantum key distribution under turbulence

Abstract

High-dimensional quantum key distribution (HD-QKD) encoded by orbital angular momentum (OAM) presents significant advantages in terms of information capacity. However, perturbations caused by free-space atmospheric turbulence decrease the performance of the system by introducing random fluctuations in the transmittance of OAM photons. Currently, the theoretical performance analysis of OAM-encoded QKD systems exists a gap when concerning the statistical distribution under the free-space link. In this article, we analyzed the security of QKD systems by combining probability distribution of transmission coefficient (PDTC) of OAM with decoy-state BB84 method. To address the problem that the invalid key rate is calculated in the part transmittance interval of the post-processing process, an intelligent threshold method based on neural network is proposed to improve OAM-encoded QKD, which aims to conserve computing resources and enhance system efficiency. Our findings reveal that the ratio of root mean square (RMS) OAM-beam radius to Fried constant plays a crucial role in ensuring secure key generation. Meanwhile, the training error of neural network is at the magnitude around 10⁻³, indicating the ability to predict optimization parameters quickly and accurately. Our work contributes to the advancement of parameter optimization and prediction for free-space OAM-encoded HD-QKD systems. Furthermore, it provides valuable theoretical insights to support the development of free-space experimental setups.

1 Introduction

By utilizing the principles of quantum mechanics, quantum cryptography ensures secure key distribution by focusing on physical properties rather than computational complexity. Unlike traditional cryptographic approaches, quantum cryptography achieves provable security at the level of information theory. As the core of quantum cryptography, quantum key distribution (QKD) establishes a bit string of common true random bits between two legitimate parties, to realize secure communication combined with One-Time Pad (OTP) Vernam encryption algorithm [1–3]. Currently, most practical QKD uses the degree of freedom such as polarization and phase of photon, the information capacity is limited to 1 bit per photon due to the intrinsically bounded Hilbert space [4–8]. To break through the system information entropy threshold, high-dimensional QKD (HD-QKD) offers improved communication capacity and enhanced noise resilience [9–14]. One common approach to achieve HD-QKD is through the utilization of the orbital angular momentum (OAM) of photons [15–19]. The OAM freedom allows photons to be mapped to an infinite dimensional Hilbert space, providing an effective resource for implementing HD-QKD.

Optical waves carrying OAM in the form of a helical phase front can be adversely affected by turbulence, leading to distortions. These distortions cause the transmission coefficient of the optical OAM channel to fluctuate randomly, resulting in random degradation of the transmitted signal. However, the existing statistical distribution models primarily focus on describing the intensity fluctuations of optical waves, which is insufficient for accurately representing the complex nature of the transmission coefficient fluctuations in turbulent optical OAM channels. Characterizing these fluctuations in the transmission coefficient is crucial for analyzing the security of OAM-encoded QKD systems [20].

Currently, there is a notable absence of comprehensive theoretical analysis on the integration of the probability distribution of transmission coefficient (PDTC) of OAM and QKD in atmospheric turbulence. The fluctuations in transmittance resulting from atmospheric turbulence lead to time-varying quantum bit error rate (QBER), posing a significant challenge in quantum communication systems [21, 22]. In order to address the challenge, enhancing the signal-to-noise ratio (SNR) through post-selection with high transmittance has been proposed as a solution [23–26]. This approach involves optimizing the transmittance threshold after receiving all signals. However, the calculation method often imposes a substantial computational burden due to its complexity. To mitigate the computational challenge and enhance system performance, optimizing QKD system parameters through machine learning techniques has emerged as a promising strategy [27–29]. By utilizing neural network methods for parameter optimization, it becomes possible to improve operational efficiency, facilitate parameter prediction, and ultimately enhance the overall performance of the system [30]. Under the grand vision of promoting the realization of global quantum Internet [31–33], the intelligent methodology holds particular significance for applications such as airborne free space QKD missions, where speed, accuracy, and reliability are paramount considerations.

The paper is organized as follows: In Sect. 2, we begin by generating random samples of transmission coefficients using Monte Carlo simulation. Based on these samples, a suitable transmission coefficient statistical distribution model is proposed which accurately captures the characteristics of turbulent optical OAM channels. Subsequently, we establish a basic framework for calculating the key rate of OAM-encoded QKD systems in Sect. 3. Finally, an artificial neural network method is used to forecast the optimal transmittance threshold to improve system in Sect. 4. The conclusions of the present paper are given in Sect. 5.

2 Preliminaries

Characterization of the statistical distribution of OAM transmission in turbulence is the primary task of studying OAM-encoded QKD. In this section, we will use Monte Carlo simulation and dual Johnsom ${S_{B}}$ distribution to get the PDTC model of OAM.

2.1 Instantaneous transmittance of OAM channels

Given the initial normalization of light intensity, the transmission coefficient of OAM is directly correlated with the received optical power. However, it is essential to note a fundamental distinction between the mathematical models for detecting power in OAM optical channels and traditional free-space channels. The variance implies that the probability distribution function governing light intensity fluctuations induced by atmospheric turbulence may not be applicable when describing the fluctuation of transmission coefficients in turbulent OAM optical channels. Unlike classical Gaussian light, the transmission of OAM in turbulent conditions does not conform to the commonly employed log-normal and Gamma-Gamma distribution models.

In general, establishing a statistical distribution model directly tailored for turbulent optical OAM channel transmission coefficients through analytical mathematical derivation poses significant challenges. To overcome the limitation, our approach involves utilizing Monte Carlo simulation to generate a random sample of transmission coefficients. This methodology enables us to propose a suitable distribution model that accurately reflects the fluctuations in the transmission coefficient. Drawing insights from existing Ref. [20], we begin by theoretically deriving the instantaneous transmission coefficient:

$$ \eta ( t ) = \int _{0}^{\infty }{\hat{\rho}(\hat{r})} W( \hat{r}{r_{0}})\,d\hat{r}, $$

(1)

with

$$\begin{aligned}& \hat{\rho}(\hat{r}) = \hat{r}{\biggl( \frac{{2\sqrt { \vert l \vert + 1} }}{\Omega }\biggr)^{2}} \frac{1}{{ \vert l \vert !}}{ \biggl[ { \frac{{\hat{r}\sqrt {2( \vert l \vert + 1)} }}{\Omega }} \biggr]^{2 \vert l \vert }}\exp \biggl[ { - \frac{{2 ( { \vert l \vert + 1} ){{\hat{r}}^{2}}}}{{{\Omega ^{2}}}}} \biggr]\, \end{aligned}$$

(2)

$$\begin{aligned}& W(r) = {(2\pi )^{ - 2}} { \biggl\vert { \int _{0}^{2\pi } {d\theta \exp \bigl[i \varphi (r, \theta )\bigr]} } \biggr\vert ^{2}}, \end{aligned}$$

(3)

where $\hat{r} = r/{r_{0}}$ represents the ratio of Laguerre-Gaussian (LG) beam width r to the Fried constant ${r_{0}}$, $\Omega = {w_{z}}\sqrt {( \vert l \vert + 1)} /{r_{0}}$ denotes the nondimensional ratio between the root-mean-square(RMS) OAM-beam radius and the Fried constant, and the RMS OAM-beam radius reflects the divergence characteristic [34]. l is the OAM index. The OAM beam under consideration has a radial index set to zero. ${w_{z}} = {w_{0}}\sqrt {1 + {z^{2}}/z_{R}^{2}}$, ${w_{0}}$ is the beam waist, ${z_{R}} = 0.5kw_{0}^{2}$ is the Rayleigh range (here $k = 2\pi /\lambda$, k and λ represent wavenumber and wavelength respectively).

The aforementioned data can be provided based on the actual scenario. The crucial aspect revolves around the implementation of the random phase screen $\varphi (r,\theta )$. The random generation of phase screens based on sparse spectrum is a commonly employed method due to its ability to be easily adapted to uniform spatial sampling in polar coordinates. The procedure for realizing $\varphi (r,\theta )$ through sparse-spectrum-based method and Monte Carlo simulation is detailed in Appendix A.1.

2.2 Model of PDTC

In order to conduct a mathematical characterization of the PDTC, a dual Johnson ${S_{B}}$ distribution with four independent control parameters is fitted to the random fluctuations of the measured signal in the turbulent optical OAM channel. The probability density function (PDF) of dual Johnson ${S_{B}}$ distribution can be expressed as:

$$ {P_{d}}( \eta |{\gamma _{1}},{\delta _{1}},{\gamma _{2}},{ \delta _{2}}) = \frac{1}{2}\bigl[{P_{J}}( \eta |{\gamma _{1}},{ \delta _{1}}) + {P_{J}}( \eta |{\gamma _{2}},{\delta _{2}})\bigr], $$

(4)

with

$$ {P_{J}}( \eta |\gamma ,\delta ) = \frac{\delta }{{\sqrt {2\pi } }} \frac{1}{{\eta (1 - \eta )}}\exp \biggl\{ { - \frac{1}{2}{{ \biggl[ {\gamma + \delta \ln \biggl( \frac{\eta }{{1 - \eta }}\biggr)} \biggr]}^{2}}} \biggr\} , $$

(5)

where ${P_{J}}(\eta \vert \gamma ,\delta )$ is the PDF of Johnson ${S_{B}}$ distribution. The four independent control parameters ${\gamma _{1}},{\gamma _{2}},{\delta _{1}},{\delta _{2}} > 0$, statistically characterize the simulated samples. For practical application of the dual Johnson ${S_{B}}$ distribution, it is advisable to establish a direct correlation between four control parameters and the propagation conditions along with OAM modes. The connection between $( {\Omega ,l,{r_{0}}} )$ and $( {{\gamma _{1}},{\delta _{1}},{\gamma _{2}},{\delta _{2}}} )$ can be perceived as a mapping from three to four dimensions. While formulating a mathematical expression to articulate the mapping remains challenges, Monte Carlo simulations and probability distribution fitting utilizing maximum likelihood estimation (MLE) can elucidate the relationship.

To further investigate the mechanism of how Ω and ${r_{0}}$ influence the PDTC, we analyze the PDF of transmission coefficients under different Ω and ${r_{0}}$ conditions. As shown in Fig. 1, when keeping Ω constant and varying ${r_{0}}$, the PDF curve of the transmission coefficient exhibits a similar trend with slight differences. However, when keeping ${r_{0}}$ constant and varying Ω, there is a significant discrepancy in the PDF curve. It indicates that the impact of Ω on the PDF of the transmission coefficient is stronger than ${r_{0}}$. Moreover, as Ω increases, the PDF curve’s “peak” gradually shifts to the left, indicating a decrease in the probability of achieving high transmittance η. It suggests that turbulence-induced channel attenuation is intensified. Notably, when Ω is close to 1 (as depicted in Fig. 1(b)), the PDF curve exhibits a pronounced tailing phenomenon that is difficult to describe with existing mathematical models.

Figure 1

PDFs of transmission coefficient η with $l \equiv 1$ and different Ω and ${r_{0}}$. (a) $\Omega = {10^{ - 1.0}}$; (b) $\Omega = {10^{ 0}}$; (c) $\Omega = {10^{ 0.6}}$

Considering the transport properties of OAM in the atmosphere and practical implementation, it is more reasonable to set ${r_{0}} = 10\text{ cm}$. As depicted in Fig. 2(a), the PDF curves of different OAM indices are essentially merged, suggesting that when Ω is small, l has no discernible influence on the PDF. However, in Fig. 2(b) and Fig. 2(c), when $\Omega \ge 1$, differences between the curves are evident, indicating that the PDTC of OAM exhibits a notable state-dependent effect. The state-dependent turbulence in this scenario comes from two folds: (1) high order OAMs have larger beam cross section which leads to a worse turbulence; (2) high order OAMs have multi-fold phase structures which are more sensitive to phase distortions.

Figure 2

PDFs of transmission coefficient η with ${r_{0}} \equiv 10\text{ cm}$ and different Ω and l. (a) $\Omega = {10^{ - 1.0}}$; (b) $\Omega = {10^{ 0}}$; (c) $\Omega = {10^{ 0.6}}$

3 Secure key rate analysis

Currently, the relationship between the security key generation rate R and the transmittance coefficient η in turbulent optical OAM channels lacks theoretical substantiation. Under the premise of fixed QKD system parameters, building upon the analysis in Sect. 2 and incorporating the Gottesman-Lo-Lütkenhaus-Preskill (GLLP) formula, we formulate the key rate of the decoy-state BB84 protocol as a function of $R(\eta )$. Consequently, assessing the system security rate in turbulent channels involves determining the integral of $R(\eta )$ over the probability distribution of η.

Ratewise integration is a prevalent method for determining the secure key rate, involving the integration of $R(\eta )$ over the entire range of the PDF, incorporating all available information regarding the channel transmission probability distribution. Initially, we examine the asymptotic scenario of transmitting an infinite number of pulses. As depicted in Fig. 1 and Fig. 2, we have measured the channel transmittance η corresponding to each signal transmission period and acquired comprehensive transmittance information. The rate of the ratewise integration model can be expressed as:

$$ {R^{\mathrm{Ratewise}}} = \int _{0}^{1} {R(\eta )P(\eta )\,d\eta }. $$

(6)

As the spatial degree of freedom of the light field, OAM possesses the ability to map an infinite-dimensional Hilbert space. In the quantum state space of high dimensions, the SUP basis and the OAM basis form two mutually unbiased bases (MUBs), which guarantees the unconditional security of QKD. OAM basis is composed of different OAM states, and the Fourier conjugate SUP basis consists of an equal superposition of OAM states with fixed relative phase between adjacent OAM components. The OAM basis and SUP basis can be represented as:

$$ \begin{gathered} \{ {| {{\varphi _{\mathrm{OAM}}}} \rangle } \} = \bigl\{ { \vert { - l} \rangle , \vert { - l + 1} \rangle ,\ldots,| 0 \rangle ,\ldots,| {l - 1} \rangle ,| l \rangle } \bigr\} \\ \{ {| {{\psi _{\mathrm{SUP}}}} \rangle } \} = \Biggl\{ {\frac{1}{{\sqrt {d} }}\sum _{l = - L}^{L} {\exp \biggl(i \frac{{jl2\pi }}{d} \biggr)\big| {{\varphi _{\mathrm{OAM}}}} \rangle } } \Biggr\} \end{gathered} $$

(7)

where d is the dimension of the Hilbert space and L is the maximum OAM quantum number in use, which satisfies the relation $2L + 1 = d$. Subsequently, Bob uses the results with OAM basis as the key bits and the results with SUP basis as the testing bits, and then they testify the security of the key distribution. After error correction and privacy amplification, they can distill a secure key [35]. For weak coherent sources (WCS), the secure key rate of decoy-state HD-QKD can be estimated by [16]:

$$ R \ge {q_{m}} \bigl\{ { - {f_{\mathrm{EC}}}\bigl(E_{\mu }^{\mathrm{OAM}} \bigr)Q_{\mu }^{\mathrm{OAM}}{H_{d}}\bigl(E_{ \mu }^{\mathrm{OAM}} \bigr) + Q_{1}^{\mathrm{OAM}}\bigl[{{\log }_{2}}d - {H_{d}}\bigl(e_{1}^{\mathrm{SUP}}\bigr)\bigr]} \bigr\} \ $$

(8)

where ${q_{m}}$ represents the probability of choosing basis and depends on the QKD protocol. ${f_{\mathrm{EC}}}(E_{\mu }^{\mathrm{OAM}})$ is the error correction efficiency of the signal state. $Q_{\mu }^{\mathrm{OAM}}$ and $E_{\mu }^{\mathrm{OAM}}$ are the gain and QBER of the signal states, respectively. $Q_{1}^{\mathrm{OAM}}$ and $e_{1}^{\mathrm{SUP}}$ are the gain and the error rate of the single-photon states, which can be estimated using decoy-state, see Appendix A.2 for detailed analysis. ${H_{d}}(x) = - (1 - x){\log _{2}}(1 - x) - x{\log _{2}} [ {x/(d - 1)} ]$ is the d-dimensional Shannon entropy.

Due to the rotation invariance of the vortex light field, it effectively avoids the problem of the reference frame alignment, and provides a new coding scheme for the construction of high-speed and stable mobile quantum communication networks. The most likely application in the future is, for example, the scenario of drone-based quantum communication, with a typical distance range of kilometers. According to the above theoretical analysis, targeting 10 km free space link scenarios, the simulation method is used to analyze the performance of OAM-encoded HD-QKD system. The settings of the QKD system parameters [36] are outlined in Table 1.

Table 1 Parameters used for the numerical simulation of OAM-encoded HD-QKD

We initially simulate the impact of the OAM index on the security key rate. Figure 3 illustrates the correlation between Ω and the security key rate R. As Ω increases, the security key rate exhibits a gradual decline, suggesting that Ω to some degree reflects the disturbance and attenuation characteristics of atmospheric turbulence channel. Nonetheless, variations in the OAM index l result in negligible effects on the secure key rate. Therefore, it can be further considered to apply a high-dimensional quantum system to improve the information capacity under the condition of changing Ω, especially when $\Omega \le 1$.

Figure 3

Secure key rate R of Ω with different l

Figure 4 depicts the correlation between the security key rate R and Ω within a d-dimensional QKD system. The figure exclusively illustrates the trend in the variation of the security key rate surface for d-dimensional QKD as Ω changes. From the visualization, it is evident that with an increase in the dimensionality of the QKD system, both the security key rate and the transmission distance rise. The observation suggests that the QKD system can effectively leverage the benefits of high-dimensional coding when Ω is considered a variable. Due to the existence of state-dependent diffraction (SDD) [35] and other effects in high-dimensional QKD system transmission in the atmosphere, the receiving efficiency mismatch leads to security vulnerabilities and security key rate decrease. According to the results of Fig. 3 and Fig. 4, we can use more high-dimensional quantum systems in the scenarios of minor Ω, because OAM index l has a subtle influence on key generation at this time. To get closer to practicality, the finite-size effect is analyzed in Appendix A.3.

Figure 4

Secure key rate R of Ω with different dimension d

The above analysis provides a theoretical basis for the performance optimization method of threshold selection. First, by combining the transmission PDF of OAM with HD-QKD, a theoretical model is found for the statistical distribution of OAM-encoded QKD; Second, the advantages of HD-QKD encoding are explored in incorporating the finite-size effects, and a theoretical reference is provided for the practical airborne application of OAM-encoded QKD.

4 Performance optimization

In addressing the challenges posed by random fluctuations in channel transmission and low key generation rates resulting from atmospheric turbulence in free-space experiments, this section presents strategies and recommendations aimed at enhancing system performance. Through the implementation of optimal threshold selection, computational resource consumption is minimized. Additionally, a neural network model is employed to train and predict discrete datasets, thereby enhancing operational efficiency.

4.1 Threshold selection method

During the process of ratewise integration method, there are instances where key generation can be negligible under certain transmission efficiency conditions due to the PDTC in turbulent OAM optical channels. However, in practical scenarios, the data segment necessitates post-processing, leading to a squandering of computational resources. Considering the time-varying transmission characteristics induced by free-space atmospheric turbulence, employing threshold selection method as a simplified method to filter information and post-selection becomes imperative. This approach helps diminish the volume of data computation, thereby enhancing the secure key rate per pulse and overall system performance. The simplified security key rate for threshold selection can be expressed as follows:

$$ {R^{\mathrm{Simplified}}} = \int _{{\eta _{T}}}^{1} {R(\eta )P(\eta )\,d\eta }. $$

(9)

According to the Eq. (9), adjusting the threshold value ${\eta _{T}}$ and observing the resultant alteration in the security key generation rate is imperative. As depicted in Fig. 5, when the OAM index $l = 1$, the security key rate that we set the threshold is almost the same as the static value obtained by the ratewise integration in the initial range, followed by a marked decline. We can find an optimal ${\eta _{T}}$ according to the actual scenario to simplify the calculation, for example, in the scenarios of minor Ω, opting for a higher threshold can be advantageous in enhancing the efficiency of data processing.

Figure 5

Secure key rate R of ratewise integration method and threshold selection simplified method

In order to further depict the selection of thresholds more vividly, we set up three-dimensional images of Ω, l and ${\eta _{T}}$ to visually observe the effects of different variables on ${\eta _{T}}$. As illustrated in Fig. 6(a), it is evident that the impact of Ω on ${\eta _{T}}$ surpasses that of l, with the optimal threshold gradually diminishing as Ω increases, aligning with the findings in Fig. 5. However, when Ω exceeds 1, a more intricate relationship among the three variables is discerned, as depicted in Fig. 6(b), signifying a shift in their interplay. At this juncture, both l and Ω exert a significant influence on the determination of ${\eta _{T}}$, and with the escalation of l, the selection of the optimal threshold declines correspondingly. Consequently, when Ω is less than 1, the influence of l on the selection of the optimal threshold may be disregarded. Conversely, when Ω surpasses 1, due consideration must be given to the impact of l on the optimal threshold selection.

Figure 6

Optimal threshold ${\eta _{T}}$ under different Ω and l conditions. (a) $- 2 \le {\log _{10}}(\Omega ) \le 2$; (b) $0 \le {\log _{10}}(\Omega ) \le 2$

4.2 Neural network method

The PDTC of OAM in turbulence is based on data obtained through Monte Carlo simulation, which is discrete and exhibits a degree of volatility. To further enhance the optimization of the threshold, an adaptive threshold selection method employing an artificial neural network is proposed. The error back propagation (BP) neural network, as a widely utilized model, has been chosen for optimization and prediction.

Based on the analysis in Sect. 2, 3 and 4.1, Fig. 7 shows the procedure of obtaining the optimal threshold ${\eta _{T}}$ from the initial parameters $(\Omega ,l,{r_{0}})$. This procedure involves a clear concept, but it entails significant computational complexity. By employing intelligent methods to train the initial parameters, it is possible to not only obtain the final result ${\eta _{T}}$ but also achieve data prediction, showcasing its practical advantages.

Figure 7

A comparison of traditional and intelligent procedures

The process of using BP neural network to achieve model training and parameter prediction is shown in Fig. 8. First, the data set is divided into three parts, the training set, the testing set, and the primary set. The BP neural network model is obtained by inputting the large-capacity data which can embody the data characteristics into the trainer. In order to verify the performance of the model, the testing set is brought into the network for calculation, and the predicted set is obtained. Finally, the data of the predicted set and the primary set are compared to judge the performance of the system.

Figure 8

The process of model training and parameter prediction by BP neural network

The network architecture, as illustrated in Fig. 9. The parameters of the input layer represent the main characteristics of OAM and turbulence, which shows the effect of random disturbance. The number of hidden layers is 3, and the number of neurons is 15, 10 and 1, respectively. The data of the output layer is the optimal threshold ${\eta _{T}}$. The transfer function for the hidden layer is the tansig transfer function, while the output layer employs a purelin function. The learning algorithm employed is the Levenberg-Marquardt (LM) variable gradient algorithm, with the mean square error (MSE) function used as the performance metric. The training process includes 2000 iterations, with a learning rate set at 0.01.

Figure 9

The structure of BP neural network

The dataset with the optimal threshold ${\eta _{T}}$ of 18*17 samples has been constructed, where the ratio Ω of the RMS OAM-beam radius and Fried constant ranges from 10⁻² to 10^1.4 and the OAM index l varies from 0 to 16. $({\eta _{T}},\Omega ,l)$ $(2 \times 18 \times 16)$ is selected as the training set to train the neural network model. The last 18 data $[ {l = 16,\Omega = {{10}^{ - 2.0}} \sim {{10}^{1.4}}} ]$ are selected for prediction. The Imitative effect and training error between the predicted data and the primary data are compared, as shown in Fig. 10.

Figure 10

Optimal threshold prediction results using BP neural network. (a) Imitative effect; (b) Training error

Figure 10(a) shows the fitting curve. The horizontal axis represents the number of samples selected by the neural network, while the vertical axis illustrates the sample value of the optimal threshold selection. By standardizing the data in training, normalization facilitates effective data processing and contributes to faster convergence during program execution. The practice is fundamental for ensuring that the neural network can efficiently learn and adapt to the given dataset, ultimately improving its performance in prediction and analysis tasks. The solid blue line with dots corresponds to the primary data, whereas the solid red lines with triangles depict the predicted data. Figure 10(b) shows the training error curve, and the ordinate represents the training error between the model’s predicted value and the primary data value.

From Fig. 10(a), it can be analyzed that the change pattern of the primary data and the predicted data follows a periodic trend. The imitative effect can be attributed to the method of selecting the optimal threshold during the collection of original data, where the thresholds are chosen in increasing order of the same OAM index l. In other words, when selecting Ω for a specific OAM index, the subsequent OAM index with Ω values is considered until 18 OAM index points have been covered.

The fitting curve effect of the predicted data generated by the BP neural network aligns closely with the primary data curve, underscoring the model’s accuracy in capturing and predicting the trends within the dataset. Moreover, Fig. 10(b) reveals that the training error of the sample set is around 10⁻³, indicating the high prediction accuracy of the neural network. It should be noted that in the scenario of high Ω, the optimal threshold accuracy at this time reaches $1 \times {10^{ - 2}}$, so the training error at this time is also an important basis for judging the network performance. The training error of the last three predicted data is in the order of 10⁻⁴, which proves the training effect of the network. The intelligent model can effectively support the adaptive optimal threshold selection in the OAM-encoded QKD system, enhancing data processing speed and providing valuable guidance for system parameter prediction and optimization.

5 Conclusion

The analysis of the statistical distribution of OAM transmission characteristics in the atmosphere plays a pivotal role in advancing high-dimensional quantum communication. The benefits offered by OAM-encoded QKD, such as communication capacity and rotationally invariant property, have propelled its application in short-distance information transmission within local hotspots. Due to the complex attenuation and disturbance caused by OAM transmission in atmospheric turbulence, it is difficult to use classical mathematical models to characterize the statistical distribution of transmission characteristics at the receiver. Currently, there remains a gap in comprehensive theoretical analysis concerning the statistical distribution of OAM-encoded QKD in free space. In this paper, the transmittance PDF of OAM transmitted in the atmosphere is integrated with the HD-QKD secure key rate formula, incorporating the finite-size effects to effectively evaluate system performance. The results show that the ratio Ω of RMS OAM-beam radius to Fried constant plays the most prominent role in determining control parameters and security key generation. If the value of Ω exceeds 1, the effect of OAM index l cannot be ignored. Furthermore, the study briefly explores the coding advantages of HD-QKD, shedding light on the potential benefits and applications of high-dimensional quantum key distribution in secure communication protocols. The comprehensive analysis contributes valuable insights into the optimization and performance enhancement of OAM-encoded QKD systems in challenging atmospheric environments.

Aiming at the problem of the free space OAM-encoded QKD system that the invalid key rate is calculated in the part transmittance interval of the post-processing process and the high-dimensional protocol needs to accumulate a large amount of data, the method combining threshold selection and neural network is used to train discrete data points. The results indicate a close alignment between the original data curve and the predicted data, with a training error around 10⁻³. This achievement signifies the model’s capability to perform intelligent parameter prediction effectively. Furthermore, the model’s success in achieving high accuracy showcases its potential to enhance system running speed and key distribution performance. Ultimately, this approach offers a practical solution for the intelligent advancement of quantum secure communication systems.

Appendix

1.1 A.1 Generation of random phase screen

It should be emphasized that $\varphi (r,\theta )$ in Eq. (3) actually denotes the phase perturbation described via a random phase screen, resulting in $\eta (t)$ taking on a random value. Clearly, the key to obtaining random samples of $\eta (t)$ is to acquire a random implementation of the phase screen $\varphi (r,\theta )$. Due to the advantageous feature of integration over θ, we have opted for sparse-spectrum-based method [37] to generate the phase screen:

$$ \varphi (\vec{r}) = {\mathop{\mathrm{Re}}\nolimits} \Biggl[\sum _{n = 1}^{N} {{a_{n}}\exp (i\vec{K} \cdot \vec{r})} \Biggr], $$

(A1)

where ${a_{n}}$ represents a random complex amplitude, $\vec{r} = (r,\theta )$ is a position vector in a two-dimensional plane, and ${\mathop{\mathrm{Re}}\nolimits} (z)$ denotes the real part of z. K⃗ signifies a random wavenumber vector, the probability density function (PDF) for the magnitude of vector K⃗ is:

$$ {p_{n}}(K) = \frac{5}{3} \frac{1}{{K_{n - 1}^{ - 5/3} - K_{n}^{ - 5/3}}}{K^{ - 8/3}}, $$

(A2)

where ${K_{n - 1}} \le K \le {K_{n}}$, ${K_{n}} = {\kappa _{0}}\exp [(n/N)\ln ({\kappa _{m}}/{\kappa _{0}})]$ and $1 \le n \le N$, ${\kappa _{0}} = 2\pi /{L_{0}}$, ${\kappa _{m}} = 2\pi /{l_{0}}$, ${l_{0}}$ and ${L_{0}}$ are the inner and outer scales of turbulence, respectively. The direction angle of vector K⃗ is assumed to follow a uniform distribution between π and −π. Moreover, ${a_{n}}$ is a random complex amplitude that follows a normal distribution and obeys:

$$ \langle {{a_{n}}} \rangle = 0, \langle {{a_{n}} {a_{m}}} \rangle = 0, \bigl\langle {{a_{n}}a_{m}^{*} } \bigr\rangle = 3.695r_{0}^{ - 5/3}\bigl(K_{n - 1}^{ - 5/3} - K_{n}^{ - 5/3}\bigr){\delta _{mn}}, $$

(A3)

where the angle brackets indicate statistical averages, and ${\delta _{mn}} = 1$ if $m = n$ and ${\delta _{mn}} = 0$ otherwise. By utilizing N to generate a random implementation of ${a_{n}}$ and K⃗, it becomes straightforward to calculate $\varphi (r,\theta )$ at any given position $\vec{r} = (r,\theta )$ based on Eq. (A1).

It is noteworthy that in the Monte Carlo simulation, to align our spectral intervals with the standard pure Kolmogorov spectrum, we have set the inner and outer scales of the turbulence to 1 mm and 100 m, respectively.

1.2 A.2 Decoy-state HD-QKD rate model

For decoy-state HD-QKD system, the total system transmittance ${\eta _{\mathrm{sys}}}$ of single photon signal consists of the transmission efficiency η and detector efficiency ${\eta _{d}}$. Thus, $Q_{\mu }^{\mathrm{OAM}}$ can be calculated by:

$$ Q_{\mu }^{\mathrm{OAM}} = {Y_{0}} + 1 - {{ \mathrm{{e}}}^{ - \mu {\eta _{\mathrm{sys}}}}}, $$

(B1)

where ${Y_{0}} = 2{p_{d}}$ is the yield when Alice does not send a photon. ${p_{d}}$ is the dark count of the detector. μ and v are the mean photon number of signal state and decoy state respectively. Meanwhile, $E_{\mu }^{\mathrm{OAM}}$ can be given as

$$ E_{\mu }^{\mathrm{OAM}} = \frac{{{e_{0}}{Y_{0}} + {e_{d}}(Q_{\mu }^{\mathrm{OAM}} - {Y_{0}})}}{{Q_{\mu }^{\mathrm{OAM}}}}. $$

(B2)

Different from the general two-dimensional QKD system, ${e_{0}} = (d - 1)/d$ is the error rate of the dark count in HD-QKD. By using “vacuum + weak decoy state” method, the yield and the error rate of the single-photon states $Y_{1}^{\mathrm{OAM}}$ and $e_{1}^{\mathrm{SUP}}$ can be estimated by:

$$\begin{aligned}& Y_{1}^{\mathrm{OAM}} \ge \frac{\mu }{{\mu v - {v^{2}}}}\biggl(Q_{v}^{\mathrm{OAM}}{e^{v}} - Q_{ \mu }^{\mathrm{OAM}}{e^{\mu }}\frac{{{v^{2}}}}{{{\mu ^{2}}}} - \frac{{{\mu ^{2}} - {v^{2}}}}{{{\mu ^{2}}}}{Y_{0}}\biggr), \end{aligned}$$

(B3)

$$\begin{aligned}& e_{1}^{\mathrm{SUP}} \le \frac{{E_{v}^{\mathrm{SUP}}Q_{v}^{\mathrm{SUP}}{e^{v}} - {e_{0}}{Y_{0}}}}{{Y_{1}^{L,v,0}v}}. \end{aligned}$$

(B4)

1.3 A.3 Finite-size analysis

Due to the disparity between HD-QKD and 2D-QKD in terms of key analysis, we conducted a security analysis of HD-QKD under asymptotic conditions. However, in practical QKD systems, the number of pulses transmitted by the quantum signal is limited, leading to statistical fluctuations between observed and measured values. To further validate the security of the HD-QKD system, it becomes crucial to analyze the finite-size effects resulting from statistical fluctuations. To obtain the expected confidence interval of the observed parameter, the Chernoff inequality is employed [38]. The upper and lower limits of the confidence interval ${E^{U}} [ \zeta ]$ and ${E^{L}} [ \zeta ]$ can be expressed as:

$$ {E^{L}} [ \zeta ] = \frac{\zeta }{{1 + {\delta ^{L}}}}, \qquad {E^{U}} [ \zeta ] = \frac{\zeta }{{1 - {\delta ^{U}}}}, $$

(C1)

with

$$ { \biggl[ { \frac{{{{\mathrm{{e}}}^{{\delta ^{L}}}}}}{{{{ ( {1 + {\delta ^{L}}} )}^{1 + {\delta ^{L}}}}}}} \biggr]^{\frac{\zeta }{{1 + {\delta ^{L}}}}}} = \frac{\varepsilon }{2}, \qquad {\biggl[ { \frac{{{{\mathrm{{e}}}^{ - {\delta ^{U}}}}}}{{{{ ( {1 - {\delta ^{U}}} )}^{1 - {\delta ^{U}}}}}}} \biggr]^{\frac{\zeta }{{1 - {\delta ^{U}}}}}} = \frac{\varepsilon }{2}, $$

(C2)

where $\zeta > 0$ is an observed value, δ is the statistical fluctuation which used to represent the estimated interval of the expected value. ε is the failure probability to estimate the confidence interval of the expected value.

Based on Eq. (C1)-(C2) and Eq. (B3)-(B4), the upper and lower bounds of the statistical fluctuations for $Y_{1}^{\mathrm{OAM}}$ and $e_{1}^{\mathrm{SUP}}$ can be obtained. The security key length of OAM-basis of HD-QKD system under finite-size effect is obtained:

$$ [{K^{\mathrm{OAM},L}} \ge M_{1\mu }^{\mathrm{OAM},L}\bigl[1 - {H_{d}} \bigl( {e_{1\mu }^{\mathrm{SUP},U}} \bigr)\bigr] - M_{\mu }^{\mathrm{OAM}}{f_{\mathrm{EC}}}\bigl(E_{\mu }^{\mathrm{OAM},U} \bigr){H_{d}} \bigl( {E_{ \mu }^{\mathrm{OAM},U}} \bigr)\, $$

(C3)

where $M_{1\mu }^{\mathrm{OAM},L}$ is the lower limit number of OAM-basis sifted key from the single-photon signal state. $M_{\mu }^{\mathrm{OAM}}$ is the number of OAM-basis sifted key from the overall signal state. $e_{1\mu }^{\mathrm{SUP},U}$ is the upper limit of the phase error rate. $E_{\mu }^{\mathrm{OAM},U}$ is the upper limit of the QBER in the OAM-basis sifted key. Therefore, for a given data size N, the final secret key rate of the practical HD-QKD system is:

$$ {R^{\text{Finite-size}}}(\eta ,N) \ge \frac{{{K^{\mathrm{OAM},L}}}}{N}. $$

(C4)

Consequently, the formula for the security key rate of the system with finite-size effects can be recalculated:

$$ {R^{\text{Finite-size}}} = \int _{0}^{1} {{R^{\text{Finite-size}}}(\eta ,N)P( \eta )\,d\eta }. $$

(C5)

Figure 11 examines the security key rate under finite-size effects. The figure presents a comparison of key rate curves across various data sizes and asymptotic ($N = \infty$) conditions. It is evident from the illustration that as the data size gradually increases, the security key rate follows suit. When the number of quantum pulses N reaches or exceeds 10¹², the key rate curve closely aligns with the asymptotic case. Taking into account practical implementation and cost optimization, we can select $N = {10^{12}}$ as the pulse transmission scheme for the HD-QKD system in practical scenarios to conserve photon resources.

Figure 11

Secure key rate R of Ω with finite-size effects