1 Introduction

Amidst the rapid evolution of the digital landscape, safeguarding digital images has emerged as an imperative task. In today’s interconnected world, where technology pervades every facet of life and industries rely heavily on digital data, ensuring the security of digital assets, especially images, has become increasingly crucial. As businesses and organizations pivot towards digital solutions and embrace digital communication channels, the need for secure data transmission has become more pronounced than ever before [1, 2]. This study introduces an advanced image encryption algorithm meticulously designed to address existing limitations, specifically focusing on pixel location and value preservation. The proposed algorithm is named PanAAVA:Affine Algorithm and Vigenere Algorithm Encryption with PTA-Based Key Generation. Positioned as a pioneering response to the escalating challenges in information security, particularly in image encryption, the PanAAVA seeks to bridge critical gaps in security practices and set new standards for encryption in the digital realm. The characterization of optimal security underscores a steadfast commitment to achieving the highest security standards, particularly in safeguarding sensitive digital data across diverse industry sectors. The successful journey of the recommended algorithm through in-depth study reveals the capabilities to overcome existing limitations and elevate security standards, particularly within various industry sectors.

By integrating advanced methods and algorithms such as the Pan-Tompkins Algorithm (PTA), Zigzag scanning, the Affine Image Encryption Algorithm (AA), and the Vigenere Image Encryption Algorithm (VA), this algorithm is designed to be versatile and applicable across various. Leveraging PTA for key generation and incorporating the Least Significant Bit (LSB) stenography method enhance security effectiveness, minimize visual distortion, and streamline secure key distribution processes.

This study meticulously identifies fundamental limitations in prevailing encryption methodologies and proposes effective solutions through a comprehensive analysis tailored to the specific security needs of industry applications. The successful journey of the recommended algorithm through in-depth scrutiny underscores the ability to overcome existing limitations and elevate security standards, particularly within various industry sectors. In essence, the significant advantages of the new algorithm, showcasing the pioneering effect in overcoming the limitations of current encryption approaches, are summarized as follows:

  • Current encryption methods and algorithms often require improvements, particularly in preserving the integrity of pixel positions and values throughout the encryption process. This study introduces an innovative image encryption algorithm that offers enhanced effectiveness and security to address the limitations observed in the PanAAVA.

  • The primary focus of this proposed encryption algorithm centers on presenting an innovative image encryption algorithm that seamlessly integrates advanced methods and algorithms, including Zigzag scanning, AA, VA, and LSB.

  • Furthermore, the PTA is utilized for key generation for the first time in the literature. Steganography is introduced to the image encryption algorithm by incorporating LSB and keys into the image. This enhancement results in a more robust encryption.

  • The meticulous presentation and comparison of all encryption results and thorough analysis contribute to establishing a robust security representation for the algorithm. Undoubtedly, this study positions the proposed algorithm as a trailblazer, revealing distinctive and optimal security capabilities and innovations in image encryption.

  • Additionally, PanAAVA is designed to be compatible with quantum computing systems. Therefore, the potential of PanAAVA for significant future developments in quantum computing is demonstrated.

The study unfolds in several sections. Section II presents a literature review. Section III provides a concise overview of essential methodologies, including AA, VA, LSB, PTA for R-Peak Detection, and Zigzag scanning, setting the foundation for the encryption algorithm. Section IV delves into a comprehensive explanation of the proposed encryption algorithm. The PanAAVA’s performance is scrutinized in Section V, evaluating the effectiveness across diverse color images in terms of security and efficiency. Section VI examines the test results, discusses potential impacts and applications, and emphasizes the algorithm’s contribution to the advancement of image encryption. The study concludes by summarizing the findings and providing observations in the final section.

2 Literature review

The burgeoning importance of security in digital environments catalyzed the evolution of various techniques, prominently featuring pixel permutation, pixel substitution, and image encryption employing Gray code. Notably, a noteworthy advancement in encryption algorithms elevated the role of the Affine Image Encryption Algorithm (AA) by seamlessly integrated Gray code-based bit-plane analysis with the pixel permutation methodology, thereby highlighting the pivotal role of AA in shaping and fortifying encryption strategies [3]. In hyperspectral data encryption, an innovative approach was devised utilizing the Fractional Fourier Transform (FrFT) framework. This method entailed segmenting data into bands, applying AA, and encrypting within the FrFT domain, leveraging AA and FrFT parameters to ensure security in both spatial and spectral data [4]. Several encryption algorithms were introduced, including the Random Matrix Affine Algorithm (RMAA), 2-dimensional Fractional Hartley Transform (RP2DFrHT), and 2-dimensional Arnold maps. RMAA enhanced security in coordinate and geometric domains, while RP2DFrHT simplified complexity, yielding digitally suitable real-valued encrypted data. Integration of the 2D Arnold map further fortified security and expanded the key space [5]. Another significant contribution was the Improved Affine Algorithm (IAA), which combined AA, Linear Feedback Shift Register (LFSR), and XOR encryption algorithm [6]. Additionally, a color image encryption algorithm incorporated a 2D Hénon map, a three-dimensional logistic map with XOR operation, and AA augmented system security [7]. Further diversification was evident in a hybrid image encryption algorithm, amalgamating traditional methods with an extended-dimensional chaotic map, enhancing security measures. Comparative analysis of optimization results for transposition methods against classical algorithms yielded improved image encryption outcomes [8]. Improved VA schemes and chaotic permutations were explored in various studies to address recurring key issues and develop innovative password-strengthening strategies [9]. One study combined VA with Huffman Encryption to devise a robust encryption algorithm, segment images into blocks, and intricately embed data in pixel group [10]. Real-time image encryption employed modified VA and chaotic maps to extend 8-tuple keys to 8 × 8 blocks, encrypting pixel blocks through XOR operations, Arnold transformations, and Baker maps [11]. Another encryption algorithm for RGB images used unique S-box arrangements over a Galois field, while AA scrambled image data, further strengthening security [12]. A novel image encryption algorithm was introduced in this study, utilizing the improved the VA alongside LSB method [13]. Another study proposed a hybrid method derived from VA and Beaufort cipher algorithm modifications. This method utilized two random keys: one composed of 8-bit integer values and the other consisting of binary values [14]. As proposed in another study, the algorithm significantly improved traditional VA encryption by two steps. Leveraging advanced dynamic permutation matrices and chaotic maps established robust encryption [15]. In another study, the combined usage of Hash Least Significant Bit (H-LSB) and VA methods was proposed to enhance image security. Additionally, a system incorporating a multi-layered security technique utilizing steganography was suggested [16]. In the literature review focusing on Zigzag scan, a color image encryption algorithm was developed using a CNN chaotic system. The algorithm introduced randomness by utilizing encrypted key matrices generated with LFSR. Combining DNA encoding, bit-plane decomposition, and high-bit-plane Zigzag scan achieved robust security [17]. The proposed alternative algorithm facilitated image encryption using a complex, chaotic system and changed pixel positions through the zigzag scan. The robust ergodicity, complex, chaotic behavior, and stable Lyapunov exponential spectrum inherent in the chaotic system presented [18]. Upon scrutiny of LSB literature, an alternative algorithm was proposed, amalgamating Advanced Encryption Standard (AES), Rivest-Shamir-Adleman (RSA), and LSB methodologies [19]. A dual-layered approach was employed in a separate study, initially encoding a digital image using LSB steganography, followed by an extra layer of security through AES encryption [20]. Furthermore, a study focused on enhancing data transmission speed by selectively encrypting bits in the Triple Data Encryption Standard (T-DES), ensuring security and performance [21]. The recent studies in image encryption are summarized in Table 1.

Table 1 Some recently developed image encryption algorithms
Full size table

3 Background

This section examines The Affine Image Encryption Algorithm (AA), The Vigenere Image Encryption Algorithm (VA), Least Significant Bit (LSB) embedding, the Electrocardiogram and the Pan-Tompkins Algorithm (PTA) for R-Peak Detection, and the Zigzag scan.

3.1 Affine image encryption algorithm

The Affine Image Encryption Algorithm (AA) is utilized for plaintext encryption, altering pixel values and pixel locations within the image [27]. Equation 1 utilizes the AA to encrypt each pixel in the image.

$$\begin{array}{cc}C=\left(a\times p+b\right)\,\text{mod}\, 256,& 0\le p\le 255\end{array}$$
(1)

where \({\text{C}}\) represents the encrypted pixel value, p denotes the original image’s pixel, and \(a\) and \({\text{b}}\) are the encryption keys. The value 256 signifies the maximum of a pixel in an 8-bit grayscale image. The decryption process employs the subsequent formulas:

$$\begin{array}{cc}p=\overline{a}\times \left(c-b\right)\,\text{mod}\, 256,& 0\le p\le 255\end{array}$$
(2)
$$\overline a\equiv a^{\mathrm\varphi\left(256\right)-1}\,\text{mod}\,256$$
(3)

where \(\overline{a}\) represents the modular multiplicative inverse of \({\text{a}}\) modulo 256, to compute the modular inverse of a modulo 256, Euler’s totient function, designated as φ(256), is employed. φ(256) provides the count of positive integers less than 256 that are coprime with 256. In simpler terms, \({\text{p}}\) determines the number of integers between 0 and 255, not share any common factors with 256 [28]. Euler’s totient function for 256, φ(256). The modular inverse of a concerning 256 is determined by raising a to the power of φ(256)-1 and subsequently taking the modulus 256 of the result.

3.2 The vigenere image encryption algorithm

The Vigenere Image Encryption Algorithm (VA) is a polyalphabetic cipher used for encryption. VA employs a tabular form, the VA table, consisting of elements. The first row contains \(n\) different elements, while subsequent rows are formed by left cycle shifting of the elements from the previous row. Decryption involves looking up the ciphered element in the row corresponding to the key element, with the column representing the decrypted output or the original letter. Originally developed for encrypting alphabetical texts, recent studies have adapted the VA concepts for image encryption [29]. The VA encryption and decryption of an image matrix of order \({2}^{{\text{N}}}\). The encryption equation used in the VA is as follows:

$$a\,(\text{M},\text{K})=\text{C}=\text{M}+\mathrm K\,\mathrm m\mathrm o\mathrm d\,2^\text{n}=\left[{\text{m}}_\text{ij}+{\text{k}}_\text{ij}\right]\left(\text{mod}\,2^\text{n}\right)$$
(4)

where \(a\) represents the encryption image, \({\text{i}}\) and \({\text{j}}\) denote the image block’s \({\text{i}}\)-th row and \({\text{j}}\)-th column, \({\text{M}}\) and \({[{\text{m}}}_{{\text{ij}}}]\) represent the original image matrix, \({\text{K}}\) and \({[{\text{k}}}_{{\text{ij}}}]\) represent the key matrix, \({\text{C}}\) and \({[{\text{c}}}_{{\text{ij}}}]\) denote the encryption matrix, with \({{\text{c}}}_{{\text{ij}}}\) representing elements, and \({2}^{{\text{N}}}\) refers to the size matrix. Equation 5 is used for decryption by the VA.

$$\overline{a }(C,K)=M=C-K\,\text{mod}\,{ 2}^{n} = \left[{c}_{ij}-{k}_{ij}\right] \left(\text{mod}\,{2}^{n}\right) = \left[{c}_{ij}+{2}^{n}-{k}_{ij}\right] \left(\text{mod}\,{2}^{n}\right)$$
(5)

where \(\overline{a }\) represents the decryption image, \({\text{n}}\) is a positive integer [11].

3.3 Least significant bit steganography

Steganography, an ancient method dating back to ancient Greece, involves concealing information within digital media while preserving the overt characteristics of the media. Typically, steganography uses a cover image and a hidden message, creating a stego image. One common method, Least Significant Bit (LSB) insertion, alters pixel values to hide the embedded message [30]. The length of the encryption key plays a critical role in ensuring security, with longer keys providing greater protection. In this process, the key encrypts the message, and the encrypted key is subtly incorporated into the RGB values of the cover image using LSB steganography. This method ensures that the alterations are imperceptible, making detection challenging. Only authorized recipients possessing the decryption key can extract the hidden message, ensuring high levels of security and confidentiality. The encryption key is indispensable in LSB steganography, guaranteeing the confidentiality and security of the transmitted message. During the embedding process, message bits replace the LSBs of pixels, randomly or sequentially selected from the cover image [31]. Equation 6 elucidates the method of utilizing LSB to embed data.

$$y_i=2\left(\frac{{\text{x}}_i}2\right)+{\text{m}}_i$$
(6)

where \({{\text{x}}}_{{\text{i}}}\) and \({{\text{y}}}_{{\text{i}}}\) are the \({\text{i}}\)-th chosen pixel values before and after embedding, and \({{\text{m}}}_{{\text{i}}}\) stands for the \({\text{i}}\)-th message bit.

3.4 The electrocardiogram and The Pan-Tompkins Algorithm for R-Peak detection

The Electrocardiogram (ECG) is a fundamental tool in monitoring cardiac health, capturing the heart’s electrical activity through a waveform consisting of key components like the P wave, QRS complex, and T wave. The QRS complex holds particular significance among these components, representing ventricular depolarization events. To precisely detect QRS complexes and identify R-wave peaks, crucial in clinical diagnosis, the Pan-Tompkins Algorithm (PTA) emerges as a pivotal solution. Tailored for ECG signal analysis, PTA employs a series of filters to enhance statistical characteristics and minimize noise, facilitating accurate interpretation of ECG data [32]. The PTA is widely used for detecting R-peaks in ECG signals. The PTA consists of several stages:

  • 1. Band-pass Filtering: The ECG signal is initially filtered using a band-pass filter to remove unwanted noise and interference while preserving the essential components of the signal within the frequency range associated with the QRS complex.

  • 2. Differentiation: Following filtration, the signal undergoes differentiation to amplify the steep slope characteristic of the QRS complex. This process enhances the detection of rapid voltage changes associated with the depolarization of the ventricles.

  • 3. Squaring: The squared signal is computed to accentuate the amplitude of the QRS complex. Squaring the signal emphasizes the high-energy components of the QRS complex. This stage aims to increase the distinguishability of the QRS complex from other components present in the ECG waveform.

  • 4. Integration: The squared signal undergoes integration over a specified window to obtain an average value. Integration smooths out the signal and highlights the areas with sustained high energy, such as the QRS complex.

  • 5. Thresholding: A threshold is established to distinguish QRS complexes from background noise. Signals exceeding this threshold are identified as potential QRS complexes, with the R-peak being the highest point within each complex.

  • 6. Refractory Period: A refractory period is implemented to prevent the detection of multiple R-peaks within a brief time interval. Additional R-peaks are ignored during this period, ensuring that only the most prominent peak is detected.

In summary, the PTA serves as a pivotal tool in precisely identifying QRS complexes, especially R-wave peaks, within ECG signals.

3.5 Zigzag scan

The Zigzag scan, recognized for the contribution to enhanced compression and transmission efficiency in image and video processing, extends the significance into image encryption, a pivotal component in safeguarding sensitive content. While the primary role is to optimize compression and transmission, including image encryption introduces an additional layer of complexity, making it challenging for unauthorized individuals to decrypt images and access confidential information [33].

A Zigzag scan is a process whereby the elements of an image are sequentially scanned in a specific pattern during processing. This method contributes to the efficient utilization of processing resources by enabling effective handling of image data. The process of performing parallel zigzags from top to bottom and right to left involves representing the image in matrix form. Subsequently, the elements of this matrix are processed sequentially following a predetermined zigzag pattern. Executing this process in parallel involves simultaneously processing different segments using multiple processing units or parallel processing techniques. Such an approach reduces processing time while optimizing processing power usage. The zigzag scan is commonly favored in fields such as image processing, video coding, and related domains.

4 Proposed algorithm

The intricately designed PanAAVA consists of five stages, each meticulously crafted to fulfill a specific purpose aimed at robust and secure image encryption. PTA takes center stage in the initial phase by expertly identifying R-peaks within an ECG signal. These identified points form the basis for generating unique and random cryptographic keys, paving the way for subsequent encryption processes. This innovative approach provides a durable foundation for the security architecture of the algorithm, enabling the derivation of highly secure keys. In the second stage, the RGB layers of the image are passed through a meticulous Zigzag scanning process. Characterized by sophistication, this step not only increases the efficiency of encryption and decryption but also significantly increases the overall security and usability of the algorithm. In the third and fourth stages of the algorithm, AA and VA-based encryption algorithms are introduced, respectively. The fifth and final stage of the algorithm involves inserting the keys obtained from R-peak detection into the encrypted image using the LSB method. Figures 1 and 2 provide a comprehensive overview of the algorithmic steps by providing understandable visual representations of the encryption and decryption processes. The PanAAVA is a testament to the effectiveness of combining innovation with security in various applications.

Fig. 1
figure 1

The block diagram of the PanAAVA

Full size image
Fig. 2
figure 2

The block diagram of the PanAAVA

Full size image

  • (i) Generation of the Proposed Algorithm’s Keys

  • In this study, the PTA is specifically chosen for ECG data due to the proposed algorithm’s unparalleled accuracy in QRS detection, robustness against noise, real-time processing capabilities, documented success in medical applications, simplicity, computational efficiency, and adaptability to signal nuances. PTA’s flexibility in processing a variety of signal sources, together with PanAAVA’s proven effectiveness in extracting necessary information from ECG data, highlights the algorithm’s applicability in a wide range of scenarios. This study utilizes the PTA to generate image encryption keys in a proposed algorithm by leveraging an ECG-based key dataset. With the precision of the PTA, 256 unique R-peak positions are extracted and seamlessly integrated as encryption keys into the AA and VA for a selected image. This integration represents a significant advancement in image encryption algorithms. The PanAAVA effectively secures sensitive image data while preserving image quality, offering a robust and effective solution for image encryption.

  • (ii) Zigzag Scan

  • The PanAAVA elevates the security of color image encryption by integrating a Zigzag scan, a pivotal step that significantly enhances the encryption process. The algorithm ensures comprehensive coverage and robust image data protection through individual scanning of the RGB layers. The algorithm generates input data for the subsequent encryption phases meticulously outlined by applying the Zigzag scan to the original image. This multifaceted encryption PanAAVA encompasses Zigzag scanning, pixel location, and pixel value encryption, each contributing to the overall security framework. By incorporating diverse methods and different algorithms, the proposed algorithm fortifies defenses against potential threats, ensuring the utmost confidentiality and integrity of the encrypted image data.

  • (iii) AA-based Pixel Value and Location Encryption with R-peak Value Key

  • The AA encryption in the PanAAVA ensures robust image security through a two-step process. Initially, RGB layers are independently encrypted for pixel locations using key sets (\({a}_{1}\), \({b}_{1}\)) and (\({a}_{2}\), \({b}_{2}\)), thereby adding an extra layer of security. In the subsequent stage, pixel values are encrypted using six unique keys: (\({a}_{3}\), \({b}_{3}\)), (\({a}_{4}\), \({b}_{4}\)), (\({a}_{5}\), \({b}_{5}\)). This process significantly increases decryption complexity, providing a strong defense. The initial 10 distinct 256 keys derived from the ECG signal R-peaks for AA encryption are strategically selected. This approach maintains complexity and reliability while minimizing computational and storage requirements. Incorporating the 10 distinct 256 keys derived from the ECG signal’s R-peaks for AA encryption is pivotal, enhancing both effectiveness and efficiency and rendering the algorithm ideal for applications necessitating critical image security.

  • (iv) VA-based Pixel Value and Pixel Location Encryption with R-peak Value Key

  • 256 keys derived from the ECG signal’s R-peaks, both pixel locations and values undergo encryption, significantly bolstering the algorithm’s security. The utilization of the R-peak value key for VA-based encryption introduces a level of unpredictability and complexity crucial for applications emphasizing image security. The choice to utilize 256 encryption keys, an additional layer of security, is introduced through VA-based encryption. Employing the PTA is strategic, as employing the PTA establishes a vast key space that effectively thwarts brute force attacks and ensures robust protection for encrypted data. This decision adheres to industry standards, as 256-bit encryption is widely acknowledged for robust security features. The selected key length creates a computationally infeasible environment for attackers attempting brute-force attacks. Consequently, opting for 256 encryption keys enhances the overall security posture of the algorithm, providing a high level of protection against potential threats and vulnerabilities for the encrypted data.

  • (v) Key Steganography with LSB

  • In the culminating step of the proposed algorithm, paramount attention is directed towards the secure embedding of keys, a critical element ensuring the confidentiality and reliability of the transfer process. The PanAAVA employs Eq. 5 as a foundational framework, leveraging a sophisticated stenographic approach with the LSB to integrate 256 distinct keys seamlessly. This advanced method provides a robust and highly effective means of concealing cryptographic keys within the fabric of images, ensuring the utmost level of security. Integrating steganography with the LSB method renders the embedded keys virtually indiscernible, thereby fortifying the concealment against potential intruders.

Figure 3 provides a comprehensive visual representation of the pseudo-code encapsulating the intricacies of the proposed algorithm. The utilization of steganography, particularly through LSB manipulation, underscores the algorithm’s commitment to encryption and the covert integration of keys, enhancing the overall security posture against adversarial threats.

Fig. 3
figure 3

The decryption block diagram of the PanAAVA

Full size image

5 Results and performance analysis of the proposed algorithm

This study implemented the PanAAVA using the MATLAB 2022(a) programming language. Encryption keys are found by utilizing the PTA to find the R-peak locations of the ECG images from a large dataset. This dataset, compiled from the Ch. Pervaiz Elahi Institute of Cardiology in Multan, Pakistan, is publicly accessible through this link: (https://data.mendeley.com/datasets/gwbz3fsgp8). Specifically, images labeled MI (111), MI (114), and MI (128) from the "ECG Images of Myocardial Infarction Patients (240 × 12 = 2880)" file are chosen within the study’s framework. The encryption process integrated the extracted ECG keys into the algorithms and Lena, Baboon, Airplane, and Pepper. After encryption, the resulting encrypted images underwent comprehensive comparative and evaluative analyses. The PanAAVA’s effectiveness is assessed through meticulous evaluations against Lena, Baboon, Airplane, and Pepper images alongside established encryption methods operating under similar conditions. The study thoroughly documents the complete results of the algorithmic analysis. Figure 4 visually illustrates the evaluation process, encompassing original, encrypted, and decrypted images, providing a holistic evaluation of the PanAAVA’s performance.

Fig. 4
figure 4

Key images for PanAAVA and (a) Original image; (b) Encrypted image; (c) Decrypted image

Full size image

The study’s results unequivocally affirm the PanAAVA’s success in various aspects. The meticulous evaluation process, as showcased in Fig. 6, consistently demonstrates the algorithm’s exceptional performance and robustness. The encryption algorithm, ingeniously incorporating ECG images as encryption keys, has proven to be a masterstroke. This innovative study enhances the security of the encryption process and introduces an additional layer of complexity that significantly bolsters the algorithm’s efficacy. The successful decryption of the encrypted images back to the original states is a testament to the reliability and excellence of the algorithm. The fidelity with which the decrypted images match the unencrypted counterparts showcases the algorithm’s proficiency in preserving image quality while ensuring utmost security.

5.1 Histogram analysis

Histogram analysis is important for understanding an image’s pixel values and distribution. In this study, histogram analysis is employed to evaluate the vulnerability of encrypted images to hacking attempts by examining the histograms’ frequency data and statistical characteristics. A uniform distribution of pixel intensities in the histogram analysis of encrypted images signifies the strength of the encryption algorithm used. The study utilizes histogram analysis plots to visualize the pixel density of the original and decrypted images in both grayscale and color formats. Typically, the pixel intensities in the input image’s histogram exhibit a non-uniform distribution. However, in an effective encryption algorithm, the histogram of the encrypted image should demonstrate a uniform distribution to withstand statistical analysis attacks [34]. Figure 5 in the study showcases the histograms of the original and decrypted images, providing visual representations of the distribution of pixel intensities.

Fig. 5
figure 5

Histogram analysis (a) Original image; (b) Encrypted image; (c) Decrypted image

Full size image

Histogram analysis of the images presented in Fig. 5(a), (b), and (c) provides valuable insights into the performance of the proposed encryption algorithm. The original and decrypted images (Fig. 5(a) and (c)) exhibit a non-uniform distribution of pixel values, with the histogram showing certain clusters of pixel values. The fluctuating shape of pixel density in the histogram, observed during the analysis of the original and decrypted images, signifies variations in pixel values. In contrast, the histogram of the encrypted image shown in Fig. 5(b) demonstrates a uniform distribution of pixel values, with histogram pixels having equal heights distributed evenly across the entire range of pixel values. The flat distribution of pixel values in the encrypted image histogram makes extracting meaningful information from the image extremely challenging for unauthorized users. Thus, the suggested encryption technique effectively protects the information’s secrecy while keeping the image’s integrity. Moreover, the PanAAVA safeguards sensitive information. PanAAVA serves as a testament to the ingenuity and dedication driving advancements in data security, ensuring that privacy and integrity remain steadfastly protected in the face of evolving threats.

5.2 Entropy analysis

Information entropy analysis is used in information theory to measure the amount of information. The entropy is \({\text{N}}\) for a true random source of 2N symbols [35]. The expression expresses the entropy calculation procedure in Eq. 7.

$$H\left(s\right)=\sum\nolimits_{i=0}^{2^W-1}p\left(s_i\right)\times{log}_2\left(\frac1{p\left(s_i\right)}\right)$$
(7)

where \(\text{p(}{\text{s}}_{\text{i}}\text{)}\) represents the frequency of symbol \({\text{s}}_{\text{i}}\), \({\text{w}}\) represents the total number of symbols \({\text{I}}\), and the column by \({\text{j}}\) represents the row. With 28 possible symbols, a completely random source would produce those symbols with a uniform distribution. The predicted entropy result is 8 (H(s)≈ 8) [29]. Table 2 of this study’s results of the entropy analysis is displayed. Table 3 contrasts the suggested encryption algorithm’s entropy analysis with existing algorithms.

Table 2 Entropy analysis
Full size table
Table 3 Comparison of entropy analysis
Full size table

The proposed algorithm demonstrates exceptional proficiency in analyzing and processing images, earning consistently high scores across the Lena, Baboon, and Pepper images. With scores of 7.9997 for Lena, 7.9996 for Baboon, and 7.9994 for Pepper, showcasing remarkable precision and effectiveness. Other studies [36,37,38] showcase commendable endeavors in the field of image processing; however, the PanAAVA consistently surpasses the other studies, exhibiting superior performance with higher scores across all assessed images. The algorithm’s ability to maintain superiority is evident in consistently elevated performance. Other studies [39,40,41] also contribute to the field, albeit with slightly lower scores than the proposed algorithm. The latter, with scores of 7.9991 across Lena and Baboon images, indicates a commendable performance but needs to catch up to the precision exhibited by the PanAAVA. In conclusion, the PanAAVA is an exemplary solution, consistently achieving top scores in image processing across all evaluated images. Unparalleled precision and unwavering consistency underscore the algorithm’s potential as an advanced and highly effective tool in image analysis.

5.3 Differential attack analysis

The Unified Average Changing Intensity (UACI) and Number of Pixels Change Rate (NPCR) are two metrics used to evaluate the performance of image encryption algorithms, particularly the resistance against differential attacks [28]. Higher NPCR and lower UACI values indicate better resistance to differential attacks. Higher NPCR and lower UACI values indicate better resistance to differential attacks. UACI is calculated using Eq. 8.

$$UACI=\frac{1}{M\times N}\left[{\sum }_{i=1 j=1}^{i=Mj=N}\frac{\left[C\left(i,j\right)-\acute{C}\left(i,j\right)\right]}{255}\right]\times 100\mathrm{\%}$$
(8)

where C represents the encrypted image of the original image, and \({\text{C}}^{\prime}\) represents the encrypted image corresponding to the original image after a minor alteration. UACI involves calculating the absolute difference between the intensity values of corresponding pixels in the original and encrypted images, normalizing by dividing by 255 (the highest intensity value for an 8-bit image) [40]. UACI is a metric commonly used to evaluate the quality of image encryption algorithms, with lower values indicating higher encryption strength. Equation 9 is used to calculate the \({\text{D}}\) parameter.

$${D}_{R,G,B}\left(i,j\right)=\left\{\begin{array}{cc}0& if\,C\left(i,j\right)=\acute{C}\left(i,j\right)\\ 1& if\,C\left(i,j\right)\ne \acute{C}\left(i,j\right)\end{array}\right.$$
(9)

where \({\text{D}}_{\text{R,G,B}}\left(\text{i, j}\right)\) is a binary value indicating whether a corresponding pixel in the encrypted image differs from the original image. Specifically, \({\text{D}}\) is set to 0 when the intensity levels of the relevant pixels are identical, and \({\text{D}}\) is set to 1 when they differ. These metrics are employed to assess the pixel change rate and average density difference between the original and encrypted images, respectively, as part of evaluating the robustness of an encryption algorithm against differential attacks [40]. NPCR is further calculated using Eq. 10.

$$NPCR=\frac{{\sum }_{i=1 j=1}^{i=Mj=N}D\left(i,j\right)}{M\times N}\times 100\%$$
(10)

where the intensity levels of corresponding pixels are aligned, an indicator of whether the corresponding pixel in the encrypted image deviates from the original image is denoted as \({\text{D}}\). Specifically, \({\text{D}}\) is assigned the value 0 inequality cases. In contrast, the \({\text{D}}\) parameter is assigned the value 1 in cases of inequality. The \({\text{D}}\) parameter assumes a critical role in the computation of NPCR and UACI, both of which are fundamental metrics utilized in assessing the robustness of an encryption algorithm against potential differential attacks [40]. Table 4 shows the NPCR and UACI results of the PanAAVA in detail. Additionally, Table 5 compares the NPCR and UACI results between the proposed algorithm and others.

Table 4 The proposed algorithm’s UACI and NPCR results
Full size table
Table 5 UACI and NPCR results of the proposed algorithm and other algorithms
Full size table

The PanAAVA excels in various key performance metrics, particularly in terms of NPCR and UACI, showcasing outstanding capabilities in image processing. For NPCR, the proposed algorithm achieves remarkable percentages across Lena, Baboon, and Pepper images, with scores of 99.751%, 99.7503%, and 99.7323%, respectively. The high NPCR percentages indicate the proposed algorithm’s ability to induce significant pixel changes, demonstrating the effectiveness in preserving image content while introducing subtle alterations. Concerning UACI, the PanAAVA consistently achieves excellence, yielding 33.3755%, 33.4198%, and 33.417% for Lena, Baboon, and Pepper images, respectively. The low UACI values underscore the algorithm’s proficiency in minimizing the average change intensity, emphasizing the prowess in maintaining image fidelity during processing. Comparatively, when evaluating other algorithms [37, 38, 41,42,43], the PanAAVA consistently outperforms them in both NPCR and UACI metrics. This superior performance across various image types highlights the proposed algorithm’s robustness and adaptability. The PanAAVA is an exemplary solution, achieving exceptional NPCR and UACI percentages across diverse images and the ability to induce meaningful pixel changes while preserving image integrity positions as a highly effective algorithm in image processing.

5.4 Mean square error

Mean Square Error (MSE) is a widely utilized measure for quantifying the discrepancy or error between two images. MSE is computed using the average squared differences between corresponding pixel values in the original and decrypted (or encrypted) images. A lower MSE value signifies a smaller disparity between the original and decrypted images, generally considered more desirable. Equation 11 represents the formula for calculating MSE.

$${MSE}_{R,G,B}=\frac{1}{M\times N}{\sum }_{i=1}^{M-1}{\sum }_{j=1}^{N-1}\left[{\left|{C}_{\text{R,G,B}}\left(m\Delta x, n\Delta y\right)-{{\acute{\text{C}}}}_{\text{R,G,B}}\left(m\Delta x,n\Delta y\right)\right|}^{2}\right]$$
(11)

where \({\text{C}}_{{\text{R}},{\text{G}},{\text{B}}}\), the decrypted image by \({{\acute{\text{C}}}}_{\text{R,G,B}}\), the image size by \({\text{M}}\times {\text{N}}\), \(\Delta {\text{x}}\), and \(\Delta {\text{y}}\) by the pixel size. Suppose the MSE values of the proposed algorithm are lower than others. In that case, the proposed algorithm better minimizes the differences between the original and decrypted images, indicating higher success in image encryption [6]. Table 6 displays the MSE results of the proposed and other algorithms.

Table 6 MSE results (%)
Full size table

In particular, the proposed algorithm achieves lower Mean Squared Error (MSE) values across the Lena, Baboon, Airplane, and Pepper images compared to the values reported in the other algorithms [39, 44, 45]. Compared with the algorithm [44], the PanAAVA minimizes Mean Squared Error (MSE) values, demonstrating superior performance in maintaining color channel coherence and reducing errors. Similarly, when contrasted with [45], the proposed algorithm generally outperforms, showcasing the ability to achieve lower MSE values, ensuring higher fidelity in color channels. In another study [39], the PanAAVA exhibits a significant advantage, especially in the Baboon and Pepper images. This highlights the proposed algorithm’s effectiveness in achieving superior color channel alignment and minimizing errors in specific images. In summary, the PanAAVA consistently outperforms other studies regarding MSE values, affirming proficiency in image processing tasks and the capability to deliver high-quality results.

5.5 Peak signal to noise ratio

The Peak Signal to Noise Ratio (PSNR) is a widely employed metric for quantifying the ratio, expressed in decibels (dB), between the maximum potential power of a signal, in this case, the original image, and the power of the difference between the original and either encrypted or decrypted images. A higher PSNR value signifies a superior quality image, whereas the opposite holds. Equation 12 delineates the formula for PSNR [46].

$$\text{PSNR(}{{{\acute{\text{I}}}}_{({\text{R}},\mathrm{ G},\mathrm{ B}}, {{{\text{I}}}_{{\text{R}},{\text{G}},{\text{B}}}}_{ }}_{ }\text{) = 10}\;{\text{log}}_{10}\left(\frac{\left({\text{M}}-1\right)\times \left({\text{N}}-1\right)}{{\sum }_{\forall {\text{M}},\mathrm{ N }}^{ }\left[ {\acute{\text{C}}}_{{\text{R}},{\text{G}},\mathrm{B }}\left({\text{M}},{\text{N}}\right) - {{{\text{C}}}_{{\text{R}},{\text{G}},{\text{B}}}}_{ }\left({\text{M}},{\text{N}}\right) \right]{ }^{2}}\right)$$
(12)

where \(\acute{\text{I}}_{\left(\text{R,G,B}\right)}\) is the original image represented in RGB format, and \({\mathrm I}_{\mathrm R,\mathrm G,\mathrm B}\) is the decrypted (or encrypted) image. \({\text{M}}\) and \({\text{N}}\) are the image’s dimensions, corresponding to the number of rows and columns. \({{\text{C}}}_{{\text{R}},{\text{G}},\mathrm{B }}\left({\text{M}},{\text{N}}\right)\) is the RGB color value for the pixel at location \(\left({\text{M}},{\text{N}}\right)\) in the decrypted (or encrypted) image. \({{{\text{C}}}_{{\text{R}},{\text{G}},{\text{B}}}}_{ }\left({\text{M}},{\text{N}}\right)\) is the RGB color value of the original image’s pixel at location \(\left({\text{M}},{\text{N}}\right)\) [50]. Table 7 presents the PSNR analysis of the proposed algorithms and other algorithms.

Table 7 PSNR analysis of the proposed algorithm and other algorithms
Full size table

For the Lena image, the PanAAVA achieves an outstanding Peak Signal-to-Noise Ratio (PSNR) of 10.6237, surpassing not only [39, 44, 47, 48] but also [49]. The suggested algorithm demonstrates an exceptional capability to meet, if not surpass, expectations regarding image quality. In the case of the Baboon image, the PanAAVA demonstrates an even higher PSNR of 10.7692, outperforming the counterparts proposed in [39, 44, 47,48,49]. In the case of the Baboon image, the PanAAVA demonstrates an even higher PSNR of 10.7692, outperforming the proposed algorithm counterparts in [39, 44, 47,48,49]. The ability of the given algorithm to succeed in challenging scenarios highlights reliability and versatility. Similarly, in the case of the Pepper image, the PanAAVA demonstrates a noteworthy PSNR of 9.6796, once again outclassing counterparts. This underscores the algorithm’s consistent ability to deliver high signal-to-noise ratios, ensuring images of exceptional quality. In summary, the PanAAVA emerges as a trailblazer, consistently surpassing benchmarks in PSNR and setting new standards in image processing. The PanAAVA success on various images is a testament to the unwavering commitment to excellence.

5.6 Key space analysis

In cryptography, a key is crucial for data encryption and decryption. The key’s strength, directly tied to the key space size, determines the security of encrypted data. A larger key space implies more potential key combinations, making deciphering the data through brute-force methods challenging for attackers. Key space analysis is vital for assessing an algorithm’s resilience against such attacks. The PanAAVA, with secret keys involving parameters like (\(a\), \(b\), \(c\), \(d\)), initial values, and fractional-order system parameters, boasts a key space estimated at around 2209. Substantial key space enhances resistance to brute-force attacks, surpassing the minimum required key space of 2100. At the same time, a larger key space contributes to security by considering additional factors like chaotic map strength and overall implementation for comprehensive encryption security [51, 52]. Table 8 provides a key space comparison using the suggested technique against various algorithms.

Table 8 Key space size comparison
Full size table

In the realm of encryption, the PanAAVA emerges as a standout contender, boasting a substantial key space size of 2209. This numerical prowess is a powerful indicator of the proposed algorithm’s capability to ensure robust encryption, establishing a strong foundation for secure data protection. Close behind, [53] asserts the encryption prowess with a commendable key space 2149, contributing to the competitive landscape of encryption algorithms but failing to outperform the suggested algorithm. Meanwhile, a study [54] with 2128 key spaces demonstrates strong encryption capacity in the competitive environment but still needs to beat the PanAAVA. However, a noteworthy contrast is observed with [55], where the key space significantly dwindles to 294. Additionally, [44] presents a key space 2128, aligning within the range observed among the various algorithms. A comprehensive exploration of the key space spectrum reveals intriguing insights. Additionally, [56, 57] exhibit varying key spaces ranging from 2169 to 2187. The PanAAVA has attained notable success in encryption, showcasing robust capabilities in providing strong encryption and establishing a solid foundation for secure data protection. Compared to other studies, the PanAAVA demonstrates leadership in this domain, surpassing the competitors. Despite the achievements of other algorithms proven in encryption strength, the unquestionable superiority of the PanAAVA remains evident. In conclusion, all results show the proposed algorithm’s resilience and effectiveness in security and data protection.

5.7 Correlation analysis

The correlation analysis, ranging from -1 to + 1, measures the degree of correlation between adjacent pixels in an image. A value of -1 indicates a perfect negative correlation, signifying that as one pixel value increases, the other decreases. Conversely, a coefficient of + 1 indicates a perfect positive correlation, where an increase in one pixel value corresponds to an increase in the other. A correlation coefficient of 0 indicates the absence of a linear relationship between consecutive pixel values [58]. Equation 13 calculates and expresses the horizontal, vertical, and diagonal correlation coefficients between adjacent pixels in an image.

$${C}_{r}=\frac{N\sum_{j=1}^{N} \left({X}_{j}\times {Y}_{j}\right)-\sum_{j=1}^{N} {X}_{j}\times \sum_{j=1}^{N} {Y}_{j}}{\sqrt{\left(N\sum_{j=1}^{N} {X}_{j}^{2}-{\left(\sum_{j=1}^{N} {X}_{j}\right)}^{2}\right)\times \left(N\sum_{j=1}^{N} {Y}_{j}^{2}-{\left(\sum_{j=1}^{N} {Y}_{j}\right)}^{2}\right)}}$$
(13)

where \({\text{N}}\) is the number of adjacent pixels taken from the image to calculate the correlation, and \({\text{X}}\) and \({\text{Y}}\) are the intensities of two nearby pixels in the case of color images. The distribution of correlation analyses in the horizontal, vertical, and diagonal directions and across the RGB channels of Lena’s image is observed in Figs. 6, 7, and 8. Furthermore, in Table 9, a comparative analysis is presented between the correlation results of the proposed algorithm and other algorithms.

Table 9 The correlation analysis findings in comparison
Full size table

The PanAAVA showcases successful performance based on the correlation analysis findings for both the Baboon and Lena images. For the Baboon image, the algorithm reveals the strength in encryption with a low correlation value of -0.0111 in the horizontal direction. This indicates reduced similarity between the encrypted and original images concerning horizontal features, highlighting the algorithm’s secure alteration of the Baboon image’s original structure. Similarly, with a low correlation of -0.0161 in the vertical direction, the algorithm demonstrates decreased resemblance in vertical features post-encryption, showcasing the ability to encrypt the Baboon image vertically while maintaining security. In Lena’s image, the algorithm displays substantial similarity with a high correlation value of 0.0063 in the horizontal direction.

Conversely, with a low correlation of 0.0158 in the vertical direction, the algorithm illustrates diminished similarity in vertical features after encryption, signifying the proficiency in securely encrypting the Lena image vertically. Additionally, with a low correlation value of -0.0022 in the diagonal direction, the algorithm indicates reduced similarity in diagonal features, further emphasizing the ability to alter the original structure in this aspect. In conclusion, based on the correlation analysis results for the Baboon and Lena images, the PanAAVA receives positive evaluations for providing secure and effective encryption. The PanAAVA sets a remarkable standard in cryptographic performance, as evidenced by the correlation analysis results for both the Baboon and Lena images. In the image of Lena, another algorithm [59] falls short with a lower correlation value of 0.0005 in the horizontal direction, starkly contrasting the PanAAVA’s impressive 0.0063. This discrepancy indicates a superior ability of the proposed algorithm to maintain a higher similarity after encryption, showcasing the finesse in preserving horizontal features.

Similarly, in the vertical direction, [59] registers a meager correlation of 0.001, while the PanAAVA boasts a robust 0.0158. This notable difference underscores the PanAAVA’s capacity to significantly alter the vertical features, ensuring heightened security in the encryption process. Overall, the proposed algorithm securely reshapes the original structure, outperforming [59] and establishing the frontrunner in cryptographic efficacy. The comparison with the another algorithm [60] reveals parity in correlation values for Lena in both horizontal and vertical directions, suggesting similar performance in preserving certain features during encryption. However, the real difference emerges in the Baboon image. With lower correlation values in horizontal and vertical directions, the proposed algorithm represents a more secure transformation of the original structure and highlights the superiority of providing robust encryption. For Lena, the another algorithm [61] demonstrates correlation values comparable to the PanAAVA across all directions, indicating similar performance in preserving the integrity of Lena’s features during encryption. However, in the case of the Baboon image, the PanAAVA consistently outshines another algorithm [61] by presenting lower correlation values in both horizontal and vertical directions, underlining the ability to replace the original structure of the Baboon image safely.

As a result, the PanAAVA performs outstandingly in correlation analysis, demonstrating the prowess in providing secure encryption for both Lena and Baboon images. The superiority over alternative algorithms [59,60,61] is evident in the meticulous preservation of image features and the robust transformation of the original structures, underlining the excellence in cryptographic endeavors.

Fig. 6
figure 6

Correlation analysis of Lena’s red layers. (a) Horizontal original image distribution; (b) Horizontal encrypted image distribution; (c) Vertical original image distribution; (d) Vertical encrypted image distribution; (e) Diagonal original image distribution; (f) Diagonal encrypted image distribution

Full size image
Fig. 7
figure 7

Correlation analysis of Lena’s green layers. (a) Horizontal original image distribution; (b) Horizontal encrypted image distribution; (c) Vertical original image distribution; (d) Vertical encrypted image distribution; (e) Diagonal original image distribution; (f) Diagonal encrypted image distribution

Full size image
Fig. 8
figure 8

Correlation analysis of Lena’s blue layer. (a) Horizontal original image distribution; (b) Horizontal encrypted image distribution; (c) Vertical original image distribution; (d) Vertical encrypted image distribution; (e) Diagonal original image distribution; (f) Diagonal encrypted image distribution

Full size image

Figures 6, 7, and 8 exhibit the remarkable efficacy of the proposed algorithm in encrypting the Lena image, yielding a notably high correlation nearing 0. This exceptional correlation analysis serves as a robust validation of the dependability and security inherent in the proposed encryption algorithm. As delineated, the proposed algorithm surpasses anticipated performance levels, showcasing adeptness in upholding both data integrity and confidentiality. The sustained efficacy underscores the superior encryption outcomes facilitated by PanAAVA and its robust architecture. Consequently, PanAAVA emerges as the optimal choice for ensuring data security.

5.8 Structural similarity index measure

The Structural Similarity Index Measure (SSIM) determines the degree of similarity between the cover image and the stego image [62]. SSIM has a value that varies from 1 to + 1. When the cover image and stego image are the same, SSIM equals 1, which is also the ideal value of SSIM [63]. The SSIM formula is given in Eq. 14.

$$\mathrm{SSIM }\left({{\text{I}}}_{1}, {{\text{I}}}_{2}\right)= \frac{(2\overline{{\text{p}} }\overline{{\text{q}} }+ {{\text{c}}}_{1})({2\upsigma }_{{\text{x}},{\text{y}}}+{{\text{c}}}_{2})}{ \left[{(\overline{{\text{p}} })}^{2}+{(\overline{{\text{q}} })}^{2}+{{\text{c}}}_{1}\right] ({{\upsigma }^{2}}_{{\text{x}}}+{{\upsigma }^{2}}_{{\text{y}}}+{{\text{c}}}_{2})}$$
(14)

where \(\overline{{\text{p}} }\) and \(\overline{{\text{q}} }\) are the average pixel values of the cover and stego images, respectively, \({\text{x}}\) and \({\text{y}}\) indicate the standard deviations of the cover image and the stego image, respectively, whereas \({{\upsigma }^{2}}_{{\text{x}}}\) and \({{\upsigma }^{2}}_{{\text{y}}}\) represent the covariance between the cover image and the stego image and \({{\text{c}}}_{1}=2.55\), \({{\text{c}}}_{2}=7.6\) (Yang et al., 2020). The SSIM outcomes of the suggested algorithm are displayed in Table 10.

Table 10 Purposed algorithm’s SSIM
Full size table

In Table 10, the SSIM values, ranging from 0 to 1, denote the degree of similarity to the proposed algorithm, where a score of 1 signifies perfect resemblance. For the Lena image, the proposed algorithm consistently attains high SSIM values across all RGB color channels, registering scores of 0.9995, 0.9995, and 0.9997, respectively. This indicates an exceedingly close correspondence between the original and encrypted images, highlighting the algorithm’s adeptness in preserving both structural integrity and finer details. Regarding the Airplane image, PanAAVA maintains elevated SSIM scores of 0.9899, 0.9899, and 0.9890 for the RGB channels, affirming PanAAVA’s capacity to uphold structural information and visual accuracy. Similarly, concerning the Baboon image, the proposed algorithm demonstrates noteworthy SSIM scores of 0.9992, 0.9991, and 0.9995 for the RGB channels, respectively, underscoring its efficacy in retaining image features and color fidelity. In comparison, [64] presents competitive SSIM values; however, PanAAVA consistently surpasses each channel across all images, indicating its superior capability in preserving image quality across diverse layers.

5.9 National Institute of Standards and Technology Analyzes

The National Institute of Standards and Technology (NIST) analyzes and offers a set of statistical tests designed to evaluate the randomness of binary sequences. These tests analyze the sequences for patterns and randomness. A significance threshold 0.01 is set, indicating a 1% chance of a false positive result. If the P-value of the test exceeds the significance threshold, the sequence passes the test and is considered 99% certain to be random. Conversely, if the P-value falls below the significance threshold, the sequence fails the test and is considered 99% certain to be non-random. The NIST tests are widely used to assess the performance of random number generators and cryptographic algorithms [65,66,67]. In this study, Table 11 displays the NIST test results for the suggested algorithm, showcasing the performance regarding the randomness of the generated sequences.

Table 11 The results of the recommended algorithm’s NIST results
Full size table

Table 11 shows the outstanding performance of the PanAAVA through a comprehensive set of NIST tests that evaluate cryptographic robustness and adherence to randomness standards. The PanAAVA excels in frequency distribution across all channels but particularly shines in the blue channel with a score of 0.94958. In block frequency analysis across RGB channels, the algorithm showcases superior uniformity, with a standout score of 0.93050 in the blue channel, emphasizing effectiveness in maintaining well-balanced block frequencies. A standout feature of the PanAAVA is exceptional performance in avoiding correlations between consecutive sequences, a fundamental requirement for cryptographic applications. With consistently high scores across RGB channels (0.7129), the algorithm demonstrates the capability to thwart predictable patterns, further fortifying cryptographic prowess. The PanAAVA impressively handles spectral domain testing, revealing the capacity to generate diverse frequency components in encrypted data. High scores in the universal test underscore adaptability and performance across various statistical ensembles. Exceptional performance in multiple serial tests attests to effectiveness in resisting predictability in consecutive sequences. A commendable score in the entropy test highlights success in generating sequences with high entropy, a critical attribute for cryptographic applications. Balanced cumulative sum forward and reverse tests underscore cryptographic strength. In summary, Table 11 provides a comprehensive view of the recommended algorithm’s consistent success across various NIST tests, underscoring excellence and suitability for cryptographic applications. The capacity to meet and surpass rigorous criteria establishes the proposed algorithm as an exceptionally effective and dependable option among cryptographic algorithms.

5.10 Next-generation technology and security: quantum computing

Quantum computers leverage the principles of quantum mechanics to surpass classical computers, utilizing qubits in a quantum superposition of 0 and 1. Key Rate indicates the system’s key delivery speed in quantum communication. A higher Key Rate enhances communication efficiency. Quantum Bit Error Rate (QBER) expresses errors; a lower QBER ensures reliability. Asymmetry measures quantum key distribution system performance, contributing to a secure communication environment. Crystal Length and Time of the Day variables are analyzed for system optimization. Analyzing different crystal lengths and time intervals reveals quantum communication system performance. QBER and Asymmetry assess reliability and security. Valuable insights guide future system designs. Quantum communication systems, crucial for secure communication, rely on low QBER and desired asymmetry levels. Data from studies inform researchers, engineers, and decision-makers, aiding quantum technology development [68,69,70]. Tables 12, 13, and 14 detail quantum communication metrics with variable crystal lengths and time of day.

Table 12 Quantum communication performance
Full size table
Table 13 Crystal length, time frame, QBER, and asymmetry levels results
Full size table
Table 14 Key rate and asymmetry levels results
Full size table

The results obtained from the proposed algorithm demonstrate the remarkable efficacy of optimizing key parameters for quantum communication systems. Specifically, for a crystal length of 20 mm during the day, the algorithm achieves a key rate of 47.5 ± 0.8 kHz, coupled with an impressively low QBER of 4.77 ± 0.02 and a well-balanced asymmetry of 49.81 ± 0.02. The algorithm’s adaptability is further evident at night, maintaining a competitive key rate of 50.9 ± 0.7 kHz with consistent QBER and asymmetry values (4.77 ± 0.02 and 50.14 ± 0.03, respectively).

The proposed algorithm’s effectiveness extends to experiments involving a 30 mm crystal length, showcasing the versatility in various scenarios. The proposed algorithm optimally adjusts parameters for daytime experiments, resulting in a key rate of 32.9 ± 2 kHz, QBER of 4.77 ± 0.02, and a balanced asymmetry of 50.1 ± 0.07. Similarly, during nighttime experiments with the same crystal length, the algorithm achieves a commendable key rate of 35.0 ± 3 kHz, maintaining a competitive QBER of 50.06 ± 0.03 and asymmetry of 50.1 ± 0.06. Furthermore, the proposed algorithm showcases the precision by fine-tuning parameters based on the time of day and crystal length. Notably, during the day with a 20 mm crystal length, the algorithm achieves a QBER of 4.8 ± 0.02 and an asymmetry of 50.06 ± 0.09, providing further evidence of the robust performance. Nighttime experiments with the same crystal length yield equally impressive results, with a QBER of 4.8 ± 0.02 and an asymmetry of 50.02 ± 0.12. While some asymmetry values remain unspecified in certain experiments, the proposed algorithm consistently delivers noteworthy key rates and QBER results. Additional asymmetry results further underscore the algorithm’s success, achieving key rates of 53.5 ± 0.4 kHz and 52.5 ± 0.5 kHz with corresponding asymmetry values of 50.2 ± 0.07 and 50.2 ± 0.06, respectively. Although the asymmetry for key rates of 65 ± 2 kHz and 61 ± 3 kHz is not specified, the algorithm’s overall performance is commendable, reflecting the potential for optimizing quantum communication systems in diverse scenarios.

Finally, in the realm of secure communication, quantum communication systems pivot around maintaining low QBER levels and achieving desired asymmetry. The findings of this study furnish invaluable data to researchers, engineers, and decision-makers, propelling advancements in quantum technology.The results gleaned from the proposed algorithm underscore its efficacy in optimizing key parameters for quantum communication systems. Notably, experiments conducted during nighttime maintain competitive key rates and consistent QBER and asymmetry values, underscoring the algorithm’s adaptability. The PanAAVA consistently yields significant key rates and QBER results, reaffirming its success in optimizing quantum communication systems, even in scenarios with unspecified asymmetry values. Furthermore, additional asymmetry results further validate the algorithm’s performance, suggesting promising avenues for enhancing quantum communication systems across diverse scenarios. Overall, the proposed algorithm holds immense potential in bolstering the efficiency and security of quantum communication technologies, thus significantly advancing their real-world applications.

6 Conclusions

This study introduces a novel and effective image encryption algorithm named PanAAVA:Affine Algorithm and Vigenere Algorithm Encryption with PTA-Based Key Generation. PanAAVA has been developed through an innovative approach, incorporating advanced features such as key generation using the Pan-Tompkins Algorithm (PTA), embedding keys within encrypted images using steganography with the Least Significant Bit (LBS) method, and integrating advanced algorithms like Zigzag scanning, Affine Image Encryption Algorithm (AA), and Vigenere Image Encryption Algorithm (VA). These features significantly enhance the security level of the algorithm.

Comprehensive comparative analyses conducted on commonly used images such as Lena, Baboon, Airplane, and Pepper demonstrate PanAAVA’s superior performance in terms of security and encryption strength. These results underscore PanAAVA’s pivotal role in safeguarding sensitive image data and fostering a secure communication environment.

In conclusion, PanAAVA emerges as an exceptional tool for enhancing security standards and safeguarding sensitive image data. Furthermore, the utilization of PanAAVA not only in image encryption but also in quantum communication marks a significant leap forward in secure communication practices. Ultimately, PanAAVA serves as a cornerstone for supporting secure communication protocols. With its innovative features and robust security measures, PanAAVA stands out as a significant advancement in the field, promising versatile applications across various domains.

Future studies will delve deeper into the performance and adaptability of the PanAAVA, paving the way for continued advancements in secure quantum communication technologies. The suggested algorithm for advancing encryption and quantum communication protocols will drive secure communication to continue evolving through cooperative efforts and interdisciplinary research endeavors.