1 Introduction

While technology has many advantages, it may also lead to harassment, violence, and disgrace by encouraging hackers to target computer systems. Concerns regarding cybersecurity and personal security arise as a result of technological innovations’ dual nature [122]. Advancements in Artificial Intelligence (AI) are transforming the role of information security, presenting both opportunities and challenges [8]. This manuscript explores the critical intersection of AI with information security, highlighting how AI technologies such as machine learning can enhance security frameworks and address complex cybersecurity threats [114]. Moreover, the deployment of AI in security applications raises important ethical considerations, necessitating a balanced approach to ensure these technologies are used responsibly and fairly. The objectives of this paper are to review the application of AI technologies in enhancing information security measures, analyze the strengths and limitations of these technologies, and discuss the ethical implications of their deployment. We aim to provide a thorough understanding of the potential of AI to revolutionize security practices, along with the associated risks and ethical concerns. In conclusion, our findings reveal that while AI offers significant benefits for security, such as improved threat detection and adaptive defense mechanisms, it also requires careful consideration of ethical issues, including privacy, bias, and accountability. We propose recommendations for integrating ethical AI practices in security applications, aiming to guide future research and implementation in this field.

1.1 The evolving role of technology and information security

As the Internet and data volume have significantly expanded over the years, the corresponding increase in cyber risks presents a threat to businesses heavily dependent on data. Information security can be defined as the protection of information and information systems from unauthorized access, use, disclosure, disruption, modification, or destruction, aiming to ensure confidentiality, integrity, and availability.Footnote 1 Information is critical in governing and sustaining any organization’s behavior. Information security strategies traditionally deploy both rule-based and manual methods to protect data and systems from threats. Rule-based methods rely on predefined algorithms or protocols that automatically enforce security measures based on specific conditions. For example, a rule-based intrusion detection system (IDS) might automatically block an IP address after detecting five failed login attempts within a minute, highlighting a system-driven, consistent enforcement approach. Conversely, manual methods require human intervention and decision-making, typically involving security personnel actively monitoring potential threats and making decisions based on real-time data analysis and intuition [114]. An example of a manual method would be a security analyst manually sifting through security logs to identify unusual activity, such as an unusually high volume of data transfer happening at odd hours, which might indicate a data breach or an insider threat [77]. This approach benefits from human expertise but is often slower and less scalable than rule-based methods.

It is critical for both enterprises and individuals, to draw a varied range of stakeholders with the aim of preventing the irreversible impacts of rising security concerns [122]. These days, many corporations seek technology services for faster and more efficient processes [9]. AI technologies have significantly streamlined and enhanced various business processes, offering notable improvements in speed and efficiency. A prime example of this transformation is evident in the banking sector. Traditionally, accessing bank accounts and applying for financial services involved time-consuming procedures and in-person interactions. However, with the integration of AI, these processes have become more user-friendly and efficient. For instance, AI-driven chatbots now facilitate 24/7 customer service, allowing customers to check balances, schedule payments, and even apply for loans without human intervention [49]. However, the limitation lies in the traditional information security-related approaches tend to overlook the human factor, assuming systems operate strictly logically, which may pose security challenges [53]. Although advancements in technology have led to faster and more efficient processes, a significant drawback is the frequent oversight of the human factor. This oversight results in vulnerabilities within information security systems, introducing weaknesses that can lead to security challenges not previously considered [61]. Additionally, challenges like the lack of standardization in handling big data complexities and the ever-evolving diverse nature of cyber threats further contribute to complexities in information security [80]. To ensure optimal performance, these systems require robust protection from threats, emphasizing the need to maintain information security as computer and internet usage has risen, and the significance of information security has grown [111]. Consequently, numerous journals, annual conferences, and workshops now focus on the security aspects of information systems and computing. These forums bring together experts in areas such as cryptology [2], computer science, electrical and computer engineering [43], and information systems, acting as meeting places for professionals to contribute and discuss information security concepts. Traditionally, information security relied on rule-based and manual methods [31].

The increase in data volume and complexity of cyber threats exposed the limitations of these approaches, resulting in the disclosure of vulnerabilities to the threat of physical attacks such as breaches and fake identities [4], as well as cyber-attacks such as DDoS [68], phishing [6], and Password Cracking, as well as issues such as Sensor Failure and Budget Failure [22] in the rise of AI. The following Fig. 1 outlines the general life cycle approach to information security, risk assessment identifies potential threats, guiding the development of policies and requirements. Policy development establishes guidelines, followed by control implementation. Continuous monitoring of operations and effective event management ensure a proactive approach to maintaining robust information security measures. The deployment of AI-driven technologies is increasingly crucial in identifying and mitigating information security threats, demonstrating enhanced performance. AI systems, utilizing Machine Learning (ML) algorithms, are adept at examining vast datasets, and identifying anomalies or irregularities indicative of potential security issues. This capability is particularly valuable in addressing complex information security challenges, where traditional security measures might overlook new or sophisticated threats. Additionally, AI significantly improves risk assessment processes. Predictive models, informed by AI, can anticipate potential information security risks based on historical data, thereby aiding in crafting more strategic policies and implementing effective controls. The integration of AI into information security signifies a notable transition from conventional rule-based and manual methods to more proactive, predictive, and automated strategies, effectively meeting the increasing complexity and volume of information security threats.

Fig. 1
figure 1

Information security life cycle

Full size image

Consequently, numerous journals, annual conferences, and workshops now focus on the security aspects of information systems and computing. These forums bring together experts in areas such as cryptology, computer science, electrical and computer engineering, and information systems. Notably, contributions from the USENIX Security Symposium [1, 72] and Privacy Enhancing Technologies Symposium (PETS) have been instrumental in shaping our understanding of the ethical implications and security challenges in AI-driven systems [60]. These venues serve as pivotal meeting places for professionals to contribute and discuss the evolving landscape of information security and ethics.

1.2 AI and information security

AI involves developing computer systems that can execute tasks traditionally requiring human intelligence, like learning, problem-solving, and decision-making. The definition of AI has evolved over different decades. According to [92], AI is a concept with fluid boundaries, where the focus is on the essence of the content rather than specific terminology, underscoring the language independence of these definitions and the gradual establishment of its distinct meaning. Chowdhary et al. [29] describe AI as a subset of science and technology aimed at developing intelligent machines to automate manual tasks, significantly influencing various sectors by boosting efficiency and productivity. Wang et al. [133] characterize AI as a fundamental course in computer science, extensively incorporated across various engineering fields. This includes areas like automation, language interpretation, robotics, and several expert systems, emphasizing AI’s versatility and its broad relevance to multiple disciplines. AI integrates ML techniques, such as supervised, unsupervised, and RL, into the landscape of information security [83]. Supervised learning involves training models on labeled datasets, generating tasks that map inputs to chosen outputs [110]. Unsupervised learning explores patterns within unlabeled data, involving the automated grouping of data into clusters without prior classification or categorization [10]. Reinforcement Learning (RL) utilizes a reward-based system to make decisions in dynamic environments, solving optimization problems by dynamically adapting parameters through interaction with the environment [128]. The following Fig. 2 illustrates the fundamental types of ML,

Fig. 2
figure 2

Types of ML algorithms

Full size image

This survey addresses substantial security risks related to the use of AI in information security, such as vulnerability to adversarial attacks [25]. The research aims to enhance the robustness and reliability of AI systems in protecting sensitive information by investigating and proposing solutions for potential threats. The research also helps to solve privacy concerns associated with the use of massive datasets, ensuring a thorough and secure deployment of AI in information security practices.

1.3 Goals of the paper

  1. 1.

    Review of AI Applications: This survey provides a comprehensive overview of how AI technologies are currently applied in information security.

  2. 2.

    Strengths and Weaknesses: To perform analysis and evaluate the strengths and limitations of using AI in information security.

  3. 3.

    Future Research Direction: A comprehensive survey of recent advancements in AI algorithms for information security, focusing on their effectiveness in cyber threat detection and response, and examining the ethical implications associated with their deployment, including data privacy and bias considerations.

  4. 4.

    Practical Insights for Professionals: This study explores valuable insights and proposed solutions for professionals and researchers in the cybersecurity field to address cybersecurity challenges.

  5. 5.

    Ethical and Societal Impact: In the concluding part, we explore the transformative impact of AI on society and address ethical considerations in AI development.

1.3.1 Understanding rule-based and learning-based systems

The role of AI in information security spans various methodologies, primarily categorized into rule-based and learning-based systems. Rule-based AI systems operate on predefined and explicitly programmed rules. For example, a rule-based intrusion detection system might use rules such as ‘block any IP address that attempts failed logins more than five times in 1 min’. In contrast, learning-based AI systems, including machine learning models, learn from data. They adjust their responses based on patterns they detect in the data, without explicit programming of the rules [130]. For instance, a learning-based intrusion detection system might analyze historical traffic data to learn to identify patterns that indicate potential security breaches.

2 AI application in information security

The integration of AI and information security has been extensively researched. This section evaluates existing literature, providing insights into the current state of knowledge and major results linked to the confluence of AI and information security. In order to understand the necessary solution categories to protect against cyberattacks, a well-known cybersecurity framework developed by the National Institute of Standards and Technology (NIST) was implemented in Shen [108]. The framework facilitates a better understanding for cybersecurity professionals and researchers of the various phases: security of information, cybersecurity defense, detection, reaction, and protection [63]. A number of studies have explored the practical implementations of AI algorithms, showcasing their effectiveness in enhancing security measures.

2.1 Enhanced threat detection

Machine Learning (ML) is a core component of AI technologies that significantly advance information security by enabling more sophisticated, adaptive threat detection systems [105]. For example, ML-based algorithms can analyze patterns from vast amounts of data to identify potential threats more rapidly and accurately than traditional methods [14]. This capability is crucial for proactive security measures, adapting to new threats as they emerge.

AI-driven strategies use ML, statistical models, and algorithms for proactive threat identification. They can detect patterns and anomalies that traditional methods might miss. AI-driven threat detection is a proactive strategy that uses ML, statistical models, and algorithms to find and address cybersecurity risks [71]. Lee et al. [71] proposed an approach for threat detection utilizing both traditional ML-based methods such as Support Vector Machine (SVM) [19, 45], Random Forrest (RF) [5], Naive Bayes (NB) [107], and Decision Tree (DT) and Deep Neural Networks (ANNs) including Convolutional Neural Networks (CNNs) [46], Fast CNN (FCNN), Long Short-Term Memory (LSTM) [48]. Their method was evaluated on two real-world datasets, namely NSLKDDFootnote 2 and CICIDS2017.Footnote 3 The researchers aimed to establish a generalizable security event analysis technique by training on a substantial amount of collected data. Their proposed work involved learning normal and threat patterns while taking into account the frequency of their occurrences. The term ‘generalizable security’ refers to the development of security models that maintain their effectiveness across different environments and types of data, not just the conditions they were originally trained on. This generalizability is crucial for AI systems in security because threats are constantly evolving and vary significantly across different systems and applications. For example, training AI models on a substantial amount of collected data from diverse sources enables these models to learn and recognize a wide range of threat patterns, thereby improving their ability to generalize and function effectively in different situations that were not part of their initial training set. However, it is important to note that while a large dataset can enhance the potential for generalization, it does not guarantee it. Effective generalization also requires careful selection of training examples, robust model validation methods, and continuous updating of the model to adapt to new threats.

A similar approach for threat detection was used by Le et al. [70] utilizing the CERTFootnote 4 dataset. This publicly available dataset contains information related to Traffic Capture, Firewall Logs, Email, and user activities. They employed ML-based methods including Logistic Regression (LR) [5, 52], XGBOOST [62] with different granularity levels. In their study, Sajja et al. [101] introduced a methodology aimed at enhancing the performance of Intrusion Detection Systems (IDS). Intrusion detection and prevention are security measures employed to identify and avert cybersecurity risks to computer systems, networks, infrastructure resources, and more [82]. Their research study utilized both rule-based techniques and learning-based algorithms for the purposes of intrusion detection and classification. Their research work utilized KDD99-DATASETFootnote 5 using conventional ML-based methods such as SVM and RF. Fu et al. [38] introduced a Deep Learning (DL) based Network Intrusion Detection (DLNID) approach. Their study utilized the NSL-KDD public benchmark dataset for NID. They applied the Adaptive Synthetic Sampling (ADASYN) method to expand minority class samples, achieving a more balanced dataset. Feature extraction was performed using CNN, and the newly extracted features from an attention mechanism were subsequently fed into a Bi-Directional LSTM (Bi-LSTM) [47], resulting in a notable 0.91 F1 score. In their work, Wu et al. [136] introduced an attention mechanism in DL-based models for intrusion detection, leveraging two publicly available datasets: CICIDS2017 and CIC-DDoS2019.Footnote 6 Their proposed model, the Transformer-based Intrusion Detection System (RTIDS), achieved an impressive F1 score of 0.99.

2.2 Anomaly detection

AI excels in identifying unusual activities or patterns in data, which are crucial for spotting potential security threats. Anomaly identification, sometimes referred to as outlier detection or novelty detection in data analysis, is the process of identifying uncommon objects, occurrences, or observations that substantially differ from the majority of the data and fail to fit into a predetermined definition of regular behavior [26, 42]. Generative Adversarial Networks (GANs), serving as unsupervised learning algorithms, have seen widespread application in anomaly detection due to their ability to make abnormal inferences through the adversarial learning of sample representations [137]. Girish et al. [40] introduced a method for detecting anomalies in OpenStack cloud computing. They applied a stacked Bi-LSTM-based model to a dataset collected from OpenStack using collectd.Footnote 7 The dataset includes 10 features along with class labels. Their proposed model achieved an accuracy score of 0.94. In their research, Hasan et al. [44] applied traditional ML-based techniques and DL-based algorithms, including SVM, RF, DT, LR, and ANN. Their objective was to identify anomalies in IoT devices using the DS2OS traffic traces dataset.Footnote 8 This dataset comprises traces recorded in the IoT environment of DS2OS. Through feature extraction using label encoding [143] and one-hot encoding [136], the researchers achieved remarkable results, attaining a 99% F1 score and accuracy. This highlights the efficacy of their approach in effectively discerning anomalies in IoT network traffic.

Ullah et al. [127] presented a robust and efficient framework that makes use of the capabilities of AI of Things (AIoT) to discover anomalies within Surveillance Big Video Data (BVD). They utilized the dataset created by Sultani et al. [121], which encompasses temporal annotations within videos. This dataset encompasses 13 instances of real-world anomalous activities, such as road accidents, theft, assaults etc. In total, the dataset comprises of 1900 untrimmed surveillance videos, categorized into 950 anomalous and 900 normal videos. In their research, BiLSTM yielded an Area Under Curve (AUC) score of 68%, and the optimization was performed using the Adam optimizer. Hooshmand et al. [54] presented a method for network anomaly detection utilizing a one-dimensional CNN model. Their proposed approach involves segmenting network traffic data into Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and other protocols. Their research study conducted by the authors was founded on the UNSW_NB15 dataset.Footnote 9 This dataset comprises a total of two million and 540,044 records. Notably, their achievement included an impressive 97% F1 score specifically for the UDP protocol, demonstrating the effectiveness of their methodology in accurately detecting anomalies in network traffic. Notably, their proposed work achieved an impressive 97% F1 score specifically for the UDP protocol, demonstrating the effectiveness of their methodology in accurately detecting anomalies in network traffic. Xu et al. [139] proposed a data-driven approach for multi-class classification in intrusion and anomaly detection. The dataset employed for their analysis was the KDDcup99Footnote 10 dataset. To enhance the quality of the training dataset, they employed the Synthetic Minority Oversampling Technique (SMOTE) algorithm along with mutual information. Various algorithms were utilized to process and filter the data, and ML-based methods such as K-Nearest Neighbors (KNN), SVM, DT, and a bagging classifier. Among all of these techniques, the ensemble method yielded a remarkable accuracy score of 99.7%.

Although high accuracy rates in anomaly detection algorithms are often highlighted, it is crucial to understand that accuracy alone does not guarantee effective security in real-world systems. High accuracy can indicate that the model is proficient at identifying anomalies within the specific dataset it was trained on. However, this does not necessarily mean that the system will perform equally well in practical scenarios where unexpected or novel threats occur. Furthermore, the effectiveness of anomaly detection is highly dependent on the relevance and quality of the features selected during the model training phase. It is essential to incorporate domain knowledge and continual human oversight to ensure that models are not only accurate but also relevant to the evolving nature of security threats. To address this, we advocate for a balanced approach where machine learning assists human analysts by flagging potential anomalies, while humans remain integral in the decision-making process to interpret and validate these findings.

2.3 Malware detection

AI algorithms are effective in identifying and classifying malware, offering significant improvements in protecting against malicious software. Malware, short for malicious software, is code created with the intention of causing harm and is frequently used to infiltrate or exploit a system. The introduction of malware into a computer network environment can yield various effects, contingent on the malware’s intended purpose and the configuration of the network [85, 90]. Urooj et al. [129] propose a framework for analyzing reverse-engineered Android applications using ML methods. Their approach focuses on identifying vulnerabilities within smartphone applications. To facilitate their work, they employed various datasets, including MalDroid [76], DefenseDroid,Footnote 11 and GD. After pre-processing, they utilized Androguard,Footnote 12 an open-source tool, to extract essential features. The research involved training up to six ML algorithms, namely AdaBoost, SVM, DT, KNN, NB, and RF, with the goal of accurately classifying these ML algorithms. Mahdavifar et al. [76] employed a semi-supervised DL-based technique for the classification of Android malware categories. They curated the CICMalDroid2020 dataset,Footnote 13 comprising 17,341 of the latest samples across five distinct Android app categories: SMS, Banking, Adware, Benign, and Riskware. Their proposed Pseudo-Label Deep Neural Network (PDNN) algorithm yielded an F1 score of 98%. In addition to achieving a high F1 score, the creation of the CICMalDroid2020 dataset contributes significantly to the field, offering a comprehensive resource for the study and analysis of diverse Android app categories.

In their study, Mohapatra et al. [81] proposed an AI-based approach for malware detection. Their research comprised three primary stages: data processing, decision-making, and detection of malware using a dataset of malware files. To achieve this, they implemented several algorithms, such as RF, LR, DT, KNN, NB, LightGBM [64], and CatBoost [57]. They attained the highest F1 score of 98% in their proposed research study. Vinayakumar et al. [132] introduced ScaleMalNet, a scalable and hybrid DL-based approach designed for real-time deployments, facilitating effective visual malware detection. This method encompasses static, dynamic, and Image Processing (IP) components within a big data framework. The datasets utilized in their research comprised both publicly accessible and private-public datasets [12, 66, 98]. Their research work incorporated a range of traditional ML algorithms, including RF, DT, LR, NB, and KNN. Additionally, DL-based methods such as CNN, GRU, and LSTM were employed. The collective efforts resulted in a noteworthy F1 score of 99%. Notably, their work not only achieves a high F1 score but also emphasizes the significance of combining traditional ML techniques with advanced DL methods for comprehensive malware detection. Yuxin et al. [142] adopted a comparable approach for malware detection, employing the unsupervised Deep Belief Network (DBN) as their proposed method. Their experimentation involved the preparation of four datasets, each comprising 850 malicious files and 850 benign files. In their study, they employed WEKA, KNN, and SVM. Notably, among these methods, DT emerged as the most effective, providing an accuracy score of 97% when utilizing n-gram feature extraction.

The application of ML in reverse-engineering Android applications, as discussed in this section, provides a compelling example of AI as a tool to aid security efforts. While these technologies can significantly streamline the process of identifying vulnerabilities, they also come with trade-offs. One key consideration is the balance between automation and human intervention. While fully automated systems can process vast datasets rapidly, they may lack the nuanced understanding that human experts bring, particularly in complex scenarios involving new or sophisticated attack vectors. An optimal security system often involves some form of human-in-the-loop configuration where machine learning algorithms are used to handle routine analyses and flag anomalies, and security experts step in to provide deeper insights and confirmations. This hybrid approach leverages the speed and efficiency of AI while maintaining the critical judgment and expertise of human analysts.

The following Figs. 3, 4 and 5 show the count of publications from ScopusFootnote 14 database over the last five years that focus on the synergy of AI and information security across various subjects, including the energy sector, computer science, engineering, agriculture, education, mathematics, physical science, material science, social science, climate change, and others. Specifically, in the domains of threat detection, malware detection, and anomaly detection (Table 1, Fig. 6).

Fig. 3
figure 3

Threat detection

Full size image
Fig. 4
figure 4

Malware detection

Full size image
Fig. 5
figure 5

Anomaly detection

Full size image
Table 1 Documents by type
Full size table

The following Table 2 represents the comparative analysis of the current state-of-the-art (SOTA) methods along with Fig. 7, which shows the percentages of produced documents by subject area.

Fig. 6
figure 6

Related SOTA studies

Full size image
Fig. 7
figure 7

Documents by subject area

Full size image

The following Table 2 shows the summarized review of the existing work in the domain of intrusion detection, anomaly detection, malware detection, and threat detection.

Table 2 Comparative analysis of selected studies
Full size table

Table 2 represents the comprehensive analysis of ML, DL, and advanced AI methods in cybersecurity, particularly for anomaly, threat, and malware detection. The detailed table, covering scientific work, datasets, feature sets, models used, and evaluation measures, serves as a crucial reference for understanding the diverse applications and effectiveness of these methods. This work not only highlights the versatility of AI-based techniques in cybersecurity but also lays the groundwork for future innovations in this rapidly advancing field.

3 AI and society: transformative impact and ethical considerations

The integration of AI into information security practices not only enhances capabilities but also introduces complex ethical issues that warrant thorough evaluation [97]. The ethical concerns arise primarily because AI systems, by their nature, operate with a level of autonomy that can influence decision-making processes directly [118]. This autonomy, if not properly managed, can lead to outcomes that are unintentionally biased, discriminatory, or infringe on privacy. Moreover, AI systems need to follow moral and ethical rules primarily because their decisions can have significant real-world impacts on individuals and communities. While organizations that create and use AI are ultimately responsible for ensuring these systems are ethical, the systems themselves must be designed from the outset to adhere to ethical principles to prevent harm. This is particularly important in sectors like banking and health, where decisions can affect financial stability and well-being. Regarding compliance, while it ensures that systems operate within legal frameworks, ethical AI goes beyond mere compliance. It involves embedding fairness, accountability, transparency, and respect for user privacy into the AI system’s design and operation. Compliance ensures legality, but ethics seeks to ensure morality and fairness, which may not always be covered by existing laws. The transformative impact of AI on society spans diverse fields, influencing daily life in areas such as personalized advertising [73], self-driving machinery [36], employment dynamics, and breakthroughs in healthcare [39]. A major challenge in this area is making AI systems that follow moral and ethical rules. To address this, industries need to focus on two things: understanding AI Ethics and finding out how to build Ethical AI. As AI systems become more independent, it is increasingly important to find the right balance between technological growth and what society values as right and wrong. Issues around privacy and respecting human rights and societal norms are important to think about when developing and using new technologies in the field of Information and Communication Technology (ICT) [117, 123].

3.1 Independence of AI systems

When we say that AI systems become more independent, it means that we are referring to the increasing capability of these systems to perform tasks without human intervention, thanks to advancements in AI technologies [115]. This increased automation necessitates robust ethical guidelines and regulatory oversight to ensure that automated decisions are just and fair. While regulations may mandate certain ethical safeguards, the inherent capabilities of AI to learn from vast datasets and adapt over time mean that ongoing monitoring and governance are critical to ensure these systems do not deviate from ethical norms.

3.2 Ethical concerns in AI

The biases in the learning algorithm cause discrimination, the prediction of sensitive personal data such as sexual preferences [120], and the potential of political manipulation through AI highlight a wide range of ethical concerns. Discrimination by algorithms occurs when biased data or biased decision-making criteria are used in AI models, resulting in unfair treatment of certain groups based on race, gender, age, or other characteristics [30]. This form of discrimination is often not a deliberate choice by the company but rather an unintended consequence of using historical data that may reflect past prejudices. Companies are ultimately responsible for the outputs of their AI systems and can be held liable if their systems perpetuate discrimination [65]. It is crucial for organizations to actively monitor, audit, and update their AI systems to mitigate these biases and ensure fairness in automated decisions. Furthermore, it is essential for companies to implement rigorous testing phases to detect and correct biases before deploying AI systems in real-world applications. Implementing ethical AI practices involves not only technical solutions but also governance frameworks that hold organizations accountable for their AI systems’ behavior.

One major source of worry is algorithmic bias, which can result in biased consequences. Biases in recruiting algorithms, for example, may perpetuate existing inequities, raising concerns about fairness and equal opportunity in the workplace [23, 51]. Furthermore, the widespread use of AI raises the possibility of political manipulation. The ability of AI algorithms to process huge amounts of data and generate targeted content raises concerns about its possible use for political, societal, economic, and other reasons. This emphasizes the need for robust ethical frameworks to ensure the responsible use of AI, preventing its misuse in manipulating public opinion or democratic processes [18, 94, 100]. With the increasing prevalence of AI, it becomes crucial to confront challenges related to ethics, impact assessment, and broader societal implications. Striking a balance between the advantages and potential drawbacks of AI is imperative in navigating the ethical areas of its increasing prevalence. AI has a wide range of good effects and contributes to societal well-being [32]. Its applications improve living standards, speed up legal processes, generate income, strengthen public safety, and mitigate the environmental and climate implications of human activity [56].

The use of AI in security contexts can have broader implications, including geopolitical outcomes. For instance, technologies such as deep fakes and sophisticated hacking tools can be employed to create and spread propaganda, influencing public opinion and potentially disrupting democratic processes [93]. These activities can be linked to larger geopolitical strategies, making it imperative for discussions on AI and security to consider the potential misuse of these technologies in political arenas.

In addition, ensuring transparency, accountability, privacy, and fairness are essential components in building ethical AI systems, emphasizing the need for comprehensive guidelines and practices in these areas [79, 112]. This involves open communication about AI processes, clear accountability mechanisms, protection of user privacy, and the establishment of fair practices to address potential biases. The five fundamental elements of ethical AI are displayed in the following figure 8.

Fig. 8
figure 8

Ethical AI specifications

Full size image

3.3 Ethical AI in information security

Integrating ethical AI into security practices involves ensuring that AI systems operate transparently, accountably, and without bias, particularly when processing personal or sensitive information [35]. Ethical considerations in AI-driven security are vital to maintaining user trust and complying with regulatory standards [88]. For instance, when ML is used to detect fraudulent activities, it must also protect the privacy and data rights of individuals, adhering to ethical guidelines to prevent misuse and discrimination.

3.4 Fair AI across diverse domains

In the pursuit of ethical AI, considerations extend to various domains, each demanding fair practices and responsible development. In employment, fair AI strives for unbiased hiring practices and equal opportunities [69, 106], cultivating an inclusive workforce. In healthcare, fair AI contributes to unbiased diagnostics, treatment recommendations, and resource allocation [27, 126], ensuring equitable healthcare access for diverse populations. The financial sector sees fair AI preventing discriminatory practices in lending and decision-making [103], promoting financial inclusivity. Education benefits from fair AI with unbiased assessments and equitable access [75], establishing a level playing field for all learners. Criminal justice systems benefit from fair AI, mitigating biases in risk assessments and sentencing [15], striving for justice without prejudice. Retail and advertising industries benefit from fair AI, ensuring unbiased targeting and recommendations [99], cultivating a diverse marketplace. On social media, fair AI practices mitigate biases in content moderation and information dissemination [41], providing a welcoming online environment.

It is critical to recognize that the financial sector’s compliance with fair AI and anti-discriminatory practices is not solely a matter of ethical choice but also a legal requirement [124]. Financial institutions are legally obligated to ensure that their AI systems do not engage in discriminatory practices, as failure to do so can result in significant legal liabilities. However, the commitment to fair AI goes beyond adhering to legal standards. While compliance ensures that financial entities do not violate regulations (such as those pertaining to equal credit opportunities), adopting ethical AI practices involves a proactive approach to fairness that seeks to surpass these regulatory minimums [35, 67]. Ethical AI practices in the financial sector involve designing AI systems that not only avoid discrimination but also actively promote inclusivity and fairness, regardless of legal compulsion.

The following Figs. 9, 10, 11, 12, 13, and 14 show the distribution of publications across six distinct domains concerning fair and ethical AI practices. This visualization offers insights into the prevalence of research in these areas, reflecting the growing emphasis on responsible AI development and deployment.

Fig. 9
figure 9

Impact of AI on educational security practices

Full size image
Fig. 10
figure 10

AI innovations in the financial security landscape

Full size image
Fig. 11
figure 11

AI’s role in employment security

Full size image
Fig. 12
figure 12

Advancements of AI in healthcare security

Full size image
Fig. 13
figure 13

AI deployment in criminal justice systems

Full size image
Fig. 14
figure 14

AI applications in retail and social media security

Full size image

The following Table 3 shows the number of documents produced from 2000 to 2023 in the fields of computer science and engineering for ethical and fair AI.

Table 3 Documents by type
Full size table

4 Ethical governance and privacy protocols for AI development

Maintaining ethical norms and protecting privacy are crucial in the development of AI. This section offers key frameworks such as PbD and the UNESCO Ethical Guidelines, highlighting the significance of EIA in encouraging fairness and accountability in the growth of AI.

4.1 Privacy by design

Privacy by Design (PbD) emerges as a critical ethical paradigm in this era, arguing for the proactive incorporation of privacy protection throughout AI development [21], providing openness, user empowerment, and deep respect for privacy rights [7, 74]. PbD is a concept advocating for the integration of data protection considerations during the system design phase. This approach aims to offer a practical solution that effectively addresses the concerns of data subjects and ensures privacy. The following figure 15 illustrates PbD, highlighting its essential components: making privacy the default setting, being proactive rather than reactive, embedding privacy into the design, maintaining full functionality, ensuring end-to-end security, and promoting visibility and transparency.

Furthermore, PbD emphasizes the significance of including privacy considerations throughout the AI system life-cycle, from original design to deployment and beyond. To address ethical concerns in AI, it’s essential to adopt a broad approach that considers the social and ethical aspects of data use, not just the technical side [78]. This strategy helps build trust with users and ensures compliance with global data protection laws, integrating it into the development process from the start [137].

Fig. 15
figure 15

PbD architecture

Full size image

The following Fig. 16 represents the count of documents generated between the years 2000 and 2024 about studies on PbD with ethical considerations.

Fig. 16
figure 16

Studies on PbD and ethical concerns

Full size image

4.2 UNESCO’s ethical framework

UNESCO’s recommendation on the Ethics of AI serves as a guiding framework to align AI developments with human rights, dignity, environmental sustainability, fairness, inclusion, and gender equality.Footnote 15 Complementing this, UNESCO has introduced two instrumental tools, the Readiness Assessment Methodology [125], and the Ethical Impact Assessments (EIS) in different sectors such as research and education [55, 86], which are designed to promote the incorporation of these moral values into technology breakthroughs from the beginning and ensure responsible and value-driven AI implementation [50].

4.3 Ethical impact assessments

Implementing EIA is a crucial step towards fostering responsible AI development and deployment [59]. Similar to a Privacy Impact Assessment (PIA), EIA may also serve as a method to ensure that stakeholders thoroughly scrutinize ethical implications before deployment [135]. This allows for the implementation of necessary mitigating measures. Ethical Impact Assessments focus on evaluating the potential impacts of AI systems on individuals and society, considering factors such as fairness, safety, privacy, transparency, and accountability [34, 113]. EIA’s role in AI extends beyond mere compliance and risk mitigation.

It creates an environment of ethical awareness and proactive responsibility. By systematically evaluating AI systems’ ethical implications, EIA ensures that the technology’s development aligns with societal values and norms. Figure 17 illustrates the percentage distribution of studies conducted using EIA with AI from 2000 to 2023 across various domains. It is evident that a significant proportion of these studies has been implemented in the field of computer science. Additionally, Fig. 18 presents the number of studies conducted by the top 14 countries in the realm of EIA with AI.

Fig. 17
figure 17

Documents by subject area

Full size image
Fig. 18
figure 18

Global distribution: EIA with AI studies by top 14 countries

Full size image

5 Proposed data security framework in different sectors

Figure 19 outlines our proposed centralized data security management framework. This framework is adaptable to various sectors including Law Enforcement, Smart Infrastructure, Health, and Education, ensuring tailored security measures that meet the unique needs of each domain. It has been thoughtfully designed and implemented by the Multidisciplinary Research Group on Privacy and Data Protection (MR PET) at the Norwegian University of Science and Technology (NTNU).Footnote 16 This initiative reflects the group’s commitment to advancing data security technologies while addressing the complex challenges of privacy and protection across diverse fields. The secure repository at the heart of this framework is critical for managing sensitive information across these diverse fields. Law Enforcement applications may involve highly confidential data requiring security protocols, while Smart Infrastructure would necessitate resilient and scalable data protection measures to protect interconnected systems.

In the Health sector, privacy and compliance with regulations like HIPAA [13] are paramount, and for Education, ensuring the confidentiality of student and faculty information is essential [33, 87]. Each sector feeds into and draws from the central repository, with security measures tailored to their unique data sensitivity and regulatory needs. The framework explains the flow of data between these sectors and the repository, highlighting the need for specialized security measures tailored to each sector’s requirements. Law Enforcement requires robust protocols to protect classified information, while Smart Infrastructure demands resilient defenses for its networked systems. In Health, privacy and regulatory compliance are crucial, and in Education, safeguarding personal records is key.

Fig. 19
figure 19

Data security framework in different sectors

Full size image

6 Challenges and limitations of AI in information security

6.1 Adversarial attacks

The ability of AI systems to withstand and effectively counter adversarial attacks is referred to as adversarial AI resilience. Adversarial attacks represent deliberately changing input data in order to mislead AI models, causing them to make inaccurate or unexpected predictions [104]. Developing AI systems resilient to these attacks is an ongoing challenge, requiring innovative defensive strategies and constant adaptation to emerging attack techniques. Moreover, the evolving nature of adversarial attacks poses a significant limitation, as attackers continuously develop more sophisticated methods to exploit vulnerabilities in AI systems [96]. This arms race between attackers and defenders in AI necessitates not only advanced technical solutions but also a fundamental rethinking of AI model architectures and training methodologies. Additionally, the requirement for extensive datasets to train AI models for adversarial resilience often raises concerns about data privacy and accessibility, further complicating the development of robust AI defenses[91].

6.2 Bias and fairness

AI algorithms may unintentionally reproduce biases existing in training data, resulting in biased results. It is an ethical duty to address bias in AI systems and ensure fairness in decision-making processes. To achieve justice, AI models must be continuously monitored, evaluated, and improved [69]. Justice in AI models refers to the principle of fairness in how AI systems make decisions that affect individuals ensuring that no group or individual is unfairly disadvantaged by automated processes [65]. This concept is closely linked to the broader goal of achieving equity in AI outcomes across diverse demographic groups.

In addition to technical measures, addressing bias in AI necessitates a deep understanding of the socio-cultural contexts from which data originates. This involves identifying and mitigating biases not just in the data, but also in the algorithms’ design and implementation processes. Addressing ‘bias in the data’ involves identifying and correcting skewed data that may lead AI systems to make prejudiced decisions [27]. Mitigation strategies often involve revising the data collection and preparation processes to reflect a more balanced perspective or adjusting the algorithmic model to counteract known biases. However, mitigating bias does not automatically guarantee fairer outcomes. There is a complex trade-off between mitigating bias and maintaining the integrity and usability of the data. Over-correcting for bias, for example, can lead to new forms of biases, sometimes at the expense of other important outcomes such as accuracy or predictive reliability.

Moreover, the subjective nature of what constitutes ‘fairness’ adds another layer of complexity, as different stakeholders may have varying perspectives on fair outcomes. Achieving consensus on these definitions is crucial but challenging [37, 131]. Furthermore, even with continuous monitoring and updating, the inherent limitations in data representation and the ever-evolving societal values make achieving absolute fairness an elusive goal. This highlights the need for ongoing dialogue and collaboration between technologists, and other stakeholders in society.

6.3 Resource intensiveness

Creating powerful AI models for real-time threat detection might take time and money. This raises issues about AI system performance, energy utilization, and overall scalability, particularly when faced with the challenges of limited or inadequate datasets. These constraints, which are especially significant for smaller firms with limited resources, may impede the adoption of cutting-edge AI solutions and present issues in keeping up with increasing security threats and technological breakthroughs [3, 11].

The need for high-performance computing resources to process and analyze large amounts of data in real-time further emphasizes the resource intensiveness of advanced AI models [28, 102]. This not only increases operational costs but also contributes to higher energy consumption, raising environmental concerns. Smaller organizations, in particular, may find it challenging to justify the high initial investment and ongoing costs associated with such sophisticated systems [24, 89, 109]. Additionally, the reliance on high-end hardware and software can create dependencies on specific vendors, potentially leading to issues with interoperability and flexibility in integrating with existing security infrastructures. Furthermore, the challenge of ensuring that these resource-intensive AI systems are resilient to disruptions and capable of operating under varying conditions adds another layer of complexity, especially in scenarios where resources are constrained or in fluctuating demand.

It is important to also focus on the specific challenges posed by generative AI and large language models (LLMs). These technologies, which include models like Generative Pre-trained Transformer (GPT) and other similar architectures, are increasingly used in security applications for tasks such as automated threat detection, simulation of cyber attacks, and natural language processing for security protocol compliance [140, 141]. Generative AI and LLMs are particularly resource-intensive, requiring significant computational power not only for initial training but also for ongoing operations [138]. This leads to substantial energy consumption and, consequently, a larger carbon footprint, which is a critical concern in the context of global efforts to reduce greenhouse emissions [58]. The use of these models in security applications can exacerbate environmental impacts, especially as their deployment scales across industries.

7 Top cybersecurity threats by ENISA

The Fig. 20 “Top 10 Emerging Cyber-Security Threats for 2030” from The European Union Agency for Cybersecurity (ENISA)Footnote 17 maps out the expected major cybersecurity challenges of the next decade.

Fig. 20
figure 20

Top 10 emerging cyber-security threats for 2030 by ENISA

Full size image

It highlights the risk of attackers targeting the supply chain to tamper with software components and the issue of misinformation campaigns disrupting public discourse. Privacy is at stake due to increased digital tracking. Human mistakes and outdated systems pose significant security risks, especially as the cyber and physical worlds converge. The exploitation of smart device data can lead to precise and damaging cyber attacks.

There’s a noted concern over the security of space-based assets, such as satellites, essential for global communication. The graphic points to the emergence of complex, multi-layered threats and the shortage of trained cybersecurity professionals. Dependency on international ICT providers could lead to significant systemic failures. Lastly, it flags the potential misuse of AI in cyber attacks, emphasizing the need for vigilant and comprehensive security measures.

8 Future directions

Several intriguing paths for future research and development arise as the fields of AI and information security continue to evolve. Addressing these directions can help to improve the effectiveness, efficiency, and ethical considerations of AI in information security.

8.1 Adversarial AI resilience

The goal of adversarial AI resilience is to create AI systems that can maintain their performance and accuracy even when confronted with well-constructed adversarial inputs [84]. Building AI models that are resistant to adversarial attacks is an important goal for the future. Researchers should focus on improving the robustness of AI systems, exploring advanced adversarial training methods, and developing creative architectures that can withstand intricate assaults. Understanding adversarial AI’s fundamental principles and creating solutions to mitigate weaknesses will be crucial [20].

8.2 Hybrid AI defense strategies

Future efforts should be devoted to developing hybrid AI models that combine the strengths of rule-based systems and ML-based methods. This strategy is intended to provide a comprehensive defense mechanism against constantly changing cyber attacks. Hybrid models can provide increased threat detection capabilities by using the interpretability of rule-based systems and the adaptability of ML [116, 134]. Integrating these hybrid approaches across multiple areas such as education, healthcare, finance, and critical infrastructure has the potential to improve many organizations’ overall security situation. Organizations can create robust and context-aware defense systems against emerging cyber threats by adapting hybrid AI models to the specific problems and requirements of each industry [95, 119].

8.3 Ethical and explainable AI practices

As the integration of AI with information security becomes more prevalent, prioritizing ethical considerations and emphasizing the importance of explainability becomes critical. Future research must concentrate on creating AI models that follow ethical norms, ensuring fairness, transparency, and responsibility [16, 46]. Explainable AI (XAI) approaches should be developed to provide explicit insights into AI decision-making processes, increasing user trust and making it easier to identify any ethical concerns [17].

9 Conclusion

This study summarizes how AI technologies like ML and DL have revolutionized threat detection and response mechanisms, offering more efficient, proactive, and adaptive cybersecurity solutions. We briefly mention the challenges, such as data requirements, vulnerability to adversarial attacks, and the need for continuous learning and adaptation in AI models. This survey highlights the importance of addressing ethical issues such as data privacy, bias in AI algorithms, and the need for transparent AI operations in cybersecurity. In conclusion, even though AI has the potential to completely transform information security, its responsible and successful implementation depends on recognizing and resolving these issues and constraints. To overcome these obstacles and guarantee that AI in information security is in line with moral standards, protects privacy, and strengthens cybersecurity overall, researchers, business leaders, and legislators must work together. As technology develops further, utilizing AI’s advantages while reducing its inherent drawbacks will require a proactive and flexible strategy.