1 Introduction

Healthcare AI is the use of artificial intelligence (AI) technologies in healthcare. AI technologies can be used to automate tasks, improve decision-making, and personalize care. Some of the ways that AI is being used in healthcare include:

Automating tasks: AI can be used to automate tasks such as data entry, scheduling appointments, and reviewing medical records. This can free up time for healthcare providers to focus on patient care.

Improving decision-making: AI can be used to improve decision-making by analyzing large amounts of data to identify patterns and trends. This can help healthcare providers to make more informed decisions about patient care.

Personalizing care: AI can be used to personalize care by taking into account a patient's individual needs and preferences. This can lead to better outcomes for patients.

Healthcare AI is a rapidly growing field with the potential to transform healthcare. By automating tasks, improving decision-making, and personalizing care, AI can help to improve the quality, efficiency, and affordability of healthcare. The global healthcare artificial intelligence (AI) solution market is expected to grow at a rate of 41.4% from 2020 to 2027, reaching $51.3 billion [1]. This growth is driven by a number of factors, including rising venture capital investment for AI in the healthcare sector, growing demand for personalized therapy, investment impact due to COVID-19, rising demand to reduce healthcare costs, and the rise in volume and complexity of healthcare datasets. North America was reported to command the largest share of the overall healthcare AI market in 2020 [1]. However, the number of eHealth start-ups on the African continent has also been increasing since 2015 [1].

AI refers to computer systems that have the ability to perform tasks without constant user supervision and can improve the performance of such tasks over time. It comprises sub-areas such as machine learning (ML), neural networks, speech processing, computer vision, natural language processing (NLP), robotics, evolutionary computation, among others. The operation of AI solutions is dependent on the consumption or processing of huge amounts of data, often referred to as “big data”. Big data is often defined by five key qualities: volume, velocity, variety, veracity, and value.

Many current AI solutions are developed in the West, in the USA in particular, and even though the targeted market is global in scope, much of the data being used to build AI solutions for health/medical-based applications are not globally representative. This lack of representativeness in the data and applications is partly due to biases found in many common AI training datasets, specifically where the data was collected, and by whom ([2]). This means that AI researchers are often claiming "state of the art" findings for applications that will be applied to global populations based on biased, non-representative training datasets. However, another major issue with the data used in globally deployed health-related applications is the general difficulty procuring data in the first place, i.e. due to privacy regulations that vary across jurisdictions and sometimes do not exist at all (raising ethical concerns) or even a lack of interaction with target groups due to access challenges in healthcare. Additionally, creating high quality, labeled healthcare datasets takes time, expertise and access to a diverse patient population which comes at a cost often higher than the low-cost labor used for such tasks, such as requiring radiologists for labeling X-rays/radiography to be used in machine vision problems. Because of these issues, and more, high-quality medical application datasets tend to be small, resulting in the continued use of larger, publicly available training datasets without consideration of their limitations. Finally, given that many healthcare technologies are developed by teams of engineers, developers, technologists, and business leaders, their interests in developing cutting-edge technology and maximizing model performance and profits may be prioritized above the clinician’s goal of maximizing quality healthcare.

This review aims to propose a mechanism of healthcare AI solution development that considers a more global application, including marginalized communities. The review also highlights factors that often lead to gaps in data that exclude marginalized communities. Figure 1 illustrates the applications and challenges considered in our review.

Fig. 1
figure 1

Mindmap of research considerations

Full size image

2 AI medical applications

Applications of AI solutions hold a lot of promise for the healthcare field. This section provides an overview of some such applications and highlights how these need further development to be inclusive of all demographics and suitable for global use.

2.1 Applications of machine learning

The use of artificial intelligence (AI) in healthcare is rapidly growing. AI has the potential to improve the quality, efficiency, and affordability of healthcare by automating tasks, improving decision-making, and personalizing care. However, there is a potential for AI to exacerbate health disparities if it is not developed and used in a way that is inclusive and equitable.

In the context of artificial intelligence, bias refers to the tendency of an AI algorithm to produce results that are systematically prejudiced due to erroneous assumptions in the machine learning process. This can be caused by a number of factors, including the data that the algorithm is trained on, the way that the algorithm is designed, and the way that the algorithm is used.

There are a number of different types of bias that can affect AI algorithms. Some of the most common types of bias include:

Data bias: This occurs when the data that an AI algorithm is trained on is not representative of the population that the algorithm will be used to serve. For example, if an AI algorithm is trained on a data set that only includes white patients, it is more likely to make biased decisions about patients of color.

Algorithmic bias: This occurs when the design of an AI algorithm is biased. For example, if an AI algorithm is designed to predict the risk of recidivism, it may be biased against people of color if the data that it is trained on is not representative of the population that it will be used to serve.

Usage bias: This occurs when an AI algorithm is used in a way that is biased. For example, if an AI algorithm is used to make decisions about who is eligible for a loan, it may be biased against people of color if the algorithm is not used in a way that takes into account the unique challenges that people of color face.

Bias in AI can have a number of negative consequences. It can lead to unfair treatment of individuals and groups, it can undermine trust in AI, and it can perpetuate social inequities. It is important to be aware of the potential for bias in AI and to take steps to mitigate it.

One example of the potential for AI bias in healthcare is in the use of computer vision algorithms to distinguish between benign and malignant moles. A study found that algorithms that were trained on data sets that did not include a diverse range of skin tones were less accurate at identifying malignant moles in patients of color [3]. This is because the algorithms were more likely to misclassify moles as benign in patients of color, which could lead to delayed diagnosis and treatment.

Another example of the effects of bias and AI solutions in medicine, researchers used ML to predict patients' perceived knee pain levels from osteoarthritis, rather than predicting the radiologist-graded osteoarthritis severity (a grading scale that was found to have racial disparities). When assuming patients’ perceived pain level as truth rather than a historically biased clinical grading scale, they found a reduction in some racial disparities previously found in osteoarthritis prediction models. By modelling towards the patients' perceived pain levels rather than the physicians grading, much of the unexplained racial disparities in pain was reduced [4]. This underestimation of pain is problematic in diagnosing conditions prevalent in certain racial and ethnic groups, such as sickle-cell and fibroids, where part of the diagnosis and care in a crisis requires an estimation of the patient’s pain and reported medical history. This is problematic when Black patients are systematically likely to have their pain underestimated [5].

The DREAM Mammography challenge used a large cohort from Western Washington State and validated using a Swedish cohort (both predominantly white populations). This study fared poorly when applied to the more diverse population in the UCLA health system [6]. The paper stated that model specificity was significantly lower among Hispanic women compared with the radiologist performance, and both the model and radiologist had lower sensitivity in Asian women compared with women of other races. These findings led to a recommendation for model transparency and fine-tuning of AI models for specific target populations prior to their clinical adoption to counter the non-generalisability.

2.2 Internet of things (IoT) devices for health application

The COVID-19 pandemic has highlighted the potential for AI bias in healthcare. For example, two large study cohorts found that at-home oximeters used to measure oxygen levels in the blood were often inaccurate for patients of color, with Black patients experiencing nearly three times the frequency of occult hypoxemia not detected by pulse oximetry as White patients [7]. This false negative risk is mirrored in wrist-worn devices used to measure physical activity for cardiovascular medicine and raises the need for caution in their use as part of health improvement programmes in diverse populations [8]. The main reason why these devices do not work well on darker skin is that they use green light (which is cheaper and has been more widely adopted) in their biometric sensors rather than amber, which is more accurate for darker skin tones. This design choice potentially endangers a marginalized community. However, it is noted that the rapid pace of technological advancement in biometric sensors makes it difficult for consumers to determine which color light manufacturers are using in their latest heart rate trackers. Few manufacturers make this information publicly available, and public data sets detailing error rates of such technology or the algorithms used are not readily available to determine if the manufacturers are addressing the problem [9].

In another AI application, the Apple Watch has been used to monitor the symptoms of Parkinson's disease. The Motor Fluctuations Monitor for Parkinson's disease (MM4PD) uses an accelerometer and gyroscope data from the Apple Watch to detect the presence of resting tremors and other hallmark involuntary movements associated with the disease. Some involuntary movements are actually a side effect of medication used to treat the disease, so this monitoring system allows for ongoing measurement of “on” and “off” patterns of the medication’s effects. This application of the Apple Watch can potentially help spot symptoms missed in regular care and identify changes after subjects undergo surgery for deep brain stimulation. The paper also suggests the tool helped pinpoint people who slipped on medication adherence, as well as cases in which a person might benefit from a modified medication regimen [10]. However, the monitoring algorithm does not monitor other non-motor symptoms of the disease or even account for other conditions with similar tremors, such as Tourette's disease.

2.3 Health records, representation in data

Reliable and accurate healthcare records and patient data are essential for ensuring that technology implementations in healthcare have the desired outcome. This is because healthcare decisions, such as treatment options, provision of care, and even access to care, are often based on this data.

Electronic health records (EHRs) are a valuable source of healthcare data. They are generated over time and contain a large amount of information about patients, such as their medical history, symptoms, and medications. EHRs can also be augmented with data from smart devices, which can gather data about patients throughout their day. However, EHRs have some limitations. They are not always accessible to patients, and they may not be complete. For example, EHRs may not include data from patients who do not have access to healthcare providers or internet connectivity. Additionally, EHRs may contain errors, such as wrong diagnoses.

AI technologies can be used to improve the reliability and accuracy of healthcare records and patient data. For example, AI can be used to identify errors in EHRs, to fill in missing data, and to make predictions about patient outcomes. However, AI technologies do not remove the need for thoughtful design of data collection, curation, and planning inclusive of all stakeholders. This is because AI technologies can also introduce new biases into healthcare data. For example, if an AI algorithm is trained on a data set that is not representative of the population that it will be used to serve, it may be biased against certain groups of people.

2.4 Regulation

The Global Data Protection Regulation (GDPR) covers all 27 European Union member states, and some individual countries have similar approaches. In Africa, Nigeria has the Nigerian Data Protection Regulation (NDPR), which closely reflects the GDPR. In total, 66% of countries have legislation around data protection and privacy, 10% have draft legislation, 19% have no legislation, and 5% of countries have no data. Figure 2 illustrates the stages of the existing legislature [11].

Fig. 2
figure 2

Source: Data Protection and Privacy Legislation Worldwide | UNCTAD [11]

Full size image

In the countries which have regulation, these regulations often do not specify data management as applicable to the healthcare space. In the US, there are early attempts in an action plan document from the Food and Drug Administration (FDA) which outlines how the FDA is considering regulating AI solutions and machine-learning medical devices [12]. How data are collected, processed, labeled, and managed is important for ensuring that the artificial intelligence systems built to leverage this data result in outcomes that are fair and safe for all patients. The European Union has set precedence in AI solution regulation by setting out a legal framework to guide the responsible development and implementation of AI solutions [13]. This framework along with the proposed European strategy for data detailed in the summary report [14] are intended to form the bases for establishing rules to ensure trustworthy technologies and give businesses confidence and means to digitise in multiple areas, including healthcare. It has been pointed out that since the AI framework proposed by the EU relies on the development of algorithmic auditing, said auditing practices should be consistent across markets and geographies where the AI-enabled solutions are applied, and grounded in principles of ethics and justice [15]. This also raises the question of a consolidated way of addressing ethics that is respectful of all intended audiences in a wide range of possible areas of use, acknowledging the lack of balance. Currently, the dangers of data colonialism and surveillance capitalism are enforced by the power dynamics displayed by large multinational companies when they push technological advances that benefit the West without acknowledging the interconnectedness of society and the non-neutrality of the technology since data, resources and power are centralised among a few companies, countries and continents [16]. This is demonstrated in how data in medical solutioning is still very biased and limited in scope. A study of research papers published between 2015 and 2020 and the datasets these papers used (4,384 datasets and 60,647 papers) illustrated that 50% of dataset usage in AI research can be attributed to 12 institutions with most data from the United States, despite the size and population of other countries [17].

2.5 Use of data in medical solutioning

Precision medicine is an approach to disease treatment and prevention that takes into account individual variability in genes, environment, and lifestyle for each person. It is often used interchangeably with "personalized medicine" and "genomic medicine." [18]. When applied in technology, an algorithm designed to predict outcomes from genetic findings may be biased if it is trained on data that is not representative of the population that the algorithm will be used to serve. For example, attempts to use data from the Framingham Heart Study (predominantly White, male individuals of Western European descent) to predict the risk of cardiovascular events in non-white populations have led to biased results, with both overestimates and underestimates of risk [19]. How do these studies then scale in global technology systems when local environments influence healthcare provision and conditions? The collective medical mind is becoming the combination of published literature and the data captured in health care systems, as opposed to individual clinical experience. Although this shift presents exciting opportunities to learn from aggregate data, the electronic collective memory may take on an authority that was perhaps never intended. In precision medicine, where a deep understanding of inter-individual differences in health and disease due to genetic and environmental factors is of high importance, the design of the majority of algorithms ignores the sex and gender dimension and its contribution to health and disease differences among individuals [20]. It has been suggested that design, comparison, and analysis of disease networks can inform epidemiology and disease biology thereby facilitating the understanding of the risk of various risk diseases and aiding the characterization of their shared disease architectures [21].

Biological databases and patient data from electronic health records (EHR) have emerged as a powerful tool for understanding the complexity of disease relationships. However, racial and ethnic groups remain understudied, with most of the current research being based on populations of European background. Tackling this data issue is difficult due to limited EHR data available in many low to middle-income countries. The compilation and analysis of data from databases with diverse demographics and racial groups should be a priority since racial background represents an overt source of variability in disease risk and mortality [22].

Several studies have found significant numbers, ranging from 20 to 40% of all COVID-19 patients, to be classified as “long-haulers”, i.e. patients who exhibit symptoms that can last for months [23]. However, a recent study from Kings College and reported by CNN reported a much lower estimate of 2% of patients with long-term symptoms [24]. The discrepancy, it turns out, stemmed from the fact that the symptom tracking app used by the Kings College study was often dropped by long-haul patients due to a time-consuming and frustrating user experience, leading to the underrepresentation of accurate long-hauler data [25]. When a group of patients is already dealing with scepticism from their healthcare providers about the veracity of their ongoing symptoms, an ill-designed study further casts into doubt their lived experience. Data does not exist in a vacuum and can be affected by processes of data collection, sampling, cleaning, feature engineering, and further elements along the data processing pipeline. “As AI researcher Inioluwa Deborah Raji wrote, “Data are not bricks to be stacked, oil to be drilled, gold to be mined, opportunities to be harvested. Data are humans to be seen, maybe loved, hopefully taken care of.”[25]

AI technologies have the potential to transform access, quality of healthcare, and improve outcomes for patients globally. This is especially true in resource-poor settings where it can be complemented by health informatics and EHR for acquiring, storing, retrieving and using healthcare information while leveraging cloud computing to scale management, access and processing of data [26]. However, having given examples of the practical implications of the need for more representative data, the following section will explore and discuss the inclusive design of AI solutions and highlight the current challenges in the underlying data.

3 Ethical considerations

3.1 Reidentification of patients

Ethical issues in healthcare data may include the generation, recording, curation, processing, dissemination, sharing, and use of data. EHR data in this case is data that is already collected longitudinally when a patient is admitted to a hospital, for example. This data includes patient history, demographics, intake notes, delivered medications, nurse and physician notes, procedures performed, image data, etc. EHRs are often used by clinicians to make diagnoses and decisions of care, as well as for insurance claims purposes in many jurisdictions across the world. But what happens when this data is collectively used to build models for future health decisions? Can the identity of individuals in a large database of EHRs, especially for patients with rare and/or serious conditions be protected without compromising their needs? Re-identification through data mining is a distinct possibility. It has been repeatedly demonstrated that so-called anonymous datasets can be re-identified and that relatively few attributes are often sufficient to re-identify with high confidence individuals in heavily incomplete datasets [27].

3.2 Re-appropriation of data/context of data

One of the concerns raised in an opinion piece by Stanford medical researchers is that “data used to create algorithms can contain bias that is reflected in the algorithms and in the clinical recommendations they generate [28].” In addition, “algorithms might be designed to skew results, depending on who’s developing them and on the motives of the programmers, companies or healthcare systems deploying them [28].” For example, if the data used to build the diagnostic algorithm is gathered and developed by a healthcare system with the goal of improving diagnostics to save money, an ML algorithm can be biased towards diagnostic decisions with the most profitable outcomes, even if the individual patient-level outcome is not maximized (i.e. early detection, decreased mortality, etc.). Other biases in this situation could include treatment decisions based on insurance status or ability to pay.

3.3 Managing informed consent, data privacy and ownership

Informed consent, data privacy and ownership present practice and ethical challenges in AI-driven healthcare [29]. Health AI applications, such as imaging, diagnostics, and surgery can transform the patient–clinician relationship, raising the question of how the use of AI solutions to assist with the care of patients will interface with the principles of informed consent. This is a pressing question that we believe has not received enough attention in the ethical debate, even though informed consent will be one of the most immediate challenges in integrating AI solutions into clinical practice. Both doctors and patients will require education and re-orientation in this regard. Diversity of patient populations makes this especially challenging. Given global education disparities, when AI solutions are globally deployed, how is informed consent acquired? What does informed consent mean in a global context? In [30], Floridi and Taddeo address the ethics of practices (issues of consent, user privacy, and secondary use), particularly “concerning the responsibilities and liabilities of people and organizations in charge of data processes, strategies and policies and query at what point during a hospital visit patient consent is sought to use their records in a larger ML algorithm…they also question whether patients should expect their data to be used after discharge or death.” Floridi and Taddeo go on to suggest an opt-in system but point out additional risks with such an opt-in system, such as the potential for introducing data bias based on whether certain groups are more likely to opt-in or not. What information should be given to individuals when opting into such medical data systems, including the use of medical assistant apps and chatbots? Do consumers sufficiently understand that the future use of the AI-enabled health app or chatbot may be conditional on accepting changes to the terms of use? How closely should user agreements resemble informed consent documents? What would an ethically responsible user agreement look like in this context? How does data labelled in one region reflect the experiences of users in other regions? Tackling these questions is tricky, and they become even more difficult to answer when information from patient-facing AI-enabled health apps or chatbots is fed back into clinical decision-making. In most of Africa, for example, there are no regional or national strategies for defining the implications or responsibilities around digital or AI technologies, thereby leaving both healthcare professionals and patients vulnerable. Additionally, in many low to middle-income countries, there is inadequate infrastructure, associated cost of artificial intelligence applications, and fragmented data and quality standards. [31, 32].

The importance of proper risk assessment around the use of digital apps that collect huge amounts of data was demonstrated during the COVID-19 pandemic where the low uptake of the contact tracing apps deployed by governments was attributed to a lack of trust, concerns around third-party access to data, and technical requirements on user’s phones [33].

3.4 Demographic data and disease relationships

Data in EHRs is not collected consistently, and varies greatly between medical providers, clinical settings, healthcare systems, and countries. In many parts of the world, this data does not exist at all. To get a complete data set, would it then be appropriate to combine EHRs from diverse settings into one algorithmic diagnostic system? Treatments that target health conditions where the patient is from a minority ethnic background, has a disability or other lifetime health condition, is non-CIS gendered, or in/from a developing country is less likely to capture the nuance introduced by these intersectionalities as this is not often represented in the available data. In making the argument for recording demographic data, Douglas et.al argue that current data generated in EHR systems used Health Information Technology for Economic and Clinical Health Act (HITECH) in the US, which does not specifically capture granular information such as race, ethnicity, or disability status and are insufficient to address known health disparities in vulnerable populations, including individuals from diverse racial and ethnic backgrounds, with disabilities, and with diverse sexual identities [34].

4 Data accuracy and security

The methods that are often used to annotate datasets used in machine learning, such as crowdsourcing, are often prone to errors which lead scientists to draw false conclusions about the performance of their models. This is demonstrated in a paper published by MIT where the authors found an average of 3.4 errors across all the datasets they reviewed [35]. This follows a previous study where authors concluded that about 20% of ImageNet photos, for example, contain multiple objects leading to reduced accuracy of up to 10% in models trained using the dataset [36]. An aggregated repository of images that would be needed for clinical diagnosis would require even more precision.

5 AI research and design

Current technologies such as AI and ML hold promise for improving healthcare outcomes in low to middle-income countries through improvements in healthcare access and accelerating diagnosis and treatment. However, the dearth of data, gaps around regulations for the healthcare sector, and infrastructural challenges in power and network connectivity, lack of social systems for healthcare, education, and sometimes high levels of pollution (air and water) particularly outside of urban areas, pose challenges to the potential universal impacts of such technologies. Technology challenges also exist in the West particularly around ensuring the sustainability of the technologies used to support the volumes of data collected and processed.

In summary, some key healthcare-related data and AI technology design challenges include:

  • Current “state-of-the-art” algorithms trained on biased/non-representative data that leads to poor performance when applied to a global population.

  • Limited availability and access to EHR data in low to middle-income countries, leading to further biases in healthcare AI applications.

  • Lack of data and model transparency in current healthcare AI solutions and IoT tools used for diagnostic purposes, leading to a lack of understanding and accountability for AI diagnostic decisions.

  • Lack of clinician and patient access to comprehensive EHR for improved quality of care, and potential application of specialized medicine based on complete, accurate, and up-to-date medical records. Patient health records are often spread over multiple, disparate sources, or not available at all.

  • Lack of globally cohesive up-to-date healthcare AI regulations that could help guide responsible development and implementation of healthcare AI solutions.

  • Collecting more, and better/more representative data leads to concerns about data privacy and security

6 Discussion and recommendations

This review aims to create a discussion mechanism in the medical technology space by better capturing marginalised communities, and in low to middle-income countries in general, in the development of such technology. The review also highlights factors that often lead to gaps in data that exclude marginalised communities. There are many areas that need to be addressed for the use of AI methods to result in fair outcomes for all patients. Starting from addressing existing inequities in access to healthcare, representation in data, diversity of application/software developers, and education of medical practitioners to mention a few. This review focused on one, albeit broad, application (healthcare) for technologists to build solutions in today’s environment with an understanding of the challenges they face. Having explored many of the factors that lead to gaps in data that exclude marginalised communities, we recommend using these considerations in the development of prototype healthcare AI solutions to better capture the needs of a global population.

6.1 People and teams

Rather than simply focusing on whether AI solutions are good or fair based on de-biasing data, AI solution providers should ensure their teams include diverse groups of people in the AI-building process who have normally been excluded from and disproportionately affected by the outcomes of AI solutions. In the design and development of healthcare technologies enhanced by AI, it is valuable at the ideation stage to include multiple perspectives engaged with extensive involvement of practising clinicians to ensure accuracy and clinical relevance [37]. Across many countries in the western world where healthcare decisions are increasingly being influenced by non-medical staff and factors to improve efficiency, costs, reduce litigation and other non-medical factors, this is especially important to ensure patient safety. Given that technology amplifies existing biases and issues, there needs to be a rigorous process where those who influence the design and data represent all potential patients and medical staff across practitioners and support staff. The definition of the problem to be solved needs to factor in who benefits from the solution—who is empowered and who is not. In shifting the power of AI-enhanced solutions, the focus should rather be on building AI to serve the data subjects, aka communities and individuals. When building “fair and transparent” systems, the question is: fair and transparent for whom [38]? Inclusive design means that there needs to be a clear design intent, definition of intended users and their context of use, as well as an assessment of any risk of potential harm.

6.2 Application scope definition

The importance of defining the intent of the solution proposed, scope of audience, and context of use cannot be underestimated as once ML–based decision support is integrated into clinical care, withholding information from electronic records will become increasingly difficult since patients whose data aren’t recorded can’t benefit from ML analyses. Specifically, authors of a 2018 study conclude that they “believe that challenges such as the potential for bias and questions about the fiduciary relationship between patients and machine-learning systems will have to be addressed as soon as possible. Machine-learning systems could be built to reflect the ethical standards that have guided other actors in health care — and could be held to those standards. A key step will be determining how to ensure that they are — whether by means of policy enactment, programming approaches, task-force work, or a combination of these strategies [39].”

While acknowledging the potential of AI as a tool for achieving health-related targets as set out in the SDGs, [26] advocates a human-centred design along with consideration of legal and ethical questions that include privacy, confidentiality, data security, ownership and informed consent.

6.3 Data collection, privacy, sharing

To achieve positive impacts globally using algorithms that require data (in terms of the task to be performed, reliability of results across all intended populations), it is important to have access to data that reflects the diversity of the population on whom the algorithmic solution will be used. To do this requires addressing fundamental issues that prevent this data from being collected such as local infrastructural challenges, lack of resources and connectivity, understanding of the intended population to capture nuances and characteristics of different groups within it. Acknowledging that this is not simply a technical challenge, it is important to continue to address the data gap to improve conditions while addressing the fundamental issues to ensure that the solutions are fit for purpose and do not harm any segments in the population.

Suggested temporary mitigation techniques which can be considered include the use of synthetic data to preserve statistical characteristics of the real data (data signal), allowing for data to be shared and models trained on realistic data without revealing sensitive information. A synthetic dataset is one that closely resembles the real dataset, and is created by learning the statistical properties of the real dataset. This can address multiple modelling challenges and provide a viable solution for education, training and/or testing when real data is expensive, scarce or unavailable [40]. Other data security and privatisation techniques, such as hashing and anonymization, mask or destroy potentially useful information. Synthetic data preserves the statistical characteristics of the real data, allowing for data to be shared and models trained without revealing sensitive information. It is important to note that synthetic data use is best advised where there is an understanding of what a more representative set would look like- this is not always the case with healthcare data- for example, how symptoms of a condition might appear on various skin tones, or in certain populations, or how symptoms may present in non-typical cases of less common conditions or less familiar populations with non-typical co-presenting conditions as could be the case when dealing with regional populations.

6.4 Contexts of application

Furthermore, effective implementation will require understanding local social, epidemiological, health system, and political contexts. For example, a hypothetical algorithm may use state-of-the-art image classification and NLP architectures, with custom improvements for complex medical image data, to make patient-level predictions using EHR and medical images. In terms of Floridi and Taddeo’s data-algorithms-practices taxonomy, this hypothetical application can be considered within all three spheres [30]. The overall ethical focus of their taxonomy is that of data-centric level of abstraction (LoAD) in which “it is not the hardware that causes ethical problems, it is what the hardware does with the software and the data” that poses ethical problems. From this approach already, it can be surmised that an algorithm for medical diagnostics is not unethical, but it poses potential data use issues and patient-level medical outcomes that should be considered from an ethical standpoint.

6.5 Change management and education

The implementation of AI-enhanced solutions in any aspect of the clinical workflow has the potential to disrupt other areas in the ecosystem. There need to be defined education and awareness programmes to prepare practitioners and administrative staff for the technologies as well as the new practices they might enable. An evaluation of changing work practice will need to be conducted and any reskilling completed. This is especially important as an AI solution that improves efficiencies in one aspect of the medical “supply chain” is likely to have an impact in another area with the possible result of increasing skill needs in other areas.

Additionally, in implementing AI solutions in various markets, providers will need to be educated in local regulatory and ethical frameworks and ensure not only adherence but also centralise the concept of considering power structures and centering those usually marginalised in the chain of development to impact of use.

7 Conclusion

This review of challenges of AI in health applications for technologists has outlined many of the current healthcare-related data and AI technology design challenges pertaining to solutions for global deployment. Ethical and AI development considerations to address these challenges have been proposed. The proposal has applied the 5 ethical principles that emerged via a global review of AI ethics guidelines by Jobin, et al. [41]. These principles incorporate the basic principles of the African concept of ubu-Ntu [16, 42] which in place of capitalism (or communism), finds a middle ground where elements of capitalism and socialism are balanced, thereby minimising the likelihood of transplanted ethical norms and values which can collide with those of the communities where technological solutions are deployed. These are:

  • Non-maleficence → AI-enabled healthcare solutions should be used for good and not for causing harm, i.e. development of healthcare AI solutions should be human-centered where being human-centered means making allowances for diverse users and reducing inequity with their solutions.

  • Responsibility & accountability → Regulations guiding the responsible development of AI should also include who is accountable, and to what degree, when AI causes harm, such as when FDA in the USA regulates harm from pharmaceuticals, and the proposed AI guidelines proposed in Europe.

  • Transparency & explainability → Both clinicians and patients should understand how AI healthcare applications reach decisions that affect medical care, how data is collected and used and by whom.

  • Justice & Fairness → improvements to data collection, representativeness of training data used for healthcare AI applications, and representation of potentially marginalized groups in the development of such AI solutions to ensure fair and non-discriminatory application. This is where people from disenfranchised societies are placed in positions of power within the companies.

  • Respect for human rights → Healthcare AI solutions should respect and promote human rights, and be considered from the perspective of diverse stakeholders, especially those in marginalized communities in low to middle-income countries.

These principles can be used as a starting point to guide healthcare AI solution development in a way that considers a more global application, including marginalised communities.