Introduction

Today, applications of artificial intelligence (AI) in radiology have become too important to ignore. The AI applications developed in the radiology domain are growing rapidly [1]. Besides the challenges [2], these applications increasingly enter the clinical practice and can significantly impact the radiology work [3]. In this situation, the radiologists’ role does not limit to being only the “users” of the applications. Radiologists, as domain experts who have deep insights into medical image diagnosis, need to actively participate in the development process [4].

The literature on AI development in general [5], and in the domain of radiology in particular, has proposed various roles that radiologists can and should play during the development of AI applications. As domain experts, they have the tacit knowledge, which can lead the AI developers towards medically relevant use cases [2, 6]. They can advise developers on how their tools can effectively contribute to the radiology work. At the same time, by being present in the development process, radiologists can better understand the principles of AI systems and identify the potential pitfalls and limitations of these applications [4]. Furthermore, including radiologists in the development can enhance the trust in AI applications, as well as the acceptance of these systems among radiologists [7]. A few studies suggest that we need to dig deeper into the development steps to better understand how radiologists can contribute to each step (see Table 1).

Table 1 How radiologists can contribute to AI development process
Full size table

Despite these suggestions, we still have a limited understanding of how these roles actually happen in the real world and whether radiologists play other roles in the development of AI applications. We also need to investigate how the roles of radiologists may change depending on the organizational conditions (e.g., between small startups versus large companies).

In this article, we examine the various roles of radiologists during the development of AI solutions, across eight companies, specializing in the development of AI applications for the field of radiology. We show that radiologists play a wider range of roles than what the current literature proposes. These roles, which occur in different steps of the AI development, are contingent to organizational conditions, such as the size of the organization, the background of the company, and the forms of interactions with radiologists.

In the next section, we describe how we conducted our study. Then, we report the roles of radiologists in various steps of AI development. Finally, we discuss the implications of our findings for the radiology community and their participation in the development of AI.

Methods

We followed a multiple case-study approach [8] to capture the diversity of the AI developers and the ways they interact with radiologists in the development process. We followed the principle of “maximum variety” [8] (see Table 2). The inclusion criteria for selecting cases were based on maximizing the variety of the cases in terms of (1) company size (large, multinationals to small startups); (2) technological and business backgrounds (e.g., dedicated to AI development and general corporations producing medical products); and (3) geographical locations (representing distinct economic, legal, and cultural contexts).

Table 2 Overview of the selected AI vendors (ordered based on company size)
Full size table

We collected data through semi-structured interviews with key managers and technical employees, who were involved in the development projects (see Appendix 1 for the interview protocol). Qualitative interviews enabled us to dig into when, how, why, and under which conditions radiologists played certain roles in each case [9]. We conducted 17 interviews, taking 60 min on averageFootnote 1. All the interviews were voice-recorded and transcribed verbatim. We also collected rich information about each company and the development projects based on public and internal documents (see Table A1 for an overview of data). Initially, several radiologists were consulted about their opinions on this research, and current practices of development for artificial intelligence within radiology. These radiologists were either employed by or affiliated to the companies. These consultations happened in an informal setting and offered background information about the roles of radiologists.

To systematically analyze data, we followed four steps. First, we integrated the data from the interviews and the documents into a detailed case-study report for each case. Second, we coded each report based on the inductive thematic analysis approach [10] to identify various roles of radiologists in the five steps of the development. We used the list of roles discussed in the literature (Table 1) as the initial codebook, yet openly explored other roles in our open coding. Third, for each role, we identified the tasks, official functions, and the forms of interaction between radiologists and the development team. Fourth, we compared the cases based on the roles that radiologists played in different steps to examine what conditions impacted when and how each role occurs.

Results

Across the companies, we identified 9 roles that radiologists played in different steps of the development process, which we describe in the following section (see Table 3 and Tables A2-A7). Figure 1 offers an overview of the radiologists' roles in different development steps

Table 3 Roles of radiologists during the development steps
Full size table
Fig. 1
figure 1

The overview of the radiologists’ roles in AI development steps

Full size image

Step 1: Defining use case and the conceptual design

In the first step, developers define the medical use case: a problem to be solved and certain improvements to be achieved in radiology work. The use case should be medically relevant and offer a viable solution that can be implemented and worked with. We observed three roles of radiologists in this step. First, as problem finders, radiologists can initiate the idea of the use case by identifying a particular problem, need, challenge, or improvement, which can be supported, expedited, or fully automated by AI solutions. Second, radiologists may be engaged after the definition of the problem, to offer their insights and feedback on the potential tool’s functionalities, features, considerations, limitations, and challenges of the idea (problem shaper). Third, radiologists sometimes become the dominant actor in making the final decisions on requirements and features (problem dominator). We observed that these three roles are often based on occasional consultations, instead of continuous, ongoing interactions. Although radiologists were involved in the definition of the use case, they were often absent in specifying the conceptual design of the solution, such as defining the operations, designing the interface, and determining how to integrate the application to the radiology workflow.

Step 2: Data sourcing and curation

Relevant, rich data, in the forms of images and scans, should be collected and prepared in order to train the algorithms and test them. It is crucial for the developers to recognize the relevant types of data and have a deep understanding of their attributes (what information is there), as well as the limitations and challenges of sourcing and working with them (e.g., the missing parameters and the biases in the data). Radiologists, who have the knowledge of medical data (especially medical images), can play an important role in finding and selecting relevant, rich data sources. In addition, the sourced data need to be checked, wrangled, and organized in such a way that can be fed into the algorithms (data curation). For that, data engineers consult radiologists to ensure that their curation of data does not violate the informational richness and accuracy of the data (data champion).

Most of the companies occasionally consulted external radiologists for sourcing and curating data (e.g., through their connections with medical institutes). Nevertheless, large companies had resources to hire radiologists to play this role. Radiologists were able to use their professional relations with PACS managers, and data warehouse managers in radiology departments, to get the data that are needed for the development of the AI solution. Sometimes, they had to decide how to collect and curate data. They also facilitated the interaction with privacy officers and ethical committees for ensuring secure and eligible handling of data.

Step 3: Labeling and establishing the ground truth

Before training the algorithm, the acquired data should be labelled in order to set a performance standard for the algorithm for (semi-)supervised algorithms. Next to labeling, developers need to establish the ground truth, against which they can assess the accuracy and performance of their algorithm. AI vendors considered the roles of data labeler to be played by radiologists. Experienced data labelers came forward from long-lasting relations with medical institutions. Sometimes, vendors used automated techniques or used trained coders (who are not radiologists) for labeling data. These trained labelers were part of the technological team (e.g., software developers, data scientists), who were already familiar with the field of radiology and with the conceptual design of the AI application. This was mostly the case for small vendors, who had limited resources to hire radiologists for labeling. Then, they relied on radiologists as data quality controllers who checked the labeled data for accuracy and consistency and resolved the complex cases.

Step 4: Training the algorithm

Once the data is curated and labelled, they are used for training the algorithm. The algorithm learns how to recognize certain patterns through detection, classification, and segmentation of the medical cases (e.g., detecting cancerous tumors). Training should be based on the datasets that consist of a wide range of representative cases (e.g., different types of patient profiles). Training is often performed by data scientists and programmers who select the type of algorithm, configure it, set the parameters, define the thresholds, execute the various iterations, and decide on stable, accurate configurations. These decisions are based on certain assumptions about how medical practice should take place and what is a relevant, valid result. There, radiologists played the role of algorithm shaper, by offering their insights and feedback on whether the technical decisions are aligned with the medical practice (e.g., are the thresholds consistent with the medical categories) and whether the results of the training are accurate, correct, and understandable from the perspective of domain experts (e.g., how to show the different classifications of lung nodules to be medically meaningful).

Step 5: Validating the AI application

After training, the developers need to ensure that the algorithm is accurate and stable when it is used on broader datasets that represent the diversity and complexity of the real-world settings (validation) and can be effectively worked with under the clinical conditions (testing). During this step, developers need to critically check the algorithm and its application based on feeding multiple, heterogeneous datasets. For that, radiologists perform the role of algorithm tester by suggesting how to validate the algorithm on real-life datasets (e.g., different patient profiles and different modalities), checking the biases and sensitivity of the algorithm, examining the false-positive and false-negative cases and identifying common issues, testing the algorithm on complex cases, and specifying the boundary conditions and limitations of it. Since there is no universal standard database and procedure for validating algorithms, radiologists play a crucial role in defining the composition of validation datasets and the procedures that should be followed to ensure an accurate and mindful validation. All these enable the technical team to revise the application and algorithm in order to reach the expected performance. In addition, vendors relied on radiologists as AI researchers to design and execute scientific tests and produce scientifically sound evidence to endorse the performance of the AI application. In this role, they are also active in writing scientific publications (especially when the development is in the research phase) and crafting scientific sections of the legal documents (when the application is going to be approved, e.g., by FDA for commercial use).

Discussion

Looking across the companies and different development steps, we see that radiologists play a diverse range of roles.

Beyond annotating and labeling roles

We see that radiologists play several other roles than merely labeling data. In fact, companies are increasingly relying on other ways to annotate their data than only recruiting radiologists (e.g., using semi- or unsupervised algorithms, using trained employees to annotate data, using automatic tools for data labeling, crowdsourcing the labeling). Hence, companies can benefit from the scarce, valuable knowledge of radiologists as domain experts in other steps of the development. Many of these roles reappear along the way of using AI. For instance, evaluation and validation of AI tools is not a one-time event, but an ongoing process, since regular updates of the software and using new types of equipment require revalidation of the algorithm. Radiologists should define how the tools can and should be evaluated in the clinical environment.

More than only offering expertise

Next to offering their expertise, radiologists can play important roles in establishing connections between the companies and medical institutions (e.g., for getting access to data) and creating trust among their fellow radiologists. Next to their medical knowledge, their technical engagement is also essential to guide the development process. In addition, they can provide guidance with the seamless integration of an application in the radiology workflow, as well as ongoing evaluation of the AI at work. This way, radiologists can contribute to the establishment of relations with the prospective clients, the introduction of the AI solutions to the clinical practice, and the definitions of viable business cases.

Different modes of interactions with radiologists

We see that companies deploy various modes of collaboration with radiologists. When companies prefer active collaborations and they have enough resources and significant development projects, they tend to hire radiologists as part of their development team, with dedicated roles. Sometimes, companies rely on occasional interactions with external radiologists, especially when they have limited resources to hire radiologists. In some cases, radiologists are actively engaged by continuously interacting with the development teams, although they do not have any formal responsibility. When companies co-develop their products with certain clients, some radiologists from the clients are engaged in the process. Even though these radiologists may not be formally affiliated to the developing company, they can play all of these roles.

Factors influencing the active engagement of radiologists

Looking across the cases, we see that several factors influence the active engagement of radiologists in the development process. Companies with strong medical background and connection with radiology departments engage radiologists from the beginning of the development process. Nevertheless, when companies are dominated by the technological teams and have a tech-heavy background, they tend to engage radiologists limitedly and in later steps of the development. This sometimes causes major challenges down the roadFootnote 2. As organizations become more mature in the development of their products, the roles of radiologists become more diverse throughout the development process. In addition, having resources available for hiring radiologists and systematically engaging them in the development is a key factor. Especially companies cannot rely only on occasional consultations and expect radiologists to help them besides their full-time medical work. Next to the resources, the active engagement of radiologists is sometimes hampered due to limited expertise of radiologists (e.g., knowing how deep learning algorithms work or how data characteristics can impact the outcomes of the algorithms), combined with their skepticism regarding how AI applications can help them.

Conclusions

Our study showed that radiologists play a diverse range of roles in the development process. Next to these roles, there are still further engagement opportunities for radiologists especially in shaping the production of AI applications, designing interfaces, and configuring and aligning them into radiology workflow. As more diverse databases become available and stricter quality levels are demanded for the algorithms, radiologists can play a stronger role in sourcing data and advising on the development and validation of the algorithms. We expect that radiologists play new roles as organizations mature in the development, data sources become more diverse and available, and more diverse use cases are identified.

Over time, organizations should not limit their interactions with radiologists to only occasional consultations; rather, they should actively and systematically bring the radiologists on board, with responsibilities and authorities to become part of the development team. This close collaboration is crucial for iterative co-creation process between the developers and the clients. At the same time, AI developers need to recognize various roles that radiologists can play and find ways to more effectively and actively engage radiologists all through the development steps and facilitate their engagement through various forms of collaboration (e.g., external advisor, affiliated medical specialists, internal team members, and collaborative teams between data scientists and radiologists).

For radiologists, it is important to recognize the large, diverse, and yet open field of contribution. Radiologists should proactively engage in the development process and define their roles and contributions. Not all radiologists need to become developers. Yet, they need to learn enough about the development of AI applications to be able to bring their valuable medical experience into the development process. Radiologists should know the basic principles, limitations, risks, and pitfalls, and the techniques and technologies used for developing these applications and learn how to evaluate and monitor the algorithms [11]Footnote 3.

Limitations and avenues for future research

Although we examined a diverse sample of companies, future studies can expand the empirical scope and diversity of the cases by comparing how radiologists play roles across geographical contexts and different modes of production (e.g., in-house development, commercial companies, and co-creation). Future studies can dig into the skills and capabilities that radiologists (need to) learn (before and) during their engagement in the development process. We can still learn from following the various career trajectories of radiologists who pursued different ways of engaging in the development of AI (e.g., as external advisers, as in-house medical experts, or as leading actors in the development). Radiologists can play important roles in the “implementation” and “use” phases of the AI systems (e.g., as local champions and AI supervisors), which deserves further investigations. Finally, it is important to study the radiology community to see how they themselves perceive and play various roles in the development of AI applications.