Computational design uses artificial intelligence (AI) to algorithmically design a variety of artefacts, ranging from software to physical products. The combination of additive manufacturing (for example, 3D printing) and computational design is already being explored as a means of developing patient-specific surgical tools (Desai et al., 2019). 3D printing is used for a variety of medical applications, including creating organ models to plan surgery, and fabricating permanent implants (Ahangar et al., 2019; Tuomi et al., 2014; Yan et al., 2018). Computational design allows surgical tools to be optimised for the patient’s anatomy (Geng & Bidanda, 2021). Such tools are already being developed to assist clinicians in performing knee arthroscopies (Razjigaev et al., 2019) and laparoscopies (Brecht et al., 2020). For example, flexible laparoscopic shafts and instruments may be designed using computational design to optimise the size of the accessible workspace within a specific patient for the instruments, minimise their geometric dimensions and maximise the manipulability of the tools in the accessible workspace (Brecht et al., 2020). A snake-like manipulator attachment for a surgical robot may be computationally designed to optimise its dexterity in particular orientations for operating within a specific patient (Razjigaev et al., 2019). These tools may be called patient-specific medical instruments or bespoke surgical tools.

Incorporating AI into the design process raises questions about the responsibilities of those involved in designing, producing, and using these tools. AI and machine learning (ML) systems that develop their own models to process input into output create the possibility of a ‘responsibility gap’, as the system’s designer did not create the model and may be unable to predict the outputs from the model the AI develops (Matthias, 2004). This is especially true of AI and ML systems that incorporate evolutionary algorithms, which are a key method of computational design and a powerful means of optimising industrial designs (Eiben & Smith, 2015; Matthias, 2004). This capability for unanticipated designs means there is a possibility, however small, that the system may produce a design with harmful characteristics that the system’s designers could not predict. This possibility creates uncertainty about responsibility for the design of the tools themselves.

Here we consider how computational design may affect the responsibilities of stakeholders who are involved in the broader process of design, production, adoption and implementation of computationally designed products. We extend our focus on the nature of responsibility beyond the computational system designer to examine how the design of such products and their production, regulation and use by other stakeholders are perceived and may also be affected. In this research, we focus on the emerging use of computational design to produce bespoke surgical tools as a case study for exploring this question. Bespoke surgical tools are a useful example as they are computationally designed, 3D printed products with clear high stakes consequences for failure.

The structure of this paper is as follows. After briefly describing the relevant features of computational design, 3D printing, stakeholders and responsibility, we describe our interviews with representatives of relevant stakeholder groups on how they attribute responsibility within the process of creating and use bespoke surgical tools. We present our findings and discuss the major themes identified in the participants’ responses. Our findings indicate that most stakeholders have responsibilities across multiple stages of the process of creating and using bespoke surgical tools, and the process should also include deciding whether to use a bespoke surgical tool and evaluating the tool’s effectiveness by tracking patient outcomes after surgical use. We also find that collective ethical responsibility may be appropriate for parts of the process that require close collaboration between stakeholders, even if the process should emphasise the individual responsibilities of the stakeholders involved to prevent uncertainty about responsibility if the stakeholders’ role responsibilities are contested.

Theoretical concepts

Computational design and 3D printing

Computational design employs AI to automate some or all of a design process that human designers would otherwise perform. Forms of computational design include parametric design and generative design. Parametric design uses changeable parameters and rules that define the limits and relationships between parameters to define a design space where designs may be created by changing the parameters (Caetano et al., 2020). Generative design uses algorithms that process input data to produce a design iteratively until the design produced meets the user-defined selection criteria (Caetano et al., 2020). Generative design systems (particularly those incorporating evolutionary algorithms) have the potential to produce surprising and unexpected results (Caetano et al., 2020; Lehmann et al., 2020).

The basic process for creating and using 3D printing for clinical applications can be described as four stages: pre-printing, printing, post-printing, and application (Geng & Bidanda, 2021).Footnote 1 The pre-printing stage includes the diagnosis that motivates creating a 3D printed object for clinical use, the medical imaging and patient scans that affect the characteristics of the object, the design and customisation of a CAD (computer-aided design) model of the object, the choice of materials and 3D printing method, the simulation of the necessary characteristics of the object to ensure it is fit for purpose, and the export of the CAD model into the necessary format for the 3D printer (Geng & Bidanda, 2021). The printing stage covers the actual creation of the physical object, while the post-printing stage covers the post-processing necessary to clean and remove excess material from the object so that it is fit for purpose. For clinical applications, this also includes sterilising the object (Geng & Bidanda, 2021; Salmi, 2021). Finally, the application stage covers the clinical use of the object.

To highlight the important features of the process of creating and using bespoke surgical tools, the pre-print stage may be divided into ‘scan’ and 'design’ stages, and the print and post-print stages combined into a ‘fabrication’ stage. The scan stage involves scanning the patient and creating medical images. The design stage covers the conversion of the medical images into a 3D model that is used as input for the computational design system, and the operation of the computational design system itself. The computational design system runs multiple times until the tool design is optimised for the requirements of the surgery and the features of the patient’s scan. The print and post-print stages are combined into a fabrication stage, which covers the 3D printing process and the post-processing of the 3D printed tool to make it suitable for clinical use. The application or use stage is the use of the bespoke surgical tool in surgery. Figure 1 illustrates this process.

Fig. 1
figure 1

The Design and Use Process for Bespoke Surgical Tools (3D printing stages from Geng and Bidana’s (2021) model shown in grey)

Full size image

Bespoke surgical tools are just one of a variety of medical applications for 3D printing, such as creating anatomical models for planning surgery and creating implants and prostheses (Ahangar et al., 2019; Geng and Bidana, 2021). Surgical tools have specific requirements that may not be shared by other medical applications. Anatomical models, while also based on patient scans, do not come into contact with the patient’s body. Implants may also be customised for specific patients, but unlike surgical tools, implants are integrated into the patient’s body and need to withstand the stresses of the patient’s movements without harming them. Bespoke surgical tools are single-use, will only be in contact with the patient’s body during the surgical procedure, and they must be durable enough to perform their surgical function.

Stakeholders and responsibility

Stakeholders are persons or groups connected with an action or process. They may participate in the action or process, be able to influence it through defining or enforcing policies, or are affected by the actions or process. Most of the stakeholders involved in the process of creating and using computationally designed products will have role-responsibilities within it. Role responsibilities in this process are duties that accompany having a particular professional role (Hart, 2008). Other stakeholders in this process, such as patients, are affected by the actions of those with professional role responsibilities.

The responsibilities that accompany a role may be ethical, legal, or merely descriptive (Hart, 2008). Role responsibility itself is a descriptive conception of responsibility, as is causal responsibility for an event or action taking place (van de Poel, 2011). Normative conceptions of responsibility can be further classified as being backward-looking if they relate to actions in the past (such as accountability, blameworthiness, and liability), or forward-looking if they refer to the potential to be held responsible in the future (such as obligation and the virtue of being responsible) (van de Poel and Fahlquist, 2013).

The ethical responsibilities of stakeholders that we consider in this research are accountability, blameworthiness, and obligation. Accountability is the duty to give an account of one’s actions or inactions, blameworthiness is the duty to accept ethical condemnation for wrongdoing, and obligation is the duty to perform actions and accept responsibility for them in the future. For example, surgeons have ethical responsibilities towards the patients they treat, which are to improve the patient’s condition through planned surgical interventions that are in the patient’s best interests (Cook, 1980). They are accountable for their treatment of their patients, blameworthy for any wrongdoing in their work, and are obliged to ensure that their treatments are effective and that they maintain their skills and professional knowledge to ensure that they can make well-informed decisions for their patients (Cook, 1980). We will focus on the ethical responsibilities that accompany the roles of stakeholders in the process of creating and using 3D printed surgical tools. Ethical responsibility does not necessarily correspond with legal responsibility or liability (Hart, 2008). Someone may be ethically blameworthy in how they performed their role without necessarily being legally liable for it. We will not be considering legal responsibility (such as liability) in this paper.

Introducing AI into the computational design process raises questions of how this may affect the ethical responsibilities of those involved, and whether existing attributions of responsibilities need to be modified to reflect the use of AI in designing the surgical tool. Traditionally, designers and engineers understand how the products they create operate and can predict how they will operate when in use due to through testing (Porter et al., 2018). For ML systems (such as the forms of computational design that would be used to design bespoke surgical tools), how the system interprets input data and produces output data may change (Porter et al., 2018). As the evolutionary algorithms used in generative design depend on processes that incorporate randomness, there is the possibility that the system will produce unpredictable outputs. Identifying the stakeholders and their role responsibilities is one approach for determining how ethical responsibility should be assigned for errors when AI is used in health care (Schiff & Borenstein, 2019). Such stakeholders may include designers, medical device companies, clinicians, and hospitals (Schiff & Borenstein, 2019).

Methods

This research uses the example of 3D printed, computationally designed bespoke surgical tools as a case study to explore the nature of responsibility for computationally designed products. The case study approach is appropriate for developing in-depth knowledge and understanding of particular research contexts with the goal of gaining insights that may be generalisable to (or instructive for) other similar contexts (Hadorn, 2017; Yin, 2018). This case study is an exemplar of how 3D printing and computational design may be used in high consequence clinical situations. It was conducted via qualitative interviews with 21 representatives of the stakeholder groups who were identified as most likely to have a role in a process for creating and using bespoke surgical tools.

Participant selection

We identified key stakeholder groups for this research through initial discussions with a research team developing 3D printed bespoke surgical tools using computational design. The identified stakeholder groups include surgeons, radiologists, computational system designers, fabricators involved in 3D printing, patients, and regulators. Medical insurers were also identified but no representatives of this group responded to requests to participate in this research.

This research used a purposive sampling approach, where coverage is determined by the range of stakeholders represented by the sample group (Patton, 2015). Potential participants from the relevant stakeholder groups were identified through an online search for organisations operating in the medical 3D printing field, surgeons with experience of 3D printing, patient advocacy groups, professional organisations representing stakeholders, relevant regulatory agencies, and researchers working in computational design. The search for participants was limited to Australia to maintain a common legal and regulatory background to their responses. Potential participants were invited by email to take part. A distinction worth noting is that while patients are a direct stakeholder in the process with responsibilities pertaining to their provision of informed consent, they do not have professional role responsibilities in the process. The responses from the patient representative are integrated into the analysis of the professional responsibilities within the system.

Snowball sampling was used to identify additional participants for the case study and to ensure independent and diverse views were represented (Singleton & Straits, 2005). Bioethicists were identified through this method as an additional stakeholder group for inclusion in the research. While bioethicists are not stakeholders in the process, their perspective is useful for understanding the ethical impact of new clinical technologies. A total of 21 structured interviews were conducted between August and November 2020. Table 1 presents the distribution of research participants by stakeholder group.

Table 1 Distribution of participants by stakeholder group
Full size table

Data collection

We used qualitative interviews in this research to draw on the experience and expertise of these stakeholders in the contexts where bespoke surgical tools may be applied, and to gain a range of perspectives on responsibility (Hoepfl, 1997). The interviews were semi-structured with an interview guide (Sankar & Jones, 2008). Semi-structured interviews allow for follow-up questions inspired by participant responses and to rearrange the order of questions if the participant raised the topic of a planned question at an earlier point in the interview. These aspects of semi-structured interviews allow the interviewer to gain richer information about the participant’s perspective by giving the interviewer greater leeway in engaging with the participant’s responses to questions. A simple diagram illustrating various stages in the creation and use of bespoke surgical tools was developed for use in the interviews (see Fig. 2). The diagram was based on the initial discussions with the research team developing a computational design system for creating bespoke surgical tools using 3D printing. The diagram identifies two specific stakeholders (radiologists and surgeons) within specific stages of the process (scan and use, respectively). The patient is situated as the beginning and end point for the process.

Fig. 2
figure 2

The bespoke surgical tool creation and use process diagram presented to participants

Full size image

Figure 2 was shared with participants before the interview. The interviewer introduced the diagram to participants during the interview along with a verbal description of the process read from the prepared interview guide to ensure consistency across interviews. If it became apparent during the interview that the participant had misinterpreted this description (such as understanding the 3D printed item to be a surgical implant rather than a tool), the interviewer guided them towards considering surgical tools once this became clear. This was necessary as while surgical implants are another application of medical 3D printing, they have a different set of requirements (such as biocompatibility and durability) compared to single-use surgical tools (Ahangar et al., 2019). The interviewer’s description emphasised that the diagram was a simple representation, and the interviewer was seeking their observations on its accuracy and completeness. The participant was asked if there were any omissions in the process, where they would situate themselves within the system, and about the responsibilities different stakeholders would have in this process.

Alongside these responses, the interview questions broadly covered: (i) participants’ current role and level of familiarity with the use of robotics and bespoke surgical tools; (ii) invited their comment on the key steps and processes involved in designing, manufacturing and using bespoke surgical tools; (iii) invited their assessment of the roles and responsibilities of themselves and others in this context; and (iv) to identify any potential risks in the process and how such risks would be appropriately mitigated. In this study, all interviews were conducted via telephone or video call. Each interview took approximately 29 min and with permission, were audio recorded and transcribed for analysis. Informed consent was sought and obtained for each interview. The transcripts were thematically analysed by the lead author and presented for discussion with co-researchers (Braun & Clarke, 2006). Themes and subthemes were recorded in a spreadsheet (Microsoft Excel 365), with the participant number, interview question, transcript page and line number, and the relevant quote from the transcript recorded. Duplicate and overlapping themes and subthemes were merged during analysis by the lead author in consultation with co-researchers.

Results

We identified four major themes in the data: (1) stakeholders have responsibilities across multiple process stages; (2) stakeholder responsibilities change as a computationally designed tool moves from R&D to adoption; (3) some responsibilities are shared between stakeholders; and (4) some stakeholders have additional responsibilities in the creation and use stages. The observations of the stakeholders interviewed for this research are identified in parentheses in the presentation of the results.

Stakeholder responsibilities exist across multiple process stages

The description given to participants explicitly identified only two specific stakeholders (other than the patient), radiologists and surgeons, and these stakeholders were presented alongside specific process stages (scan and use, respectively). Nonetheless, participants identified a range of additional stakeholders that included both individual professional roles and organisations as having responsibilities that extended across multiple process stages. The responsibilities are backward-looking if the stakeholder is actively involved in the process at that stage, and forward-looking if their responsibilities affect stages that follow their active involvement. In terms of ethical responsibility, stakeholders are accountable for their actions in the stages where they participate in the process, and blameworthy for any negligence in performing their role responsibilities. Responsibilities that follow the stakeholder’s actions in the process are obligations.

The additional stakeholders with roles in the creation and use of bespoke surgical tools that we identified in the interviews are listed in Table 2, along with the process stages where participants attributed responsibilities to them. The specific responsibilities of each stakeholder group are described in further detail below with the professional roles of the participants who expressed these views identified in brackets.

Table 2 Stakeholders and stages containing their responsibilities
Full size table

Designers

Designers are the creators and maintainers of the computational design system that interprets the data collected from patient scans and uses it to create a surgical tool design optimised for the patient and the intended surgical procedure. As the computational design system itself is only directly involved within the design stage of the process, the responsibilities attributed to these stakeholders fall largely within this stage. Designers are responsible for the computational design system itself (Designers 1 and 4) and for its configuration (Designers 2 and 3). Participants also attributed to designers responsibilities to understand the scan inputs used by the system (Designer 2) and to mitigate any risks arising from using computational design to design the surgical tool (Bioethicist 1). Designer 2 also attributed responsibility for design failure and for the tool’s suitability for its intended purpose to the designer. Designers would therefore be accountable for these responsibilities and blameworthy for negligence in fulfilling them.

However, the responsibilities of designers are not limited to the design stage. One participant noted that designers are responsible for supplying a clear definition of the data required during the scan stage (Designer 3). Like their responsibilities in the design stage, designers are also accountable for their definition of the data required and blameworthy for any negligence in completing this definition. Some responsibilities for the tool itself during the use stage were also attributed to the designer. Several participants (Designers 1 and 4, Regulator 1, and Surgeon 2) attributed responsibility to the designer for mechanical failure of the tool during its use in surgery, while Designer 2 also attributed responsibility for material failure of the tool during surgery to the designer. These responsibilities are obligations for the designer.

Fabricators

Most of the responsibilities attributed to fabricators (those operating 3D printers) are unsurprisingly related to the fabrication stage. Fabricators are responsible for fabricating the tool (Surgeon 2) and the post-processing necessary to complete it (Fabricator 1), identifying designs able to be 3D printed (Designer 3), and for the materials used for 3D printing (Surgeon 4 and Designer 1). They are also responsible for tool quality (Patient 1) and any mechanical failure of the tool (Radiologist 1) during the fabrication stage. Fabricators are also expected to be aware of the relevant regulatory requirements for surgical tools and clinical uses of 3D printing (Regulator 2), as well as for tool strength and specification (Regulator 1). Fabricators are accountable for these responsibilities and blameworthy for failing to adequately fulfil them.

In addition to their fabrication stage responsibilities, fabricators were also identified as responsible for any material (Designer 2) and mechanical (Radiologist 1, Regulator 1, and Surgeon 2) failures with the 3D printed tool during the use stage. Like the designer’s responsibilities in the use stage, these are obligations for the fabricator.

Hospitals/Medical institutions

Hospitals and medical institutions were identified as being responsible for maintaining the equipment used during the scan and use stages (Bioethicist 1). If the 3D printing of the tools occurs on-site, the hospital would also be responsible for creating the tool (Fabricator 4), and oversight of the 3D printing process (Bioethicist 2). The hospital or medical institution employing clinical staff who use tools created using this process is also responsible for training them in the use of bespoke surgical tools (Bioethicist 3). These responsibilities are obligations for the hospital or medical institution.

Radiologists

Unlike the other stakeholders identified in this process, the identified responsibilities of radiologists are confined to a single stage (the scan stage). Their responsibilities are performing the necessary patient scans (Designer 4 and Fabricator 2), interpreting these scans (Bioethicist 1 and Surgeon 2), verifying that the scans are suitable for use (Fabricator 2), and mitigating the inherent risks of performing medical scans on patients (Bioethicist 1). As these responsibilities occur during the stage where the radiologist is an active participant in the process, they are accountable for them and blameworthy if they are negligent in performing them.

Regulators

Regulators were identified as having responsibilities within every process stage. Approval of the process by the relevant regulatory body for medical devices (such as the Therapeutic Goods Administration (TGA) in Australia) would be necessary (Fabricator 5, Regulator 2, and Surgeon 1). Regulatory approval would also be necessary at each stage of the process (Surgeon 1). This includes the scanning method used in the scan stage (Fabricator 5 and Surgeon 1), the computational design system (Regulator 1 and Surgeon 1) used in the design stage, and the materials used for 3D printing the tool (Bioethicists 2 and 3). Regulators were identified as having a role in the use stage through regulation of medical devices (Fabricator 3). Like hospitals, regulators are not active participants in the process, so these responsibilities are obligations.

Surgeons

Surgeons were attributed a range of responsibilities across various stages of the process. The surgeon is responsible for interpreting (Fabricator 2) and approving (Fabricator 5 and Surgeon 1) the patient scan acquired by the radiologist during the scan stage. The surgeon is also responsible for approving the tool after fabrication (Designer 4, and Fabricator 5, and Surgeon 3), and for design faults in the tool (Designer 2). The surgeon is also responsible for the patient during surgery (Bioethicist 2 and Surgeon 1), for the outcome of the surgical procedure (Designer 1 and Surgeon 2), for any unintended damage to the patient that occurs during surgery (Surgeon 4), and for mitigating any risks of surgery (Bioethicist 1). As the surgeon is an active participant in the process, the surgeon is accountable for how they perform these responsibilities and blameworthy if they are negligent in performing them.

Surgical colleges

Surgical colleges are professional organisations that accredit and represent surgeons. They are involved in surgical education, developing best practice guidelines, and in defining codes of conduct for their members (Whyte, 2019). These organisations were identified as having a role in the process by influencing and defining the standards expected by their members, and by representing their members’ interests to governments, regulators, and other groups (Bioethicist 1 and Surgeon 2). Surgical colleges would play a role by auditing the creation and use process as being suitable for use by their members (Bioethicist 1). They would also contribute to establishing processes and responsibilities for surgeons within the use stage (Surgeon 2). Like hospitals and regulators, surgical colleges are not active participants in the process, so these responsibilities are obligations.

Changes in role responsibilities from R&D through to adoption

A second theme we identified in the data is that the role responsibilities of stakeholders (and the stakeholders themselves) will change as the process for creating and using bespoke surgical tools moves from R&D through initial trials and into wider adoption. The initial R&D is likely to be performed by a closely knit team (Bioethicist 3), which would consist of designers, surgeons, and fabricators.

During the initial R&D of the computational design system, the system’s designers are responsible for providing a clear definition of the input data required (Designer 3), configuring the system (Designers 2 and 3), mitigating the risks associated with using the system (Bioethicist 1), and for the system’s designs being fit for purpose (Designer 2). The surgeons involved will guide the team in developing the process to ensure that it produces tools that will be practical for clinical use (Surgeon 1). The fabricators involved in developing the process are responsible for the materials used to produce the tools (Designer 1 and Surgeon 4), identifying viable tool designs (Designer 3), the fabrication (Surgeon 2) and the post-processing of the tool (Fabricator 1), and the tool’s quality (Patient 1). Fabricators are also responsible for mechanical failures of the tool that result from its manufacture (Radiologist 1), the strength and specification of the tool (Regulator 1) and need to be aware of the relevant regulatory requirements (Regulator 2).

Hospitals, patients, and radiologists will become involved once human trials of the process begin. Hospitals will be responsible for approving the trials performed by surgeons at their sites (Bioethicist 3) and for confirming that the trials are performed in accordance with their policies on new technologies and procedures (Regulator 2). If the tools are fabricated on the hospital site, the hospital will also be responsible for oversight of 3D printing (Bioethicist 2) and on-site 3D printing itself (Fabricator 4). The patients involved in these trials will need to give informed consent (Bioethicist 2, Patient 1, Regulator 1, and Surgeon 2). Surgeons will also need to note the effects of the new tool on existing surgical procedures (Bioethicist 3), any differences in use compared to existing tools (Regulator 1), and the surgical team will need to be aware that the tool is experimental (Bioethicist 2).

As trials continue, regulators will be responsible for approving the process itself (Fabricator 5, Regulator 2, and Surgeon 1), as well as its individual aspects. These include the scanning process performed in the scan stage (Surgeon 1), the computational design system (Regulator 1 and Surgeon 1), the 3D printing material used to create the tool (Bioethicist 3), and approving the 3D printed tools as suitable for clinical use (Fabricator 3). Regulators will also be responsible for collecting evidence of the tool’s effectiveness (Bioethicist 1). Surgical colleges will also be responsible for establishing processes and responsibilities relating to the clinical use of the process (Surgeon 2).

Stakeholder collaboration and shared responsibilities

Multiple participants emphasised the importance of collaboration between stakeholders for the overall success of the process. The most frequently mentioned collaboration was between the radiologist and surgeon (Designer 2, Radiologist 1, and Surgeons 1 and 2). This reflects existing collaborative practices between radiologists and surgeons about the scans necessary for the surgeon’s requirements (Radiologist 1). The need for collaboration between the radiologist, fabricator, and surgeon was identified by Designer 3 and Surgeon 2. Other collaborations identified by participants were between the designer, fabricator, and surgeon (Fabricator 2), the designer and radiologist (Designer 2), the designer, radiologist, and surgeon (Designer 2), and between the radiologist and the fabricator (Designer 2).

However, some responsibilities were attributed to different stakeholders by participants. Responsibility for interpreting the patient scan was attributed to both radiologists and surgeons (Bioethicist 1 and Surgeon 2). Failures in the design of the tool produced by the computational design system were identified as the responsibility of the designer and the surgeon by Designer 2. Similarly, Designer 2 also assigned responsibility for failures due to the material used for 3D printing the tool to both the designer and the fabricator. Five participants (Designers 1 and 4, Radiologist 1, Regulator 1, and Surgeon 2) attributed responsibility for mechanical failures in the tool to the designer of the computational design system and to the fabricator. How responsibility is shared between stakeholders was interpreted as either a collective (or group) responsibility (Bioethicist 1), or as individual responsibilities within a group (Designer 4 and Surgeon 2).

Responsibilities beyond the creation and use stages

Finally, some of the responsibilities participants identified did not easily fit within the described stages of the creation and use process. Several of these responsibilities relate to the decision to adopt the process itself. Hospitals and medical institutions have new technology or process policies to assist them in adopting new technologies such as bespoke surgical tools (Regulator 2). Regulators would also play a role in decisions to use these tools by assessing the evidence of the tool’s effectiveness (Bioethicist 1) and the effects of using bespoke surgical tools compared to existing ones (Fabricator 4).

For surgeons, the additional responsibilities include identifying that a patient has a suitable condition for treatment using a bespoke surgical tool (Designer 1, Patient 1, Radiologist 1, and Surgeons 2, 4 and 5), communicating the risks of surgery and of using a bespoke surgical tool to the patient (Bioethicist 2 and Regulator 1), and describing to the patient surgical alternatives and costs (Bioethicist 2). The surgeon is also responsible for seeking and obtaining informed consent from the patient for using a bespoke surgical tool (Bioethicist 2, Patient 1, Regulator 1, and Surgeon 2).

Other responsibilities occur following the tool’s use in surgery. Surgeons are responsible for diagnosing the cause of any faults in the tool that emerged during use (Bioethicist 1, Fabricator 2, and Surgeon 2), and for providing feedback about improvements to the creation and use process (Surgeon 2). Collecting data about design faults, mechanical failures, and surgeon feedback is also important for identifying whether faults are occurring regularly (Bioethicist 3). The potential use of data collected during the process and patient monitoring by surgeons and regulators would also occur after the tool’s use (Bioethicist 2, Fabricator 1, and Regulator 1). Information about patient outcomes would be used by the regulator to collect ongoing evidence of the tool’s effectiveness (Bioethicist 2). Surgical colleges may also consult with regulators about the effectiveness of the process and the tools it creates (Bioethicist 2).

Implications and discussion

Based on the themes identified in the data, we identify three implications for responsibility and computationally designed products as follows: (1) the importance of extending the creation and use process to cover responsibilities that precede and follow creating and using the tool; (2) the expansion of stakeholders’ role responsibilities and their collaborations in the process; and (3) how role responsibilities change as the creation and use process moves from R&D to wider adoption.

Extensions to the creation and use process

The participants’ responses identified that two additional stages are required to accurately represent the range of responsibilities in the process: a consultation stage situated before the scan stage, and a post-operative stage that follows the use stage. The consultation stage is where the decision to use a bespoke surgical tool is made and the patient provides informed consent to be treated with such a tool, and the post-operative stage covers the evaluation of the tool’s safety and effectiveness following its clinical use. Figure 3 presents a revised process diagram incorporating these two additional stages.

Fig. 3
figure 3

The revised bespoke surgical tool creation and use process

Full size image

The professional stakeholders involved in the additional consultation stage are surgeons, hospitals, and regulators. The surgeon’s initial identification of a patient having a condition suitable to treatment with a bespoke surgical tool occurs in this stage. Surgeons are also responsible for communicating the risks of performing surgery and of using a bespoke surgical tool to the patient, explaining any alternatives to surgery, and for seeking and obtaining the patient’s informed consent. Surgeons are accountable for fulfilling these duties, and blameworthy if they fail to do so. Failing to obtain informed consent will prevent the process from progressing further. It is the patient’s responsibility to decide whether they accept surgery, and to give informed consent to the surgeon if they agree to the surgery and the use of bespoke surgical tools during the procedure.

As mentioned in the results section, hospitals and medical institutions have responsibilities to implement and follow policies for adopting new technologies, such as bespoke surgical tools. Regulators would also assess the evidence of the tool’s effectiveness, and how using bespoke surgical tools compares to existing ones. These responsibilities would occur in the consultation stage.

The post-operative stage covers the data collection and evaluation that participants described in their responses to the original process diagram. This additional stage covers the diagnosis of faults in the tool by the surgeon, and their feedback on improving the creation and use process itself. Regulators are responsible for collecting data about design faults, mechanical failures, and surgeon feedback about the process. This data would be used to establish the tool’s effectiveness compared to alternative surgical methods and tools, and to determine whether there are any patterns in patient outcomes that indicate a problem with the tools created using the process. Surgical colleges would also consult with regulators on these matters. The updated list of stakeholders and the revised set of stages where they bear responsibilities is presented in Table 3.

Table 3 Stakeholders and stages containing their responsibilities in the revised process
Full size table

The addition of these stages recognises the need to adequately account for the decisions made prior to admitting a patient for this type of treatment, and the need for ongoing monitoring after the surgery has concluded.

Expanding role responsibilities and collaborations

The process to create and use bespoke surgical tools has a significant effect on the role responsibilities of surgeons. The involvement of surgeons throughout the process has the potential to impose a significant burden on them, as their regular responsibilities for patient diagnosis and treatment now extend to supervising the design and fabrication of a surgical tool. This expansion of responsibility is due to the surgeon’s responsibility for the patient’s welfare. While surgeons may be willing to accept this burden if bespoke tools offer them a significant benefit in performing surgery, the process of creating and using them will need to be efficient and relatively straight-forward to incorporate into existing surgical practice for it to be widely adopted. Excessive workloads and inefficient work processes are recognised factors in clinician burnout (West et al., 2018). The benefits of bespoke surgical tools will need to be considered in the context of the existing workloads of surgeons and radiologists, and risks to patient care and clinician health that any additional burden that the creation and use process will place on those involved.

The responsibilities of regulators throughout the process highlights their significance to adoption. As they are responsible for both regulating each stage of the process (medical imaging, surgical tool design, medical 3D printing, and surgical practice itself) and the whole process itself, they also need to be involved in its development. During development, regulators serve as consultants and reviewers for the process. Once regulators approve the process, they perform the role of monitoring it in use. This monitoring would reasonably involve reviewing data on the implementation of the process, including effectiveness of the bespoke tools, that may be provided by the hospitals and medical institutions where the process is adopted, the developers of the computational design system, or collected by the regulators themselves. In the case of bespoke surgical tools, a blanket regulatory approval will not relieve surgeons and other stakeholders of their responsibilities in assessing the safety and effectiveness of each tool manufactured and used, thus highlighting the importance of ongoing monitoring.

Collaboration between stakeholders is also necessary for the process to be effective. The frequent presence of radiologists as collaborators (with either designers, fabricators, or surgeons) highlights their importance to the process, despite having responsibilities in only one stage (the scan stage). The significance of collaborations highlights how stakeholders are interconnected throughout the process and suggest that there are also collaborative moments across the system that may contribute to collective responsibility arrangements. Some stakeholders may share specific responsibilities, or a stakeholder’s role in the process depends on the work completed by a stakeholder active in an earlier stage, and these interdependencies should be made explicit.

As mentioned in the results, the participants interpreted shared responsibility as either as a collective (or group) responsibility or as individual responsibilities within a group. These interpretations imply collective ethical responsibility (where all group members are equally ethically responsible for the group’s actions) or individual ethical responsibilities within a group (Ludwig, 2020). The first interpretation places significant burdens on all the stakeholders involved in the process: negligence by any stakeholder would make all the stakeholders as ethically blameworthy, even if they were not active participants in the process. This interpretation may be made less demanding by refining the collective responsibility to only those stakeholders who are active in completing specific responsibilities within the process. Similarly, a radiologist and a surgeon collaborating on completing the scan stage of the process may share collective responsibility for that stage. The surgeon involved in this process may share responsibilities with other stakeholders (such as the fabricator, for instance) throughout the process. Any given stakeholder would share collective responsibility for the collaborations they are involved in during the process. A surgeon may share collective ethical responsibility for their collaborations with the radiologist and the fabricator, but the fabricator would not share collective ethical responsibility for the radiologist unless they collaborate directly with them during the scan stage. Instead of all the stakeholders involved in the process being considered as a group with collective ethical responsibility for the process, collective ethical responsibility exists for the series of collaborations between stakeholders.

The second interpretation resembles the concept of ‘shared responsibility’ for passenger safety in aviation, and for patient safety in healthcare (Sittig et al., 2018). For example, Sittig et al. (2018) describe how responsibility for the safety of electronic health records can be shared between developers, users and healthcare organisations, and regulators. Distributing responsibility within a group of individuals raises the ‘distributive question’ of how ethical responsibility should be allocated between group members (Ludwig, 2020). Shared responsibility would attribute responsibility to the stakeholder best positioned to respond to problems that emerge (Sittig et al., 2018). For groups with a limited number of members (such as the process for creating and using bespoke surgical tools), the dilution principle where the ethical responsibility of a group member “is proportional to the causal contribution of that member to the harms (or benefits)” caused by that group is a useful approach (Ludwig, 2020). In this context, the stakeholders’ responsibilities describe their contributions to the outcome of the process. For example, radiologists are ethically responsible for performing their own responsibilities and would not share any ethical responsibility for errors made by other stakeholders, even if they are collaborating with them. The interpretations will frequently overlap in how they allocate responsibility. When they diverge in allocating responsibility (for instance, when there are collaborations between stakeholders where their contributions are significantly unequal), the second interpretation (where responsibility is proportional to the causal responsibility for a stakeholder for the result) should be preferred as it offers a means of determining ethical responsibility if it is contested between stakeholders. This will encourage stakeholders to address ambiguities about the extent of their role responsibilities.

Changes in role responsibility from R&D to adoption

It became apparent from the interviews that participants identified the role responsibilities of some stakeholders differently depending on the stage of the process ranging from R&D, through initial human trials and finally, to regulation and adoption. The roles of designers and fabricators will change as the process is tested and the characteristics of the computational design system and the necessary settings and materials for reliably 3D printing surgical tools become clearer. The changes in role responsibilities for designers and fabricators represent the maturity of the process as it progresses from being experimental to becoming a practical option for wide adoption. While the individual designers, fabricators and surgeons involved in developing the system would no longer be active stakeholders in the process once the system matures, they would continue to be ethically responsible for their work in developing the system. They would continue to have the backward-looking responsibilities of accountability and blameworthiness. For the forward-looking responsibility of obligation, the stakeholders developing the system may either continue to maintain it and correct any errors in the system revealed during its use, or pass on this obligation to another party able to fulfill this obligation. For example, the R&D team may sell the system to a company that takes responsibility for supporting it.

The designer’s role will change from heavy development of the computational design system during testing and clinical trials to maintenance, support, and updating the system to keep pace with other changes that may affect the process (such as changes in medical imaging and 3D printing). While the designer and the computational design system are prominent in the development, testing, and early deployment of the process, the computational design system will effectively replace the designer as a participant in the process as it becomes widely adopted. The computational design system itself is unable to bear ethical responsibility for the designs it creates (Douglas et al., 2021). While the designer will not be an active participant in the process once the system is widely deployed, they will continue to be accountable for the computational design system, blameworthy for any negligence in developing it, and have an obligation to maintain the system to ensure that it continues to be fit for purpose as circumstances change.

Fabricators will continue to have an active role in the process as it moves from development to wider adoption, as they are required to perform the 3D printing itself and the post-processing necessary to create a tool suitable for clinical use. Nonetheless, the fabricator will be able to rely to the settings and materials that are found to be most effective during the development of the process. The fabricator will have an obligation to revise the settings and materials recommended to those using the system if better alternatives are identified.

The role of surgeons will also change as the surgeons involved in the process change from being those participating in the research and development team to surgeons interested in using the process to create tools for treating their patients. While surgeons will necessarily have to consider the unique characteristics of each patient and each planned operation, decisions about the range of possible tool designs and other details about the process itself will not need to be repeated for each operation.

The conception of collective ethical responsibility discussed in the previous section captures the shared responsibilities of the development team. The designer, fabricator, and surgeon working together during the R&D phase of the system’s development share collective responsibility for the computational design system they develop.

Limitations and further research

A limitation of this study is the absence of medical insurers as a stakeholder group. Medical insurance plays a key role in the adoption of new medical technologies. Their perspective on the responsibilities of those involved in creating and deploying this technology would be a welcome addition to this research. While clinicians, fabricators, and designers were well represented in this study, a greater representation of patient advocates and regulators would also be useful for further research.

There were few unprompted commentaries on the presence of AI within this process. No participants attributed responsibility to the computational design system for designing bespoke surgical tools. Directly asking the participants about the significance of AI within this process is necessary to establish whether the concern about responsibility gaps is uncommon among stakeholders, or if responsibility gaps are more prominent in other applications (such as autonomous vehicles).

This research could be expanded by considering other application domains for computationally designed products. While the significant consequences of clinical use of computational design and 3D printing bring questions of responsibility into focus, these consequences may also imply that rigorous regulatory approval methods and testing of the design and use process has limited the risks of harmful designs being produced. This may differ in other application domains. Similarly, considering how stakeholders attribute responsibilities in other high consequence domains (such as aviation and road vehicles) would also be useful to determine if the responsibilities of designers, fabricators, and relevant domain stakeholders change significantly. Exploring how responsibilities are shared in other domains of technology development and use is another possible expansion of this research.Footnote 2

Conclusion

Bespoke surgical tools are an example of how computational design using AI and 3D printing may be used to create new products for high-consequence applications. Creating and using these tools will involve a variety of stakeholders, such as surgeons, radiologists, designers, and fabricators, who will bear responsibilities within this process. To better understand how using computational design for product design may affect the responsibilities of those who play a role in the creation and use process, we interviewed 21 representatives of stakeholder groups who would be involved in the deployment and use of bespoke surgical tools.

In this research, we found that the process needs to include role responsibilities that precede and follow the creation and use of bespoke surgical tools. The consultation stage involves surgeons, hospitals, and regulators. Surgeons are responsible for diagnosing the patient’s condition and for communicating to the patient the risks of surgery and of using a bespoke surgical tool. Patients need to provide informed consent for a bespoke surgical tool to be used, and the process cannot continue past the consultation stage without patients giving this consent. Hospital policies for assessing new technologies and the responsibility of regulators to assess bespoke surgical tools would also occur in the consultation stage. The post-operative stage involves regulators, surgeons, and surgical colleges. It covers surgeons providing feedback on using the tool and diagnosing faults in the tool that occurred during use, regulators collecting data for assessing the tools’ effectiveness and looking for patterns in patient outcomes, and surgical colleges would consult with regulators about the usefulness of these tools.

Apart from radiologists, stakeholders also have role responsibilities across several stages of the process. The collaborations between stakeholders throughout the process also mean that collective ethical responsibility may exist between the collaborating stakeholders involved at various process stages. Role responsibilities will also change as the process itself moves from being experimental and under development to wider adoption by surgeons. The stakeholders who comprise the R&D team behind the computational design system (designers, fabricators, and surgeons) will also share collective ethical responsibility for the system both during its development and once it is deployed, even though their roles change from developers to maintainers of the system. Finally, the role responsibilities of stakeholders (especially surgeons) are expanded by introducing bespoke surgical tools. Surgeons will have to perform additional tasks and collaborate with other stakeholders (such as fabricators) to use these tools. For the process of creating and using these tools to be widely adopted, it must be straight-forward to incorporate into existing surgical procedures and institutional structures. Otherwise, the burdens of expanding these responsibilities would discourage stakeholders from adopting computational design systems into their practices. The ethical responsibilities of stakeholders also begin before the system is used and endure after the created tools are used in surgery.

The broader implications of this study are twofold. First, the examination of the stakeholder system in this research identified that introducing computational design at one stage of a tool development process led to the identification of a series of collaborative moments among stakeholders that lead to a type of collective or ‘shared responsibility’ alongside their existing professional responsibilities. This may have been emphasised in the case of bespoke surgical tools because the case study centred around patient safety, but it suggests that changes such as incorporating computational design have ‘flow on’ effects within other technology development and deployment pipelines that warrant closer examination. Second, there was also evidence of expanding responsibilities among some of the stakeholders arising from process changes to the overall system. Part of this expansion of responsibilities related to the importance of being able to account for how responsibilities may change or endure ‘after the fact’. For example, the transition of a technology process from R&D to approved use is generally accompanied by a shift in the regulatory requirements and in this case, the ongoing monitoring by regulators was identified as a potential change or expansion of stakeholder responsibilities that would be required. There is also a risk that the burdens of expanding these responsibilities would discourage stakeholders from adopting computational design systems into their practices. For decisions about incorporating computational design and the associated AI and ML techniques, it is essential that responsibility is understood at both the stakeholder and system levels. It is in this way that the interconnected responsibilities of developing, adopting, using and evaluating such technologies can be made explicit and transparent to all involved.