5.1 Introduction

Regulatory science is ultimately a matter of trust. You need to trust that certain evidence, when obtained with certain methodologies, is sufficient to inform about a new medical product's safety and/or efficacy. Trust is formed based on previous experience but is also informed by the educational background of the experts involved and, in particular, how they decide when a belief can be considered true. And when previous experience is scarce, the educational background drives the decision to trust a new methodology.

Medical device regulators build their regulatory science using an epistemology that is at least partially that of physical sciences. In this context, it is common to expect quantitative experimental results, measurement methodologies mostly free of systematic errors (unbiased), and prior knowledge from fundamental laws of physics and chemistry to be frequently informative. Under these expectations, the inference is mostly Bayesian in that posterior probability is the product of the likelihood probability observed through controlled quantitative experiments and the prior probability that existing knowledge provides. There is also an expectation that the prior probability and the likelihood are quite similar because the prior knowledge in use has frequently resisted extensive falsification attempts, which is the theoretical basis of the concept of validation. This opens the door to using in silico methodologies to reduce, refine, and partially replace experimental methodologies.

Drug regulators build their regulatory science using an epistemology that derives from natural and social sciences. In this context, it is common to expect experimental results that are qualitative or semi-quantitative. Even when quantitative results are available, there is an expectation that they may be affected by considerable systematic errors caused by selection, information, and confounding biases. There is also the expectation that prior knowledge is scarcely informative due to the complexity and the non-linearity of the phenomena under investigation. Inference is mostly frequentist, under these expectations. Prior knowledge (and thus in silico methodologies based on it) can, at most, be used to inform the design of experimental studies, and replace them only when experimental studies are impossible.

As medical products (and the technology to test them) evolve, these expectations need to change. But such change will not happen overnight. Trust in the in silico methodologies will grow as they demonstrate their validity when used as clinical technologies and as clinical research tools in pre-regulatory settings. But in parallel, it is also necessary to break down the cultural walls that separate the regulatory science for medical devices from that for drugs. Scientific advisory panels should become more interdisciplinary and represent all expertise. Targeted re-training programs are also necessary for the staff of regulatory agencies that inform them of the opportunities and risks those innovative technologies pose.

Another possibility discussed in this chapter is to modify the current regulatory qualification pathways for in silico methodologies. This might allow for the optimal use of expertise within regulatory agencies to provide a thorough and balanced qualification process. In the following sections, we discuss possible alternative pathways to provide elements for reflection to regulatory agencies.

5.2 Pre-certification as Predictive SaMD

Most regulatory authorities nowadays recognise software with a medical purpose as a special class of medical devices called Software As a Medical Device (SaMD). The International Medical Device Regulators Forum (IMDRF) defines SaMD as “software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device.” FDA CDRH and the EU CE-marking process both include established regulatory pathways for these technologies. A special case is that of SaMDs with predictive capabilities. Examples of this new class of SaMD are solutions for fractional flow reserve (Zarins et al., 2013), planning software for transcatheter aortic valve replacement (Halim et al., 2021), or software to predict the risk of hip fracture from CT data (Keaveny et al., 2020). A recent FDA draft guideline confirms that even for these solutions, the ASME VV-40: 2018 can be used to assess the credibility of these predictive models.

A first possible regulatory pathway for in silico methodologies could be to require that any evidence supporting the marketing authorisation of a new medical product (whether the medical product is a medical device, a drug, or an ATMP) if obtained in silico, should be produced with technologies certified as predictive SaMD. Once the in silico methodology is certified as a predictive SaMD, its qualification as a medical device or drug development tool would be limited to the clinical validation aspects.

The main limitation of this approach is that not all in silico methodologies are patient-specific models, and thus their framing into a medical purpose might be impossible. Another potential issue is that some safety requirements that are indispensable for medical purposes might not be necessary when the model is used as an in silico methodologies solution; thus, this pathway might be unnecessarily severe for some solutions. On the other hand, it would simplify the regulatory process for solutions intended as SaMD and in silico methodologies, as the SaMD certification would cover the technical validation in the qualification process.

5.3 Certification of the Technical Validity

A more limited version of the SaMD pathway could be a certification of the technical validity according to the ASME VV-40:2018 or other similar standards or technical guidance provided by FDA CDRH or EU notified bodies. Once an in silico methodology has such certification, the qualification as a medical device or drug development tool would focus only on clinical validation.

The main limitation of this approach is the need to establish an accreditation process for bodies with the relevant expertise that can produce a credibility certification according to some internationally accepted technical standard.

5.4 Towards an Ad Hoc Qualification Pathway for In Silico Methodologies

A third possible strategy could be to recognise that the qualification of in silico methodologies requires a specialised panel, regardless of whether they are used to develop drugs or medical devices. This would imply creating an ad hoc process that cuts through most regulators’ current organisations built on the distinction between drugs and devices. The scientific advisory panel would include the same expertise normally found in qualification panels but also experts of in silico methodologies, who are qualified to evaluate the most technical aspects.

The main limitation of this approach is that an ad hoc qualification pathway would need to be created. In the US, such a scenario could be realised through a collaboration between CDER, CBER, and CDRH, where the management of such an ad hoc qualification pathway could be delegated to one of the three FDA centres or operated under a collaborative model. This approach would be more complicated in Europe, given that no central authority for medical devices exists.

5.5 Adapting the Existing Qualification Pathways to In Silico Methodologies

The least disruptive approach would be to embed the technical review process into the existing qualification pathways. The FDA provides qualification pathways for medical device and drug development tools, whereas the EMA provides one only for drug development tools. Qualifying a new methodology is not mandatory but highly recommended, especially for innovative methodologies. Seeking qualification for a method provides an early engagement with the regulatory agencies and will facilitate the integration of this tool into various product development processes.

Currently, a new methodology is qualified for regulatory use by first requesting qualification advice on the process intended to be used to demonstrate the validity of the new methodology in that CoU. If the authority agrees with the approach, the next step is to conduct the planned validation studies and request a formal qualification opinion. A positive draft qualification opinion is then made public to debate the adequacy of the validation evidence. The new methodology is confirmed in its final form if no criticisms emerge from the expert review. A developer can use that methodology to produce evidence in a marketing authorisation application for a new product without providing additional information on the methodology.

Existing qualification pathways are separated by the type of medical product: so there are pathways for drug development tools (e.g., small molecules, biologics, ATMPs, microbiome-derived products), and for medical device tools. They currently focus on clinical validation of the methodology rather than on its technical validation. For example, in a recent qualification opinion of EMA on a digital health methodology,Footnote 1 the only reference to the technical validity of the new methodology is in a footnote, and the only quantitative requirement is: “The length and velocity of the strides should be accurately measured with an error at 1 sigma (68% confidence interval) under 2.5%.”

One major limitation of this approach is that qualification pathways are not available for all types of products worldwide. Qualification procedures exist both in the EU and in the US. However, it should be noted that while the FDA provides a qualification procedure for new methodologies used to develop new drugsFootnote 2 and new medical devices,Footnote 3 such a pathway is available in the EU only for methodologies used to develop new drugs.Footnote 4 There are no qualification pathways for medical device development tools in Europe, a major hurdle when advanced, complex, innovative methodologies such as in silico methodologies are being proposed. Another issue is the focus of existing scientific advisory panels on the clinical validation aspects. In silico methodologies require a thorough credibility assessment, requiring experts to evaluate the dossier properly. Therefore, using existing qualification pathways also for in silico methodologies would require the extension of these panels to include experts in computational methodologies.

Another issue is that in silico methodologies are sometimes developed to address a specific use case relevant only to that product. In this scenario, it would be more convenient to include the evidence of credibility for the in silico methodology in the marketing authorisation dossier rather than undertaking a separate qualification procedure.

5.6 Essential Good Simulation Practice Recommendations

  • Regulatory agencies should increase the interdisciplinarity of scientific advisory panels and develop targeted staff re-training programs on the opportunities and risk those innovative technologies pose.

  • Regulatory agencies should explore whether existing qualification pathways should be adapted to include in silico methodologies properly or if creating new qualification pathways for these methodologies is more prudent.