Keywords

1 Introduction

Health Technology Assessment (HTA) involves the systematic evaluation of the properties, effects, and/or impacts of health technology. Its main purpose is to inform decision-makers to better support the introduction of new health technologies [1]. New digital health solutions, such as digital health applications, AgeTech, Digital Therapeutics (DTx), artificial intelligence (AI), and robotics, enable further development of healthcare services, but their introduction should follow the same criteria as other healthcare methods. They must provide evidence-based benefits and be safe to use, and their impacts on patients and organizations need to be clarified [2]. In case of digital health, the data security and privacy of the products must also be ensured in all situations, and they should be user-friendly for all assumed user groups [2, 3].

The new and innovative digital health products also set new demands on HTA models as well [2]. In Finland, the need for new models to support the HTA work of digital health was identified. Therefore, in 2018, the Finnish Ministry of Social Affairs and Health commissioned the development of a new HTA model for digital health [4]. A new HTA model, named Digi-HTA, that supports a wide range of digital health products such as digital health applications, AgeTech, AI, and robotic solutions, was published in 2019 [2,3,4]. The Digi-HTA model utilizes the Digi-HTA assessment framework as well as criteria developed in the Kyber-Terveys project, which are used for assessing data security and protection aspects [2,3,4]. Since 2020, Digi-HTA has been part of the daily HTA activities of the Finnish Coordinating Center for Health Technology Assessment (FinCCHTA), and Digi-HTA assessments have been published on various digital health products, such as digital health applications, medicine dispensing, and rehabilitation robotics, as well as digital platform solutions [5].

HTA for digital health is still a growing trend globally, not only in Finland, and new models are constantly being developed, with some of them already in use [6, 7]. Some of these models are national, such as German Fast-Track process for digital health applications, while others are developed for international use, such as the CEN-ISO/TS 82304-2:2021 Health software – Part 2: Health and wellness apps – Quality and reliability technical specification (hereinafter referred to as “the CEN-ISO/TS 82304-2:2021”) [8, 9]. Some models aim to address the assessment needs of a specific region, such as the Nordic Digital Health Evaluation Criteria (NordDEC) model developed for the Nordic countries [10].

In 2019, Germany enacted the Digital Healthcare Act (Digitale-Versorgung-Gesetz), which defines the so-called Fast-Track procedure for the assessment and reimbursement of digital health applications. Digital health applications covered by the German Fast-Track process are referred to as “DiGAs” (“Digitale Gesund-heitsAnwendungen”). [8] The details of the requirements for the DiGA are regulated in the Digital Health Applications Ordinance (Digitale Gesundheitsanwendungen-Verordnung, DiGAV) (Bundesministerium für Gesundheit, 2022) [11]. The German Federal Institute for Drugs and Medical Devices (BfArM) is the body that carries out assessments and approvals for DiGA [8].

The CEN-ISO/TS 82304-2:2021 was published in July 2021. The development was motivated by the fact that the number of digital health applications had already exceeded 300.000, yet there was no standard in place for assessing their quality. The background of the development was a commission from the European Commission, and the development was carried out in collaboration with the International Organization for Standardization (ISO). [9] The adoption of CEN-ISO/TS 82304-2:2021 is being promoted in the Label2Enable project funded by the European Union (EU) [12].

The goal of the NordDEC assessment model is to support the assessment of digital health applications in the Nordic countries and enable cross-border assessment work [10]. The requirements of the NordDEC assessment process for digital health applications are defined in the Nordic Digital Health Evaluation Criteria [13]. The development of NordDEC is managed by the Nordic Interoperability Project, jointly funded by Nordic Innovation and the Nordic health tech industry. The assessment model is developed and operated by the Organisation for the Review of Care and Health Apps (ORCHA) [10].

Since many assessment models have been developed from national or regional perspectives, such as DiGA or NordDEC, there may be significant differences or emphasis variations in the requirements of different models [8, 10]. It has been recognized among EU member states that voluntary cooperation is needed to harmonize these models, and one example of this collaboration is the European Taskforce for Harmonised Evaluations of Digital Medical Devices (DMDs) [14]. In Finland as well, it has been recognized in the EU-funded Finnish Recovery and Resilience Plan program that the existing Digi-HTA model should be further developed. For that reason, understanding the key features and requirements of available HTA models for digital health is crucial. In this study, the models selected for evaluation were considered relevant in the context of Finland. DiGA was chosen because it has already become a benchmark for assessing DTx applications and integrating assessments into reimbursement processes since 2020. As the purpose of CEN-ISO/TS 82304-2:2021 is to serve as a global criterion for digital health applications, it provides a valuable point of comparison in a global context. The NordDEC assessment model is based on the long-standing ORCHA assessment model and is designed to meet the needs of the Nordic countries, making it a good point of comparison from a Nordic perspective. Through this study, it is possible to develop and harmonize the Finnish HTA model at the national level, as well as utilize these results in international harmonization efforts.

2 Aim of the Study

  1. 1.

    To evaluate the features, domains, and aspects that are included in the DiGA, CEN-ISO/TS 82304-2:2021, and NordDEC assessment models.

  2. 2.

    To identify the similarities and differences between the evaluated assessment models and the Digi-HTA model.

3 Materials and Methods

Information about the key features of different assessment models was gathered from the websites of organizations conducting assessments, guidelines, and scientific articles [8,9,10, 12]. The information about the assessment frameworks of DiGA and NordDEC models was collected from information available on their websites [11, 13]. The DiGAV criteria, which were available on the website as of January 23, 2023, were included in the comparative work [11]. The comparative work included the version of the Nordic Digital Health Evaluation Criteria that was last updated on June 15, 2022 [13]. Information about the requirements included in CEN-ISO/TS 82304-2:2021 was obtained directly from the technical specification, which was published on August 20, 2021 [15]. The comparative work included the Digi-HTA assessment model criteria that was in use in the process between May 2022 and April 2023 [2, 3]. During this period, there were no changes in the criteria.

In the first phase, the key features of each assessment model and its associated process were listed. This included, for example, what product categories the assessment process supported and whether the assessment were linked to reimbursement processes. In the next phase, the assessment frameworks and the included domains were compared. Each assessment framework was reviewed at the level of individual questions. After that, the questions were grouped into key identified domains. However, it should be noted that different naming practices were in use for the domains that mainly addressed the same issues, such as technical stability in Digi-HTA and NordDEC, and robustness in DiGA. Therefore, these were attempted to be consolidated under the same domain. With regard to data security and protection, the comparison was conducted based on the product requirement categories and category groups presented in the article ‘Common cybersecurity requirements in IoT standards, best practices, and guidelines’ [16]. Subsequently, a comparison was made between each individual assessment model and the Digi-HTA assessment model. The individual comparative works were carried out between May 2022 and April 2023. This study includes the final top-level synthesis between different assessment models based on three individual comparative reports [17,18,19].

4 Results

Figure 1 illustrates the key elements included in typical assessment processes for digital health products. In all evaluated models, the technology company initiates the process. Afterward, the product is assessed using an assessment framework that includes detailed questions about the product being assessed. These questions are divided into different HTA domains. In addition to the questions, other documentation, such as research studies, is required, providing sufficient evidence to support the claims. Assessment is carried out by the entity responsible for assessments in each model. The assessment team may include various types of expertise, such as HTA experts and cybersecurity experts. Completed assessments are published on the web portal. There are two different scenarios for utilizing the assessments. Completed assessments can lead to a formal product reimbursement process, or assessments can be used more freely as part of procurement or product introductions.

Fig. 1.
figure 1

Typical assessment process for digital health products

Full size image

The key features of evaluated assessment models are presented in Table 1. In all evaluated models, the assessment frameworks have been published and are available. The assessment model is still under development for the CEN-ISO/TS 82304-2:2021 in Label2Enable project. Only the DiGA assessment model includes a clear reimbursement model, while in others, it is still under development. There were differences in the supported product categories among the assessment models.

Table 1. Features of evaluated models
Full size table

The domains included in different assessment frameworks are presented in Table 2. In the examined assessment frameworks, there were a lot of similarities in terms of the key assessment domains they included. The main differences were related to effectiveness, costs, robotics, AI, ethics and consumer protection.

Table 2. Domains included in evaluated assessment frameworks
Full size table

5 Discussion

The aim of this study was to investigate the key features and requirements of existing well-known HTA models for digital health. This study synthesized the similarities and differences between the models. The results of the study are intended to facilitate the further development of the Finnish Digi-HTA model. The goal is to ensure that Digi-HTA covers as many perspectives of existing well-known assessment models as possible and to identify key aspects that ensure regulated market access in different countries. The results of this study can also be utilized as part of international harmonization efforts. The comparison included the DiGA, CEN-ISO/TS 82304-2:2021, and NordDEC assessment models, which were found to be the most relevant in the Finnish context.

According to this study, the published assessment framework was available for all models, but the assessment process was still under development for CEN-ISO/TS 82304-2:2021 in the Label2Enable project. Only DiGA assessments were linked to a formal reimbursement process, while in others, this was still under development. DiGA focused solely on Class I and IIa medical devices, whereas others covered both medical and non-medical devices. The Digi-HTA model covers the widest range of different digital health products, such as digital health applications, AI, robotics, and various digital platform solutions, while others primarily focus on native applications, web-based applications, or websites. However, in the DiGA process, it is stated that products may include hardware components, but the primary functionality must be digital. After completed assessments, it is crucial that information about the conducted assessments is also publicly disseminated to all those who need assessment information. All models, except for CEN-ISO/TS 82304-2:2021, had an existing publication portal where completed assessments could be viewed. The goal of the development of CEN-ISO/TS 82304-2 is that the quality label obtained through assessments would become a part of app stores or libraries, or it would be incorporated into trusted websites used by patients or clinicians [9].

Traditionally, key domains of HTA have included effectiveness, costs, and safety [20]. However, digital health products introduce new key aspects that should also be considered in addition to these traditional domains [2]. The key observation of this study was that there was a great deal of similarity in the key domains of all assessment models, although there were differences in emphasis within these domains. This may indicate that the entities developing the models have each identified the essential domains that should be considered in the adoption of digital health products. For example, all models assess the usability of digital health products, and according to research, the ease of use of digital health products has been identified as a factor that promotes their use [2, 21]. Since the DiGA process evaluates only products classified as medical devices, the safety and functionality of the products are assumed to be demonstrated by the CE marking. However, additional evidence from the product manufacturer may be required if necessary. In other models, there were more detailed requirements for product safety or safety-related company processes. Only the Digi-HTA model included the evaluation of costs as part of the assessment process. In other models, there was a requirement that the costs associated with using the product should be communicated transparently to end users. Even though only DiGA included its own domain on consumer protection, in other models as well, except in Digi-HTA, these perspectives had been partially addressed. For instance, CEN/ISO TS 82304-2:2021 required that age restrictions for applications should be clearly communicated to consumers.

All models assessed data security and privacy issues, which should fundamentally be in order for all digital health products to ensure user trust and prevent the leakage of sensitive information to unauthorized parties [21, 22]. In the domain of data security and privacy, the Digi-HTA model had the broadest coverage. For example, CEN/ISO TS 82304-2:2021 focuses on digital health applications, while Digi-HTA covers the entire IT system. The former has very few requirements beyond applications. Digi-HTA model’s data security and protection requirements covered 108 categories and 23 different category groups, while NordDEC’s requirements were the most limited, encompassing 63 categories and 18 category groups.

Digital health products or services have the potential to offer benefits to patients, but also to healthcare service providers, for example, through improved efficiency in care processes [23]. Each evaluated model assesses effectiveness/clinical benefit from the patients’ perspective. The DiGA process emphasizes, in all aspects of product benefits, that the achieved benefits must be relevant to the patients. Benefits solely from the perspective of healthcare organizations are not sufficient evidence of effectiveness in the DiGA procedure. However, in other models, benefits obtained solely from the organization’s perspective, such as improvements in care processes, are also considered. CEN-ISO/TS 82304-2:2021 and NordDEC assessment models define the required evidence of product benefits based on the Evidence Standard Framework (ESF) developed by the National Institute for Health and Care Excellence (NICE). According to the ESF, products are classified into three different categories (Tier A, B and C) based on the potential risk they may pose. The higher the risk classification, the more compelling evidence is required. Digital health products that do not have direct outcomes related to patient health or care, but instead provide system services aimed at saving time or cost, are included in Tier A. [24] The DiGA process emphasizes that studies should be conducted in Germany or companies must demonstrate that research results from other countries can be transferred to the context of German healthcare. In other models, the origin of research results is not precisely defined. However, in the Finnish Digi-HTA process, it is always assessed on a case-by-case basis whether the results can be transferred to the context of Finnish healthcare.

The three key separate comparative works, on which the synthesis of this study is based, were conducted between May 2022 and April 2023 [17,18,19]. At the time of the study, these three assessment models, namely DiGA, CEN-ISO/TS 82304-2:2021, and NordDEC, were considered the most relevant for conducting the comparative work in the Finnish context. However, since then, new assessment models have been published, with one of the most significant being the French Early Access to Reimbursement for Digital Devices (PECAN) assessment and reimbursement model released in the spring of 2023. The process defines assessment and reimbursement models for products that can be included in the categories of DTx and remote monitoring. The PECAN process is designed for products classified as medical devices. However, unlike the German DiGA process, products from all risk classes can be included in the process [7, 25]. The ongoing development of models emphasizes the need to continue comparative work to identify all key perspectives that should be included in HTA models assessing digital health products.

6 Conclusion

In this study, the key features and requirements of four different assessment models for digital health were analyzed. The study included the Digi-HTA, DiGA, CEN-ISO/TS 82304-2:2021, and NordDEC assessment models. There was a great deal of similarity in the evaluated models, although certain differences in emphasis were found. The key differences relate to reimbursement process, maturity of the assessment process and supported product categories as well as cost and effectiveness evaluation. The information from this study can be utilized in the harmonization efforts of HTA models for digital health.