Background

Given limited health professional resources and limited (spatial) access in rural areas, DiHA is seen as having the potential to improve health care delivery [1,2,3]. DiHA includes cooperative or interactive applications of modern information and communication technologies aiming to improve health care provision and population health. However, especially in the light of evidence-based health care, assessing DiHA is crucial to ensure safe, efficacious and effective health care. In Europe, the requirements under the Medical Device Regulation (MDR) to obtain market authorization (Conformité Européenne, CE) do not require proof of efficacy and effectiveness for DiHA that fall into the lower risk classes I and II. Uncertainty regarding benefits of DiHA is thus high, especially in the context of coverage decisions. This may also lead to a partially reserved acceptance among physicians, for instance in Germany [4].

Although Germany has been less advanced in terms of digitisation in health care compared to other countries so far [5], and digitisation is also insufficient in the hospital sector [6], it was the first country to launch a DiHA “fast-track” pathway for coverage decisions of digital health applications in ambulatory care. This has led to changes advancing development and evaluation of DiHA [7]. Germany has been a pioneer for many countries but the use of DiHA is also increasingly supported by the governments of other European countries [8]. France, for instance, will follow suit to foster innovation and provide access to DiHA for patients [9]. Many countries are currently developing national digital health strategies to include DiHA into the benefit basket of public health systems [10]. Several approaches for categorisation and decision making whether to include DiHA in public benefit baskets exist in European countries. Examples include an approach for categorisation, evaluation, and pricing/reimbursement developed to support of the German “Digital Health Care” (DVG) legislative process [11], the “Evidence standards framework for digital health technologies” concept developed by the National Institute for Health and Care Excellence (NHS) of England [12], a platform and a validation pyramid for CE-marked mobile applications in Belgium [13], and the French guide on clinical evaluation of a medical device in the context of eligibility decisions for reimbursement [14] complemented by a French Health Authority (HAS) system for classification of digital solutions according to their intended use, their ability to provide an individual response and their autonomy [15]. However, the evidence base for coverage decisions is a frequently and highly discussed topic. Increasingly, models are being developed to evaluate DiHA [16,17,18] also due to methodological gaps and difficulties in assessing the value of DiHA, including the question for an adequate control group to be included in clinical trials and the level of evidence to be used [19].

Although a process of assessing DiHA with regard to coverage decisions was set up in Germany in 2019, there is no study analysing its findings. Therefore, this study aims to investigate the first results of the fast-track pathway while especially focusing on the evidence evaluation of DiHA. Additionally, risk of bias of clinical trials used in the decision-making process of the German fast-track pathway was assessed.

Procedure and reimbursement of DiHA in Germany

In Germany, the Digital Health Care Act (Digitale-Versorgung-Gesetz, DVG) was passed in November 2019 [20]. It describes the formalisation of the DiHA, which can be prescribed by physicians and psychotherapists or reimbursed directly by sickness funds upon request of insured persons provided the corresponding diagnoses was made by the physician. DiHA may include standard software, software as a service, mobile as well as browser-based applications. Insureds are entitled to certain DiHA, which must meet certain requirements in order to be covered by the German statutory health insurance (SHI).

The subsequent Ordinance on Digital Health Applications (Digitale Gesundheitsanwendungen-Verordnung, DiGAV) describes the procedure [21]. DiHA-manufacturers must apply to the German Institute for Medicines and Medical Devices (Bundesinstitut für Arzneimittel und Medizinprodukte, BfArM) to be considered for coverage through SHI. Their application must meet requirements for safety, quality, functionality, privacy, and data security. Furthermore, evidence of positive health care effects must be shown. The latter include medical outcomes and patientcentred structural and procedural outcomes to be demonstrated by corresponding endpoints in comparative clinical trials. The manufacturer must decide between two possibilities of the fast-track pathway: an application to be considered (1) permanently or (2) conditionally into SHI directory to be eligible for reimbursement. (1) If evidence of positive health care effects is available at the time of application, the DiHA can be included in the official DiHA directory of SHI. (2) For DiHA following the application process to be conditionally included into the official directory for a limited time period (usually twelve months, max. 24 months), manufacturers have to submit plausible justification of DiHA’s contribution to positive health care effects and a scientific evaluation concept prepared by a manufacturer-independent institution to demonstrate health care effects. Following the concept of coverage with evidence development, manufacturers must generate respective evidence of positive health care effects, while the DiHA is conditionally reimbursed (Fig. 1). After inclusion in the directory, manufacturers are basically free to set their own prices for the first year. From the 13th month after inclusion into the DiHA directory, the price negotiated between the manufacturer and Federal Association of Sickness Funds applies.

Fig. 1
figure 1

Overview of process for digital health applications under the DVG (own illustration)

Full size image

Methods

Figure 2 gives an overview on the methods of the study. We identified between 08/2021 and 02/2022 all DiHA listed in the DiHA directory and respective decision documents of BfArM [22] used during the fast-track evaluation between 09/2020, when the first decisions were made, and 02/2022.

Fig. 2
figure 2

Methodological approach (own illustration)

Full size image

First, all DiHA permanently and conditionally included in the German DiHA directory were investigated descriptively according to predefined criteria: name of DiHA, registry inclusion, company, platform and price details. Data were extracted from the publicly available data sources of the BfArM [22]. Prices represent costs covered by SHI and were analysed descriptively and presented graphically in a boxplot.

Second, clinical trials on the permanently included DiHA identified in the respective documents of BfArM (summary section: information on positive health care effects) for each DiHA [22] were examined with regard to the study design, the endpoints investigated and the survey instruments used, and whether an intention-to-treat analysis was performed. In addition, data was extracted on diagnostic category and health problem, type and level of evidence of the identified trial, the number of trial participants randomised, type of interventions in experimental and control arm, planned time points of the longest follow-up, and evaluated outcomes and survey instruments used for the measurement of the outcome. The level of evidence (LoE) of identified clinical trials was determined based on classification according to chapter 2, Sect. 3, § 11, Nr. 2 of the rules of procedure of the Federal Joint Committee [23]. Data availability on evidence for DiHA conditionally included was analysed descriptively and classified according to the information in the DiHA directory [22]. The data extraction was performed by two persons (HL and AC).

Third, the risk of bias in the identified randomised controlled trials (RCT) was assessed independently by two researchers (HL, HE) using the revised Cochrane risk of bias tool for randomised trials (RoB II) [24]. Disagreements during the assessment were discussed until agreement was reached. The ROB II tool covers five domains that can introduce bias. The domains consider aspects of trial design, conduct, and reporting. The domains “randomisation process” and the “effect of assignment to an intervention” were assessed at trial level, while the domains “missing outcome data”, “measurement of the outcome” and the “selection of reported results” were assessed at outcome level (morbidity) of each trial. Each domain is subdivided into several aspects and each aspect is assessed against a series of questions that help to identify the risk of bias. The assessment of each aspect is finally combined into a judgement for one domain and for the entire trial at the outcome level [25]. To assess ROB II, all available publications on a trial, trial protocols and trial registry entries, if available, were used.

Results

Overview of DiHA included in the German DiHA directory

As of February 2022, 30 digital health applications were listed in the DiHA directory of BfArM. The majority of the DiHA (n = 20/30) had been conditionally included while ten DiHA were listed permanently [22]. Additional file 1 gives an overview on DiHA listed in the directory. Two-thirds of DiHA were admitted in 2021 (n = 18/30), one-third in 2020 (n = 10/30) and two DiHA in early 2022. The manufacturers are most often start-up companies, small to medium-sized companies. Employee numbers range up to 140. Almost half (n = 14/30) of the DiHA are web-only applications, 13 of 30 DiHA are Apple iOS and Google Android applications only and three of 30 DiHA are accessible via of these three platforms. Most DiHA (n = 19/30) applied the concept of cognitive behavioural therapy in the therapeutic field of mental diseases and disorders [22]. However, the conditions targeted by the DiHA are diverse, ranging from cognitive behavioural therapy for tinnitus or alcohol dependence, to the application of individual elements of behavioural therapy for diabetes self-management (Fig. 3).

Fig. 3
figure 3

Clinical conditions of reimbursable DiHA, n = 30 (own illustration; data base: BfArM, 2022b)

Full size image

For a few DiHA (n = 4/30), there may be additional costs to be paid by the user if they wish to use certain additional services that are not covered by the prescription. Two thirds of the DiHA (n = 22/30) were only available in German language; only some in one additional language (n = 4/30) and several languages (n = 4/30) [22].

The costs of DiHA covered by SHI are listed in the directory. Those include costs of start-up package and costs for the use of the first 90 days.In some cases, there are cost for an additional 90 days of use (n = 4/30). Overall, the average price for the first 90 days is € 443.80 per quarter with a price spectrum ranging from € 119.00 to € 743.75. The most expensive DiHA thus cost 6.3 times as much as the cheapest. The average cost of a DiHA permanently listed is € 469.82 per first quarter (min = € 203.97, max = € 743.75). The average cost of the DiHA listed conditionally is € 430.80 per first quarter (min = € 119.00, max = € 718.20). No extra payments by patients are required for any DiHA listed in the DiHA directory. Figure 4 shows the cost of all DiHA in time trend by half-year. The prices are widely distributed within each half-year and, on average, increase slightly over time.

Fig. 4
figure 4

Quarterly cost of DiHA at the time of inclusion in the DiHA registry (own illustration)

Full size image

Data availability on evidence of health care effects for DiHA conditionally included in the registry

With the requirement to provide evidence of positive health care effects, 20 DiHA were provisionally listed (Additional file 1). For the DiHA CANKADO PRO-React Onco, an RCT is ongoing (NCT03220178). For the DiHA Novego, three RCTs were already published [26,27,28] and another RCT is currently ongoing. For the other DiHA, the current evidence is limited to efficacy data, systematic data evaluations, preliminary clinical trials, and “data from a sample” [22]. For all DiHA, an assessment of the medical outcome is planned. For five DiHA (CANKADO PRO-React Onco, Cara Care für Reizdarm, Kranus Edera, Mindable, Rehappy), an additional demonstration of patient-centred structural and procedural outcomes such as health literacy, patient sovereignty, adherence, and patient satisfaction is planned [22]. The manufacturers of the DiHA provisionally included state that they want to conduct an RCT (n = 19/20), and a multi-centre, prospective, two-arm study (Cankado) (n = 1/20).

Data availability on evidence of health care effects for DiHA permanently included in the registry

Table 1 gives an overview on permanently listed DiHA (n = 10/30), and clinical trials used in the fast-track pathway. RCTs were conducted for all DiHA with two clinical trials still in the process of publication (Kalmeda and Vivira). The number of trial participants randomised was on average 281 (max = 1,013; min = 56). The intervention and control groups were always similar in number.

Table 1 Overview of the evidence of the permanently included DiHA; 09/2020 – 02/2022 (based on information of DiHA directory, clinical trials, and study registers)
Full size table

Medical outcomes were investigated for all DiHA. In all published and unpublished trials (n = 13/13), at least one secondary medical outcome was investigated in addition to the primary medical outcome. Medical outcomes mostly include morbidity such as reduction of the disease symptoms, and improvement of quality of life. Patient-centred structural and procedural effects were investigated in clinical trials of two DiHA (Velibra and Vorvida), including reduced therapy-related costs and burdens for patients and their relatives, and increased patient sovereignty. In three trials (n = 3/13) [26, 36, 40], at least one outcome was measured by different survey instruments. In two trials [29, 38], a survey instrument was used to investigate primary and secondary outcomes. The survey instruments were partly validated instruments, such as Patient Health Questionnaire-9 (deprexis, Kalmeda), the General Depression Scale (HelloBetter Diabetes and Depression), and the Hospital Anxiety and Depression Scale (Elevida).

The intervention of the experimental arm in the clinical trials was either guided, i.e. with professional support via e-mail or other contact, or unguided, i.e. without professional support via e-mail or other contact, web- or mobile-based intervention via application, or care as usual and the application. The control group was either “Waitlist” or “Care as usual”.

Risk of bias assessment of identified RCTs

Risk of bias in eleven published RCTs of eight of ten permanently included DiHA was assessed based on the RoB II tool [24]. The agreement rate between the two assessors (HL, HE) across all clinical trials was 93.4%. Clinical trials of two DiHA were at the time of assessment in publication process (Kalmeda and Vivira). Nine of eleven RCTs were judged to be at high risk of bias and two clinical trials raise some concerns. Visual presentations of overall risk of bias per assessment category for included clinical trials can be found in Fig. 5, and a justification of the evaluation in the Additional file 2.

Fig. 5
figure 5

Evaluating risk of bias of included clinical trials (own illustration)

Full size image

Randomisation

The randomisation process was associated with a low risk of bias in three clinical trials. These clinical trials showed random allocation sequence generation and concealed allocation. About two-thirds (n = 7/11) of the clinical trials were associated with some concerns as no information on the allocation sequence was available. One study [38] resulted in a high risk of bias, as baseline differences between intervention groups suggest a problem with the randomisation process.

Deviation from the intended intervention

For deviations from intended interventions, almost all clinical trials (n = 10/11) had low risk of bias, as no deviation from the intended intervention was observed. Almost all authors conducted an intention-to-treat analysis (n = 10/11). The authors of one study [39] did not provide information on the analyses conducted to estimate the effect of assignment to intervention.

Missing outcome data

For around two-thirds of the clinical trials (n = 7/11), there was substantial amount of missing outcome data. There is reason to suggest that missing outcome data is related to its true value, which is why the potential for bias in this category was estimated to be high. Sensitivity analyses was performed for two clinical trials [33, 35]. Those were assessed with a low risk of bias in the domain. In two clinical trials [26, 29], it was not likely that missing outcome data depended on its true value—this led to some concerns.

Measurement of the outcome

All eleven clinical trials raised some concerns as participants’ knowledge of their assignment to the intervention or control could theoretically lead them to over- or understate their outcome measurements; however, there is no evidence that such bias occurred.

Selection of the reported results

Almost all of the clinical trials (n = 10/11) had a low risk of bias in the selection of the reported results as a pre‐specified protocol was provided and data produced was analysed in accordance with the pre‐specified analysis plan (Additional file 2). Almost one third of the clinical trials (n = 3/11) raised some concerns as it remains unclear if selective reporting occurred (Additional file 2). One third of the clinical trials (n = 3/11) had a high risk of bias as outcomes that were supposed to be investigated according to the protocol were not mentioned in the study or were not evaluated (Fig. 5).

Discussion

European countries are struggling to advance DiHA adoption due to several reasons. Germany is the first country in Europe where DiHA have been systematically included into the benefit basket of SHI and therefore can be prescribed by office-based physicians and psychotherapists. With the fast-track pathway aiming at evidence-based inclusion of DiHA into SHI’s benefit basket, DiHA are available to respective patient groups Germany-wide instead of being available only for patients through certain selective programmes of individual sickness funds. Although, the fast-track-pathway can serve as an example for other countries on how to advance the adoption of DiHA with a simultaneous focus on improving health care effects [7], we identified several shortcomings that should be taken into account by policy makers and industry to finally pave the way for evidenced-based decision-making regarding DiHA. (1) Reporting quality in studies often is insufficient. (2) Shortcomings have been identified, e.g. regarding an adequate control group. (3) Patient-centred structural and procedural effects are given only little consideration, although those might be important in real world settings. (4) There is a lack of transparency as to whether and to what extent prices reflect benefits. These points are discussed in more detail below.

About one third of all DiHA listed in the SHI’s directory is permanently listed. A comparative study must have been conducted to prove positive health care effects [43]. Although the manufacturer is free in selecting the study design, the study design depends on the type of DiHA and the health care effects to be evaluated [43]. Our study shows that all permanently included DiHA (n = 10) in the directory are based on RCTs (n = 13); two are still in the publication process. However, a high risk of bias was found for eleven clinical trials (seven DiHA) and there were some concerns for two clinical trials. In particular, the clinical trials did not score well in terms of missing outcome data and measurement of the outcomes. Regarding the control group, the Federal Association of Sickness Funds criticises that often there is no active control group, but instead a waiting list [44]. Therefore, it is not possible to determine a treatment advantage of the DiHA compared to other DiHA or conventional treatment. Manufacturers argue that a control group that does not receive guideline-compliant therapy would reflect the common care reality in Germany [45]. It is questionable with regard to the comparison group whether an DiHA plus standard treatment or standard treatment alone would be sufficient [46]. A comparison group could be a waiting group approach, a guideline-conform face-to-face treatment or a regular treatment approach, which represents the most promising approach as the other two approaches are ethically less justifiable or represent a clear difference to the DiHA [47]. It would be conceivable to evaluate continuously changing variants of the same DiHA that are constantly compared with each other in a randomised way [48]. However, also the results of the experimental arm could be biased because the authors did not provide information on concomitant treatment. Trial authors must therefore indicate all concomitant treatments that the patients receive during the trial. In addition, the small number of study participants randomised in some clinical trials should be viewed critically, as this can limit the validity of the study [49].

It is remarkable that all clinical trials of the permanently included DiHA reported outcomes with regard to a medical outcome, but only a few evaluate a patient-centred structural and procedural outcomes. This could be due to the fact that the clinical trials were conducted or started before the DiHA legislation, and the operationalisation of patient-centred structural and procedural outcomes is complex [50, 51]. For provisionally listed DiHA, more manufacturers plan to prove patient-centred structural and procedural outcomes [22]. Therefore, increased evaluation of these outcomes can be expected in future, necessitating new study designs and paving the way for the use of real-world data. Using real-world-data and thus alternative study designs in the evaluation of health care effects of DiHA would also have been possible and is also particularly suitable for the evaluation of the frequency of use and compliance of DiHA use. Therefore, guidance documents on the use of real-world data and respective study designs may help to set up appropriate studies. However, an early exchange with the BfArM is recommended in order to plan the evaluation concept accordingly [46]. Until now, new study designs are not yet widely used, but will be relevant in future [52]. There is a need for appropriate study characteristics to support the generation of evidence [19]. Advantages in pragmatic randomized trials can be seen because this study design is feasible and suitable for the characteristics of DiHA [48, 53].

The reimbursement prices for DiHA are characterised by a wide range. This led to disagreements between the SHI and the DiHA-manufacturers, with the Federal Association of Sickness Funds criticising the current legal basis [44]. According to this, the current law place little emphasis on the assessment of the benefit of a DiHA for patients and leads to excessive prices. However, with the new Framework Agreement on negotiation of maximum reimbursement amounts [54], DiHA will in future be grouped according to the indication and their positive health care effect. After calculation of the group specific maximum reimbursement price, the product-specific maximum reimbursement amount applicable from October 2022 will depend both on the DiHA status (conditionally versus permanently included DiHA) and on the number of redeemed prescription codes/activation codes [54]. In addition, the Framework Agreement specifies the calculation of cumulative thresholds, below which reimbursement by SHI becomes possible without an additional negotiation. If the price set by the manufacturer is above the maximum reimbursement price, out-of-pocket payments may arise for the patient.

There are similarities with the procedure for new pharmaceuticals entering the market, e.g. free price setting in the first year, or price negotiations between the manufacturer and the Federal Association of Statutory Health Insurance Funds with valid prices from the 13th month after market approval for pharmaceuticals (DiHA: price negotiations with valid prices from the 13th month after inclusion in the DiHA registry) [55]. However, the transparency of the fast-track assessments of the benefit of DiHA at the BfArM should be improved. Until now it is unclear whether the body of evidence included in the assessment is complete and to what extent the quality of the evidence and the extent of the positive health care effect are included in the reimbursement decision, while pharmaceuticals are subject to price negotiations according to the level of the additional benefit.

The main limitation of this study needs to be mentioned. The quality assessment is based on those clinical trials used by BfArM for decisions on inclusion in the DiHA directory. It is possible that these are not all clinical trials that exist on the corresponding DiHA.

Conclusion

This is the first study to provide both a descriptive overview of first results of the fast-track pathway to include DiHA into SHI’s benefit basket, to examine the body of evidence on health care effects used in the fast-track evaluation and to assess the quality of these clinical trials. Results show that primarily the medical outcome was evaluated to show health care effects. As the results of the present study are rather sobering regarding trial quality, improvements are required also with respect to the reporting quality in the clinical trials. Furthermore, the use of patient-centred structural and procedural outcomes in addition to medical outcomes should be strengthened, but may require guidance on the use of real-world data and appropriate study designs. In the future it is also important that prices and benefits of DiHA are in reasonable proportion to each other. Although, the fast-track pathway encourages digital innovations and improves patient access to them, in terms of assessing effectiveness and pricing, systematic structures should be implemented to collect and assess respective data for this purpose.