Advertisement

Cost-utility Analysis of Opportunistic and Systematic Diabetic Retinopathy Screening Strategies from the Perspective of the Brazilian Public Healthcare System

  • Ângela J. BenEmail author
  • Jeruza L. Neyeloff
  • Camila F. de Souza
  • Ana Paula O. Rosses
  • Aline L. de Araujo
  • Adriana Szortika
  • Franciele Locatelli
  • Gabriela de Carvalho
  • Cristina R. Neumann
Open Access
Original Research Article

Abstract

Objective

To perform a cost-utility analysis of diabetic retinopathy (DR) screening strategies from the perspective of the Brazilian Public Healthcare System.

Methods

A model-based economic evaluation was performed to estimate the incremental costs per quality-adjusted life-year (QALY) gained between three DR screening strategies: (1) the opportunistic ophthalmology referral-based (usual practice), (2) the systematic ophthalmology referral-based, and (3) the systematic teleophthalmology-based. The target population included individuals with type 2 diabetes (T2D) aged 40 years, without retinopathy, followed over a 40-year time horizon. A Markov model was developed with five health states and a 1-year cycle. Model parameters were based on literature and country databases. One-way and probabilistic sensitivity analyses were performed to assess model parameters’ uncertainty. WHO willingness-to-pay (WHO-WTP) thresholds were used as reference (i.e. one and three times the Brazilian per capita Gross Domestic Product of R$32747 in 2018).

Results

Compared to usual practice, the systematic teleophthalmology-based screening was associated with an incremental cost of R$21445/QALY gained ($9792/QALY gained). The systematic ophthalmology referral-based screening was more expensive (incremental costs = R$4) and less effective (incremental QALY = −0.012) compared to the systematic teleophthalmology-based screening. The probability of systematic teleophthalmology-based screening being cost-effective compared to usual practice was 0.46 and 0.67 at the minimum and the maximum WHO-WTP thresholds, respectively.

Conclusion

Systematic teleophthalmology-based DR screening for the Brazilian population with T2D would be considered very cost effective compared to the opportunistic ophthalmology referral-based screening according to the WHO-WTP threshold. However, there is still a considerable amount of uncertainty around the results.

Key Points for Decision-Makers

Systematic teleophthalmology-based screening would be considered very cost effective compared to an opportunistic screening based on the WHO willingness-to-pay thresholds.

However, there is still considerable uncertainty around the results.

1 Introduction

Diabetic retinopathy (DR) is a microvascular complication of diabetes mellitus and one of the major causes of preventable blindness among working-age adults, affecting the quality of life [1, 2]. The prevalence of DR among people with diabetes mellitus is around 35% worldwide [1, 3]. Since individuals have no symptoms until severe retinal damage, prevalence estimates of DR are likely to be even higher than reported. Further increase in the prevalence of DR is also expected due to the rising prevalence of type 2 diabetes mellitus (T2D), especially in low- and middle-income countries [4, 5, 6]. Therefore, it is important that healthcare systems effectively organize eye care for this population.

DR evolves through recognizable states over time, accurate diagnosis technologies, and effective treatments exist [7]. Given that, various model-based economic evaluations have been conducted worldwide to assess the long-term cost effectiveness of screening strategies for DR, especially in terms of quality-adjusted life-years (QALYs, i.e. cost-utility analysis) [8, 9, 10, 11, 12]. Studies have compared various strategies and the comparisons made are highly dependent on the studies’ context [7, 11]. For instance, they compared strategies including a combination of screening types (e.g. opportunistic, systematic); screening intervals (e.g. 1, 2 or more years); screening technologies (e.g. indirect ophthalmoscopy, retinal photographs); screening setting (e.g. primary care, secondary care) and the kind of healthcare provider conducting the screening (e.g. ophthalmologist, endocrinologist, general practitioner, software) [7, 11]. Most studies have shown that systematic screening is more expensive but improves the quality of life compared to no screening or opportunistic screening. However, this conclusion may change according to the social context and the willingness-to-pay threshold of the country.

In Brazil, DR screening is mainly delivered opportunistically (i.e. offering ophthalmology referral to secondary care for individuals who seek medical attention at primary care). The main drawbacks of this strategy are (1) low access rates, (2) referral waiting lists are not based on the risk of vision loss, and (3) long waiting time for ophthalmology appointments at secondary care [13, 14]. To enhance access rates, a systematic population-based screening alternative has been proposed, offering ophthalmology referral to the complete diabetes mellitus population covered by the primary care services instead of only to those seeking medical care. Nevertheless, this alternative is likely to boost waiting lists, increasing waiting time for appointments at secondary care and may incur additional use of resources and costs.

One alternative to optimize a systematic population-based screening process is teleophthalmology. In teleophthalmology, retinal photographs are taken from the population at primary care, which are subsequently remotely assessed, and only individuals identified as being at high risk of vision loss are referred to secondary care [15]. This strategy has optimized eye care delivery in rural areas in other countries and/or in situations where the number of ophthalmology appointments is insufficient to supply the demand [16]. Additionally, the sensitivity of the teleophthalmology exceeds 80% justifying the use of this screening technology [15].

To the best of our knowledge, there is no study in Brazil comparing the current opportunistic ophthalmology referral-based screening strategy with systematic population-based alternative strategies. Considering the potential to increase population access to DR screening, the results of a cost-utility analysis comparing the aforementioned strategies could potentially aid decision makers in allocating the limited public healthcare resources to the most cost-effective strategy. Therefore, this study aims to perform a cost-utility analysis of diabetic retinopathy screening strategies from the perspective of the Brazilian Public Healthcare System.

2 Methods

2.1 Target Population and Perspective of Analysis

The target population included individuals with T2D, starting at the age of 40 years, without DR with a 40-year time horizon. Analyses were performed from the perspective of the Brazilian Public Healthcare System (SUS). The study was approved by the Ethics Committee of the Hospital de Clínicas de Porto Alegre (protocol number 1.063.244).

2.2 Setting and Location

The SUS is based on the primary care model, which aims to provide universal access and comprehensive healthcare, coordinate and expand coverage to the other healthcare levels (e.g. specialist care and hospital care) [17]. The Family Health Strategy (FHS) (i.e. the public primary care program at SUS) has been scaled up throughout the country covering 63% of the Brazilian population [18]. Despite the SUS moving towards universal coverage through the FHS, DR screening is mainly delivered opportunistically. According to the Brazilian National Health Survey [19], only 36% of individuals with diabetes mellitus reported attendance to DR screening.

2.3 Comparators: DR Screening Strategies

Detailed information about the three DR screening strategies themselves as well as their advantages and disadvantages is provided in Supplementary Table 1. In brief, the opportunistic ophthalmology referral-based screening is currently usual practice and consists of offering ophthalmology referral to secondary care for individuals with T2D who seek medical attention at primary care. The systematic ophthalmology referral-based strategy consists of offering ophthalmology referral to secondary care for all individuals with T2D covered by the FHS. For both strategies, annual screening was assumed.

The systematic teleophthalmology-based screening consists of offering retinal photographs to all individuals with T2D covered by the FHS. Retinal images are then sent to a remote ophthalmology center for evaluation. Individuals without DR are advised to undertake biannual screening at primary care. Individuals with DR or unreadable retinal photographs are referred to an ophthalmologist. Readable retinal photographs rate was based on data from a pilot project of DR teleophthalmology-based screening implemented at a public primary care service in Southern Brazil [20]. More detailed information about teleophthalmology-based screening training can be found elsewhere [20].

2.4 Model Structure

A model-based economic evaluation was performed to estimate the incremental costs per QALY gained over time between three DR screening strategies: (1) the opportunistic ophthalmology referral-based screening; (2) the systematic ophthalmology referral-based screening, and (3) the systematic teleophthalmology-based screening. Similar accuracy was assumed for all three screening strategies. A systematic review and meta-analysis found that the sensitivity of teleophthalmology exceeded 80% in detecting the absence of DR compared to the gold standard 7-field stereoscopic photographs (7F-ETDRS) [15]. Additionally, evidence suggests a substantial agreement (kappa 0.67–0.74) between teleophthalmology and the indirect ophthalmoscopy (i.e. eye examination using a magnifying lens and a light to diagnose retinopathy considered the usual practice in Brazil) [21]. The model did not include other diabetes mellitus complications. Figure 1 shows the model structure. The model structure was based on the workflow of a pilot project of DR teleophthalmology-based screening implemented at a public primary care service in Southern Brazil [20].
Fig. 1

Model structure. A square node represents the decision three between the three diabetic retinopathy screening strategies. Circles represent chance nodes (i.e. the probability of being screened or not). M Circled represents the Markov Model. Individuals could transit between the five health states: NoDR no diabetic retinopathy, Non-STDR non-sight-threatening diabetic retinopathy, STDR sight-threatening diabetic retinopathy, BB bilateral blindness, p_opport probability of attending the opportunistic screening, p_system probability of attending the systematic screening strategies, MortalityrateDM[40+_stage] relative risk of mortality associated with diabetes starting at age of 40 years-old, tp_ transition probability, p_RP probability of gradable retinal photographs

The model starts with a decision tree between the three DR screening strategies (Fig. 1). Each screening strategy was divided into two branches (i.e. screened and not screened) by a chance node. This chance node represents the probability of attending each screening strategy, which was 0.36 [95% confidence interval (CI), 0.33–0.38] for the opportunistic strategy according to the Brazilian National Health Survey [19] and 0.82 (95% CI, 0.70–0.92) [22] for the systematic population-based alternatives based on uptake DR screening rates found in countries with universal health coverage [7, 22].

In the teleophthalmology branch (Fig. 1), a correction was made for the fact that some of the patients’ retinal photographs were unreadable by the teleophthalmology-based screening. Subsequently, the probability of patients with unreadable retinal photographs attending the screening at secondary care is assumed to be 0.36 (i.e. patients are as likely to attend the screening at secondary care as in the opportunistic ophthalmology referral-based screening branch).

A node corresponding to a Markov model emerges from each screened and not screened branches (i.e. the circled M in Fig. 1). The structure of a Markov model is suitable to represent the DR condition as it allows to model repetitive health states over time [23]. Figure 2 shows the five possible DR health states, which were defined based on literature [8, 24]: (1) No Diabetic Retinopathy (NoDR), (2) Non-sight-threatening Diabetic Retinopathy (Non-STDR), (3) sight-threatening Diabetic Retinopathy (STDR), (4) bilateral blindness (BB) and (5) Death. Individuals start on NoDR and might transition between health states at each 1-year cycle based on transition probabilities extracted from the literature [25]. A half-cycle correction was adopted to compensate for the fixed time transitions at the end of the cycle. For the Markov model validation, a cohort analysis [26] was carried out to verify consistency with the incidence of DR reported in the literature [27, 28, 29]. The cohort analysis showed that about 60% of individuals would be either at Non-STDR or STDR health states, 2% would be at BB health state after approximately 20 years of follow-up, which is aligned with previous observational studies [27, 28, 29] (Supplementary Fig. 1). Supplementary Table 2 shows the number of cases of BB avoided by DR screening per 100.000 individuals with T2D.
Fig. 2

Markov Model structure. The DR health states are represented by ovals and possible transitions between states are shown by arrows. NoDR no diabetic retinopathy, Non-STDR non-sight-threatening diabetic retinopathy, STDR sight-threatening diabetic retinopathy, BB bilateral blindness

2.5 Model Parameters

2.5.1 Transition Probabilities

A non-systematic search on literature was performed to find DR health states’ transition probabilities. A combination of search terms such as systematic review, cost-utility analysis, cost effectiveness, diabetic retinopathy, and Markov models was used in PubMed and Google Scholar. The United Kingdom Prospective Diabetes Study (UKPDS 33) was referenced by at least two model-based economic evaluations comparing DR screening strategies [30, 31]. The UKPDS 33 was a randomized controlled trial designed to compare the effects of intensive and conventional diabetes mellitus treatment on the risk microvascular and macrovascular in patients with T2D over 10 years [25]. The transition probabilities between DR health states were based on data from this study [25] since similar longitudinal data were not available for the Brazilian diabetes mellitus population.

The transition probabilities from the conventional diabetes mellitus treatment group were chosen because the proportion of patients with controlled diabetes mellitus (i.e. 40% of patients had glycated hemoglobin of 7.9%) was similar to diabetes mellitus control rates at primary care in Brazil [32]. In the aforementioned group, the probability of DR progression found was 0.487 and the probability of blindness was 0.019 in 12 years [25]. The resulting annual probabilities of DR progression and blindness were 0.538 and 0.0016, respectively (Table 1). To take into account the increased risk of blindness according to DR severity, the relative risk (RR) of blindness was multiplied by the transition probabilities of the Non-STDR and STDR health states [33] (i.e. individuals at STDR health state had twice the risk of becoming blind compared to those at Non-STDR health state, Table 1). The RR of blindness related to different DR health states was based on literature [30, 33] and on the experience of ophthalmologists who work at SUS [20].
Table 1

Model parameters

Variable name

Parameters

Baseline

Sensitivity analysis

References

Attendance probability

 p_opport

Opportunistic strategy

0.36

0.33–0.38

[19]

 p_system

Systematic strategies

0.80

0.70–0.92

[22]

 p_RP

Gradable rate of RP

0.91

0.86–0.97

[20]

Annual transition probabilities

 tp_NoDR_NonSTDR

From NoDR to Non-STDR

0.0541

0.0516–0.0555

[25, 33]

 tp_NoDR_STDR

From NoDR to STDR

0.0541

0.0516–0.0555

 

 tp_NoDR_BB

From NoDR to BB

0.0016

0.0014–0.0020

 

 tp_Non-STDR_STDR

From Non-STDR to STDR

0.0541

0.0516–0.0555

 

 tp_NonSTDR_BB

From Non-STDR to BB

0.0040

0.0034–0.0050

 

 tp_STDR_BB

From STDR to BB

0.0080

0.0068–0.0090

 

Utility values

 u_NoDR

Utility of NoDR

0.748

0.698–0.798

[38]

 u_Non-STDR

Utility of Non-STDR

0.752

0.679–0.825

 u_STDR

Utility of STDR

0.628

0.521–0.726

 u_BB

Utility of BB

0.355

0.105–0.606

Annual costs

 c_NoDR_oph_ref

Cost NoDR in ophthalmology referral

R$59

R$50–R$67

[41]

 c_NoDR_teleoph

Cost NoDR in teleophthalmology

R$16

R$14–R$19

 

 c_Non-STDR

Cost Non-STDR

R$117

R$100–R$135

 

 c_STDR

Cost STDR

R$235

R$199–R$270

 

 c_BB

Cost BB

R$37

R$31–R$43

 

 tc_NoDRNon-STDR_STDR

From NoDR/ Non-STDR to STDR

R$122

R$104–R$140

[41]

 tc_NoDRNon-STDR_BB

From NoDR/ Non-STDR to BB

R$15

R$12–R$17

 

 tc_STDR_BB

From STDR to BB

R$79

R$67–R$91

 

 tc_RP

From NoDR to other states

R$16

R$14–R$19

 

discount_rt

Discount rate

0.05

0–0.10

 

 RR_laser

RR BB (STDR treatment/no treatment)

0.49

0.37–0.64

[34]

Baseline probabilities are displayed in the table. BB bilateral blindness, NoDR no diabetic retinopathy, Non-STDR non-sight-threatening diabetic retinopathy, R$ Reais, RP retinal photographs, RR relative risk, STDR sight-threatening diabetic retinopathy

Individuals who undertake DR screening received treatment with laser photocoagulation (i.e. the first-choice treatment for individuals with STDR and currently available at SUS), while individuals not screened do not get the benefits of the treatment. According to a systematic review [34], treatment with laser photocoagulation reduces the risk of DR progression by over 50% at 12 months (RR 0.49, 95% CI 0.37–0.64, Table 1) [34]. To adjust for life expectancy, the RR of mortality associated with diabetes stratified by age [35] was multiplied by the Brazilian population mortality rate based on data from the Brazilian Institute of Geography and Statistics [36]. The RR of mortality associated with diabetes was based on data from the National Health and Nutrition Examination Survey [35].

2.5.2 Measurement of Utility Values

The EQ-5D-3L was used to estimate utility values associated with the five DR health states [37] in a convenience sample of patients who participated in a pilot project of DR teleophthalmology-based screening implemented at a public primary care service in Southern Brazil [20]. The methodology and results of this pilot study will be presented in detail in a future publication [38]. Briefly, 206 patients with T2D who attended the screening were asked to complete the EQ-5D-3L, meaning that they rated their mobility, self-care, usual activities, pain/discomfort, and anxiety/depression on a three-point scale (i.e. no problems, some problems or extreme problems) [38]. The patients’ EQ-5D-3L health states were converted into utility values using the Brazilian EQ-5D-3L tariff [37]. The mean age of the sample was 63 years (SD 10.6) and the mean utility value was 0.76 (SD 0.19). The utility values associated with the various DR health states were used to estimate the QALYs in the Markov model (Table 1). The use of QALY as an effect outcome is recommended by the Brazilian Economic Evaluation guideline [39]. A discount rate of 5% was applied to the utility values as recommended by the aforementioned guideline [39].

2.5.3 Estimating Healthcare Utilization Costs

The recommendations of the Brazilian Ophthalmology Council [40] and the American Academy of Ophthalmology [24] were used to identify the healthcare utilization of patients at each DR health state and those resources demanded when patients transit from one health state to another (i.e. a top-down approach). This information was validated by one DR expert and two general ophthalmologists from SUS to verify consistency between recommended and usual practice (Supplementary Tables 3 and 4).

To identify the resources used in the teleophthalmology-based screening and estimate the costs incurred by using teleophthalmology (i.e. costs associated with DR diagnosis by using retinal photographs), data from a pilot project of DR teleophthalmology-based screening implemented at a public primary care service in Southern Brazil [20] was used (i.e. a bottom-up approach) (Supplementary Tables 5 and 6).

Healthcare utilization costs and resources used in the teleophthalmology-based screening were valued using the 2016 SUS reimbursement table (SIGTAP) [41], the Brazilian Ophthalmology Council recommendations [40], the Hospital de Clínicas de Porto Alegre database, and the National Employment System [42]. All prices were adjusted to the year 2019 using the consumer price index [43]. To allow for international comparison, the results were also converted from BRL(R$) to International Dollars ($) based on 2019 Purchasing Power Parity (1$ = 2.19R$) [44]. A discount rate of 5% was applied to the costs as recommended by the Brazilian Economic Evaluation guideline [39].

2.6 Cost-Utility Analysis

2.6.1 Base-Case Analysis

For the base-case analysis, the costs and QALYs were estimated based on the parameters shown in Table 1. The three strategies were ordered according to increasing costs and the dominance of one strategy over another was examined. A strategy can be considered dominant if it is less expensive and more effective than the alternative. Once the dominated strategy was excluded, the incremental cost-effectiveness ratio (ICER) was calculated between remaining non-dominated strategies.

2.6.2 One-Way Sensitivity Analysis

A one-way sensitivity analysis was performed by varying a single parameter value at a time according to the ranges described in Table 1. A variation of 10% in the discount rate was used as recommended by the Brazilian Economic Evaluation guideline [39]. A variation of 15% was adopted for the cost estimates based on the average variation used in other model-based economic evaluations performed in Brazil [45, 46]. The 95% CI was used in the sensitivity analysis of utility values and RR of laser photocoagulation treatment (Table 1).

2.6.3 Probabilistic Sensitivity Analysis

The probabilistic sensitivity analysis (PSA) was performed to assess parameter uncertainty around the ICER [47]. A gamma distribution was fitted for the costs and beta distributions for the other variables. Subsequently, 5000 random draws were taken from these distributions using Monte Carlo simulation techniques. The results from the PSA were used to estimate cost-effectiveness acceptability curves (CEACs) [48]. The CEAC shows the probability of a strategy being cost-effective for a range of different willingness-to-pay (WTP) thresholds [47]. Additionally, the cost-effectiveness plane (CE-Plane) was used to show the joint distribution of the 5000 Monte Carlo simulations of the incremental costs and QALYs between non-dominated strategies. The CE-Plane shows the difference in cost effectiveness between the new strategy and the comparator given a WTP threshold [47]. The proportion of replicates of cost-effective pairs lying below the WTP represents the probability that the alternative strategy will be cost effective [23].

In the absence of an established Brazilian-recommended WTP threshold, we used the approach recommended by the World Health Organization (WHO) [49, 50], which is based on the per capita Gross Domestic Product (GDP) for Brazil. Interventions with an ICER less than the per capita GDP (R$32.747 or $14,953/QALY gained in 2018 [51, 52]) were considered as very cost effective and those with ICER between one and three times the per capita GDP (R$32.747 to R$98.241 or $44,859/QALY gained) were considered as cost effective, and interventions with an ICER above 3 times the per capita GDP (>R$98.241/QALY gained) were considered as not cost effective [53]. Analyses were performed in the TreeAge Software, Williamstown, MA 2011.

3 Results

3.1 Base-Case Analysis

The opportunistic ophthalmology referral-based screening was less expensive than the other alternative strategies (Table 2). Compared with usual practice, the systematic teleophthalmology-based screening is more expensive, with an additional cost of R$903, but more effective, with an incremental QALY of 0.042. The systematic ophthalmology referral-based screening was more expensive (incremental costs of R$4) and less effective (incremental QALY of − 0.012) compared with the systematic teleophthalmology-based strategy and, therefore, dominated by the latter. The ICER of the systematic teleophthalmology-based screening compared with the usual practice was R$21445/QALY gained ($9792/QALY gained), which is below the minimum WHO-WTP.
Table 2

Base-case analysis

Screening strategy

Cost, R$

Incr. cost, R$

QALY

Incr. QALY

ICER (R$/QALY gained)

Opportunistic ophthalmology referral-based

841

0

10.136

0

0

Systematic teleophthalmology-based

1744

903

10.178

0.042

21445

Systematic ophthalmology referral-based

1748

4

10.165

− 0.012

Dominated

C/E cost-effectiveness ratio, ICER incremental cost-effectiveness ratio, Incr. incremental, (Incr. cost/Incr. QALY), QALY quality-adjusted life-year, R$ Reais

3.2 One-Way Sensitivity Analysis

The parameters with the biggest influence on the ICER were the BB utility value, discount rate, STDR utility value, RR of laser photocoagulation and cost of STDR health state, (Supplementary Fig. 2). The upper limit of the ICER, which resulted from the variation of the parameters, did not exceed the maximum WHO-WTP threshold, except for the BB utility value. This means that the proposed model was robust given the parameters’ uncertainty, except for the uncertainty surrounding health-related quality of life of individuals with BB.

3.3 Probabilistic Sensitivity Analysis

Figure 3 shows the CEAC of the three screening strategies. The probability of the opportunistic ophthalmology referral-based screening being cost effective decreased with increasing WTP values. By contrast, the probability of cost effectiveness of the systematic screening strategies increased with increasing values of WTP. At the minimum WHO-WTP threshold, the probability of the usual practice being cost effective was 0.51; whereas the probability of the systematic ophthalmology referral-based and teleophthalmology-based were 0.09 and 0.40, respectively. At the maximum WHO-WTP threshold, the probability of the usual practice and the systematic ophthalmology referral-based and teleophthalmology-based being cost effective were 0.32, 0.12 and 0.56, respectively.
Fig. 3

Cost-effectiveness Acceptability Curve showing the probability of cost-effectiveness (y axis) for each strategy at a range of willingness-to-pay in Reais (R$), for a unit of quality-adjusted life-years (QALY) gained (x axis)

The mean costs of the non-dominated strategies, i.e. opportunistic ophthalmology referral-based screening and the systematic teleophthalmology-based alternative, were R$1393 (95% CI 212–R$2223) and R$2877 (95% CI 529–4789), respectively. The mean QALY of usual practice was 15.34 (95% CI 2.40–26.68) and of the systematic teleophthalmology-based screening was 15.44 (95% CI 2.40–27.00). The PSA shows that the ICER between non-dominated strategies would be R$14840/QALY gained ($6776/QALY gained), which is below the minimum WHO-WTP.

Figure 4 shows the CE-Plane, comparing the non-dominated strategies. 48% of the joint distribution of the cost-effective pairs were below the minimum WHO-WTP, and 67% were below the maximum WHO-WTP. That is the probability of the systematic teleophthalmology-base screening being cost effective compared to usual practice is 0.48 and 0.67 at the minimum and the maximum WHO-WTP thresholds.
Fig. 4

Incremental Cost-effectiveness Plane showing the joint distribution of the 5,000 Monte Carlo simulations. Each point represents the incremental cost-effective pairs between the non-dominated strategies (i.e. systematic teleophthalmology-based screening alternative vs. opportunistic ophthalmology referral-based screening). The ellipses represent the 95% confidence interval. The World Health Organization willingness-to-pay threshold is represented by the dashed line, with a slope of R$32747/QALY gained (a) and R$98241/QALY gained (b)

4 Discussion

This model-based economic evaluation compared three DR screening strategies from the SUS perspective. The results suggest that the systematic teleophthalmology-based screening alternative would be very cost effective compared to the opportunistic ophthalmology-referral based screening based on the WHO-suggested WTP threshold (i.e. below the Brazilian per capita GDP) in the base-case analysis. However, there is still a considerable amount of uncertainty around the probability of the systematic teleophthalmology-based screening alternative being cost effective compared to usual practice.

4.1 Comparison with Literature

Researchers worldwide have simulated the cost effectiveness of systematic screening by teleophthalmology using Markov models [8, 9, 10, 12, 30, 31, 54]. Their findings are similar to some of our results, despite considering different sociocultural contexts and models [8, 9, 10, 12, 54]. Rachapell et al. [8] found that the rural systematic teleophthalmology-based screening program was cost effective compared to no screening from the healthcare provider perspective in India. Kawasaki et al. [10] also found that DR teleophthalmology-based screening was cost effective compared to no screening at the Japanese WTP. Kim et al. [9] found that the opportunistic ophthalmology referral-based screening was dominated over the systematic teleophthalmology-based screening according to the Korean WTP.

Regarding screening intervals, Scanlon et al. [54] found the systematic teleophthalmology-based screening, defined as usual practice in the United Kingdom since 2003, was cost effective when delivered every 5 years to people at low risk of vision loss and every 3 or 2 years to those at medium and high risk, respectively. In our model, we assumed a 2-year screening interval in the systematic teleophthalmology-based screening alternative instead of annually because it would be suitable to the population with DR in Brazil given the average of diabetes mellitus control rate of 40% [32].

Our model is simple compared to the aforementioned models and focused on assessing the benefits of DR screening, especially regarding access to laser photocoagulation treatment. We assumed that individuals would already have access to diabetes mellitus treatment at primary care and we did not include other diabetes mellitus complications. One of the strengths of our model is that it was based on a real-world scenario using the resources (e.g. procedures and treatments) available in the public healthcare system, which is more likely to be of interest to decision makers. The structure of the model is available as a supplementary material and may be used or updated for researchers from other countries as a first attempt to evaluate the benefits of DR screening.

4.2 Impact on Decision Making

In Brazil, there is an estimated total of 12 million people with diabetes mellitus [55], of whom approximately 4 million have DR and 1.3 million have a vision-threatening condition [56]. In a universal health system scenario, all would have access to screening. The main issue of the opportunistic ophthalmology referral-based screening is the lower access rates (i.e. 36%). A systematic ophthalmology referral-based screening (i.e. expanding the screening coverage to 80%) would potentially increase the number of people waiting for appointments at secondary care. Usually, individuals with T2D wait an average of 5.2 years for eye examination [20]. The distribution of ophthalmologists is unequal across the country and many of them work outside the SUS. Systematically offering ophthalmology referral to diabetes mellitus population would probably not be cost effective either. Our results suggest that the systematic ophthalmology referral-based screening would incur more costs without improving health-related quality of life compared to the systematic teleophthalmology-based screening strategy.

Our model assumed that the systematic teleophthalmology-based screening alternative would screen 80% of people living in the area covered by the FHS. In doing so, we expected an increase in costs but also in health benefits as confirmed by our findings. It is worth mentioning that teleophthalmology is not a replacement for a comprehensive eye examination by an ophthalmologist. Nevertheless, it holds the potential to reduce waiting list composed of people with a higher risk of vision loss who would have more benefits from the ophthalmology referral [16]. It is up to decision makers to determine whether the presented uncertainty around our results is acceptable.

4.3 Limitations and Strengths

The Markov model assumes that all individuals in a given health state are identical and the transition probabilities depend on the current health state, while in clinical practice people usually have other comorbidities and progress differently depending on their medical background. Nonetheless, the structure of a Markov model is the most recommended modeling technique to investigate the long-term cost effectiveness of chronic conditions [23, 57].

Our model did not allow for any recovery to full health from having a more severe condition, which might be a reasonable assumption for some sight-threatening DR conditions under treatment of new medications such as anti-vascular endothelial growth factor [58]. This may underestimate the benefits of the treatment of the screening.

One-way sensitivity analysis shows that utility values have a prominent influence on the ICER. The utility values used in the model may be higher than those of the Brazilian population with diabetes mellitus [59], which may result in an underestimation of the screening effect. A protocol of a systematic review and meta-analysis aiming to maximize the generalizability of utility values associated with DR health states was published [60]. However, the results were not available by the time this study was performed.

Another limitation was that the parameter choices were not systematically searched on literature. This may lead to doubts that the most appropriate evidence to the decision population has been used. Despite the efforts made to validate the model parameters along with ophthalmologists from the public health system, the probability sensitivity analysis shows a considerable amount of uncertainty in the model’s parameter values. A future update of the model Brazilian representative data, especially regarding health-related quality of life, is warranted.

Costs related to informal care (i.e. costs incurred with caregivers, family and/or friends due to DR condition) and lost productivity costs (i.e. absenteeism and presenteeism) were not included in the model due to the adoption of the healthcare perspective [61], which is recommended by the Brazilian Economic Evaluation guideline [39]. Additionally, further research is needed regarding the budget impact of implementing systematic teleophthalmology-based screening on a national scale. It will enhance optimal decision making in the field of eye care delivery to the Brazilian diabetic population.

5 Conclusion

Systematic teleophthalmology-based screening would be considered very cost effective for the Brazilian population with T2D compared to the opportunistic and systematic ophthalmology referral-based screening strategies according to the WHO-WTP threshold. However, it is up to decision makers to determine whether the presented uncertainty around the results is acceptable. Further investigation on the budget impact of implementing the systematic teleophthalmology-based screening alternative is needed to estimate the financial consequences of its adoption on a national level.

Notes

Acknowledgements

We thank the contribution by Prof. Dr. Daniel Lavinsky and by Dr. Felipe Mallmann as experts in diabetic retinopathy.

Author contributions

All authors contributed to the study conception and design. Material preparation and data collection were performed by Ângela J. Ben, Camila F. de Souza, Ana Paula O. Rosses, Aline L. de Araújo, Adriana Szortika, Franciele Locatelli and Gabriela de Carvalho. The formal analysis was performed by Ângela J. Ben and Jeruza L. Neyeloff. The first draft of the manuscript was written by Ângela J. Ben supervised by Cristina R. Neumann and Jeruza L. Neyeloff. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Compliance with Ethical Standards

Conflict of interest

Ângela J. Ben, Jeruza L. Neyeloff, Camila F. de Souza, Ana Paula O. Rosses, Aline L. de Araújo, Adriana Szortika, Franciele Locatelli, Gabriela de Carvalho, Cristina R. Neumann declare no conflict of interests.

Funding

This study was funded by the Departamento de Ciência e Tecnologia—Ministério da Saúde do Brasil. Rede Brasileira de Avaliação de Tecnologias em Saúde: Pesquisa de Efetividade Comparativa (PEC-REBRATS). Grant no.: 57/2013 MCTI/CNPq/MS—SCTIE—DECIT.

Supplementary material

40258_2019_528_MOESM1_ESM.docx (128 kb)
Supplementary material 1 (DOCX 128 kb)

References

  1. 1.
    Yau JWY, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, et al. Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care. 2012;35:556–64.CrossRefGoogle Scholar
  2. 2.
    Trikkalinou A, Papazafiropoulou AK, Melidonis A. Type 2 diabetes and quality of life. World J Diabetes. 2017;8:120–9.CrossRefGoogle Scholar
  3. 3.
    Leasher JL, Bourne RRA, Flaxman SR, Jonas JB, Keeffe J, Naidoo K, et al. Global estimates on the number of people blind or visually impaired by diabetic retinopathy: a meta-analysis from 1990 to 2010. Diabetes Care. 2016;39:1643–9.CrossRefGoogle Scholar
  4. 4.
    Telo GH, Cureau FV, de Souza MS, Andrade TS, Copês F, Schaan BD. Prevalence of diabetes in Brazil over time: a systematic review with meta-analysis. Diabetol Metab Syndr [Internet], p 8. 2016. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5015260/. Accessed 26 Jan 2017.
  5. 5.
    Ogurtsova K, da Fernandes JDR, Huang Y, Linnenkamp U, Guariguata L, Cho NH, et al. IDF diabetes Atlas: global estimates for the prevalence of diabetes for 2015 and 2040. Diabetes Res Clin Pract. 2017;128:40–50.CrossRefGoogle Scholar
  6. 6.
    Sabanayagam C, Banu R, Chee ML, Lee R, Wang YX, Tan G, et al. Incidence and progression of diabetic retinopathy: a systematic review. Lancet Diabetes Endocrinol. 2019;7:140–9.CrossRefGoogle Scholar
  7. 7.
    Jones S, Edwards RT. Diabetic retinopathy screening: a systematic review of the economic evidence. Diabet Med. 2010;27:249–56.CrossRefGoogle Scholar
  8. 8.
    Rachapelle S, Legood R, Alavi Y, Lindfield R, Sharma T, Kuper H, et al. The cost-utility of telemedicine to screen for diabetic retinopathy in India. Ophthalmology. 2013;120:566–73.CrossRefGoogle Scholar
  9. 9.
    Kim S-W, Kang G-W. Cost-utility analysis of screening strategies for diabetic retinopathy in Korea. J Korean Med Sci. 2015;30:1723.CrossRefGoogle Scholar
  10. 10.
    Kawasaki R, Akune Y, Hiratsuka Y, Fukuhara S, Yamada M. Cost-utility analysis of screening for diabetic retinopathy in Japan: a probabilistic markov modeling study. Ophthalmic Epidemiol. 2015;22:4–12.CrossRefGoogle Scholar
  11. 11.
    Pasquel FJ, Hendrick AM, Ryan M, Cason E, Ali MK, Narayan KMV. Cost-effectiveness of different diabetic retinopathy screening modalities. J Diabetes Sci Technol. 2016;10:301–7.CrossRefGoogle Scholar
  12. 12.
    Aoki N, Dunn K, Fukui T, Beck JR, Schull WJ, Li HK. Cost-effectiveness analysis of telemedicine to evaluate diabetic retinopathy in a prison population. Diabetes Care. 2004;27:1095–101.CrossRefGoogle Scholar
  13. 13.
    IBGE, editor. Pesquisa nacional de saúde, 2013: percepção do estado de saúde, estilos de vida e doenças crônicas: Brasil, grandes regiões e unidades da Federação [Internet]. Rio de Janeiro: Instituto Brasileiro de Geografia e Estatística—IBGE. 2013. https://biblioteca.ibge.gov.br/visualizacao/livros/liv94074.pdf. Accessed 21 Oct 2019.
  14. 14.
    Oliveira DF, Arieta CE. Demand, capacity and flow of care in ophthalmology in the Campinas region, Brazil. Invest Ophthalmol Vis Sci. 2014;55:6108.CrossRefGoogle Scholar
  15. 15.
    Shi L, Wu H, Dong J, Jiang K, Lu X, Shi J. Telemedicine for detecting diabetic retinopathy: a systematic review and meta-analysis. Br J Ophthalmol. 2015;bjophthalmol-2014-305631.Google Scholar
  16. 16.
    Sreelatha OK, Ramesh SV. Teleophthalmology: improving patient outcomes? Clin Ophthalmol Auckl NZ. 2016;10:285–95.CrossRefGoogle Scholar
  17. 17.
    Massuda A, Hone T, Leles FAG, de Castro MC, Atun R. The Brazilian health system at crossroads: progress, crisis and resilience. BMJ Glob Health. 2018;3:e000829.CrossRefGoogle Scholar
  18. 18.
    Andrade MV, Coelho AQ, Neto MX, de Carvalho LR, Atun R, Castro MC. Transition to universal primary health care coverage in Brazil: Analysis of uptake and expansion patterns of Brazil’s Family Health Strategy (1998–2012). PLoS One. 2018;13:e0201723.CrossRefGoogle Scholar
  19. 19.
    Malta DC, Iser BPM, Chueiri PS, Stopa SR, Szwarcwald CL, Schmidt MI, et al. Cuidados em saúde entre portadores de diabetes mellitus autorreferido no Brasil, Pesquisa Nacional de Saúde, 2013. Rev Bras Epidemiol. 2015;18:17–32.CrossRefGoogle Scholar
  20. 20.
    Rosses APO, Ben ÂJ, Souza CF de, Skortika A, Araújo AL de, Carvalho G de, et al. Diagnostic performance of retinal digital photography for diabetic retinopathy screening in primary care. Fam Pract [Internet]. 2017. https://academic.oup.com/fampra/article-abstract/doi/10.1093/fampra/cmx020/3093574/Diagnostic-performance-of-retinal-digital. Accessed 4 Apr 2017.
  21. 21.
    Malerbi FK, Morales PH, Farah ME, Drummond KRG, Mattos TCL, Pinheiro AA, et al. Comparison between binocular indirect ophthalmoscopy and digital retinography for diabetic retinopathy screening: the multicenter Brazilian Type 1 Diabetes Study. Diabetol Metab Syndr. 2015;7:116.CrossRefGoogle Scholar
  22. 22.
    NHS. Public Health England. NHS Diabetic Eye Screening Programme: Summary Statistics for England, 1 April 2015 to 31 March 2016. Public Health England; London, UK: 2016. [Internet]. 2016. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/605040/Diabetic_eye_screening_summary_infographic_2015_to_2016.pdf. Accessed 2 Sep 2019.
  23. 23.
    Drummond MF. Methods for the economic evaluation of health care programmes [Internet]. 3rd ed. Oxford: Oxford University Press. 2005. http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&doc_number=013190365&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA.
  24. 24.
    AAO. Diabetic Retinopathy PPP—Updated—American Academy of Ophthalmology [Internet]. Am. Acad. Ophthalmol. 2017. https://www.aao.org/preferred-practice-pattern/diabetic-retinopathy-ppp-updated-2017. Accessed 19 Mar 2019.
  25. 25.
    UKPDS Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes: UKPDS 33. The Lancet. 1998;352:837–853.Google Scholar
  26. 26.
    Software T, Williamstown I +1 413-458-0104 OBS, MA, USA 01267. Evaluating Markov Models [Internet]. TreeAge Softw. 2019. http://www.treeage.com/software-feature/evaluating-markov-models/. Accessed 21 Jun 2019
  27. 27.
    Harris MI, Klein R, Welborn TA, Knuiman MW. Onset of NIDDM occurs at Least 4–7 yr before clinical diagnosis. Diabetes Care. 1992;15:815–9.CrossRefGoogle Scholar
  28. 28.
    Klein R, Klein BE, Moss SE, Cruickshanks KJ. The Wisconsin Epidemiologic Study of diabetic retinopathy. XIV. Ten-year incidence and progression of diabetic retinopathy. Arch Ophthalmol Chic Ill 1960. 1994;112:1217–28.CrossRefGoogle Scholar
  29. 29.
    Wong TY, Mwamburi M, Klein R, Larsen M, Flynn H, Hernandez-Medina M, et al. Rates of progression in diabetic retinopathy during different time periods. Diabetes Care. 2009;32:2307–13.CrossRefGoogle Scholar
  30. 30.
    Ting J, Martin D, Hass M. A Markov model of diabetic retinopathy progression for the economic evaluation of a novel DR prognostic device, Centre for Health Economics Research and Evaluation (CHERE) working paper 2007/14. Sydney: University of Technology. https://www.researchgate.net/publication/5092688_A_Markov_model_of_Diabetic_Retinopathy_Progression_for_the_Economic_Evaluation_of_a_novel_DR_prognostic_device_CHERE_Working_Paper_200714.
  31. 31.
    Vijan S, Hofer TP, Hayward RA. Cost-utility analysis of screening intervals for diabetic retinopathy in patients with type 2 diabetes mellitus. JAMA. 2000;283:889–96.CrossRefGoogle Scholar
  32. 32.
    Bertoldi AD, Kanavos P, França GVA, Carraro A, Tejada CAO, Hallal PC, et al. Epidemiology, management, complications and costs associated with type 2 diabetes in Brazil: a comprehensive literature review. Glob Health. 2013;9:62.CrossRefGoogle Scholar
  33. 33.
    Janghorbani M, Jones RB, Allison SP. Incidence of and risk factors for proliferative retinopathy and its association with blindness among diabetes clinic attenders. Ophthalmic Epidemiol. 2000;7:225–41.CrossRefGoogle Scholar
  34. 34.
    Evans JR, Michelessi M, Virgili G. Laser photocoagulation for proliferative diabetic retinopathy. In: The Cochrane Collaboration, editor. Cochrane Database Syst Rev [Internet]. Chichester, UK: John Wiley & Sons, Ltd. 2014. http://doi.wiley.com/10.1002/14651858.CD011234.pub2. Accessed 4 Nov 2016.
  35. 35.
    Roglic G, Unwin N. Mortality attributable to diabetes: estimates for the year 2010. Diabetes Res Clin Pract. 2010;87:15–9.CrossRefGoogle Scholar
  36. 36.
    IBGE. Tábua completa de mortalidade para o Brasil—2013 [Internet]. Instituto Brasileiro de Geografia e Estatística. 2014. ftp://ftp.ibge.gov.br/Tabuas_Completas_de_Mortalidade/Tabuas_Completas_de_Mortalidade_2013/notastecnicas.pdf.Google Scholar
  37. 37.
    Viegas AM, Noronha K, Kind P, Maia AC, Miranda de Menezes R, De Barros Reis C, et al. Societal Preferences for EQ-5D Health States from a Brazilian Population Survey. Value Health Reg Issues. 2013;2:405–12.Google Scholar
  38. 38.
    Ben ÂJ, Souza CF de, Locatelli F, Rosses APO, Araujo AL de, Carvalho G de, et al. Health-related quality of life associated with diabetic retinopathy in patients of a public primary care service in Southern Brazil. forthcoming. Arch Endocrinol Metab. 2019.Google Scholar
  39. 39.
    REBRATS. Diretriz de Avaliação Econômica. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Gestão e Incorporação de Tecnologias em Saúde. 2013. http://rebrats.saude.gov.br/diretrizes-metodologicas. Accessed 18 Apr 2017.
  40. 40.
    CBO. Manual de Ajuste de Condutas 2012/Conselho Brasileiro de Oftalmologia. [Internet]. 2012. http://www.cbo.com.br/novo/cms/wp-content/uploads/2012/05/Manual-de-Ajuste-de-Condutas-2012.pdf.
  41. 41.
    SIGTAP-SUS. SIGTAP—Sistema de Gerenciamento da Tabela de Procedimentos, Medicamentos e OPM do SUS. [Internet]. 2017. http://sigtap.datasus.gov.br/tabela-unificada/app/sec/inicio.jsp.
  42. 42.
    SINE. Rede SINE—Portal do Fundo de Amparo ao Trabalhador [Internet]. 2016. http://portalfat.mte.gov.br/programas-e-acoes-2/sistema-nacional-de-emprego-sine/rede-sine/. Accessed 8 Aug 2018.
  43. 43.
    Brazil Consumer Price Index [Internet]. 2019. https://ycharts.com/indicators/brazil_consumer_price_index_wdi. Accessed 15 Sep 2019.
  44. 44.
    PPP. OECD (2016), Purchasing power parities (PPP) (indicator).  https://doi.org/10.1787/1290ee5a-en. Accessed 31 Oct 2016.
  45. 45.
    Toscano CM, Zhuo X, Imai K, Duncan BB, Polanczyk CA, Zhang P, et al. Cost-effectiveness of a national population-based screening program for type 2 diabetes: the Brazil experience. Diabetol Metab Syndr [Internet]. 2015. [cited 2019 Sep 15];7. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4628345/.
  46. 46.
    Bertoldi EG, Stella SF, Rohde LE, Polanczyk CA. Long-term cost-effectiveness of diagnostic tests for assessing stable chest pain: modeled analysis of anatomical and functional strategies. Clin Cardiol. 2016;39:249–56.CrossRefGoogle Scholar
  47. 47.
    Petrou S, Gray A. Economic evaluation using decision analytical modelling: design, conduct, analysis, and reporting. BMJ. 2011;342:d1766.CrossRefGoogle Scholar
  48. 48.
    Fenwick E, O’Brien BJ, Briggs A. Cost-effectiveness acceptability curves—facts, fallacies and frequently asked questions. Health Econ. 2004;13:405–15.CrossRefGoogle Scholar
  49. 49.
    WHO. WHO | Thresholds for the cost–effectiveness of interventions: alternative approaches [Internet]. WHO. 2015. http://www.who.int/bulletin/volumes/93/2/14-138206/en/. Accessed 30 Apr 2017
  50. 50.
    Edejer TT-T, WHO, editors. Making choices in health: WHO guide to cost-effectiveness analysis. Geneva: World Health Organization. 2003.Google Scholar
  51. 51.
    Produto Interno Bruto—PIB|IBGE [Internet]. 2019. https://www.ibge.gov.br/explica/pib.php. Accessed 16 Sep 2019.
  52. 52.
    PIB do Brasil: histórico e evolução | Infográficos | Gazeta do Povo [Internet]. Infográficos. 2019. https://infograficos.gazetadopovo.com.br/economia/pib-do-brasil/. Accessed 16 Sep 2019.
  53. 53.
    IBGE, editor. Conta-satélite de saúde: Brasil, 2010–2013. Rio de Janeiro: IBGE. 2015.Google Scholar
  54. 54.
    Scanlon PH, Aldington SJ, Leal J, Luengo-Fernandez R, Oke J, Sivaprasad S, et al. Development of a cost-effectiveness model for optimisation of the screening interval in diabetic retinopathy screening. Health Technol Assess. 2015;19:1–116.CrossRefGoogle Scholar
  55. 55.
    Duncan BB, Achtjtti C, Benia LR. Socioeconomic distribution of noncommunicable disease risk Factors in Urban Brazil: the Case of Porto Alegre. Pan Am J Public Health. 1993;13.Google Scholar
  56. 56.
    Lee R, Wong TY, Sabanayagam C. Epidemiology of diabetic retinopathy, diabetic macular edema and related vision loss. Eye Vis [Internet]. pp 2. 2015. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4657234/. Accessed 22 Feb 2017.
  57. 57.
    Weinstein MC, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices-Modeling Studies. Value Health J Int Soc Pharmacoeconomics Outcomes Res. 2003;6:9–17.CrossRefGoogle Scholar
  58. 58.
    Zhao Y, Singh RP. The role of anti-vascular endothelial growth factor (anti-VEGF) in the management of proliferative diabetic retinopathy. Drugs Context [Internet], p 7. 2018. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6113746/. Accessed Apr 1 2019.
  59. 59.
    da Mata AR, Álvares J, Diniz LM, da Silva MRR, Alvernaz dos Santos BR, Guerra Júnior AA, et al. Quality of life of patients with Diabetes Mellitus Types 1 and 2 from a referal health centre in Minas Gerais, Brazil. Expert Rev Clin Pharmacol. 2016;9:739–46.CrossRefGoogle Scholar
  60. 60.
    Sampson CJ, Tosh JC, Cheyne CP, Broadbent D, James M. Health state utility values for diabetic retinopathy: protocol for a systematic review and meta-analysis. Syst Rev. 2015;4:15.CrossRefGoogle Scholar
  61. 61.
    Drost RMWA, van der Putten IM, Ruwaard D, Evers SMAA, Paulus ATG. Conceptualizations of the societal perspective within economic evaluations: a systematic review. Int J Technol Assess Health Care. 2017;33:251–60.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Ângela J. Ben
    • 1
    Email author
  • Jeruza L. Neyeloff
    • 2
  • Camila F. de Souza
    • 3
  • Ana Paula O. Rosses
    • 4
  • Aline L. de Araujo
    • 5
  • Adriana Szortika
    • 6
  • Franciele Locatelli
    • 7
  • Gabriela de Carvalho
    • 7
  • Cristina R. Neumann
    • 7
  1. 1.Department of Health Sciences, Faculty of ScienceVrije Universiteit AmsterdamAmsterdamThe Netherlands
  2. 2.Hospital de Clínicas de Porto AlegrePorto AlegreBrazil
  3. 3.Faculty of MedicineUniversidade do Vale do TaquariLajeadoBrazil
  4. 4.Department of Social MedicineFederal University of PelotasPelotasBrazil
  5. 5.Telessaúde, Universidade Federal do Rio Grande do SulPorto AlegreBrazil
  6. 6.Ophthalmology ServiceHospital Moinhos de VentoPorto AlegreBrazil
  7. 7.Postgraduate Program in Epidemiology, School of MedicineUniversidade Federal do Rio Grande do SulPorto AlegreBrazil

Personalised recommendations