Introduction

Syphilis is caused by the sexually transmitted spirochete Treponema pallidum subspecies pallidum. Without timely treatment, disease in adults can progress from the early stage (primary and secondary syphilis) to the late stage (tertiary syphilis) causing severe cardiovascular and neurological disease and death [1]. Infection in pregnancy can result in stillbirth, neonatal death, prematurity, low birth weight, and congenital syphilis in neonates [2]. Global prevalence estimates of syphilis have remained steady thanks to expanded antenatal care coverage, however congenital syphilis is still a significant contributor to burden of disease in children [3, 4]. In some high-income countries like Australia and the United States, where eradication was once a public health prospect, syphilis cases are now resurging [5, 6]. Syphilis programs are substantially underfunded compared to nearly every other infectious disease, increasing the importance of economic evaluation studies to guide investment decision-making [7].

A common measurement of the impact of the disease is quality of life, often framed within the context of quality-adjusted life years (QALYs) or disability-adjusted life years (DALYs) [8]. Both measures combine the quality of life and duration lived in that state: the QALY is a measure of the amount of time lived in any given health state (measured by a utility weight) and the DALY is a combined measure of years lived with a disability (measured by a disability weight) or illness and the years of life lost – the values range from 0 to 1: for utility weights, perfect health is given a weighting of 1 and death is given a weighting of 0, and vice versa for disability weights. The Global Burden of Disease (GBD) projects remain dominant in developing health state utility values. The disability weight was introduced by the GBD team in the 1990s [9]. The disability weight methodology and the empirical data supporting it were substantially revised and updated for the 2010 iteration of GBD, and has received iterative updates to its evidence base in subsequent years [10].

Policymaking for syphilis screening and treatment programs, and investment in new tools for syphilis prevention and management, should be informed by up-to-date, accurate and well-designed economic evaluation studies. If health states are given weights that misrepresent real-life preferences, results can significantly underestimate or overestimate the cost-effectiveness of interventions [11]. The development of utility and disability weights for specific populations can be time-consuming and expensive [12]. In lieu of generating high-quality primary studies for a population, researchers commonly extrapolate using previously published utility and disability weights.

Systematic reviews on syphilis have focused on diagnostics and treatment [13, 14], but not on economic components. We aimed to summarise studies reporting utility and disability weights for health states associated with syphilis. Furthermore, we sought to locate the primary sources of these values and report how they were developed.

Methods

We conducted a systematic review following the guidance from the Cochrane Handbook and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for reporting [15, 16]. The study was registered in the International Prospective Register of Systematic Reviews (PROSPERO, CRD42021230035).

Inclusion criteria

The criteria for inclusion for economic evaluations were: the participants were men, women, or children with syphilis; the intervention was any program or procedure to prevent, control, or treat syphilis infection; the main outcomes were either cost-per-DALY or cost-per-QALY or the valuation of the health states associated with syphilis infection. For primary studies, we included them if a valuation for health state utility values was performed or if we found them as the primary source for a study. We did not explicitly search grey literature, though we did include government burden of disease studies referenced by included economic evaluations. The search was restricted to publications from January 1 2000 to February 4 2022. We excluded qualitative studies, studies without primary data, duplicates and studies not in English.

Search strategy

Six databases (MEDLINE, EMBASE, NHS Economic Evaluation Database, Health Technology Assessment, Database of Reviews of Effects, Web of Science Core Collection) were searched on January 7, 2021, with an updated search on February 4, 2022 (see Supplementary Tables S1-7). The search was adapted from a systematic review valuing the health states associated with chlamydia [17]. Reference lists of studies were manually searched to find additional studies and to find the primary source of the utility or disability weights.

Study selection

Two reviewers (PM, CW) independently screened the abstracts which met the inclusion criteria using Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia). Any discrepancies were reviewed by a third reviewer (JO). Quality assessment or quality of reporting was conducted using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist and the Consensus Health Economic Criteria (CHEC) checklist for economic evaluations, and an appraisal checklist by Picot for health-related quality-of-life primary studies (see Supplementary Tables S8-10) [18,19,20].

Data extraction and analysis

Two reviewers (PM, CW) independently extracted data into an Excel spreadsheet. The data extracted from economic evaluations were author, year, country, study aims, study participants (gender, age, risk characteristics), study outcomes (DALY, QALY), health states related to syphilis, and utility or disability weights, and durations of disease. The data extracted from primary studies were the study population, methods used to calculate utilities, and health states valued with their results. The data extracted from the burden of disease studies were methods, the health states and their disability weights, and durations of disease. Descriptive statistics were used to summarise the characteristics of the included studies. Where information was lacking, we contacted one corresponding author who provided additional data.

Results

Across the initial and updated searches, we returned 3401 studies (3041 from the initial search and 360 from the updated search). Following title and abstract screening, a total of 93 studies were selected for full-text review. Of these, 36 studies were included in our review. In total, 22 economic evaluations were included [21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40]; fifteen had the outcome framed in DALYs [21,22,23,24,25,26,27,28,29,30,31,32,33,34, 41], and seven had the outcome framed in QALYs [35,36,37,38,39,40, 42]. Two primary studies were identified after hand-searching reference lists [40, 43]. Note that one primary study was also an economic evaluation [40]. Thirteen burden of disease studies were included as part of the review: seven studies as part of the GBD series published by the Institute of Health Metrics and Evaluation [4, 44,45,46,47,48,49], four country-related landmark burden of disease studies [50,51,52,53], and two other separate burden of disease studies [54, 55]. Fig. 1 displays the PRISMA flowchart.

Fig. 1
figure 1

PRISMA flowchart of the search strategy. *one economic evaluation using QALYs was also a primary study and thus should only be counted once toward the total

Full size image

Supplementary Table S11 summarises the key characteristics of the 36 studies reviewed. For the 22 economic evaluations, two studies were from low-income countries [21, 27], three from lower-middle-income countries [32, 33, 41], three from upper-middle-income countries [22, 29, 31], eight from high-income countries [23, 35,36,37,38,39,40, 42], and six from a mixture of countries [24,25,26, 28, 30, 34]. Fifteen studies (68%) reported the cost-effectiveness of antenatal screening for syphilis [21, 22, 24,25,26,27,28,29,30,31,32,33,34, 36, 37]. In addition to studies reporting weights for mothers and their newborns, other common populations included blood donors and patients receiving blood transfusions [23, 41, 42], and people living with HIV [35, 39].

Economic evaluations

Table 1 summarises the characteristics of the 22 economic evaluations. Of these, the primary outcome was the cost-per-DALY-averted in 15 studies and the cost-per-QALY-gained in seven studies. Health states reported in the text below and Tables 1, 2, 3, 4 and 5 reflect the choice of wording within the respective study (e.g. “mild early syphilis” or “stage one”), however with ambiguous terms we have standardised them to reflect the more widely accepted medical terminology.

Table 1 Key characteristics of health economic evaluations related to syphilis infection
Full size table
Table 2 Disability weights for health economic evaluations where the primary outcome was measured in DALYs
Full size table
Table 3 Syphilis-related health states and utility weights used in cost-effectiveness studies where the primary outcome was QALYs
Full size table
Table 4 Primary studies with utility weights used in economic evaluations evaluating the impact of syphilis infections
Full size table
Table 5 Disability weights for syphilis-related conditions in Global Burden of Disease Studiesa and Related Studies
Full size table

DALYs

Table 2summarises the disability weights and associated durations used in the 15 economic evaluations in which the primary outcome was framed in DALYs [21,22,23,24,25,26,27,28,29,30,31,32,33,34, 41]. The commonly used health states for newborns were congenital syphilis (n=13) with disability weight ranges from 0·315 to 0·316, low birth weight (n=9) with ranges from 0·106 to 0·291, and miscarriage (n=3), stillbirth (n=13), and neonatal death (n=12) all with values of 0 or 1. Health states for adult syphilis included early syphilis (n=4), tertiary syphilis (n=4), and HIV and syphilis coinfection (n=1), with disability weight ranges from 0·006 to 0·38. Eleven economic evaluations sourced their disability weights directly from a Global Burden of Diseases study [21,22,23,24,25,26, 28, 29, 31,32,33]. Of these eleven studies, only one used weights from after the 2010 update [23]. Three studies referenced other economic evaluations in our review for their disability weights [27, 30, 34]. The only economic evaluation that did not use GBD weights assumed the health state utility value, giving a disability weight of 0·12 for syphilis [41].

Eleven studies used a disability weight of 0·315 for clinical congenital syphilis [21,22,23, 25, 26, 28,29,30,31,32, 34], while two used a weight of 0·316 [24, 27]. One of the studies which used 0·316 referenced the other study [27, 56]. Six studies applied the disability weight for three years [21,22,23, 28,29,30], five applied the disability weight over the life expectancy of the newborn [24, 25, 27, 32, 34], and two did not specify the applied duration of disability [26, 31].

Nine studies contained disability weights for low birth weight as a sequelae of congenital syphilis [21, 24, 27,28,29,30,31, 33, 34], seven of which used a disability weight of 0·106 [21, 24, 27,28,29,30,31]. Two studies by the same author used a disability weight of 0·291 [33, 34]. Four studies applied the disability weight for one year [21, 28,29,30], and five studies applied the disability weight over the life expectancy of the newborn [24, 27, 31, 33, 34].

Thirteen studies included stillbirth and/or neonatal death attributed to syphilis [21, 22, 24,25,26,27,28,29,30,31,32,33,34]. In ten studies, stillbirths and neonatal deaths were counted as a full discounted life expectancy lost due to disability (i.e. disability weight of 1) [22, 25,26,27,28,29, 31,32,33,34]. Of the other three studies, one study gave neonatal death a disability weight of 1, but gave stillbirth a disability weight of 0 [21]; one study converted stillbirths and neonatal deaths to a set quantity of DALYs, calculating 4·95 DALYs per stillbirth and 9·4 DALYs per neonatal death [24]; another study applied years of life lost from stillbirths and neonatal deaths only up to 20 years of age with discounting [30]. Three studies listed miscarriage as an adverse pregnancy outcome [21, 22, 28], and two of these studies gave equal weight to stillbirth and neonatal death (i.e. disability weight = 1) [22, 28], with one study valuing miscarriage as 0 (i.e. no disability) [21]. Two studies included induced abortion as an adverse pregnancy outcome, with one study giving it a disability weight of 1 [22], and another study giving it a disability weight of 0 [29].

Four economic evaluations contained disability weights for adult syphilis [23, 28, 29, 41], using results from editions of the GBD from 1990 to 2015. The most recent of the three studies, published in 2019, used disability weights of 0·006 for mild early syphilis and 0·203 for tertiary syphilis which were taken from the 2015 GBD study [23]. One study used a disability weight of 0·12 and referenced a paper in which the utility weight for adult syphilis of 0·88 was an assumption; the duration of 90 days for adult syphilis infection was also assumed [41, 42].

QALYs

Table 3 summarises the utility weights used in the seven economic evaluations in which the primary outcome was framed in QALYs [35,36,37,38,39,40, 42]. Utility weights among these papers ranged from 0.65 to 0.9928. Four studies applied utility weights only for adults with syphilis [35, 38,39,40, 42], one study for newborns only [37], and one study for both adults and newborns [36]. One study applied the same utility weight of 0·737 regardless of disease stage (stage 1, stage 2, latent, tertiary) but varied the mean duration by stage [40]. One study used a utility weight of 0·82 for syphilis and HIV coinfection [35]; however, the referenced source did not contain the weight [39]. One study assessing transfusion recipients gave a utility weight assumption for primary syphilis of 0·88, and used a utility weight of 0·65 for tertiary syphilis taken from a catalogue which could not be verified [42, 57].

One study included a maternal perspective in the analysis, applying a reduced quality of life for women experiencing a stillbirth, neonatal death, or giving birth to a child with congenital syphilis, with utility weights of 0·92, 0·76, and 0·88 respectively, and applied these weights to a lifetime duration to generate QALYs for the mother [36].

Two studies used an adapted utility weight of 0·74 for congenital syphilis [36, 37]; this was based on studies valuing outcomes – specifically meningitis – from occult bacteraemia in children (i.e. bacteria in the bloodstream without an obvious source of infection) given the lack of a known utility weight for congenital syphilis [43, 58].

The duration for which utility weights were applied varied between all seven studies. One study applied a duration of 4·93 years for stage 1, 3·38 years for stage 2, 0·33 years for a latent period, and 0·2 years for an immune period [40]. One study which converted utility weights from disability weights used durations of 0·04 years for primary syphilis, 0·07 years for secondary syphilis, and 10 years for neurosyphilis [38]. Another study which converted utility weights from disability weights used durations of 0·7 years for primary syphilis, 3·6 years for secondary syphilis, and 7·7 years for neurosyphilis and tertiary syphilis [39]. Two studies applied all utility weights over a lifetime time horizon [36, 37]. One study assumed that if a syphilis diagnosis was missed it would be detected within the subsequent year (thus resulting in an implied duration of up to two years), and another study looking at transfusion-related transmission assumed that primary syphilis would be treated in the year following transfusion (giving an implied duration of one year) [35, 42].

Two studies framed their outcome in QALYs but used disability weights instead of utility weights to calculate QALYs [38, 39]. One study used unique disability weights to other economic evaluations included in this review: 0·0072 for primary syphilis, 0·041 for secondary syphilis, and 0·094 for tertiary syphilis [39]. One study looked at the cost-savings of removing certain sexually transmitted infections from a routine screening panel and reported cost savings per QALY lost – the QALY loss per missed adult syphilis infection was calculated to be 0·005 [38].

Half of the economic evaluations provided ranges or confidence intervals for disability or utility weights which were used in sensitivity analyses. The ranges for disability weights for the seven out of fifteen DALY studies [23, 25, 26, 28, 29, 32, 41], as well as the ranges for the utility weights for the four out of seven QALY studies are provided in Tables 2 and 3 [35, 36, 39, 42].

Primary sources and methods

Table 4 summarises the two primary sources for weights [40, 43]. The first primary source was a valuation study in which Canadian parents were asked to value health states associated with bacterial meningitis using a combination of standard gamble and visual-analogue scale. A condition-specific utility weight of 0·74 was created for “meningitis with minor brain damage”, and was subsequently used in a separate study analysing newborn screening strategies – in this study, the weight of 0·74 was applied to a health state named “mild developmental delay”. This health state and its weight was used by two separate economic evaluations to represent congenital syphilis [36, 37].

The second primary study developed its own utility weight by adapting the EuroQol-5 Dimensions (EQ-5D) questionnaire and conducting interviews with two groups: 29 syphilis-affected inmates and 67 patients from a sexually transmitted infections outpatient clinic. The mean utility weights of the two populations for primary, secondary, tertiary, and latent syphilis were 0.737. It then subsequently used the values in its own economic evaluation [40]. Further details of the methodology for developing utility weights for the primary sources can be found in Supplementary File S1.

Global burden of disease and related studies

The Global Burden of Disease project comprises seven major publications of disability weights spanning GBD 1990 and GBD 2019 [4, 44,45,46,47,48,49]. A minor GBD update (GBD 2016) was published as an update to GBD 2015 and does not provide revisions to syphilis disability weight data and so was not included in this review [59].

In 1990 and its corresponding 2004 update, the GBD team developed disability weights in small focus groups [44]. The initial set of disability weights were: 0·315 for congenital syphilis, 0·015 for primary syphilis, 0·048 for secondary syphilis, 0·283 for tertiary syphilis, and 0·106 for low birth weight. In GBD 2010 [4, 46,47,48,49], the methodology for deriving disability weights was substantially revised, using population-based surveys, wherein respondents are presented with pairs of narrative generic health state descriptions and asked to say which of two individuals they consider healthier. GBD researchers modelling the burden of specific diseases and injuries then choose the narrative generic health states that best match specific disease outcomes based on clinical expert opinion. The population-based disability weight for that generic health state is applied to the specific disease outcome. This approach seeks to value outcomes such as disfigurement or cognitive impairment equally regardless of the disease or injury responsible [10]. The suite of generic health states for which disability weights have been estimated has been expanded with subsequent surveys [60, 61], and with each new GBD publication health state assignments may be changed or combined. Combinations include, for example, “severe disfigurement and cardiovascular complications due to adult tertiary syphilis”, as a combination of the health states of “Level 3 disfigurement” and “moderate infectious disease, acute episode”, which is calculated to have a disability weight of 0·435 [10, 60]. In GBD 2019, there are ten unique health states relating to syphilis, all of which are for adult syphilis [4].

It is not uncommon for studies calculating the burden of syphilis to adopt earlier published health state utility values (and in the case of congenital syphilis and low birth weight, if authors wish to use GBD values, then they must be from prior to GBD 2010). The Victorian Burden of Disease Study 2001 [51], the Burden of Disease and Injury in Australia study 1999 [50], the State of Infectious Diseases in the Netherlands 2013 [53], Kuznik’s burden of disease study in sub-Saharan Africa in 2015 [54], and Liu’s burden of disease study in China in 2018 [55], all use the same weights as those in the initial Global Burden of Disease study in 1990. No studies provided estimations or recommendations for the durations of the different health states.

Only one related study, the Ontario Burden of Infectious Disease 2010 [52], provides new disability weights using a new quality of life instrument. For syphilis, severity weights were developed for four health states: primary syphilis (0·017), secondary syphilis (0·039), neurosyphilis (0·074), and congenital syphilis (0·139). These studies are summarised in Table 5.

Discussion

This systematic review summarised the health state utility and disability weights for syphilis. To our knowledge, this is the first systematic review of the health sequelae of syphilis infection. Similar systematic reviews have been performed for chlamydia [17], genital warts [62], and genital herpes [63]. Our findings show an overreliance on the same few weights, a lack of transparency when reporting how weights are derived, and inconsistency when applying these weights with regard to the values of the weights, the clinical stages, and the respective duration over which they are applied.

Disability weights, used as part of the calculation for DALYs, were applied in 15 of the 22 economic evaluations in our review. Fourteen studies based their weights on the 1990 Global Burden of Disease study [21,22,23,24,25,26,27,28,29,30,31,32,33,34, 64]. The methodology for developing the initial disability weights is described in Supplementary File S2. This methodology was criticised for its lack of transparency and on ethical and distributional grounds [65,66,67,68]. The methodology was substantially revised in response to these criticisms in GBD 2010, but economic evaluations of syphilis have generally not taken up the modern GBD disability weights, which are based on population-based surveys and generic health state descriptions, rather than focus groups with disease-specific expertise [10]. Aside from the weights, we found inconsistency in the duration of both disability and utility weights. For example, of 11 congenital syphilis-related studies, six applied the weighting for only three years [21,22,23, 28,29,30], and the other five applied the weight over the life expectancy of the newborn [24, 25, 27, 32, 34]. For a single case, the difference between using a disability weight for three years and a lifetime is significant (without discounting or age weighting, as per GBD 2010 onward) [67]. As an example, 0·315 multiplied by three years gives 0·945 years lost to disability for a single case, whereas 0·315 multiplied by a life expectancy of 65 yields 20·475 years lost to disability. There was a similar finding for utility weights. Of six QALY-related economic evaluations relating to adult syphilis, there were six unique sets of durations for syphilis health states ranging from weeks to total life expectancy [35, 36, 38,39,40, 42].

Our study highlights the limited evidence base for utility and disability weights and the durations they are applied for syphilis. Four economic evaluations explicitly stated a lack of validated evidence for syphilis utility weights may have led to uncertainty in their results [22, 36,37,38]. For such an uncertain variable, only half of the economic evaluations used ranges for disability or utility weights in their sensitivity analyses [23, 25, 26, 28, 29, 32, 35, 36, 39, 41, 42]. Outside of the GBD studies, we found only two primary sources for syphilis utility weights which developed their own utility weights for burden of disease calculation: one for congenital syphilis and one for adult syphilis [40, 43]. The congenital syphilis weights come from a study of meningitis in children [43]; this was used in two syphilis economic evaluations with no biomedical justification for equating congenital syphilis with childhood meningitis [36, 37]. A primary study for adult syphilis reflects a similar lack of evidence base for utility weights [40]. The study combined two Chilean prison populations (combined sample of 96 adults) to produce a single weight (0·737) that they applied to all forms of adult syphilis, from the primary stage to the life-threatening tertiary stage.

The disability weights and duration estimates constructed by the Global Burden of Disease project for the 1990 estimates have been updated. Salomon undertook work to update the weights for the 2010 GBD using a new methodology which addressed some of the concerns levelled at the original GBD 1990 methodology (see Supplementary File S3) [10, 60]. The interventions in many of the studies related to antenatal care, thus congenital syphilis and its sequelae were the most common health states to be valued in our review with a total of sixteen out of 22 economic evaluations doing so. However, there is little guidance in the updated disability weights as to which health states should be used when calculating the burden of childhood health states, such as for “congenital syphilis” and “low birth weight”, which no longer exist in GBD studies from 2010 onward [4]. For example, the 2019 GBD presents hundreds of congenital health sequelae that use niche wording such as “moderate hearing loss with ringing due to other congenital anomalies”. Given the myriad of clinical presentations associated with congenital syphilis [69, 70], guidance on the selection or pooling of relevant health states is needed to ensure studies take a consistent approach to the selection of weights for their models. We hope that future GBD studies reintroduce congenital syphilis and its sequelae to aid researchers in representing quality-of-life loss in these populations. For this systematic review, despite there being twelve economic evaluations using DALYs published after 2010, only one study used weights taken from after the GBD 2010 update [23]. Of three studies published after 2018 using GBD weights, two economic evaluations and one burden of disease study published have not used the GBD’s updated weights and have instead used the GBD 1990 weights [30, 31, 55]. Within the economic evaluations, there was minimal discussion or justification of the use of older weights, though the lack of congenital syphilis weights from GBD 2010 onward is likely contributing to the use of weights from earlier versions of the GBD estimates, especially in analyses which include the burden of congenital syphilis.

The main strength of this paper is its comprehensive overview of the use of health state utility and disability weights for economic evaluations of syphilis. This includes a comparative analysis of economic evaluations and primary sources. Limitations of this study include the omission of studies not published in English, though empirical evidence demonstrates little impact on systematic review conclusions [71, 72], and that we did not explicitly search the grey literature. We restricted our search for publications after January 2000, which may have resulted in missing some older studies. Overall, the quality assessment of economic evaluations was high – however, no studies presented a health economic analysis plan and distributional effects were seldom discussed. Another limitation is the exclusion of economic evaluations which did not explicitly provide a numerical weight for a health state utility value related to syphilis, noting that 42 out of 90 studies which were assessed for eligibility did not explicitly provide the numerical weights used in their analyses – in this way, we may be underestimating the interconnectedness of the literature and the reuse of the same weights by studies which do not have explicitly stated weights in their manuscripts but have still used them to calculate cost-effectiveness.

Future research

We have uncovered a gap in the literature which deserves further research: the creation of validated and societally representative weights for both congenital and adult syphilis to help inform economic evaluations. Consensus is required firstly for the weights of standardised stages of syphilis and secondly, on the median durations over which to apply them. This ensures economic evaluations draw accurate conclusions which then inform policymaking. Without accurate weights or durations, researchers should use sensitivity analyses for DALY and QALY calculations. Research into the secondary or collateral effects of disease is ongoing. Uniquely, one study in our review considered the maternal perspective of sequelae of congenital syphilis [36]. Poorly-captured effects of syphilis on quality of life – such as the effects of stigma or the effect of a stillbirth on a mother or family – should be researched and incorporated for more grounded results [73]. As health equity rises in importance as a policymaking agenda, it is important to recognise that syphilis is a disease which disproportionately affects those from low-income countries [74]. Equity impact analysis as a tool may further influence syphilis health state utility values and lead to more equitable policy outcomes [75].

Conclusion

Economic evaluations, which include syphilis utility or disability weights, are often recycling weights adapted from very different methods, primary sources, disease stage classifications and durations, which on further investigation, are based on limited evidence that is outdated and questionable in terms of its external validity. Aligning to recent updates for disability weights by the Global Burden of Disease initiative could be a pragmatic starting point to standardise. Given the complexity of syphilis and its wide variety of clinical health states, guidance is needed from the Global Burden of Disease team on how to correctly apply the disability weights by standardised clinical stages with consensus duration estimates. Until researchers use more accurate weights in economic evaluations, policymakers may be misinformed when considering the cost-effectiveness of syphilis programs.