Evaluating the Outcomes Associated with Genomic Sequencing: A Roadmap for Future Research

The rapid pace of discovery in genomics has been remarkable. The first human genome was sequenced as recently as 2003, costing between US$500 million and US$1 billion [1]. It then took just 5 years for costs to fall so dramatically that next-generation sequencing (NGS) approaches entered the clinical research setting. NGS approaches allow either the whole genome (via whole-genome sequencing [WGS]) or parts of it (via whole-exome sequencing [WES] or targeted panels) to be sequenced in hours, at great depth and increasing sensitivity [2, 3]. This genomic information can inform diagnosis, prognosis and clinical management for a variety of disorders, particularly cancer and rare diseases. Over the past decade, genomic sequencing research studies have increased in size such that we are now said to be living in the million-genome era [4]. Large-scale sequencing projects such as the 100,000 Genome Project in the UK and the All of Us Program in the US are collecting an unprecedented amount of genomic, clinical and healthcare resource use data on individuals with cancer or rare diseases, as well as healthy individuals.

Some of these large-scale projects are now approaching completion, and national health services will soon have to decide whether WGS and WES should be translated into clinical practice for specific disorders. In many countries, these decisions will be informed by evidence on the cost effectiveness of these technologies; however, the health economic evidence base for WES and WGS is very limited [3]. A recent literature review identified just eight economic evaluations of either WGS or WES, six of which were cost-effectiveness analyses using diagnostic yield as the outcome measure [3, 5,6,7,8,9,10,11]. Only two publications presented cost-utility analyses using quality-adjusted life-years (QALYs) as the measure of health outcomes, as recommended by most health technology assessment (HTA) agencies [12, 13]. However, neither of these cost-utility analyses provides information on health outcomes that HTA agencies can use to inform the translation of NGS technologies into clinical practice for specific disorders. Bennette et al. estimated the cost effectiveness of generating information on incidental findings using NGS technologies, but evaluated a population screening approach rather than estimating health outcomes for a particular disorder [14]. Buchanan-Hughes et al. investigated the cost effectiveness of using bacterial WGS to guide antibiotic selection for urinary tract infections, and concluded that WGS is not cost effective in this context; however, these results are reported in a conference abstract that provides no information on QALYs [15].

Methodological uncertainty among health economists is one potential explanation for the lack of evidence on the health outcomes associated with genomic sequencing. Over the past decade, health economists have repeatedly questioned whether metrics such as the QALY, which focuses on clinical utility, can fully quantify the outcomes that are important to patients when they undergo genomic testing [16,17,18,19,20]. Some applications of genomic sequencing generate information that may not improve quality of life (as measured using preference-based health-related quality of life [HRQoL] instruments such as the EuroQol-five dimensions [EQ-5D] questionnaire) or extend life expectancy. One example is the use of WGS and WES to guide diagnosis in autism spectrum disorder [10, 11]. However, such applications may impact on patient wellbeing via non-clinical routes, generating ‘personal utility’. This is a particular issue for individuals with rare diseases, who often have lengthy diagnostic journeys but few (if any) treatment options once they receive a diagnosis. This could also be an issue if individuals without known health problems undergo genomic sequencing and find out that they have an elevated risk of a disease, but no preventive action can be taken to manage this risk.

There is no consensus among health economists on whether QALYs are sufficient to capture the clinical benefits of sequencing, or on the best way to capture the non-clinical benefits of sequencing. More fundamentally, health economists are also yet to agree on whether information on personal utility should feed into resource allocation decisions at all in this context, given that, in many jurisdictions, the costs associated with genomic sequencing will be met by the healthcare budget (hence an analytical perspective that more broadly considers patient wellbeing is not necessarily appropriate). This methodological uncertainty partially explains why existing economic evaluations of genomic sequencing have not gone beyond ‘narrow’ outcome measures such as diagnostic yield.

Compounding this uncertainty are practical issues. Most existing studies evaluating the outcomes associated with WGS or WES are designed to inform ‘local’ resource allocation decisions (for example, within laboratories or in individual hospitals) instead of contributing to within-country HTAs [21, 22]. Such studies therefore have less incentive to tackle methodological issues in outcome measurement in this context, and thus go beyond diagnostic yield. In addition, many existing studies have been conducted without the involvement of trained health economists [3]; therefore, even if investigators would like to better quantify clinical or non-clinical utility in this context, they may lack the expertise to do so. Large-scale sequencing projects are, in theory, well-placed to address these practical issues by involving health economists, facilitating larger within-country studies, and collecting data on clinical and non-clinical outcomes. However, with the exception of the Cancer 2015 study in Australia (which collected some EQ-5D data [23]), large-scale studies are yet to address these issues.

A Roadmap for Future Research

Despite these methodological and practical challenges, evidence on the relative cost effectiveness of WGS and WES will soon be required to inform the translation of these technologies into clinical practice. Without this evidence, these technologies may be used in clinical contexts that reduce rather than improve population wellbeing. There are several steps that health economists can take in the coming months and years to improve the evidence base on the clinical and non-clinical utility of WGS and WES that will underlie these implementation decisions.

First, it is crucial that health economists generate evidence on the clinical utility of genomic sequencing using the methods and metrics that are recommended by HTA agencies. Here, we are primarily referring to the use of preference-based HRQoL instruments such as EQ-5D questionnaires to generate utility weights that can be used to calculate QALYs. There is a perception that such instruments will be too insensitive to capture changes in patient wellbeing before and after undergoing genomic sequencing, but this assumption should be tested in empirical studies (for an example, see Davison et al. [24]) to ensure that any evolution in methods is evidence-based. Ideally, health economists should be involved in the early stages of large-scale sequencing studies to ensure that preference-based HRQoL instruments are administered to as many participants as possible. If this is not feasible—data collection is already underway in several large-scale studies—then stand-alone studies that collect HRQoL data from smaller subgroups of patients could still provide useful information on the clinical utility of WGS and WES. If these subgroups are carefully selected, and utility weights are generated for health states that are of broad relevance (for example, possessing a diagnosis), this information could then be extrapolated to related clinical contexts.

If there is reason to believe that patient wellbeing will change after undergoing genomic sequencing (for example, supportive qualitative evidence), but commonly used HRQoL instruments show no effect, a second step would be to explore the use of alternative health-state valuation techniques to generate utility weights within the QALY framework. The time-trade-off (TTO) technique is one such approach. This approach has not only been used to calculate preference weights for health states as defined by EQ-5D survey responses but can also be used independently of the EQ-5D survey instrument to generate utility weights for health states specified in alternative ways (for example, using vignettes) [25]. Evidence from discrete choice experiments (DCEs) is also increasingly being used to supplement the findings of TTO studies that generate utility weights [26]. This evidence could be informative in this context given that patient preferences for health states generating personal utility will likely be heterogeneous.

A third step would be to make full use of existing evidence on the diagnostic yield of WGS and WES. Studies that link this evidence to patient survival and quality of life (for example, trials or observational studies with long-term follow-up), could inform decision making regarding the translation of these technologies into clinical practice.

These initial steps are grounded in the extra-welfarist QALY-led approach applied by most HTA agencies around the world, and health economists should thoroughly investigate these lines of research before recommending a move away from the existing paradigm for outcomes measurement in the context of genomic sequencing. Should health economists conclude that such a move is justified, in the first instance there are several extensions to the extra-welfarist approach that could be considered. One is the use of multicriteria decision analysis, which could facilitate the simultaneous consideration of multiple outcomes [27]. A second potential extension is Sen’s capability approach, which could broaden the evaluative space of economic evaluations of WGS and WES [28].

Moving away from the extra-welfarist framework entirely, stated preference studies could provide an alternative monetary measure of the value attached to genomic sequencing outcomes. Such approaches are appealing in this context as they allow analysts to combine information on both clinical and personal utility within a single metric. This information could then be used within cost-benefit analysis (CBA). Unfortunately, few stated preference studies of genomic sequencing have been conducted to date [29], primarily because incorporating this information into full economic evaluations would represent a notable departure from the value judgements that underpin current HTA processes (which reflect the fact that decision makers want to maximise health) [20]. However, some decision makers do now allow evidence from DCEs and CBAs to be submitted as supplementary analyses [13]. Stated preference studies may therefore be a valid approach for valuing genomic sequencing outcomes going forward, if, and only if, health economists conclude that a move away from the existing paradigm is justified.

Health economists have a vital role to play in the translation of genomic sequencing into clinical practice, and a key task is to improve the evidence base on the clinical and non-clinical utility of WGS and WES. We have outlined a roadmap for future research to build this evidence base, and, alongside all of these steps, health economists must also proactively engage with key policymakers and government agencies to highlight the importance of collecting this evidence. In doing so, health economists can ensure that appropriate and timely decisions are made regarding the allocation of scarce healthcare resources to genomic sequencing.


  1. 1.

    National Human Genome Research Institute. The Cost of Sequencing a Human Genome. 2016. Available at: https://www.genome.gov/27565109/the-cost-of-sequencing-a-human-genome/. Accessed 12 Sept 2018.

  2. 2.

    Davies S. Annual Report of the Chief Medical Officer 2016, Generation Genome. London: Department of Health; 2017.

    Google Scholar 

  3. 3.

    Schwarze K, et al. Are whole-exome and whole-genome sequencing approaches cost-effective? A systematic review of the literature. Genet Med. 2018. https://doi.org/10.1038/gim.2017.247 (epub 15 Feb 2018).

    Article  Google Scholar 

  4. 4.

    The Genomes Project Consortium. A global reference for human genetic variation. Nature. 2015;526:68–74.

    Article  Google Scholar 

  5. 5.

    Sagoo G, et al. The budget impact and cost-effectiveness of introducing whole-exome sequencing-based virtual gene panel tests into routine clinical genetics. Cambridge: PHG Foundation; 2017.

    Google Scholar 

  6. 6.

    Schofield D, et al. Cost-effectiveness of massively parallel sequencing for diagnosis of paediatric muscle diseases. NPJ Genom Med. 2017;2(1):4.

    Article  Google Scholar 

  7. 7.

    Soden SE, et al. Effectiveness of exome and genome sequencing guided by acuity of illness for diagnosis of neurodevelopmental disorders. Sci Transl Med. 2014;6(265):265ra168.

    Article  Google Scholar 

  8. 8.

    Stark Z, et al. Prospective comparison of the cost-effectiveness of clinical whole-exome sequencing with that of usual care overwhelmingly supports early use and reimbursement. Genet Med. 2017;19(8):867–74.

    Article  Google Scholar 

  9. 9.

    van Nimwegen KJ. Health technology assessment of next-generation sequencing. Nijmegen: Radboud University; 2017.

    Google Scholar 

  10. 10.

    Tsiplova K, et al. A microcosting and cost-consequence analysis of clinical genomic testing strategies in autism spectrum disorder. Genet Med. 2017;19(11):1268–75.

    Article  Google Scholar 

  11. 11.

    Tsiplova K, Zur RM, Ungar WJ. A microcosting and cost-consequence analysis of genomic testing strategies in autism spectrum disorder. The Hospital for Sick Children Technology Assessment at SickKids (TASK); 2016.

  12. 12.

    National Institute for Health and Care Excellence. Guide to the methods of technology appraisal. National Institute for Health and Care Excellence; 2013.

  13. 13.

    Pharmaceutical Benefits Advisory Committee. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee (version 5.0). Canberra: Department of Health; 2016.

  14. 14.

    Bennette CS, et al. The cost-effectiveness of returning incidental findings from next-generation genomic sequencing. Genet Med. 2015;17(7):587–95.

    Article  Google Scholar 

  15. 15.

    Buchanan-Hughes AM, et al. Investigating the cost-effectiveness of bacterial whole-genome sequencing for enabling targeted antibiotic selection in urinary tract infections. Value Health. 2015;18(7):A510.

    CAS  Article  Google Scholar 

  16. 16.

    Buchanan J, Wordsworth S, Schuh A. Issues surrounding the health economic evaluation of genomic technologies. Pharmacogenomics. 2013;14(15):1833–47.

    CAS  Article  Google Scholar 

  17. 17.

    Foster MW, Mulvihill JJ, Sharp RR. Evaluating the utility of personal genomic information. Genet Med. 2009;11(8):570–4.

    Article  Google Scholar 

  18. 18.

    Grosse SD, Wordsworth S, Payne K. Economic methods for valuing the outcomes of genetic testing: beyond cost-effectiveness analysis. Genet Med. 2008;10(9):648–54.

    Article  Google Scholar 

  19. 19.

    Pokorska-Bocci A, et al. Personalised medicine in the UK: challenges of implementation and impact on healthcare system. Genome Medicine. 2014;6(4):28.

    Article  Google Scholar 

  20. 20.

    Payne K, et al. Cost-effectiveness analyses of genetic and genomic diagnostic tests. Nat Rev Genet. 2018;19:235.

    CAS  Article  Google Scholar 

  21. 21.

    Valencia CA, et al. Clinical impact and cost-effectiveness of whole exome sequencing as a diagnostic tool: a pediatric center’s experience. Front Pediatr. 2015;3:67.

    Article  Google Scholar 

  22. 22.

    Willig LK, et al. Whole-genome sequencing for identification of Mendelian disorders in critically ill infants: a retrospective analysis of diagnostic and clinical findings. Lancet Respir Med. 2015;3(5):377–87.

    CAS  Article  Google Scholar 

  23. 23.

    Lorgelly PK, et al. Condition-specific or generic preference-based measures in oncology? A comparison of the EORTC-8D and the EQ-5D-3L. Qual Life Res. 2017;26(5):1163–76.

    Article  Google Scholar 

  24. 24.

    Davison N, et al. Exploring the feasibility of delivering standardized genomic care using ophthalmology as an example. Genet Med. 2017;19:1032.

    Article  Google Scholar 

  25. 25.

    Shen J, et al. Conducting a time trade-off study alongside a clinical trial: a case study and recommendations. Pharmacoecon Open. 2018. https://doi.org/10.1007/s41669-018-0084-1 (epub 15 Jun 2018).

    Article  Google Scholar 

  26. 26.

    Feng Y, et al. New methods for modelling EQ-5D-5L value sets: an application to English data. Health Econ. 2018;27(1):23–38.

    Article  Google Scholar 

  27. 27.

    Devlin N, Sussex J. Incorporating multiple criteria in HTA: methods and processes. Office of Health Economics; 2011.

  28. 28.

    Coast J, Smith R, Lorgelly P. Should the capability approach be applied in Health Economics? Health Econ. 2008;17(6):667–70.

    Article  Google Scholar 

  29. 29.

    Buchanan J, Wordsworth S, Schuh A. Patients’ preferences for genomic diagnostic testing in chronic lymphocytic leukaemia: a discrete choice experiment. Patient. 2016;9(6):525–36.

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to James Buchanan.

Ethics declarations

Conflict of interest

James Buchanan and Sarah Wordsworth have no conflicts of interest to declare.


No funding was received for the writing of this editorial.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Buchanan, J., Wordsworth, S. Evaluating the Outcomes Associated with Genomic Sequencing: A Roadmap for Future Research. PharmacoEconomics Open 3, 129–132 (2019). https://doi.org/10.1007/s41669-018-0101-4

Download citation