Health and Technology

, Volume 9, Issue 2, pp 153–166 | Cite as

Network among HTA ecosystem

  • Songul CinarogluEmail author
  • Onur Baser
Original Paper


This study intends to examine the main drivers of network relations among health technology assessment (HTA) organizations. Social network analysis was performed to determine the relations among HTA organizations and to visualize the main drivers of such collaboration. The members in HTA organizations such as International Society for Pharmacoeconomics and Outcomes Research, Health Technology Assessment international, International Network of Agencies for Health Technology Assessment, EuroScan, European Network for Health Technology Assessment, HTAsiaLink, and Health Technology Assessment Network for the Americas are said to create networks. Ten different HTA organizations were considered in the analysis, including the Ministry of Health (MoH) organizations, universities, for-profit organizations, and hospitals. The Fruchterman-Reingold algorithm was used to perform networking, and the average clustering coefficient and average path length were examined to measure collaborative performance. The network graph of the HTA ecosystem shows the highest collaborative frequency among HTA organizations, which are the members of MoH organizations, government agencies, universities, and nonprofit organizations. The average path length was 2.21, and the average clustering coefficient was 36.57, indicating an obvious clustering effect. The study results highlight that networking within the HTA ecosystem is driven by government organizations. Boosting the integration of the private sector into the system and creating data-sharing strategies are essential to foster HTA collaboration. Because HTA is shaped by local dynamics and no gold standard exists for HTA implementation, encouraging collaborative efforts is the only way to avoid redundant efforts and make health technologies available for everyone.


HTA Network analysis HTA ecosystem 



This study was supported by a research grant of The Scientific and Technological Research Council of Turkey (TUBITAK) with a grant number 1059B141500020. The sponsor had no role in the study design, collection and analysis of data, the writing ofthe report or the submission ofthe paper for publication.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    World Health Organization (WHO). Sixtieth World Health Assembly. 2007. Available at: Accessed on: 31.8.2018.
  2. 2.
    The World Health Report. Health systems financing: the path to universal coverage. Geneva: World Health Organization; 2010. World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland.Google Scholar
  3. 3.
    Vostalova L, Mazelová J, Samek J, Vocelka M. Health technology assessment in evaluation of pharmaceuticals in the Czech Republic. Int J Technol Assess Health Care. 2017;33(3):339–44.Google Scholar
  4. 4.
    The International Network of Agencies for Health Technology Assessment (INAHTA). Tools and resources. 2014. Available at: Accessed on: 31.6.2018.
  5. 5.
    Garcia-Altes A, Ondategui PS, Neumann PJ. Cross national comparison of technology assessment processes. Int J Technol Assess Health Care. 2004;20(3):300–10.Google Scholar
  6. 6.
    Taylor R. National Institute for Clinical Excellence (NICE): symposium on technology assessment-introduction. Int J Technol Assess Health Care. 2002;18:159–60.Google Scholar
  7. 7.
    Banta D. The development of health technology assessment. Health Policy. 2003;63(2):121–32.Google Scholar
  8. 8.
    Sampietro-Colom L, Thomas S. Rethinking stakeholder engagement and technology access in health technology assessment: reactions to policy forum discussions. Int J Technol Assess Health Care. 2016;32(4):200–2.Google Scholar
  9. 9.
    Torbica A, Tarricone R, Drummond MF. The impact of culture, values and institutional context on the methods and use of economic evaluation. Value Health. 2016;19(3):A292.Google Scholar
  10. 10.
    Torbica A, Tarricone R, Drummond M. Does the approach to economic evaluation in health care depend on culture, values, and institutional context? Eur J Health Econ. 2018;19:769–74.Google Scholar
  11. 11.
    Neumann PJ. Lessons for health technology assessment: it is not only about the evidence. Value Health. 2009;12(2):45–8.Google Scholar
  12. 12.
    Lays M. Health care policy: qualitative evidence and health technology assessment. Health Policy. 2003;65(3):217–26.Google Scholar
  13. 13.
    Ozcan Y, Khushalani J. Assessing efficiency of public health and medical care provision in OECD countries after a decade of reform. Cent Eur J Oper Res. 2017;25(2):325–43.MathSciNetzbMATHGoogle Scholar
  14. 14.
    Or Z, Cases C, Lisac M, Vrangbæk K, Winblad U, Bevan G. Are health problems systemic? Politics of access and choice under Beveridge and Bismarck systems. Health Econ Policy Law. 2010;5(03):269–93.Google Scholar
  15. 15.
    Hutton J, Trueman P, Facey K. Harmonization of evidence requirements for health technology assessment in reimbursement decision making. Int J Technol Assess Health Care. 2008;24(4):511–7.Google Scholar
  16. 16.
    Kalo Z, Gheorghe A, Huic M, Csanadi M, Kristensen FB. HTA implementation roadmap in central and eastern European countries. Health Econ. 2016;25(1):179–92.Google Scholar
  17. 17.
    Lehoux P, Blume S. Technology assessment and the sociopolitics of health technologies. J Health Polit Policy Law. 2000;25(6):1083–120.Google Scholar
  18. 18.
    Ciani O, Tarricone R, Torbica A. Diffusion and use of health technology assessment in policy making: what lessons for decentralised healthcare systems? Health Policy. 2012;108(2–3):194–202.Google Scholar
  19. 19.
    Sullivan SD, Watkins J, Sweet B, Ramsey SD. Health technology assessment in health-care decisions in the United States. Value Health. 2009;12(2):39–44.Google Scholar
  20. 20.
    International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Connections uniting science and practice, Latin America, Healthcare Technology Assessment (HTA): developing & using the evidence in health care decisions. 2016a.Google Scholar
  21. 21.
    Perry S, Thamer M. Health technology assessment: decentralized and fragmented in the US compared to other countries. Health Policy. 1997;40(3):269–90.Google Scholar
  22. 22.
    European Network for Health Technology Assessment (EUnetHTA). Accessed on: 5.12.2016.
  23. 23.
    Berntgen M, Gourvil A, Pavlovic M, Goettsch W, Eichler HG, Kristensen FB. Improving the contribution of regulatory reports to health technology assessments-a collaboration between the European medicine agencies and the European network for health technology assessment. Value Health. 2014;17(5):634–41.Google Scholar
  24. 24.
    Health Technology Assessment International (HTAi) Rome Annual Meeting. Towards an HTA ecosystem: From local needs to global opportunities. 2017. Accessed on: 1.12.2016.
  25. 25.
    International Society for Pharmacoeconomics and Outcomes Research (ISPOR). 2016b. Available at: Accessed on: 3.12.2016.
  26. 26.
    The International Network of Agencies for Health Technology Assessment (INAHTA). 2016. Available at: Accessed on: 5.12.2016.
  27. 27.
    Rowen D, Zouraq IA, Chevrou-Severac H, van Hout B. International regulations and recommendations for utility data for health technology assessment. PharmacoEconomics. 2017;35(1):11–9.Google Scholar
  28. 28.
    Ricciardi W, Cicchetti A, Marchetti M. Health Technology Assessment’s Italian Network: Origins, aims and advancement. Ital J Pub Health. Theme Papers. 2005. p. 29–32.Google Scholar
  29. 29.
    Kristensen FB. Development of European HTA: from vision to EUnetHTA. Michael. 2012;9:147–56.Google Scholar
  30. 30.
    Neumann PJ. American exceptionalism and American health care: implications for the U.S. debate on cost-effectiveness analysis. Office of Health Economics OHE briefing. 2007. No. 47, London, January 2007.Google Scholar
  31. 31.
    Papanicolas I, Woskie LR, Jha AK. Health care spending in the United States and other high-income countries. JAMA. 2018;319(10):1024–39.Google Scholar
  32. 32.
    Hass JS, Moskowitz EJ. Health technology assessment in Canada and the United States: the case of biologics. Biotechnol Healthc. 2007;4(2):47–51.Google Scholar
  33. 33.
    Hawlik K, Rummel P, Wild C. Analysis of duplication and timing of health technology assessments on medical devices in Europe. Int J Technol Assess Health Care. 2018;34:18–26.Google Scholar
  34. 34.
    Allen N, Pichler F, Wang T, Patel S, Salek S. Development of archetypes for non-ranking classification and comparison of European National Health Technology Assessment systems. Health Policy. 2013;113:305–12.Google Scholar
  35. 35.
    Garrido MV, Kristensen FB, Nielsen CP, Busse R. Health technology assessment and health policy-making in Europe. Observatory Studies Series No. 14, EUnetHTA. Available at: Accessed on: 4.9.2018.
  36. 36.
    Battista RN, Hodge MJ. The “natural history” of health technology assessment. Int J Technol Assess Health Care. 2009;25(1):281–4.Google Scholar
  37. 37.
    Leyens L, Reumnann M, Malats N, Brand A. Use of big data for drug development and for public and personal health and care. Genet Epidemiol. 2016;41:51–60.Google Scholar
  38. 38.
    Jensen PB, Jensen LJ, Brunak S. Mining electronic health records: towards better research applications and clinical care. Nat Rev Genet. 2012;13(6):395–405.Google Scholar
  39. 39.
    Marshall DA, Burgos-Liz L, Pasupathy KS, Padula WV, IJzerman MJ, Wong PK, et al. Transforming healthcare delivery: integrating dynamic simulation modelling and big data in health economics and outcomes research. PharmacoEconomics. 2016;34(2):115–26.Google Scholar
  40. 40.
    Christopoulou SC, Kotsilieris T, Anagnostopoulos I. Evidence-based health and clinical informatics: a systematic review on randomized controlled trials. Health Technol. 2018;8:137–15.Google Scholar
  41. 41.
    Chen M, Ma Y, Song J, Lai CF, Hu B. Smart clothing: connecting human with clouds and big data for sustainable health monitoring. MONET. 2016;21(5):825–45.Google Scholar
  42. 42.
    Wyber R, Vaillancourt S, Perry W, Mannava P, Folaranmi T, Celi LA. Big data in global health: improving health in low- and middle-income countries. Bull World Health Organ. 2015;93:203–8.Google Scholar
  43. 43.
    Boncz I, Vajda R, Agoston I, Endrei D, Sebestyen A. Changes in the health status of the population of central and eastern European countries between 1990 and 2010. Eur J Health Econ. 2014;2014(Suppl 1):137–41.Google Scholar
  44. 44.
    Huic M, Nachtnebel A, Zechmeister I. Collaboration in health technology assessment (EUnetHTA joint action, 2010-2012): four case studies. Int J Technol Assess Health Care. 2013;29(3):323–30.Google Scholar
  45. 45.
    Drummond MF, Bloom BS, Carrin G, Hillman AL, Hutchings HC, Knill-Jones RP, et al. Issues in the cross-national assessment of health technology. Int J Technol Assess Health Care. 1992;8(4):671–82.Google Scholar
  46. 46.
    Rottingen JA, Gerhardus A, Garrido MV. Future challenges for HTA in Europe. In: Garrido MV, Kristensen FB, Nielsen CP, Busse R (eds) Health technology assessment and health policy-making in Europe. Current status, challenges and potential. Observatory Studies Series No14, World Health Organization, on behalf of the European Observatory on Health Systems and Policies; 2008.Google Scholar
  47. 47.
    Health Technology Assesment International (HTAi). 2016. Available at: Accessed on: 30.11.2016.
  48. 48.
    Health Technology Assessment International (HTAi)/World Health Organization (WHO). Collaboration Plan Template for 2014–2016 Available at:, Accessed on:30.11.2016.
  49. 49.
    EuroScan. Available at: 2016. Accessed on:30.11.2016.
  50. 50.
    Orwat MI, Kempny A, Bauer U, Gatzoulis MA, Baumgartner H, Diller GP. The importance of national and international collaboration in adult congenital heart disease: a network analysis of research output. Int J Cardiol. 2015;195:155–62.Google Scholar
  51. 51.
    Battista RN. Expanding the scientific basis of health technology assessment: a research agenda for the next decade. Int J Technol Assess Health Care. 2006;22(3):275–82.Google Scholar
  52. 52.
    Ehlers L, Vestergaard M, Kidholm K, Bonnevie B, Pedersen PH, Jorgensen T, et al. Doing mini-health technology assessments in hospitals: a new concept of decision support in health care? Int J Technol Assess Health Care. 2006;22(3):295–301.Google Scholar
  53. 53.
    Gagnon MP. Hospital-based health technology assessment: developments to date. PharmacoEconomics. 2014;32(9):819–24.Google Scholar
  54. 54.
    Kahveci R. Commentaries, views and developments in HTA. Int J Technol Assess Health Care. 2009;25(3):425–6.Google Scholar
  55. 55.
    HTAsiaLink. 2016. Available at: Accessed on:10.11.2016.
  56. 56.
    RedETSA. 2016. Available at: Accessed on:15.11.2016.
  57. 57.
    Yu H, Kim PM, Sprecher E, Trinofov V, Gerstein M. The importance of bottlenecks in protein etworks: correlation with gene essentiality and expression dynamics. PLoS Comput Biol. 2007;3(4):e59.Google Scholar
  58. 58.
    Fruchterman TM, Reingold EM. Graph drawing by force-directed placement. Softw Pract Exp. 1991;21(11):1129–64.Google Scholar
  59. 59.
    Barrat A, Barthelemy M, Pastor-Satorras R, Vespignani A. The architecture of complex weighted networks. PNAS. 2004;101(11):3747–52.Google Scholar
  60. 60.
    Gajdos P, Jezowicz T, Uher V, Dohnalek P. A parallel Fruchterman-Reingold algorithm optimized for fast visualisation of large graphs and swarms of data. Swarm Evol Comput. 2016;26:56–63.Google Scholar
  61. 61.
    Chattopadhyay S, Rakesh S, Land LPW, Acharya UR. Studying infant mortality rate: a data mining approach. Heal Technol. 2011;1:25–34.Google Scholar
  62. 62.
    Herzlinger RE. Why innovation in health care is so hard? Harv Bus Rev. 2006;84(5):58–66.Google Scholar
  63. 63.
    Moharra M, Espallargues M, Kubesch N, Estrada MD, Parada A, Vondeling H, et al. Systems to support health technology assessment (HTA) in member states of the European Union with limited institutionalization of HTA. Int J Technol Assess Health Care. 2009;25(2):75–83.Google Scholar
  64. 64.
    Kristensen FB, Makela M, Neiktar SA, Rehnqvist N, Haheim LL, Morland B, et al. European network for health technology assessment, EUnetHTA: planning, development and implementation of a sustainable European network for Healthn technology assessment. Int J Technol Assess Health Care. 2009;25(S2):107–16.Google Scholar
  65. 65.
    Martelli F, La Torre G, Ghionno ED, Staniscia T, Neroni M, Cicchetti A, et al. Health technology assessment agencies: an international overview of organizational aspects. Int J Technol Assess Health Care. 2007;23(4):414–24.Google Scholar
  66. 66.
    Guegan EW, Cook A. Success factors for international HTA projects: evaluating EUnetHTA joint action as a exemplar. Int J Technol Assess Health Care. 2014;30(5):544–51.Google Scholar
  67. 67.
    Banta D, Jonsson E. History of HTA: introduction. Int J Technol Assess Health Care. 2009;25:1):1–6.Google Scholar
  68. 68.
    Desouza KC, Jacob B. Big data in the public sector: lessons for practitioners and scholars. Adm Soc. 2017;49(7):1043–64.Google Scholar
  69. 69.
    Krumholz HM. Big data and new knowledge in medicine: the thinking, training, and tools needed for a learning health system. Health Aff. 2014;33(7):1163–70.Google Scholar
  70. 70.
    Veillard J, Champagne F, Klazinga N, Kazandjian OA, Arah A, Guisset AL. A performance assessment framework for hospitals: the WHO regional office for Europe PATH project. Int J Qual Health Care. 2005;17(6):487–96.Google Scholar
  71. 71.
    Hanna E, Remuzat C, Auquier P, Dussart C, Toumi M. Could Healthcoin be a revolution in healthcare? Value Health. 2017;20(9):A672.Google Scholar

Copyright information

© IUPESM and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Economics and Administrative Sciences (FEAS), Department of Health Care ManagementHacettepe UniversityBeytepeTurkey
  2. 2.STATinMED ResearchNew YorkUSA
  3. 3.Columbia UniversityNew YorkUSA
  4. 4.University of MichiganAnn ArborUSA
  5. 5.MEF UniversityIstanbulTurkey
  6. 6.Journal of Health Economics and Outcomes ResearchNew YorkUSA

Personalised recommendations