Abstract
Use and acceptance of health economic evaluations (HEEs) has been much greater in developed than in developing nations. Nevertheless, while developing countries lag behind in the development of HEE methods, they could benefit from the progress made in other countries and concentrate on ways in which existing methods can be used or would need to be modified to fulfill their specific needs. HEEs, as context-specific tools, are not easily generalizable from setting to setting. Existing studies regarding generalizability and transferability of HEEs have primarily been conducted in developed countries. Therefore, a legitimate question for policy makers in Latin America and the Caribbean region (LAC) is to what extent HEEs conducted in industrialized economies and in LAC are generalizable to LAC (trans-regional) and to other LAC countries (intra-regional), respectively.
We conducted a systematic review, searching the NHS Economic Evaluation Database (NHS EED), Office of Health Economics Health Economic Evaluation Database (HEED), LILACS (Latin America health bibliographic database) and NEVALAT (Latin American Network on HEE) to identify HEEs published between 1980 and 2004. We included individual patient- and model-based HEEs (cost-effectiveness, cost-utility, cost-benefit and cost-consequences analyses) that involved at least one LAC country. Data were extracted by three independent reviewers using a checklist validated by regional and international experts.
From 521 studies retrieved, 72 were full HEEs (39% randomized controlled trials [RCTs], 32% models, 17% non-randomized studies and 12% mixed trialmodeling approach). Over one-third of identified studies did not specifically report the type of HEE. Cost-effectiveness and cost-consequence analyses accounted for almost 80 % of the studies. The three Latin American countries with the highest participation in HEE studies were Brazil, Argentina and Mexico. While we found relatively good standards of reporting the study’s question, population, interventions, comparators and conclusions, the overall reporting was poor, and evidence of unfamiliarity with international guidelines was evident (i.e. absence of incremental analysis, of discounting long-term costs and effects). Analysis or description of place-to-place variability was infrequent. Of the 49 trial-based analyses, 43% were single centre, 33% multinational and 18% multicentre national. Main reporting problems included issues related to sample representativeness, data collection and data analysis. Of the 32 model-based studies (most commonly using epidemiological models), main problems included the inadequacy of search strategy, range selection for sensitivity analysis and theoretical justifications.
There are a number of issues associated with the reporting and methodology used in multinational and local HEE studies relevant for LAC that preclude the assessment of their generalizability and potential transferability. Although the quality of reporting and methodology used in model-based HEEs was somewhat higher than those from trial-based HEEs, economic evaluation methodology was usually weak and less developed than the analysis of clinical data. Improving these aspects in LAC HEE studies is paramount to maximizing their potential benefits such as increasing the generalizability/transferability of their results.
Similar content being viewed by others
References
Singer ME. Cost-effectiveness analysis: developing nations left behind. Pharmacoeconomics 2008; 26 (5): 359–61
Drummond M, Manca A, Sculpher M. Increasing the generalizability of economic evaluations: recommendations for the design, analysis, and reporting of studies. Int J Technol Assess Health Care 2005 Spring; 21 (2): 165–71
Sculpher MJ, Pang FS, Manca A, et al. Generalisability in economic evaluation studies in healthcare: a review and case studies. Health Technol Assess 2004 Dec; 8 (49): iii-iv, 1–192
Drummond M, Barbieri M, Cook J, et al. Transferability of economic evaluations across jurisdictions: report of an ISPOR task force. Value Health 2009; 12 (4): 409–18
Barbieri M, Drummond M, Willke R, et al. Variability of cost-effectiveness estimates for pharmaceuticals in Western Europe: lessons for inferring generalizability. Value Health 2005 Jan-Feb; 8 (1): 10–23
Manca A, Willan AR. ‘Lost in translation’: accounting for between-country differences in the analysis of multinational cost-effectiveness data. Pharmacoeconomics 2006; 24 (11): 1101–19
Welte R, Feenstra T, Jager H, et al. A decision chart for assessing and improving the transferability of economic evaluation results between countries. Pharmacoeconomics 2004; 22 (13): 857–76
Craig D, Rice S, editors. NHS Economic Evaluation Database (NHS EED) handbook 2007. York: Centre for Reviews and Dissemination, University of York, 2007 Apr [online]. Available from URL: (http://www.crd.york.ac.uk/crdweb/html/help.htm) [Accessed 2009 Oct 1]
Drummond ME, Sculpher MJ, Torrance GW, et al. Methods for the economic evaluation of health care programmes. 3rd ed. Oxford: Oxford University Press, 2005
Stata Corp. Stata statistical software: release 8.2. College Station (TX): StataCorp LP, 2004
Philips Z, Bojke L, Sculpher M, et al. Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. Pharmacoeconomics 2006; 24 (4): 355–71
Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ 1996 Aug 3; 313 (7052): 275–83
Boulenger S, Nixon J, Drummond M, et al. Can economic evaluations be made more transferable? Eur J Health Econ 2005 Dec; 6 (4): 334–46
Nixon J, Rice S, Drummond M, et al. Guidelines for completing the EURONHEED transferability information checklists. Eur J Health Econ 2009 May; 10 (2): 157–65
Iglesias CP, Drummond MF, Rovira J. Health-care decisionmaking processes in Latin America: problems and prospects for the use of economic evaluation. Int J Technol Assess Health Care 2005 Winter; 21 (1): 1–14
Kuntz K, Weinstein M. Modelling in economic evaluation. In: Drummond M, McGuire A, editors. Economic evaluation in health care: merging theory with practice. Oxford: Oxford University Press, 2001: 141–71
Abdala OA, Levy RR, Bibiloni RH, et al. Advantages of video assisted thoracic surgery in the treatment of spontaneous pneumothorax. Medicina (B Aires) 2001; 61 (2): 157–60
Abizaid A, Costa MA, Centemero M, et al. Clinical and economic impact of diabetes mellitus on percutaneous and surgical treatment of multivessel coronary disease patients: insights from the Arterial Revascularization Therapy Study (ARTS) trial. Circulation 2001; 104 (5): 533–8
Akhavan D. Cost-effectiveness analysis of the leishmaniasis component of the endemic diseases control projects of the Northeast of Brazil [in Portuguese]. Rev Patol Trop 1996; 25 (2): 203–52
Akhavan D, Musgrove P, Abrantes A, et al. Cost-effective malaria control in Brazil: cost-effectiveness of a Malaria Control Program in the Amazon Basin of Brazil, 1988-1996. Soc Sci Med 1999 Nov; 49 (10): 1385–99
Arredondo A, Rangel R, de Icaza E. Cost-effectiveness of interventions for end-stage renal disease. Rev Saude Publica 1998 Dec; 32 (6): 556–65
Arroyave Loaiza G, Jarillo Soto EC, Garfias Garnica MG, et al. Cost-benefit of the pharmacologic treatment of cerebral palsy with spasticity in Mexico. Rev Esp Salud Publica 2000 Sep-Dec; 74 (5-6): 549–59
Badaro R, Nascimento C, Carvalho JS, et al. Recombinant human granulocyte-macrophage colony-stimulating factor reverses neutropenia and reduces secondary infections in visceral leishmaniasis. J Infect Dis 1994 Aug; 170 (2): 413–8
Barberis M, Harvey PD. Costs of family planning programmes in fourteen developing countries by method of service delivery. J Biosoc Sci 1997 Apr; 29 (2): 219–33
Bennett-Guerrero E, Sorohan JG, Gurevich ML, et al. Cost-benefit and efficacy of aprotinin compared with epsilon-aminocaproic acid in patients having repeated cardiac operations: a randomized, blinded clinical trial. Anesthesiology 1997 Dec; 87 (6): 1373–80
Borghi J, Fox-Rushby J, Bergel E, et al. The cost-effectiveness of routine versus restrictive episiotomy in Argentina. Am J Obstet Gynecol 2002 Feb; 186 (2): 221–8
Burckel E, Ashraf T, de Sousa Filho JP, et al. Economic impact of providing workplace influenza vaccination: a model and case study application at a Brazilian pharma-chemical company. Pharmacoeconomics 1999 Nov; 16 (5 Pt 2): 563–76
Casciano J, Doyle J, Arikian S, et al. The health economic impact of antidepressant usage from a payer’s perspective: a multinational study. Int J Clin Pract 2001 Jun; 55 (5): 292–9
Cattaneo A, Davanzo R, Worku B, et al. Kangaroo mother care for low birthweight infants: a randomized controlled trial in different settings. Acta Paediatr 1998 Sep; 87 (9): 976–85
Cavalcante MD, Braga OB, Teofilo CH, et al. Cost improvements through the establishment of prudent infection control practices in a Brazilian general hospital, 1986-1989. Infect Control Hosp Epidemiol 1991 Nov; 12 (11): 649–53
Caviedes L, Lee TS, Gilman RH, et al. Rapid, efficient detection and drug susceptibility testing of Mycobacterium tuberculosis in sputum by microscopic observation of broth cultures: the Tuberculosis Working Group in Peru. J Clin Microbiol 2000 Mar; 38 (3): 1203–8
Charnas R, Luthi AR, Ruch W. Once daily ceftriaxone plus amikacin vs three times daily ceftazidime plus amikacin for treatment of febrile neutropenic children with cancer: Writing Committee for the International Collaboration on Antimicrobial Treatment of Febrile Neutropenia in Children. Pediatr Infect Dis J 1997 Apr; 16 (4): 346–53
Cookson ST, Stamboulian D, Demonte J, et al. A cost-benefit analysis of programmatic use of CVD 103-HgR live oral cholera vaccine in a high-risk population. Int J Epidemiol 1997 Feb; 26 (1): 212–9
Crabbe F, Vuylsteke B, de Clerck M, et al. Cost-effectiveness of management strategies for acute urethritis in the developing world. Trop Med Int Health 2000 Sep; 5 (9): 640–7
Creese AL, Dominguez-Uga MA. Cost-effectiveness of immunization programs in Colombia. Bull Pan Am Health Organ 1987; 21 (4): 377–94
Dasbach EJ, Rich MW, Segal R, et al. The cost-effectiveness of losartan versus captopril in patients with symptomatic heart failure. Cardiology 1999; 91 (3): 189–94
Dayan GH, Nguyen VH, Debbag R, et al. Cost-effectiveness of influenza vaccination in high-risk children in Argentina. Vaccine 2001 Jul 20; 19 (30): 4204–13
Dias da Costa JS, Fuchs SC, Olinto MT, et al. Cost effectiveness of hypertension treatment: a populationbased study. Sao Paulo Med J 2002 Jul 4; 120 (4): 100–4
Dominguez Uga MA. Economic analysis of the vaccination strategies adopted in Brazil in 1982. Bull Pan Am Health Organ 1988; 22 (3): 250–68
Doyle JJ, Casciano J, Arikian S, et al. A multinational pharmacoeconomic evaluation of acute major depressive disorder (MDD): a comparison of cost-effectiveness between venlafaxine, SSRIs and TCAs. Value Health 2001 Jan-Feb; 4 (1): 16–31
Fariña D, Rodriguez SP, Bauer G, et al. Respiratory syncytial virus prophylaxis: cost-effective analysis in Argentina. Pediatr Infect Dis J 2002 Apr; 21 (4): 287–91
Fonseca W, Misago C, Fernandes L, et al. Use of manual vacuum aspiration in reducing cost and duration of hospitalization due to incomplete abortion in an urban area of northeastern Brazil. Rev Saude Publica 1997 Oct; 31 (5): 472–8
Fox KAA, Bosanquet N. Assessing the UK cost implications of the use of low molecular weight heparin in unstable coronary artery disease. Br J Cardiol 1998; 5 (2): 92–105
Gagliardino JJ, Etchegoyen G. A model educational program for people with type 2 diabetes: a cooperative Latin American implementation study (PEDNID-LA). Diabetes Care 2001 Jun; 24 (6): 1001–7
Garcia-Contreras F, Del-Angel-Garcia G, Ramirez Cuenca A, et al. Cost-effectiveness analysis of ceftriaxone and cefotaxime in the treatment of community-acquired pneumonia. Rev Invest Clin 2000 Jul-Aug; 52 (4): 418–26
García-Peña C, Thorogood M, Wonderling D, et al. Economic analysis of a pragmatic randomised trial of home visits by a nurse to elderly people with hypertension in Mexico. Salud Publica Mex 2002; 44 (1): 14–20
Garduno-Espinosa J, Martinez-Garcia MC, Valadez-Salazar A, et al. Cost-effectiveness analysis of treatment of E. histolytica/E. dispar cyst carriers. Arch Med Res 1997; 28 Spec No: 293–4
Gerth WC, Remuzzi G, Viberti G, et al. Losartan reduces the burden and cost of ESRD: public health implications from the RENAAL study for the European Union. Kidney Int Suppl 2002 Dec; (82): S68–72
Gomez JC, Selvaggio G, Pizarro B, et al. Value of a screening algorithm for celiac disease using tissue transglutaminase antibodies as first level in a population-based study. Am J Gastroenterol 2002 Nov; 97 (11): 2785–90
Grines CL, Marsalese DL, Brodie B, et al. Safety and costeffectiveness of early discharge after primary angioplasty in low risk patients with acute myocardial infarction. PAMIII Investigators. Primary Angioplasty in Myocardial Infarction. J Am Coll Cardiol 1998 Apr; 31 (5): 967–72
Harris RA, Owens DK, Witherell H, et al. Helicobacter pylori and gastric cancer: what are the benefits of screening only for the CagA phenotype of H. pylori? Helicobacter 1999 Jun; 4 (2): 69–76
Hourneaux G, de Moura E, Sakai P, et al. Palliative treatment of advanced esophageal cancer: comparative study. Auto-expandable metal stent and isoperistaltic esophagogastric bypass. Acta Gastroenterol Latinoam 2001 Mar; 31 (1): 13–22
Kroeger A, Gerhardus A, Kruger G, et al. The contribution of repellent soap to malaria control. Am J Trop Med Hyg 1997 May; 56 (5): 580–4
Lacerda-Filho A, Cunha-Melo JR. Outpatient haemorrhoidectomy under local anaesthesia. Eur J Surg 1997 Dec; 163 (12): 935–40
Lavalle C, Aguilar JC, Pena F, et al. Reduction in hospitalization costs, morbidity, disability, and mortality in patients with aids treated with protease inhibitors. Arch Med Res 2000 Sep-Oct; 31 (5): 515–9
Lechtig A, Townsend JW, Pineda F, et al. Nutrition, family planning, and health promotion: the Guatemalan program of primary health care. Birth 1982 Summer; 9 (2): 97–104
Lopez-Neblina F, Alvarez JH, Finkelstein LI. Highefficiency kidney transplantation: concept, technique, results, and cost analysis. Transplant Proc 2000 Feb; 32 (1): 141–2
Mark DB, Cowper PA, Berkowitz SD, et al. Economic assessment of low-molecular-weight heparin (enoxaparin) versus unfractionated heparin in acute coronary syndrome patients: results from the ESSENCE randomized trial. Efficacy and Safety of Subcutaneous Enoxaparin in Non-Q wave Coronary Events (unstable angina or non-Q-wave myocardial infarction). Circulation 1998 May 5; 97 (17): 1702–7
Murad A, de Andrade CA, Delfino C, et al. A pharmacoeconomic comparison of UFT and 5-FU chemotherapy for colorectal cancer in South America. Oncology (Williston Park) 1997 Sep; 11 (9 Suppl. 10): 128–35
Musgrove P. Cost-benefit analysis of a regional system for vaccination against pneumonia, meningitis type B, and typhoid fever. Bull Pan Am Health Organ 1992; 26 (2): 173–91
Netto Jr NR, Ikonomidis J, Zillo C. Routine ureteral stenting after ureteroscopy for ureteral lithiasis: is it really necessary? J Urol 2001 Oct; 166 (4): 1252–4
O’Brien BJ, Willan A, Blackhouse G, et al. Will the use of low-molecular-weight heparin (enoxaparin) in patients with acute coronary syndrome save costs in Canada? Am Heart J 2000 Mar; 139 (3): 423–9
Ovalle A, Martinez MA, Wolff M, et al. Prospective, randomized, comparative study of the efficacy, safety and cost of cefuroxime versus cephradine in acute pyelonephritis during pregnancy. Rev Med Chil 2000 Jul; 128 (7): 749–57
Phillips M, Sanghvi T, Suarez R, et al. The costs and effectiveness of three vitaminA interventions in Guatemala. Soc Sci Med 1996 Jun; 42 (12): 1661–8
Puschel K, Sullivan S, Montero J, et al. Cost-effectiveness analysis of a preventive program for gallbladder disease in Chile. Rev Med Chil 2002 Apr; 130 (4): 447–59
Rodriguez A, Ayala F, Bernardi V, et al. Optimal coronary balloon angioplasty with provisional stenting versus primary stent (OCBAS): immediate and long-term follow-up results. J Am Coll Cardiol 1998 Nov; 32 (5): 1351–7
Rodriguez A, Boullon F, Perez-Balino N, et al. Argentine randomized trial of percutaneous transluminal coronary angioplasty versus coronary artery bypass surgery in multivessel disease (ERACI): in-hospital results and 1-year follow-up. ERACI Group. J Am Coll Cardiol 1993 Oct; 22 (4): 1060–67
Rodriguez A, Mele E, Peyregne E, et al. Three-year followup of the Argentine Randomized Trial of Percutaneous Transluminal Coronary Angioplasty Versus Coronary Artery Bypass Surgery in Multivessel Disease (ERACI). J Am Coll Cardiol 1996 Apr; 27 (5): 1178–84
Rollan A, Giancaspero R, Acevedo C, et al. Treatment of Helicobacter pylori infection in patients with duodenal ulcer: a cost-benefit study. Rev Med Chil 2000 Apr; 128 (4): 367–77
Ruebush 2nd TK, Zeissig R, Koplan JP, et al. Community participation in malaria surveillance and treatment: III. An evaluation of modifications in the Volunteer Collaborator Network of Guatemala. Am J Trop Med Hyg 1994 Jan; 50 (1): 85–98
Ryckelynck JP, Hurault de Ligny B, Levaltier B, et al. Prevention of peritonitis in continuous ambulatory peritoneal dialysis. Nephrologie 1989; (10 Suppl.): 46–9
Salinas AM, Villarreal E, Nunez GM, et al. Health interventions for the metal working industry: which is the most cost-effective? A study from a developing country. Occup Med (Lond) 2002 May; 52 (3): 129–35
Serruys PW, de Bruyne B, Carlier S, et al. Randomized comparison of primary stenting and provisional balloon angioplasty guided by flow velocity measurement: Doppler Endpoints Balloon Angioplasty Trial Europe (DEBATE) II Study Group. Circulation 2000 Dec 12; 102 (24): 2930–7
Serruys PW, Unger F, Sousa JE, et al. Comparison of coronary-artery bypass surgery and stenting for the treatment of multivessel disease. N Engl J Med 2001 Apr 12; 344 (15): 1117–24
Serruys PW, van Hout B, Bonnier H, et al. Randomised comparison of implantation of heparin-coated stents with balloon angioplasty in selected patients with coronary artery disease (Benestent II). Lancet 1998 Aug 29; 352 (9129): 673–81
Sesso R, Eisenberg JM, Stabile C, et al. Cost-effectiveness analysis of the treatment of end-stage renal disease in Brazil. Int J Technol Assess Health Care 1990; 6 (1): 107–14
Shepard DS, Robertson RL, Cameron 3rd CS, et al. Costeffectiveness of routine and campaign vaccination strategies in Ecuador. Bull World Health Organ 1989; 67 (6): 649–62
Suarez PG, Floyd K, Portocarrero J, et al. Feasibility and cost-effectiveness of standardised second-line drug treatment for chronic tuberculosis patients: a national cohort study in Peru. Lancet 2002 Jun 8; 359 (9322): 1980–9
Szucs TD, Meyer BJ, Kiowski W. Economic assessment of tirofiban in the management of acute coronary syndromes in the hospital setting: an analysis based on the PRISM PLUS trial. Eur Heart J 1999 Sep; 20 (17): 1253–60
Tieffenberg JAWI, Del PH, Berbeglia A, et al. Decisionmaking and the health sector in Argentina: CEA and the use of nitrous oxide in anesthesia. Int J Technol Assess Health Care 1998; 4: 601–11
Vargas Flores LA, Nunez Gomiciaga E. Cost-benefit analysis of a national immunization program against measles in Mexico. Salud Publica Mex 1984 Jul-Aug; 26 (4): 373–80
Varley RC, Tarvid J, Chao DN. A reassessment of the costeffectiveness of water and sanitation interventions in programmes for controlling childhood diarrhoea. Bull World Health Organ 1998; 76 (6): 617–31
Vela-Ojeda J, Tripp-Villanueva F, Montiel-Cervantes L, et al. Prospective randomized clinical trial comparing high-dose ifosfamide+GM-CSF versus high-dose cyclophosphamide + GM-CSF for blood progenitor cell mobilization. Bone Marrow Transplant 2000 Jun; 25 (11): 1141–6
Villar Centeno LA, Niño OC, Gómez C. Post discharge surveillance of surgical wound infections: evaluations of costs and utilities [in Portuguese]. Med UNAB 1998; 1 (3): 153–58
Villar J, Ba’aqeel H, Piaggio G, et al. WHO antenatal care randomised trial for the evaluation of a new model of routine antenatal care. Lancet 2001 May 19; 357 (9268): 1551–64
Walters SJ, Whitfield M, Akehurst RL, et al. Pharmacoeconomic evaluation of simulect prophylaxis in renal transplant recipients. Transplant Proc 2001 Nov-Dec; 33 (7-8): 3187–91
Ward LS, Castelo Filho A, Menabo E, et al. Cost/effective ratio in the treatment of Basedow-Graves disease. AMB Rev Assoc Med Bras 1986 Sep-Oct; 32 (9-10): 147–54
Younes RN, Gross JL, Deheinzelin D. Follow-up in lung cancer: how often and for what purpose? Chest 1999 Jun; 115 (6): 1494–9
Murray CJ, Lopez AD. The global burden of disease: a comprehensive assessment of mortality and disability from diseases, injuries, and risk factors in 1990 and projected to 2020. Boston (MA): Harvard University, 1996: xxxii, 990
Sculpher MJ, Drummond MF. Analysis sans frontieres: can we ever make economic evaluations generalisable across jurisdictions? Pharmacoeconomics 2006; 24 (11): 1087–99
Methodological guidelines: economic evaluation of health technologies. Secretaria de CIencia, Tecnologia e Insumos Estrategicos. Departamento de Ciencia y Tecnologia. Serie A. Normas e Manuais Tecnicos. Brasilia: Ministerio Da Saude, 2009
Augustovski F, Bastardo Y, Caso A, et al. Pharmacoeconomics and outcomes research in Latin America: Argentina, Brazil, Chile, Colombia, México and Venezuela. ISPOR 1st Latin America Conference; 2007 Sep 9-11; Cartagena
Acknowledgements
The authors would like to thank Lisa Mather for her help with the search of the documents, and Alicia Domínguez Uga, Carlos Gouveia Pinto, Gabriel Carrasquilla and Joan Rovira from the NEVALAT steering group for kindly reviewing and commenting on the checklist finally used in the present study.
This work was conceived and undertaken partly with the support to Dr Augustovski from the Program Alban, Program of high level scholarships of the European Union to Latin America (identification number E03E17444AR), as well as by the Institute for Clinical Effectiveness and Health Policy, and the Centre for Health Economics at The University of York. The authors also acknowledge the helpful comments of the reviewers of the paper.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Augustovski, F., Iglesias, C., Manca, A. et al. Barriers to Generalizability of Health Economic Evaluations in Latin America and the Caribbean Region. Pharmacoeconomics 27, 919–929 (2009). https://doi.org/10.2165/11313670-000000000-00000
Published:
Issue Date:
DOI: https://doi.org/10.2165/11313670-000000000-00000