General guidance to assessors
This transferability information checklist requires users of it to provide one of four responses to each question: ‘yes’, ‘partially’, ‘no’ or ‘not applicable’. For questions involving multifaceted elements, the criteria to qualify for a ‘yes’ and a ‘no’ response have been outlined below. If only part of the required criteria has been addressed by the authors, the assessor should answer ‘partially’. However, in some circumstances where the answer has been partially and inadequately dealt with, the answer given should be ‘no’. The response ‘not applicable’ should be given when the point being addressed is not relevant to the study’s methods.
It is recognised that a certain degree of subjectivity will be present and answers may vary according to the judgement of the assessor. This subjectivity can be mitigated by the use of blinded assessors and post-assessment discussion.
Study question
Q1. Is the study question clearly stated?
Did the author(s) clearly state the hypothesis/aim/objective of the study? Did they fully describe what they were trying to demonstrate?
Q2. Are the alternative technologies justified by the author(s)?
Did the authors clearly explain the reason(s) for their choice of comparator(s)? If they simply stated their choice of comparator(s) the answer should be ‘no’. If they provided a description of potential comparators and a rationale for their selection, or if they chose current practice, the answer should be ‘yes’.
Health technology
HT1. Is the intervention described in sufficient detail?
The principal question being addressed is whether or not enough information is provided for a reader to be able to fully understand the health technology studied, and whether or not it is applicable to other contexts. To provide a ‘yes’ answer the article should provide a full description of the technology studied, which in the case, for example, of pharmaceuticals includes dosages, relevant intervals, method of administration (for example, orally, subcutaneously), information on how many times per hour/day/week, for how long, and where and by whom it was administered. If only a general description is given the answer should be ‘partially’, or ‘no’ if only the name of the health technology is given.
HT2. Is(are) the comparator(s) described in sufficient details?
See explanation above for HT1.
Setting
SE1. Did the authors correctly specify the setting in which the study took place (e.g. primary care, community)?
Give responses according to the five cases possible:
-
Correctly specified: answer ‘yes’
-
Misspecified but can be inferred: answer ‘partially’
-
Misspecified but cannot be inferred: answer ‘no’
-
Not specified but can be inferred: answer ‘partially’
-
Not specified but cannot be inferred: answer ‘no’.
SE2. Is(are) the country(ies) to which the economic study applies clearly stated?
Answer either ‘yes’ or ‘no’ as appropriate.
Perspective
P1. Did the authors correctly state which perspective they adopted for the economic analysis?
This question reflects the plausible scenarios with regard to the economic perspective (stated or not) and whether or not the cost analysis matched the stated/inferred perspective; for example, if the authors stated a societal perspective but did not include indirect costs (productivity changes), the response should be ‘partially’. Give responses according to the five cases possible:
-
Correctly specified with all relevant costs included: answer ‘yes’
-
Misspecified but can be inferred from the reporting of costs: answer ‘partially’
-
Misspecified but cannot be inferred from the reporting of costs: answer ‘no’
-
Not specified but can be inferred from the reporting of costs: answer ‘partially’
-
Not specified and cannot be inferred from the reporting of costs: answer ‘no’.
Study population
SP1. Is the target population of the health technology clearly stated by the authors or, when it is not stated, can it be inferred by reading the article?
The target population is the entire group a researcher is interested in, the group about which the researcher wishes to draw conclusions. This may differ from the actual study population and therefore affect the generalisability of the results.
SP2. Are the study population characteristics described? (e.g. age, sex, health status, socioeconomic status, inclusion/exclusion criteria)
This question refers to the population from which the sample was drawn. To answer ‘yes’ the author(s) need(s) to have described the demographic profile and disease status where relevant of the studied population. If only a general description, i.e. a description not or only partially allowing the reader to assess if it is applicable to the population of interest is given, the answer should be ‘partially’, or ‘no’ if the study population is not described.
SP3. Does the article provide sufficient detail about the study sample(s)?
To answer ‘yes’ the authors need to have reported all of the following: whether the sample size was determined prior to the selection process and if so, how; the sample selection process; sample inclusion/exclusion criteria (which may be additional to study population inclusion/exclusion criteria); the percentage of patients who refused to participate or who were excluded; the number of subjects in each group; the characteristics of the subjects.
If only some/none of the above items are reported, answer ‘partially’ or ’no’.
- •
If the clinical data are derived from a literature review, there may be many samples from many studies and this detail on the samples is unlikely to be reported, in which case the answer should be ‘not applicable’.
SP4. Does the paper provide sufficient information to assess the representativeness of the study sample with respect to the target/study population?
If the subjects in the study sample have characteristics different from those of the target/study population, these differences must be considered to be potential confounders and may bias the outcome of the study. To answer ‘yes’ the author(s) need(s) to have described the sample in sufficient detail and when and if it differs from the study population inclusion/exclusion criteria. They may also compare the characteristics, where relevant, of those who refused to be included in the study with those who participated, and determine if people who dropped out of the study were different in comparison with those who remained (if the study participants were found to be different from those who were not in the study, the results could be biased because the subjects who were potentially the ‘worst cases’ were not included in the study). If the authors did only some of the above the answer should be ‘partially’.
- •
If the clinical data are derived from a literature review and there are many samples from many studies, the above guidelines still apply.
Modelling
M1. If a model is used is it described in detail?
When a model is used, to answer ‘yes’ the authors should have clearly stated the purpose of the model, described the type of model used (e.g. decision tree, Markov model), the key assumptions of the model, provided details of the software used, and the time horizon the model is examining. ‘Partially’ should be answered only if some of these requirements ‘yes’ are provided, and ‘no’ if no description is made of the model (stating that a decision tree, for example, is used, is not enough).
- •
If no model is used, or other modelling such as regression analysis is performed, write ‘not applicable’.
M2. Are the origins of the parameters used in the model given?
To answer ‘yes’ the authors need to have provided detailed descriptions of the sources (e.g. literature, expert opinion, single trial) used to derive the model’s parameters (point estimates and ranges if appropriate). ‘Partially’ should be the answer if only some of the sources are given. Choose ‘no’ if no sources are given.
- •
If no model is used, write ‘not applicable’.
Effectiveness
E1. If a single study is used is the study design described (sample selection, study design, allocation, follow-up)?
To answer yes the authors need to have described the method of sample selection, the study design, if the study was single or multicentred, the allocation method, the duration of follow-up and the loss to follow-up (if relevant).
‘Partially’ should be the answer if any of the requirements for ‘yes’ are not met. ‘No’ should be the answer if either the type of study, whether or not it was single or multicentred or the duration of follow-up was not mentioned.
E2. If a single study is used are the methods of data analysis described (ITT/per protocol or observational data)?
Did the authors:
- •
State, in the case of an experimental study, if the study was based on intention-to-treat or treatment completers, and in the case of an observational study, if all patients included in the study were accounted for in the analysis,
- •
Report the primary health outcomes used in the analysis and any particular instrument used to measure them,
- •
Discuss the comparability of the groups.
If only some/none of these details were given answer ‘partially’/’no.’
E3. If based on a review/synthesis of previously published studies, are the review methods described (search strategy, inclusion criteria, sources, judgement criteria, combination, investigation of differences)?
There are many important steps in a review. However, it is recommended that to answer ‘yes’ the authors should undertake the following:
- •
State the inclusion and/or exclusion criteria,
- •
List the sources searched by (e.g. MEDLINE, unpublished data),
- •
Specify the methods used to combine the results of the individual primary studies (if applicable).
If only some/none of these details were given answer ‘partially’ or ’no’.
E4. If based on opinion, are the methods used to derive estimates described?
To answer ‘yes’ the authors should have reported the methods used to derive estimates of effectiveness (model parameters), e.g. consensus, experts’ opinion and authors’ assumptions.
E5. Is the level of reporting of the effectiveness results adequate?
To answer ‘yes’ the authors need to have fully provided—in terms of principal effectiveness measures—the results of the clinical trial, the study, the literature review (or parameters used in the model) or the results based on opinion.
E6. Are the side-effects or adverse effects addressed in the analysis?
To answer ‘yes’ the author(s) should provide quantitative results relating to side-effects or adverse events. However, if the health technology being studied is not associated with side-effects (although most interventions have some degree of side-effect) the answer should be ‘not applicable’.
E7. Does the article provide the results of a statistical analysis of the effectiveness results?
Did the authors present the 95% confidence intervals and/or the p values. The responses to this question will normally be ‘yes,’ ‘no’ or ‘partially’ depending on the level of reporting. This item would not normally apply to modelling studies.
Benefit measure
B1. Do the authors specify any summary benefit measure(s) used in the economic analysis?
Is a measure of benefit used for the economic analysis? For example, lives saved, numbers of life years gained, or quality-adjusted life years (QALY). If a cost-consequences analysis is performed (e.g. several clinical outcomes are reported as in the case of a surgical intervention) write ‘not applicable’.
B2. Do the authors report the basic method of valuation of health states or interventions?
This question only applies to cost-utility and cost-benefit analyses. In the case of a cost-effectiveness study (including cost-consequences or cost-minimisation studies), write ‘not applicable’.
To answer ‘yes’ the author(s) should report the method of valuation of health states (using generic or health-specific valuation tools), willingness-to-pay, human capital, etc.
B3. Do the authors specify the source(s) of health states (e.g. specific patient population or the general public)?
Answer this question only if B2 is relevant.
Answer ‘yes’ if the authors indicate whose values were used to assess health states: e.g. authors’ assumption, clinician and patients.
B4. Do the authors specify the valuation tool used?
Answer this question only if you answered ‘yes’ to B2.
To answer ‘yes’ the authors need to specify standard gamble, time trade-off, conjoint analysis, etc. If a standard generic measure is used (e.g. EQ-5D, Health Utility Index (HUI), SF-6D) write ‘not applicable’.
B5. Is the level of reporting of benefit data adequate (incremental analysis, statistical analyses)?
To answer ‘yes’ the author(s) need to have reported the appropriate results for each study subgroup, presented the incremental results (when applicable), and the results of any statistical tests. The response should be ‘not applicable’ if the answer to question B1 was ‘no’ or ‘not applicable’.
Costs
C1. Are the cost components/items used in the economic analysis presented?
To answer ‘yes’ the author(s) need(s) to have stated which costs (drug, personnel, etc.) they measured and which costs were included in the final cost figure.
C2. Are the methods used to measure cost components/items provided?
This question requires an assessment of the way each cost item was calculated. For example, was microcosting conducted or were diagnostic-related group costing used? Irrespective of the approach used, it should be fully described.
Answer ‘yes’ if the methods are fully provided,
Answer ‘partially’ if methods are provided for some costs only or if the methods are not sufficiently described,
Answer ‘no’ when no information is provided about the method of calculation of the cost items.
C3. Are the sources of resource consumption data provided?
Answer ‘yes’ if the authors provide full details of sources, which may be: prospective or retrospective study (actual data), a model, a literature review, Health Department data, etc.
Answer ‘partially’ if sources are provided for some costs only.
C4. Are the sources of unit price data provided?
Answer ‘yes’ if the author(s) detail where their unit prices come from, e.g. hospital source, published literature, or official prices,
Answer ‘partially’ if only some sources are provided.
C5. Are unit prices for resources given?
Answer ‘yes’ if the authors give, for each resource consumption item, the unit price,
Answer ‘partially’ if only some unit prices are provided.
C6. Are costs and quantities reported separately?
Answer ‘yes’ if the author(s) provide, for each resource included, individual costs and their associated quantities used,
Answer ‘partially’ if some costs and quantities were reported separately.
C7. Is the price year given?
Answer ‘yes’ if the cost data are presented (possibly reflated if resource consumption relates to different years) for a given price year.
C8. Is the time horizon given for each element of the cost analysis?
Here we want to know the period of time covered by the cost measured (e.g.: (1) the cost of rehabilitation covers a 6-month period, which corresponds to the length of follow-up, (2) the cost of drugs were measured for the lifetime of a patient).
Answer ‘yes’ if the time horizon for each cost element is known. This is important as it will enable judgement regarding the need for discounting.
C9. Is the currency unit reported?
Answer ‘yes’ if the currency is provided.
C10. Is a currency conversion rate given?
This question only applies to studies in which the results were converted from one currency to another. Answer ‘yes’ if the results were converted and the conversion rate given. This question is mainly applicable to multicountry studies when the cost figures for each country are converted into a single currency unit (in which case all conversion rates should be reported). Also applicable to single-country studies when, for example, a French study is published in an American journal and the cost data (in Euros or in Francs) are converted into American dollars. The method should also be given (e.g. exchange rate or Purchasing Power Parities, PPP).
Answer ‘partially’ if the rate is provided for some countries in the analysis but not for all,
Answer ‘not applicable’ if no conversion of the results or data was performed.
C11. Does the article provide the results of a statistical analysis of quantities/cost results?
Answer ‘yes’ if the quantities/cost data were treated stochastically and appropriate measures of precision (e.g. p values) given. Answer ‘partially’ if descriptive statistics were provided (such as mean and standard deviation), and ‘no’ if treated deterministically (i.e. only point estimates given). If a model is used, answer ‘yes’ if the methods to deal with cost uncertainty, such as sensitivity analyses or probabilistic sensitivity analyses, are provided.
Discounting
D1. Was the summary benefit measure(s) discounted?
Answer ‘yes’ if the time horizon warranted discounting and it was undertaken. Answer ‘no’ when the time horizon warranted discounting but it was not undertaken. Answer ‘not applicable’ when the time horizon is below 1 year.
D2. Were the cost data discounted?
Answer ‘yes’ if the time horizon warranted discounting and it was undertaken. Answer ‘no’ when the time horizon warranted discounting but it was not undertaken. Answer ‘not applicable’ when the time horizon is below 1 year.
D3. Do the authors specify the rate(s) used in discounting costs and benefits?
Answer ‘yes’ if these data were given and relevant. Answer ‘not applicable’ when the time horizon did not warrant discounting for costs and benefits.
D4. Were discounted and undiscounted results reported?
Answer ‘not applicable’ if discounting for costs and benefits was not relevant.
Discussion
S1. Are quantitative and/or descriptive analyses conducted to explore variability from place to place?
Answer ‘yes’ if the authors assessed quantitative variability in the data through (for example) sensitivity analysis (e.g. modelling using country-specific data in multinational studies, or applying country-specific cost data to determine results). Answer ‘partially’ or ’no’ if descriptive or no comments (variations in practice) were given.
O1. Did the authors discuss caveats regarding the generalisability of their results?
Answer ‘yes’ if, in the discussion of the paper, the authors undertook an appraisal of how the particular features and methods of their study may limit the relevance of their findings to other locations or countries.