Abstract
Background
Valuation is a critical part of the costing process in health economic evaluations. However, an overview of specific issues relevant to the European context on harmonizing methodological requirements for the valuation of costs to be used in health economic evaluation is lacking. We aimed to inform the development of an international, harmonized and multi-sectoral costing framework, as sought in the European PECUNIA (ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions) project.
Methods
We conducted a scoping review (information extraction 2008–2021) to a) to demonstrate the degree of heterogeneity that currently exists in the literature regarding central terminology, b) to generate an overview of the most relevant areas for harmonization in multi-sectoral and multi-national costing processes for health economic evaluations, and c) to provide insights into country level variation regarding economic evaluation guidance. A complex search strategy was applied covering key publications on costing methods, glossaries, and international costing recommendations augmented by a targeted author and reference search as well as snowballing. Six European countries served as case studies to describe country-specific harmonization issues. Identified information was qualitatively synthesized and cross-checked using a newly developed, pilot-tested data extraction form.
Results
Costing methods for services were found to be heterogeneous between sectors and country guidelines and may, in practice, be often driven by data availability and reimbursement systems in place. The lack of detailed guidance regarding specific costing methods, recommended data sources, double-counting of costs between sectors, adjustment of unit costs for inflation, transparent handling of overhead costs as well as the unavailability of standardized unit costing estimates in most countries were identified as main drivers of country specific differences in costing methods with a major impact on valuation and cost-effectiveness evidence.
Conclusion
This review provides a basic summary of existing costing practices for evaluative purposes across sectors and countries and highlights several common methodological factors influencing divergence in cost valuation methods that would need to be systematically incorporated and addressed in future costing practices to achieve more comparable, harmonized health economic evaluation evidence.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Background
The increased health needs and demands as well as the scarcity of resources have resulted in a more prominent role of economic considerations in evidence-based healthcare decision-making [1]. Nevertheless, conducting economic evaluations (EEs) in the healthcare setting is a complex task. For instance, a prerequisite for the validity and usability of health economic evidence is sound costing methodology. Next to the identification, definition and measurement of relevant resource use for consideration in EEs [2], a critical part of the costing process is the valuation method. Valuation implies that the different resources used for the production of a unit of resource use (e.g. service) are multiplied with their value (e.g. price) and summed up to derive its cost, also referred to as unit cost. From an economic perspective, there is a consensus that the derived unit costs should ideally capture the benefit forgone when a resource is consumed, i.e. the opportunity cost [3], also referred to as the true ‘economic cost’ [4]. However, due to the influence of governmental regulations or stakeholder negotiations, the health and social care sectors are typically considered imperfect markets and e.g. hospital charges, physician fees or drug prices do not necessarily reflect opportunity costs [4].
While these unit costs and valuation sources have nonetheless been accepted for use in EEs in the healthcare sector [5], the application of different costing approaches and definitions still seem to be ambiguous [6]. The lack of harmonization of methodological requirements for cost valuation results in numerous practical challenges in relation to costing of health-related services, with many of these challenges not resolved. For example, no universal gold standard seems to exist up to date regarding the choice of overhead allocation method [2]. An important determinant for the appropriate source of valuation is the analytical (study) perspective from which the EE (economic evaluation) is conducted. The study perspective (e.g. a) provider, b) third-party payer, c) patient, d) society (i.e. the broadest perspective [7, 8]) determines which cost components (e.g. healthcare costs, patient and family out-of-pocket expenses, costs occurring in other sectors) are to be included in an EE [3]. At the same time, it also determines which sources are appropriate to value the service use [3]. For instance, charges may well reflect the costs from a payer’s perspective, while not necessarily capture the true economic cost of service provision from a societal perspective. Overall, several comparisons have shown that unit cost estimates in the healthcare sector are sensitive to the applied costing method (e.g. [9,10,11]).
The importance of harmonization of methodological requirements in relation to costing increases when healthcare interventions influence resource use outside the healthcare sector [12]. These so-called intersectoral costs and benefits [13] include, e.g. the valuation of spill over effects on employment and work productivity. A review shows that substantial differences in the items considered and methods used are resulting in major differences in lost productivity estimates [14]. This applies likewise to patient and family costs [15]. Although still a relatively new field of research, costs in the education and (criminal) justice sectors were found to be a vital cost component in EEs conducted from the societal perspective [16], especially in the field of mental health.
In light of the methodological heterogeneity, it often remains unclear if differences in (unit) costs observed between national and international studies are e.g. due to differences in applied costing methodology or differences attributable to the service [17]. From a global viewpoint, such differences could indeed alter the cut-off point between an efficient versus a non-efficient intervention in an EE, and eventually the decision on its reimbursement and implementation. To achieve more comparability and harmonization in methods across studies, countries and sectors, identification of the areas of controversy in cross-sectoral and cross-country cost valuations is a fundamental prerequisite. This insight is a vital first step, even if differences in data availability and formal requirements regarding the analytic perspective remain heterogeneous. Beyond this, methodological harmonization is also a relevant stepping-stone towards the increased transferability of EEs internationally.
The overarching objective of this current scoping review was to inform the development of an international, harmonized and multi-sectoral costing framework, as sought in the European PECUNIA (ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions) project [18]. This review covers multiple sectors including the health and social care, criminal justice and education sectors, as well as several selected countries as represented in the PECUNIA project. The specific objectives of the current scoping review were threefold:
-
a) To establish a set of definitions of central economic terms and main concepts to determine the degree of heterogeneity that currently exists in the literature;
-
b) To generate an overview of the most relevant potential areas for harmonization in multi-sectoral and multi-national costing processes for health EEs;
-
c) To provide insights into country level variation regarding EE guidance, based on a case study of six European countries represented in the PECUNIA project.
Methods
This scoping review was guided by the approach to map the main concepts, theories, evidence, knowledge gaps and their main sources as recommended by the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network [19]. The reporting checklist by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) was followed and can be found in the Additional file 1.
Search strategy
With key grey literature and scientific peer-reviewed articles including multiple sectors (i.e. health and social care, criminal justice and education) and methodological aspects regarding costing not being always clearly identifiable as such, a comprehensive search strategy was required.
For the establishment of a set of definitions of central economic terms and main concepts to determine the degree of heterogeneity currently existing in the literature (research aim a)) we screened textbooks and online glossaries. In addition, we screened all sources identified in course of the literature search for research aim b) for definitions. The selection of the economic terms included in the glossary was based on the criteria ‘relevance’ and ‘likelihood for interpretation differences’, determined by the research team, who are experts in the field of health economics. This overview does not make any claim to comprehensiveness (i.e. providing a complete picture regarding all existing definitions and descriptions represented in the literature), but rather aims to highlight the potential variation and partial contradictions of existing definitions of costing approaches and their components. For the identification of general methodological issues regarding sector-specific valuation issues (research aim b) potentially causing amenable heterogeneity in service use costing, the following approach was adopted: Publications from previous EU projects linked to costing methods were screened (i.e. IMPACT HTA, HealthBASKET, IBenC [20,21,22]). Reference lists of these identified publications were screened for peer-reviewed publications and grey literature. An author and snowball search in Embase and PubMed were conducted to further identify relevant peer-reviewed methodological publications including books. In addition, relevant health economic and HTA websites were searched including the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) [23], the international Health Economics Association (iHEA) [24], and the Unit Costs Article database collated by the Personal Social Services Research Unit (PSSRU) at the University of Kent.
For the identification and analysis of country-specific harmonization issues (research aim c)), six European countries represented in the European PECUNIA project (i.e. Austria, Germany, Hungary, Spain, the Netherlands and England), were selected as case studies for feasibility reasons. Since these countries represent different types of health care systems (i.e. tax-funded/social insurance-funded, different levels of costing guidance for EEs and use of such evidence, availability of standardized unit cost catalogues, geographic locations within Europe) the overview is nevertheless expected to provide a fairly complete picture. At the same time, broader international insights were considered where identified. As the development of costing methods and relevant recommendations is often part of the national Health Technology Assessment (HTA) agenda, national websites were considered as most promising to screen for country-specific guidance (i.e. national EE guidelines and unit cost(ing) manuals, costing guidelines and unit cost programmes).
The searches were augmented by additional literature (e.g. national grey literature) from the PECUNIA consortium and scientific advisory board (SAB).
Study selection
The search was conducted throughout 2018 (the start of the PECUNIA project) and information extraction limited to publications from the past 10 years (i.e. 2008) to cover timely methods. Full-text publications of peer-reviewed articles and books as well as grey literature were included. In regards to EE guidelines, the latest available publications within the defined search window were considered. In addition, we conducted an update of the search until 2021 to include any major advancements after 2018 (i.e. updates of costing guidelines).
Textbooks, online glossaries, peer-reviewed publications and grey literature were selected if they included a relevant definition/description of central economic terms. Relevant health economic and HTA websites, peer-reviewed and grey literature were selected if they included main concepts or general methodological issues regarding sector-specific valuation issues, which are of relevance when conducting multi-country, multi-sectoral health EEs. National EE/HTA guidelines and manuals, available costing programmes/manuals and guidelines, and country-specific health economic and HTA websites were selected in case they included country-specific information and recommendations regarding valuation aspects for EEs.
Publications in German, Hungarian, Spanish, Dutch, and English were considered for inclusion.
Data extraction
Identified publications were screened for their definitions/descriptions of central economic terms and main concepts. These were collected, compared and organized in a table format. General methodological aspects in respect to sector-specific valuation issues were extracted and grouped according to the specific issue it related to. For the identification and analysis of country-specific harmonization issues, data extraction was conducted using a newly developed and standardized, pilot-tested data extraction form created in MS Excel. Synthesised information on service costing methods recommendations covered: analytic study perspective (e.g. healthcare, societal), costs (to be included in an EE), list of national/regional unit cost lists available, source of valuation (cost data), the hierarchy of data sources stated for valuation, costing methodology (valuation of costs), measurement of capital costs, overhead costs and operating costs (e.g. capital costs), sources used to calculate reference prices for various healthcare services (e.g. public statistics), physical units (any common units used like hours of nursing; per visit), reporting of costs (any requirement to report costs and quantities/volumes separately), time horizon, discounting, adjustment for inflation/price indexation, specific considerations per field/area of application, periodicity of costing manual, recommended level of guideline adherence (e.g. mandatory).
In each stage of the applied methodology, at least two researchers were involved. The searches in the multiple information sources were executed by two researchers (CF, NP). Identified literature was screened against the above pre-defined inclusion criteria by two researchers (CF, NP). Potential disagreements regarding inclusion were resolved by discussion or if needed, by a third reviewer (SM, JS).
Results
The initial search via previous EU projects (N = 35), relevant books (N = 2), and grey literature (N = 14) resulted in 51 hits. The search for EE guidelines, unit cost(ing) manuals/programmes/list of unit costs, health economic and HTA websites added 46 hits. This resulted in the inclusion of a total of 97 sources for information synthesis. The overall study selection process is illustrated in the PRISMA flow diagram (see Additional file 1).
Heterogeneity of definitions of central economic terms and main concepts
Varying definitions and interpretations can be found in the international context, as the understanding and usage of a definition are highly dependent on the country context and the analysist’s professional background. Table 1 presents definitions and descriptions of central economic terms and main concepts. The aim of this glossary of terms is to give an overview of the potential variation and partial contradictions of existing definition of costing approaches and their components, which may be a jeopardizing factor for harmonization efforts.
General methodological issues regarding sector-specific valuation
The following paragraphs describe the identified general sector-specific methodological aspects and cross-sectoral costing challenges, including problems as well as potential solutions and recommendations, found in the screened literature. Table 2 provides an overview of these findings and highlights the multiple open questions regarding essential methodological aspects.
Methodological issues in costing for the health and social care sectors
Common health economic costing methods of services in the health and social care sectors are micro or gross costings for cost identification, and bottom-up or top-down approaches for valuation [22, 42,43,44]. Based on the available definitions of these approaches as described in different sources (e.g. [6, 45] however, it becomes apparent that these methods are not always clearly distinguishable or uniformly defined/applied [6, 18, 42]. In addition to the application of valuation approaches to newly developed unit costs, readily available costing sources may be used for the valuation of service use, including reference unit costs, fees, charges and market prices [22, 46]. According to recently published cross-European recommendations regarding the valuation of service use in EEs, the preferred proxy measure for the opportunity costs of healthcare and supportive care/social care services are country-specific reference unit costs, when available [40]. It remains unclear, however, in what way standard unit costs differ from market prices or charges, and e.g. what costing perspective (i.e. long-run, short-run) shall be incorporated in these estimates. At the same time, the chosen method is crucial for the resulting unit cost estimate [47]. Top-down and bottom-up approaches were previously found to yield different results when calculating the unit cost of cognitive behavioural therapy [47], although the size of the impact on the results was not specified. Also other comparisons confirmed the sensitivity of unit cost estimates in the healthcare sector regarding the applied costing methodology (e.g. [9, 11]).
Bottom-up micro-costing has not been widely used in assessing the costs of healthcare services [48]. Presumably, this is mainly due to its feasibility, as this methodology is time-consuming, especially when information systems are absent or inadequate [6]. Hospital costing studies indicate that a full bottom-up methodology should be considered for healthcare services with a large component of labour or overheads as expected for mental health services. In multi-country studies, international differences between salaries of comparable professionals may have significant effects on the unit costs. Moreover, in different countries diverse professions may deliver the same services resulting in further variation in the unit cost of specific services [47, 49,50,51,52].
It is generally agreed that depending on the specific service and its role in the EE, different costing methods are appropriate [3]. Besides, depending on the purpose of the costing exercise, different time horizons and hence cost components may be considered [3, 46]. For example [53], total costs (fixed plus variable costs) are relevant whenever a service requires considerable constructional changes (e.g. addition of a new operating theatre). Using variable costs is recommended for services that happen within existing infrastructure despite requiring new investments at other levels. Marginal costs are to be considered for services that can be offered by using existing equipment, while average costs capture total costs per unit of output [53].
Overhead costs may largely vary between different organisations, as do the use of services and the method of cost allocation [2]. Generally, there are different types of overhead costs such as capital versus non-capital to consider separately. They may be related exclusively to the management and administrative services, or to maintenance (e.g. catering, cleaning, gas, water) [2]. There are also several methods for performing overhead calculations which include the allocation of i) weighted service, ii) hourly rate, iii) inpatient day, or iv) marginal mark-up [54]. The weighted service method establishes the relative cost of the individual patient. Hourly rate yields a cost per treatment minute by employing service time of the primary treatment as a proxy for consumption. When using inpatient day allocation, all patients are assumed to have the same indirect costs per day irrespective of the actual resource use. In marginal mark-up allocation, indirect costs are distributed to direct costs by raising the direct costs with a mark-up percentage. [33].
Micro-costing is not feasible for the determination of overhead costs for hospitals and other large institutions [4]. Instead, it is suggested to apply the ratio of overhead to direct expenses for a similar subdivision [4]. Nevertheless, to date, there is no universally accepted standard for the estimation of overhead costs [2]. Potential double-counting of costs is another risk acknowledged in the health and social care sectors. For example, double counting may occur when financing costs are captured in the healthcare unit cost on the one hand but also separately considered as cost on the other hand [5, 55].
Methodological issues in costing for the education and criminal justice sectors
Examples for intersectoral costs and benefits (ICBs) resulting from healthcare interventions that affect the education sector could include special education services and the costs for student transport to the education facility [56, 57]. Criminal justice inter-sectoral costs compromise costs for court proceedings, police services, or forensic services [56, 58]. The methodology for the valuation of health-related service use in the education and (criminal) justice sectors is less established. A first major step towards the valuation of such service use was recently taken by determining several methods and testing their applicability in the Netherlands [59]. The four methods suggested for consideration based on their accuracy in a hierarchical manner include i) the opportunity cost method based on micro-costing, ii) utilisation of market prices from governmental reports, iii) self-constructed unit prices based on the information given in governmental reports, and iv) hourly labour costs for the provision of the relevant services. Although the opportunity cost method based on micro-costing (method i) is very accurate in regards to the valuation of ICBs in monetary terms, its feasibility is limited by the time it takes to calculate these [13]. Method ii) can be applied to determine proxy unit prices. Examples for relevant sources include annual reports of governmental and public or private organizations, which have been granted authority and responsibility for the provision of services related to ICBs. The reliability, transparency, unrestricted availability and unambiguous referencing are a prerequisite for the validity of this method [13]. The third method (iii) suggests using the previously mentioned annual reports for the division of the total annual costs by the total annual output. Labour costs can be used as a fourth method for valuation (method iv), e.g. from national statistics and administrative data [13]. The methodological choice should be based on the availability and reliability of the underlying data [13].
Country specific requirements and recommendations
A summary of some key costing recommendations from national health economic guidelines of the selected six countries is shown in Table 3 (extended structured synthesis Additional file 1). While these guidelines are defined as being mandatory to be followed in the Netherlands and Germany, the Spanish guideline is voluntary. The application of the Austrian, English and Hungarian guidelines is (strongly) recommended. This does not necessarily imply, however, that recommendations are also relevant for the pharmaceutical reimbursement process as in the case of Austria, where the relevant institution applies its internal criteria [60].
With regards to the adopted analytical study perspective of an EE, the German and the English recommend a healthcare perspective. The Austrian guideline is not specific in this regard. While the Dutch guideline recommends a societal perspective, the Spanish one recommends both the societal and the healthcare perspectives to be applied parallel. The Hungarian guideline states that if costs and outcomes are falling mainly outside of the healthcare system (e.g. in the case of preventive health technologies) the societal perspective is suggested in addition to the healthcare perspective. That what is meant by societal is, however, is not always explicitly stated and may vary very much from analysis to analysis.
Most guidelines state which costs should be included in an EE. The majority of them use a traditional cost component typology referring to health and social care and out-of-pocket expenses as direct costs and lost productivity as indirect costs [61]. The Dutch guideline, updated in 2016, is an exception and refers to cost components based on sectors, such as costs within the healthcare sector, patient and family costs and costs in other sectors [62].
The inclusion of social care costs such as those resulting from the use of respite care and supportive care services is explicitly recommended in four guidelines (England, Germany, the Netherlands, Spain), while the inclusion of patient (and family) costs (e.g. patient out-of-pocket expenses, time costs, informal care costs and travel costs) are recommended in four guidelines (Austria, Germany, the Netherlands, Spain). In Germany, patients’ out-of-pocket expenses are included as part of the perspective of the Statutory Health Insurance (SHI)-insured community. In contrast, time costs are not standardly considered, but are examined in the course of sensitivity analyses. The Hungarian guideline specifies that the inclusion of cost types should be determined based on the adopted perspective. The inclusion of lost productivity costs is recommended in four guidelines (Austria, Germany, the Netherlands, Spain) and are to be reported separately from direct medical costs). In the Hungarian guideline the inclusion of productivity costs is recommended just a complementary element of cost calculation if majority of costs are falling outside of the healthcare sector.
There is large variation between the guidelines regarding the suggested valuation approach. All guidelines stress that the valuation method chosen should reflect opportunity costs. In addition, various valuation methods are recommended in the guidelines, varying from top-down micro-costing to gross-costing, including the use of standard unit costs/reference prices, tariffs, market prices, administrative data, and diagnosis-related groups (DRGs). All guidelines name one or more sources, which should preferably be used for the unit costing. A standard cost list containing sets of standardized unit cost estimates is officially recognized and applied by the HTA agency in the Netherlands (Zorginstituut Nederland) [63]. These have been (partly) published as part of national costing guidelines/programs and are periodically updated. In Spain, each region has its own list of official unit costs [64]. There are also national statistics for some of the unit costs (e.g. hospitalization) managed by the ministry of health [65]. In addition, the unit cost database OBILUKE is used in Spain to obtain healthcare unit costs. It is updated annually and uses primary and secondary sources, such as published articles, reports, hospital accounting systems, but access to this unit cost database is not free of charge [66]. Other countries (e.g. Germany) also have unit cost projects in place that have been conducted systematically with a clear methodology, but are not regularly updated. In Austria, a list with unit costs used in existing Austrian health economic analyses has been published [48, 67]. Updates of the national unit cost lists are published periodically spanning from annual updates (e.g. England and Wales) to updates as required for methodological reasons or timeliness, e.g. four-yearly in the Netherlands.
There is limited information in the majority of the guidelines about the valuation and allocation of capital/overhead and operating costs, i.e. costs that cannot be directly allocated. Austria and Spain do not include recommendations on this at all. The Dutch guideline names different methods that can be used to measure overheads such as average fixed costs per unit, the equivalence method, and the mark-up method. Although no specific method is recommended, it is emphasized that each of these methods has different advantages and disadvantages. Based on data from the financial statistics for Dutch hospitals, an estimation of overhead costs was conducted in 2012. The derived percentage for overheads on the directly attributable costs of medical departments was 38%. The percentage for overheads on housing and depreciation costs on the directly attributable costs of medical departments was 6%, adding up to a total of 44% for overheads. This percentage reflects an average with huge variations between individual organisations and services [63].
For the adjustment of unit costs from different calendar years, the Austrian and Spanish guidelines recommend that costs should be adjusted to a common reference year, but they fail to provide further details. The Hungarian guideline states that costs shall be uprated to the same date. The consumer price index (inflation) should be chosen as inflation rate, irrespectively of whether the costs (or savings) arise within or outside the healthcare sector, with the official publications of the Hungarian Central Statistical Office as recommended source. Both the German and Dutch guidelines specify that all costs should be inflated to the present value by using the official price index from national statistics. The Dutch guideline also tackles the aspect of correcting for inflation in case of including costs from different European countries. It is suggested that the Indices of Consumer Prices (HICP) of the European Central Bank, which were specially designed for international comparisons of consumer price inflation, need to be used.
Discussion
For EEs to be able to inform efficient resource allocation based on valid high-quality evidence, it is crucial that both outcomes and costs are assessed rigorously [68]. In contrast to outcomes, costs appear to have suffered neglect regarding methodological research, resulting in the absence of an universally accepted costing methodology for the healthcare sector [69], as well as other sectors affected by the impact of healthcare interventions such as the education and criminal justice sectors [13]. Unit cost estimates between studies and countries are often not comparable due to differences in costing methodologies [70] and the lack of detailed methodological guidance, which may also result in decision-makers’ low confidence as a barrier to the uptake of EEs [43, 71]. As of now, collections of cost estimates are not routinely available for services across European countries, especially beyond the health and social care sectors [72]. If available, it is unclear whether differences in cost estimates stem from differences in the service composition, intensity and definition (unit of analysis), differences in the unit of measurement, or differences in methodological approaches for costing including input costs (unit of valuation) [73], calling for more transparency and harmonization in this respect.
This scoping review highlights multiple methodological problems challenging the harmonization of service unit costs for (inter)national, multi-sectoral health EEs. For the health and social care sectors, these include ambiguity and feasibility problems regarding the definition and application of cost identification and valuation approaches. Several problems regarding the data source used to develop unit costs were identified: the impact of choice of the costing sources, the applied costing perspective and unavailable proxy measures, as well as the absence of a standard to estimate overhead costs.
Moreover, the limited cross-country comparability of labour costs and ambiguity concerning the application of cost types were identified as further difficulties. Challenges in respect to the education and criminal justice sectors concern the valuation of health-related service use in these sectors, unestablished valuation methods, and missing feasibility testing in the international context, as well as difficulties regarding the generation of proxy unit prices. Double counting of costing components also may not only be an issue within the health care sector as previously reported, but also cross-sectorial. For example, double counting of out-of-pocket expenses may occur as part of the unit cost of a health care service and as part of patients’ out-of-pocket costs, which may arise due to lacking transparency in the reported costing components. The analysis of national costing guidelines revealed extensive variation concerning the recommendations with respect to guideline adherence, analytical study perspective, valuation approach and inflation.
The early FP6-funded HealthBasket project (2004–2007), which focused on harmonization of costing in the healthcare sector, already concluded one decade ago that “the prerequisite for international cost comparison is mutually accepted methodological guidance (standard costing method) and reasonably good compliance with it” [22]. Furthermore, it stated that consensus alone on basic scientific principles would not be sufficient to achieve meaningful comparability. Instead, it was proposed to “standardize” the most important and frequently used methods, including resource use measurement, cost allocation methods, including allocation base and allocation techniques and valuation methods, as well as capacity utilisation and to include detailed instructions on how to implement these instruments in practice [22]. Another recent editorial also recommended that an independent group should be mandated with the production of standard country-specific unit costs available to national and international researchers and decision-makers [70]. These challenges were one of the key motivations for setting-up the European PECUNIA project, aiming among others to systematically address the above heterogeneity-causing factors in costing methods and to develop unit costs for different countries and sectors based on harmonized methods [18, 74].
At the same time, some practical limits to the harmonization of costing will remain. Firstly, unit costs are one major ingredient to the valuation aspect in EEs. Limited guidance regarding the calculation of overheads resulting in assumptions are a very much under-discussed, under-researched and under-reported area. Both, extensive sensitivity analysis or transparent overhead calculation as one harmonization aspect for unit costs would be potential ways forward. There are, however, also broader costing issues [45], such as questions about the inclusion of future medical costs [75], the choice of the discount rate [40], and the choice of the analytical study perspective [76] that may introduce systematic differences in cost estimates [77, 78]. Secondly, where newly developed harmonization strategies are not fully in line with existing national EE guidelines, especially those relevant for reimbursement decisions, their implementation most likely will face resistance. The quality and transferability of international EE studies that include multiple countries would, however, still greatly benefit from such standardisation approaches [40]. The same applies to national EEs where such perceived conflict does not exist, or the evaluation perspective is expected to be expanded to a multi-sectoral, societal one. On the other hand, the compulsory nature of EE guidelines seem to promote the availability of more comprehensive and standardised unit cost catalogues (e.g. England, Netherlands) and the use of more harmonised costing methods.
To the best of our knowledge, this is the first review on costing methods across several sectors affected by healthcare interventions focusing on a set of selected European countries and healthcare systems. Other publications focusing on differences between EE guidelines do exist (e.g. [79, 80]), but they cover different aspects (e.g. methods for price and currency adjustment, uncertainty analysis), additional countries outside of Europe (e.g. Australia, Thailand, Japan), or include older guideline versions.
The selected countries included in the PECUNIA Project differ regarding their health care systems with varying feasibility and acceptability of EEs in evidence-informed decision-making. Some countries have established national unit cost programmes/lists (DE, NL, UK), some early stage initiatives (AT, ES, HU). Availabilities of health utility value sets for outcome evaluations and requirements in terms of the study perspective also differ. Nevertheless, the country selection is not necessarily balanced in regards to the aforementioned aspects.
Study selection and data extraction was challenging in some cases, especially when slightly divergent information on one topic was identified. However, two researchers were involved in the study selection process and data extraction phase, and with two additional researchers consulted in case of disagreements, which enabled extensive discussions and thorough assessment of the identified material. Due to the broad topic area and expected spread of relevant information between different types of publications, mostly within the grey literature needed to be included. Our search strategy to identify grey literature was very comprehensive and national experts were involved to complement our findings. In addition, a search update was conducted to enhance the review’s timeliness. Although the applied strategy to identify peer-reviewed publications was expected to capture all relevant key topics, as it covered different approaches which were explained in our methods section, it cannot be ruled out that some potentially relevant material has been left out, despite our attempts to be as comprehensive as possible.
Conclusions
Several methodological issues were identified that lead to the current heterogeneity in valuation methods in health EEs used across sectors and countries. To address these explicitly in future costing guidelines and tools is a key step towards more comparability and harmonization in national and international health EEs.
Availability of data and materials
All data generated or analysed during this study are included in this published article.
Abbreviations
- BIA:
-
Budget impact analysis
- Catsalut:
-
The Catalan Health Service (Servei Català de la Salut)
- CBS:
-
Statistics Netherlands (Centraal Bureau voor de Statistiek)
- CPI:
-
The consumer price index
- DoH:
-
Department of Health
- DRGs:
-
Diagnose-related groups
- EE(s):
-
Economic evaluation(s)
- EU:
-
European Union
- EQUATOR:
-
The Enhancing the QUAlity and Transparency Of health Research Network
- FCA:
-
The friction cost approach
- FTE:
-
Full Time Equivalent
- HBCs:
-
Diagnosis Related Group, DRG (homogen betegsegcsoport)
- HCA:
-
The human capital approach
- HES:
-
Hospital Episode Statistic
- HICP:
-
The Indices of Consumer Prices
- HRG:
-
Healthcare Resource Group
- HTA:
-
Health Technology Assessment
- ICBs:
-
Inter-sectoral costs and benefits
- iHEA:
-
The international Health Economics Association
- ISPOR:
-
The International Society for Pharmacoeconomics and Outcomes Research
- IQWIG:
-
The independent Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen)
- n/a:
-
Not available
- NHS:
-
National Health Service
- NZA:
-
Dutch healthcare authority (Nederlands Zorg Autoriteit)
- OENO:
-
International Classification of Procedures in Medicine
- OSTEBA:
-
Basque Office for Health Technology Assessment
- PPP:
-
Purchasing power parities
- PSS:
-
Personal social services
- PECUNIA:
-
The European ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions
- PSSRU:
-
The Personal Social Services Research Unit
- RRISMA-ScR:
-
Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews
- SA:
-
Sensitivity analysis
- SHI:
-
Statutory Health Insurance
- UKE:
-
The University Medical Center Hamburg-Eppendorf
- VAT:
-
Value-added tax
References
Chisholm DE, Evans DB. Economic evaluation in health: saving money or improving care? J Medical Econ. 2007;10:325–37.
Razzouk D. Methods for Measuring and Estimating Costs. In: Razzouk D, editor. Mental Health Economics: The Costs and Benefits of Psychiatric Care. Cham: Springer International Publishing; 2017. p. 19–33.
Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 4th ed. Oxford: Oxford University Press; 2015.
Barnett PG. An improved set of standards for finding cost for cost-effectiveness analysis. Med Care. 2009;47(7 Suppl 1):S82–8.
Larg A, Moss JR. Cost-of-illness studies: a guide to critical evaluation. Pharmacoeconomics. 2011;29(8):653–71.
Špacírová Z, Epstein D, García-Mochón L, Rovira J, Olry de Labry Lima A, Espín J. A general framework for classifying costing methods for economic evaluation of health care. Eur J Health Econ. 2020;21(4):529–42.
Luce BR, Elixhauser A. Standards for the socioeconomic evaluation of health care services. Berlin: Springer Science & Business Media; 2012.
Drost R, van der Putten IM, Ruwaard D, Evers S, Paulus ATG. Conceptualizations of the societal perspective within economic evaluations: A systematic review. Int J Technol Assess Health Care. 2017;33(2):251–60.
Clement F, Ghali WA, Donaldson C, Manns BJ. The impact of using different costing methods on the results of an economic evaluation of cardiac care: microcosting vs gross-costing approaches. Health Econ. 2009;18(4):377–88.
Heerey A, McGowan B, Ryan M, Barry M. Microcosting versus DRGs in the provision of cost estimates for use in pharmacoeconomic evaluation. Expert Rev Pharmacoecon Outcomes Res. 2002;2(1):29–33.
Shrestha RK, Sansom SL, Farnham PG. Comparison of Methods for Estimating the Cost of Human Immunodeficiency Virus-Testing Interventions. J Public Health Manag Pract. 2012;18(3):259–67.
Simon J. Health economic analysis of service provision. (Chapter 23/136). In: Geddes JR, Andreasen NC, Goodwin GM, editors. New Oxford Textbook of Psychiatry. 3rd ed. Oxford: Oxford University Press; 2020.
Drost RM, Paulus AT, Ruwaard D, Evers SM. Valuing inter-sectoral costs and benefits of interventions in the healthcare sector: methods for obtaining unit prices. Expert Rev Pharmacoecon Outcomes Res. 2017;17(1):77–84.
Noben CY, Evers SM, Nijhuis FJ, de Rijk AE. Quality appraisal of generic self-reported instruments measuring health-related productivity changes: a systematic review. BMC Public Health. 2014;14(1):115.
Grosse SD, Pike J, Soelaeman R, Tilford JM. Quantifying Family Spillover Effects in Economic Evaluations: Measurement and Valuation of Informal Care Time. Pharmacoeconomics. 2019;37(4):461–73.
Drost R, van der Putten I, Ruwaard D, Evers S, Paulus A. Conceptualizations of the societal perspective within economic evaluations: systematic review. Int J Technol Assess Health Care. 2017;33(2):251–60.
Tan SS, Bakker J, Hoogendoorn ME, Kapila A, Martin J, Pezzi A, et al. Direct cost analysis of intensive care unit stay in four European countries: applying a standardized costing methodology. Value Health. 2012;15(1):81–6.
Simon J, Konnopka A, Brodszky V, Evers S, Hakkaart-van Roijen L, Serrano-Pérez P, et al. (Pharmaco) economic evaluations for mental health related services: the PECUNIA project. Eur J Public Health. 2020;30(Suppl 5):ckaa166. 574.
Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.
EHMA – European Health Management Association. IBenC - Identifying best practices for care-dependent elderly by Benchmarking Costs and outcomes of community care 2022. https://ehma.org/research-projects/ibenc/. Accessed 10 May 2020.
IMPACT HTA – Improved methods and actionable tools for enhancing HTA, 2021. https://www.impact-hta.eu/. Accessed 10 May 2020.
Busse R, editor. HealthBASKET: Executive Summary and Policy Recommendations. 2007. https://ehma.org/research-projects/past-projects/healthbasket/. Accessed 10 May 2020.
ISPOR – The Professional Society for Health Economics and Outcomes Research. https://www.ispor.org/home. Accessed 10 May 2018.
iHEA – international Health Economics Association. Useful Links. https://www.healtheconomics.org/page/UsefulLinks. Accessed 10 May 2020.
CDC – Centers for Disease Control and Prevention. Economic Evaluation Glossary. https://www.cdc.gov/dhdsp/evaluation_resources/economic_evaluation/docs/Economic_Evaluation_Glossary.pdf. Accessed 20 Sept 2021.
BMJ Best Practice. A glossary of health economics terms 2022. https://bestpractice.bmj.com/info/toolkit/ebm-toolbox/a-glossary-of-health-economics-terms/. Accessed 20 May 2020
Hrifach A, Brault C, Couray-Targe S, Badet L, Guerre P, Ganne C, et al. Mixed method versus full top-down microcosting for organ recovery cost assessment in a French hospital group. Health Econ Rev. 2016;6(1):1–7.
Olsson TM. Comparing top-down and bottom-up costing approaches for economic evaluation within social welfare. Eur J Health Econ. 2011;12(5):445–53.
Chapko MK, Liu CF, Perkins M, Li YF, Fortney JC, Maciejewski ML. Equivalence of two healthcare costing methods: bottom-up and top-down. Health Econ. 2009;18(10):1188–201.
Australian Government. Key financial terms 2020. https://business.gov.au/finance/accounting/key-financial-terms . Accessed 20 May 2020.
CADTH – Canadian Agency for Drugs and Technologies in Health. Guidance Document for the Costing of Health Care Resources in the Canadian Setting: Second Edition. 2016, March 30.
Karabatsou D, Tsironi M, Tsigou E, Boutzouka E, Katsoulas T, Baltopoulos G. Variable cost of ICU care, a micro-costing analysis. Intensive Crit Care Nurs. 2016;35:66–73.
Tan SS. Microcosting in economic evaluations Ioa, feasibility, consistency and generalisability. 2009. https://core.ac.uk/download/pdf/18510449.pdf. Accessed 16 Aug 2021.
Hendriks ME, Kundu P, Boers AC, Bolarinwa OA, Te Pas MJ, Akande TM, et al. Step-by-step guideline for disease-specific costing studies in low-and middle-income countries: a mixed methodology. Glob Health Action. 2014;7(1):23573.
Than TM, Saw YM, Khaing M, Win EM, Cho SM, Kariya T, et al. Unit cost of healthcare services at 200-bed public hospitals in Myanmar: what plays an important role of hospital budgeting? BMC Health Serv Res. 2017;17(1):1–12.
York Health Economics Consortium. Glossary Health Economic Terms. 2016. https://yhec.co.uk/resources/glossary/. Accessed 20 May 2020.
Sandmann FG, Robotham JV, Deeny SR, Edmunds WJ, Jit M. Estimating the opportunity costs of bed-days. Health Econ. 2018;27:592–605.
Kalman N, Hammill B, Schulman K, Shah B. Hospital overhead costs: the neglected driver of health care spending? J Health Care Finance. 2015;41(4):1–15.
Beecham J, Bonin E-M. Preventonomics unit cost calculator v1.2: Guidance document. 2016.
van Lier LI, Bosmans JE, van Hout HPJ, Mokkink LB, van den Hout WB, de Wit GA, et al. Consensus-based cross-European recommendations for the identification, measurement and valuation of costs in health economic evaluations: a European Delphi study. Eur J Health Econ. 2017;19(7):993–1008.
National Library of Medicine. National Information Center on Health Services Research and Health Care Technology (NICHSR). Glossary of frequently encountered terms in health economics. 2020. https://www.nlm.nih.gov/nichsr/edu/healthecon/glossary.html. Accessed 20 May 2021.
Tan SS, Rutten FFH, van Ineveld BM, Redekop WK, Hakkaart-van RL. Comparing methodologies for the cost estimation of hospital services. Eur J Health Econ. 2009;10(1):39–45.
Mayer S, Fischer C, Zechmeister-Koss I, Ostermann H, Simon J. Are unit costs the same? A case study comparing different valuation methods for unit cost calculation of general practitioner consultations. Value Health. 2020;23(9):1142–8.
Jacobs JC, Barnett PG. Emergent Challenges in Determining Costs for Economic Evaluations. Pharmacoeconomics. 2017;35(2):129–39.
Franklin M, Lomas J, Walker S, Young T. An Educational Review About Using Cost Data for the Purpose of Cost-Effectiveness Analysis. Pharmacoeconomics. 2019;37(5):631–43.
Mogyorosy Z, Smith P. The main methodological issues in costing health care services: a literature review. 2005. Centre for Health Economics, University of York Working Papers, No. 007cherp.
Barrett B, Petkova H. Cognitive behaviour therapy: a comparison of costs. 2013. https://www.pssru.ac.uk/pub/uc/uc2013/barrett-petkova.pdf. Accessed 10 May 2020.
Mayer S, Kiss N, Łaszewska A, Simon J. Costing evidence for health care decision-making in Austria: A systematic review. PLoS ONE. 2017;12(8):e0183116.
McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, et al. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS One. 2012;7(8):e40808.
Seivewright H, Green J, Salkovskis P, Barrett B, Nur U, Tyrer P. Cognitive–behavioural therapy for health anxiety in a genitourinary medicine clinic: randomised controlled trial. Br J Psychiatry. 2008;193(4):332–7.
Lamb SE, Hansen Z, Lall R, Castelnuovo E, Withers EJ, Nichols V, et al. Group cognitive behavioural treatment for low-back pain in primary care: a randomised controlled trial and cost-effectiveness analysis. Lancet. 2010;375(9718):916–23.
McCrone P, Knapp M, Kennedy T, Seed P, Jones R, Darnley S, et al. Cost-effectiveness of cognitive behaviour therapy in addition to mebeverine for irritable bowel syndrome. Eur J Gastroenterol Hepatol. 2008;20(4):255–63.
Moreno K, Sanchez E, Salvador-Carulla L. Methodological advances in unit cost calculation of psychiatric residential care in Spain. J Ment Health Policy Econ. 2008;11(2):79–88.
Tan SS, Van Ineveld BM, Redekop WK, Hakkaart-van RL. Comparing Methodologies for the Allocation of Overhead and Capital Costs to Hospital Services. Value Health. 2009;12(4):530–5.
Larg A, Moss JR. Cost-of-Illness Studies. Pharmacoeconomics. 2011;29(8):653–71.
Mayer S, Paulus ATG, Łaszewska A, Simon J, Drost R, Ruwaard D, et al. Health-Related Resource-Use Measurement Instruments for Intersectoral Costs and Benefits in the Education and Criminal Justice Sectors. Pharmacoeconomics. 2017;35(9):895–908.
Pokhilenko I, Janssen LM, Evers SM, Drost RM, Simon J, König H-H, et al. Exploring the identification, validation, and categorization of costs and benefits of education in mental health: The PECUNIA project. Int J Technol Assess Health Care. 2020:1–7.
Janssen LMM, Pokhilenko I, Evers S, Paulus ATG, Simon J, König HH, et al. Exploring the identification, validation, and categorization of the cost and benefits of criminal justice in mental health: the PECUNIA project. Int J Technol Assess Health Care. 2020:1–8.
Drost RM, Paulus AT, Ruwaard D, Evers SM. Valuing inter-sectoral costs and benefits of interventions in the healthcare sector: methods for obtaining unit prices. Expert Rev Pharmacoecon Outcomes Res. 2013;17(1):77–84.
Zechmeister-Koss I, Stanak M, Wolf S. The status of health economic evaluation within decision making in Austria. Wien Med Wochenschr. 2019;169(11):271–83.
Bracco A, Krol M. Economic evaluations in European reimbursement submission guidelines: current status and comparisons. Expert Rev Pharmacoecon Outcomes Res. 2013;13(5):579–95.
Kanters TA, Bouwmans-Frijters C, van der Linden N, Tan SS, van Hakkaart-van Roijen L. Update of the Dutch manual for costing studies in health care. PLoS One. 2017;12(11):e0187477.
Hakkaart-van Roijen L, Van der Linden N, Bouwmans C, Kanters T, Tan S. Methodology of costing research and reference prices for economic evaluations in healthcare [in Dutch: Kostenhandleiding: Methodologie van kostenonderzoek en referentieprijzen voor economische evaluaties in de gezondheidszorg]: Zorginstituut Nederland - National Health Care Institute; 2016. https://www.zorginstituutnederland.nl/over-ons/publicaties/publicatie/2016/02/29/richtlijn-voor-het-uitvoeren-van-economische-evaluaties-in-de-gezondheidszorg. Accessed 10 May 2020.
Artola C GM, Perez JJ, Urtasun A, Fiorito A, Vila D. Monitoring the Spanish economy froma regional perspective: main elements of analysis. Documentos Ocasionales N.1809.
INE – Instituto Nacional de Estadistica. Economcy. INEbase. https://www.ine.es/dyngs/INEbase/en/categoria.htm?c=Estadistica_P&cid=1254735570541. Accessed 21 Sept 2021.
Obiluke Consulting. eSalud Platform 2018. http://www.oblikue.com/en/esalud.html. Accessed 21 Sept 2021.
Department of Health Economics (DHE), Center for Public Health, Medical University of Vienna. DHE Unit Cost Online Database: Cost Collection from Existing Studies. Version 3.1/2019 Vienna. 2019. https://healtheconomics.meduniwien.ac.at/downloads/dhe-unit-cost-online-database/access-to-database/. Accessed 10 May 2020.
Xu X, Nardini HKG, Ruger JP. Micro-costing studies in the health and medical literature: protocol for a systematic review. Syst Rev. 2014;3(1):1.
Busse R, Schreyögg J, Smith PC. Variability in healthcare treatment costs amongst nine EU countries–results from the HealthBASKET project. Health Econ. 2008;17(S1).
Frappier J, Tremblay G, Charny M, Cloutier LM. Costing bias in economic evaluations. J Med Econ. 2015;18(8):596–9.
Cheung KL, Evers S, De Vries H, Levy P, Pokhrel S, Jones T, et al. Most important barriers and facilitators of HTA usage in decision-making in Europe. Expert Rev Pharmacoecon Outcomes Res. 2018;18(3):297–304.
Kim DD, Silver MC, Kunst N, Cohen JT, Ollendorf DA, Neumann PJ. Perspective and Costing in Cost-Effectiveness Analysis, 1974–2018. Pharmacoeconomics. 2020;38(10):1135–45.
Busse R, Schreyogg J, Smith PC. Variability in healthcare treatment costs amongst nine EU countries - results from the HealthBASKET project. Health Econ. 2008;17(1 Suppl):S1-8.
PECUNIA—ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions. http://www.pecunia-project.eu/. Accessed 10 May 2020.
van Baal P, Morton A, Meltzer D, Brouwer W. Future unrelated medical costs need to be considered in cost effectiveness analysis. Eur J Health Econ. 2019;20(1):1–5.
Blank MM, Chen L, Papageorge M, Driscoll D, Graham R, Chatterjee A. The underreporting of cost perspective in cost-analysis research: A systematic review of the plastic surgery literature. J Plast Reconstr Aesthet Surg. 2018;71(3):366–76.
Walker S, Griffin S, Asaria M, Tsuchiya A, Sculpher M. Striving for a societal perspective: a framework for economic evaluations when costs and effects fall on multiple sectors and decision makers. Appl Health Econ Health Policy. 2019;17(5):577–90.
Franklin M, Lomas J, Richardson G. Conducting value for money analyses for non-randomised interventional studies including service evaluations: an educational review with recommendations. Pharmacoeconomics. 2020;38(7):665–81.
García-Mochón L, Špacírová Z, Espín J. Costing methodologies in European economic evaluation guidelines: commonalities and divergences. Eur J Health Econ. 2021:1–13.
Sharma D, Aggarwal AK, Downey LE, Prinja S. National healthcare economic evaluation guidelines: a cross-country comparison. Pharmacoecon Open. 2021;5(3):349–64.
Acknowledgements
The authors would like to thank Valentin Brodszky, Mencia Ruiz Gutierrez Colosia, Silvia Evers,, Leona Hakkaart-van Roijen, Timea Helter and Joanna Thorn for their assistance with the analysis of the country specific documents in their national languages. We are also grateful for essential national literature provided by the PECUNIA Group.
Funding
This study was conducted as part of the PECUNIA (ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions; grant agreement No 779292) research project funded by the European Union’s Horizon 2020 research and innovation programme. All views and opinions expressed in this article are those of the authors and do not necessarily reflect the views of the funding agency.
Author information
Authors and Affiliations
Contributions
SM and JS conceived the idea for this review. CF and NP conducted the search and collated, synthesized and summarized the results with inputs from SM under the supervision of JS. JS secured the funding for the project. Based on the write-up of the results, SM together with CF drafted the manuscript, which was revised based on comments from NP and JS. All authors read and approved the final version of this manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
Authors have no conflicts to declare.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1:
Figure 1. Adapted PRISMA flow chart of study selection. Table 2. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist. Table 3. Synthesis of country-specific recommendations regarding valuation aspects from economic evaluation guidelines of six selected European countries.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Fischer, C., Mayer, S., Perić, N. et al. Harmonization issues in unit costing of service use for multi-country, multi-sectoral health economic evaluations: a scoping review. Health Econ Rev 12, 42 (2022). https://doi.org/10.1186/s13561-022-00390-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13561-022-00390-y