Refractory septic shock in children: a European Society of Paediatric and Neonatal Intensive Care definition
- 4.6k Downloads
Although overall paediatric septic shock mortality is decreasing, refractory septic shock (RSS) is still associated with high mortality. A definition for RSS is urgently needed to facilitate earlier identification and treatment. We aim to establish a European society of paediatric and neonatal intensive care (ESPNIC) experts’ definition of paediatric RSS.
We conducted a two-round Delphi study followed by an observational multicentre retrospective study. One hundred and fourteen paediatric intensivists answered a clinical case-based, two-round Delphi survey, identifying clinical items consistent with RSS. Multivariate analysis of these items in a development single-centre cohort (70 patients, 30 % mortality) facilitated development of RSS definitions based on either a bedside or computed severity score. Both scores were subsequently tested in a validation cohort (six centres, 424 patients, 11.6 % mortality).
From the Delphi process, the draft definition included evidence of myocardial dysfunction and high blood lactate levels despite high vasopressor treatment. When assessed in the development population, each item was independently associated with the need for extracorporeal life support (ECLS) or death. Resultant bedside and computed septic shock scores had high discriminative power against the need for ECLS or death, with areas under the receiver operating characteristics curve of 0.920 (95 % CI 0.89–0.94), and 0.956 (95 % CI 0.93–0.97), respectively. RSS defined by a bedside score equal to or higher than 2 and a computed score equal to or higher than 3.5 was associated with a significant increase in mortality.
This ESPNIC definition of RSS accurately identifies children with the most severe form of septic shock.
KeywordsShock Septic Paediatrics Acidosis Lactic Resuscitation Failure Cardiac
Infection remains the leading cause of paediatric mortality worldwide . In most high-income countries, access to care, vaccination campaigns and improvement in intensive care have drastically decreased deaths from infection. Similar to adults, recent paediatric studies have shown an increase in the incidence of invasive infection and septic shock and a relative decrease in mortality [2, 3, 4].
Recently, sepsis and septic shock definitions in adults were revised [4, 5, 6, 7]. The aim of the Sepsis-3 definition was to help the clinician in the detection of septic shock patients and to treat them according to their risk of death. The sepsis-related organ failure assessment (SOFA)  has a prominent place in this definition, but is neither adapted nor validated in paediatric patients. The clinical course of septic shock is affected by the age and immune state of the patient, the virulence of the pathogen, and the haemodynamic adaptation to circulatory failure. Paediatric logistic organ dysfunction (PELOD) is a paediatric organ dysfunction score validated in the severity classification of patients with sepsis . However, similar to the SOFA score, it does not impact patient management at the bedside.
The Surviving Sepsis Campaign  and the American College of Critical Care Medicine (ACCM) guidelines for hemodynamic support in neonates and children  remain the most recognized standards of care. Specific therapies such as activated protein C [12, 13] or early goal-directed therapy [14, 15] have not been associated with a consistent decrease in mortality. The term septic shock encompasses various aetiologies and immune states for which specific interventions may have variable results . However, the subset of patients who are unresponsive to standard resuscitation are often labelled as having ‘refractory septic shock’ (RSS). RSS is typified by circulatory failure due to septic cardiomyopathy [17, 18, 19] with or without vasoplegia . Importantly, effective short-term support of the circulation with newer vasoactive agents or extracorporeal life support (ECLS)  means that RSS is potentially reversible [21, 22]. To maximise the impact of these rescue therapies, a robust tool for early identification of RSS is required,
The Infection, Systemic Inflammation and Sepsis section of the European Society of Paediatric and Neonatal Intensive Care (ESPNIC) designated a taskforce to define paediatric RSS . We performed a study linking a two-round Delphi survey with the development of septic shock scores. These scores aimed at an early indentification of RSS patients with a high risk of mortality at any moment during clinical care. The scores were subsequently validated in a multicentre international retrospective cohort.
Materials and methods
The first part of the Delphi study was a clinical case-based questionnaire. Two members (L.M. and P.T.) of the Infection, Systemic Inflammation and Sepsis section of ESPNIC were assigned to create four clinical cases of septic shock patients with varying levels of shock and organ dysfunction.
The characteristics of the clinical cases were set following a review of the literature on septic shock and a case analysis of septic shock patients, and covered the whole clinical spectrum of disease severity. Five members from the taskforce (G.M., S.N., M.P., M.K., N.J.G.J.) reviewed the cases for consistency and objectivity. All clinical cases were composed of vignettes with specific clinical and biological parameters (later described as criteria and corresponding cut-offs), encompassing the evolution of sepsis. For each vignette, responders were asked to grade the occurrence of RSS (0 = no RSS to 10 = yes) (see supplementary file 1). The questionnaire was then sent to all ESPNIC members via an Internet survey supplier (SurveyMonkey Inc., Palo Alto, CA, USA).
Criteria extracted from the clinical vignettes were ranged according to the distribution of their strength of association with RSS. Criteria graded A (first quartile) and B (second quartile) were selected and compiled in four draft definitions constituted by a minimal definition alone or with up to three additional organ failures. Preliminary definitions were then tested in the second part of the Delphi study. Participants of this second part had access to the results of the first study. They were asked to choose one of the four definitions and to re-score the definition cut-off values issued from the first part’s results. Based on this second round, criteria graded A or B were incorporated into the selected RSS draft definition. (See supplementary file 2 for additional information on methodology).
Development population application and definition adjustment
The RSS draft definition was first tested in a development population. All patients less than 18 years old admitted to the Paediatric Intensive Care Unit (PICU) at Centre A for septic shock, according to consensus definition , and requiring vasopressor or inotrope therapy between January 1, 2010 and December 31, 2013 were analysed. Exclusion criteria were: (1) the presence of limitations of life support at septic shock onset; (2) postmenstrual corrected age <37 weeks; and (3) perinatal sepsis defined as sepsis occurring in the first 7 days of life.
Protocols for clinical care of septic shock patients did not change during the 3-year study period. Patients were retrospectively identified using hospital records and data collected. The French Intensive Care Society ethics committee approved this study (CE SRLF 14-38). Clinical variables included in the definition or used to describe the populations or outcomes, and occurring before the occurrence of the composite endpoint PICU mortality and/or ECLS, were recorded (see supplementary file 2). Draft definition criteria identified in the Delphi study were evaluated against two outcomes: PICU mortality, and the composite endpoint PICU mortality and/or ECLS. Multivariable Cox regression was performed to study the association between the variables used in the definition and these two outcomes, to define two RSS scores. An easy to calculate score was named “bedside RSS score” and a more complex one was named “computed RSS score”.
RSS definition assessment in a validation population
Hospitals and patients
Type of PICU
PICU admissions and mortality
Number of septic shock cases and mortality
Weight in the cohort (cases/deaths)
M, S, ECLS
3109 (4.2 %)
70 (30 %)
100 %/100 %
M, S, ECLS
4608 (3.4 %)
40 (7.5 %)
9.4 %/6.1 %
3875 (7.1 %)
165 (13.9 %)
38.9 %/46.9 %
1288 (5.6 %)
101 (6.9 %)
23.8 %/14.3 %
M, S, C
3145 (2.7 %)
18 (33.3 %)
4.2 %/12.2 %
M, S, C, ECLS
3117 (4.7 %)
57 (3.5 %)
13.4 %/4.1 %
3150 (1.9 %)
44 (18.2 %)
10.4 %/16.3 %
Continuous variables were tested for normality with Kolmogorov–Smirnov test and compared with Student’s t test or Mann–Whitney test, as appropriate. Non-continuous variables were tested with Chi-squared test or Fisher’s exact test, as appropriate. Data were described as frequencies and percentages, means and standard deviations or median and interquartile range, as appropriate. Multivariable Cox’s regression with backward-stepwise method was performed having as outcomes mortality or the composite endpoint mortality/need for ECLS. Covariates inserted in the models were the variables identified through the Delphi process and the patients’ age . Adjusted hazard ratios were used to weight covariates and included in the scores. The model goodness of fit was evaluated with Omnibus test. Receiver operating characteristics (ROC) analysis was then performed with the validation population for both scores and areas under the curves (AUC) were calculated to assess for the discriminative power of these scores. The best thresholds for these scores were obtained with the calculation of sensitivity, specificity, positive and negative predictive values and the Youden’s index (sensitivity + specificity – 1). The DeLong test was used to compare the AUC of both scores . Survival of RSS patients according to both scores have been evaluated by Kaplan–Meier curves and these latter have been tested using Logrank test. A p value of less than 0.05 was considered statistically significant. Statistical analyses were performed with SPSS for Macintosh, v.22.0 (SPSS, Chicago, IL, USA) and GraphPad Prism®, v. 5.0a (GraphPad Software, La Jolla, CA, USA) softwares.
Draft of the definition
Results of the first round of the Delphi study: association of clinical criteria with refractory septic shock diagnosis
Criteria and cut-offs
Number of iteration
ScvO2 < 70 %
Vaso-inotrope score (mcg/kg min)
Vaso-inotrope association bitherapy
Vaso-inotrope association tritherapy
Mild (P/F 200–299)
Moderate (P/F 100–199)
Severe (P/F < 100)
Cardiac index (L/min m2)
Cardio circulatory arrest
Left ventricle ejection fraction (%)
Prior PICU admission
Hepatic insufficiency (prothrombin time/factor V <50 % or INRe >2)
Need for CRRT
Procalcitonin at admission (ng/mL)
Time since PICU admission (h)
Development population application and definition adjustment
During a 4-year period between January 1, 2010 and December 31, 2013, 78 patients were admitted at the Centre A PICU for septic shock requiring inotropes or vasopressors. Eight were excluded because of limitations of life support on admission. Of the remaining 70 patients, 21 (30 %) died during PICU stay, of whom 11 (15.7 %) were classified in RSS by the draft definition with a mortality rate of 100 % (sensitivity = 52.4 %, specificity = 100 %) and a median delay from shock onset to death of 19 (12–72) h. No comorbidity was found in 47.1 % of the patients while 22 patients (31.4 %) were immunocompromised.
Computed SSS calculated as follows, (cSSS) = 1.001VIS in mcg/kg min + 1.1arterial lactate in mmol/L + 18 (in the presence of myocardial dysfunction).
Bedside SSS (bSSS) based on 5 points with coefficient ranked and rounded to have a user-friendly score:
VIS > 200 mcg/kg min = 1 point
Arterial lactate >8 mmol/L or its increase of 1 mmol/L after 6 h of care = 1 point
Myocardial dysfunction as defined above = 3 points.
Patient characteristics in the development and validation populations
Development population (n = 70)
Validation population (n = 424)
Sex ratio M/F
25 (35.7 %)
117 (27.6 %)
Absence of comorbidity
33 (47.1 %)
220 (52 %)
22 (31.4 %)
69 (16.3 %)
Use of mechanical ventilation
54 (77.1 %)
369 (87 %)
14 (20 %)
118 (27.8 %)
14 (20 %)
48 (11.3 %)
Use of CRRT
11 (15.7 %)
47 (11.1 %)
Maximal blood lactates (mmol/L)
25 (35.7 %)
100 (23.6 %)
ScvO2 <70 %
25 (35.7 %)
141 (33.2 %)
Use of ECLS
2 (2.9 %)
2 (0.6 %)
21 (30.0 %)
49 (11.6 %)
Number of days in PICU
Delay from septic shock onset to death (days)
Predictive performance of septic shock scores and their best threshold values
Positive predictive value (%)
Negative predictive value (%)
Bedside septic shock score (bSSS)
Computed septic shock score (cSSS)
The septic shock scores (bedside and computed) are the first scores that aim at diagnosing refractory septic shock in paediatric patients admitted in PICU for septic shock. The scores were effective to identify the most severely ill patients. A bedside score ≥2 or computed score ≥3.5 defined RSS with a mortality higher than 55 %. Although statistically different, the diagnostic accuracy of both scores seems clinically equivalent with an absolute difference of 0.035 in the AUC. The clinical utility for each score is different. The bSSS is a bedside tool potentially useful for stratifying the severity and assessing the imminent risk of death, helping clinicians to implement rescue therapies. Its simplicity to use may counter-balance the small loss in accuracy. The cSSS is a potentially powerful and discriminating epidemiological tool and its calculation needs a computer or a smartphone application.
Defining a study population has implications on trials outcomes and may explain contradictory outcomes from historical large-scale randomized trials [12, 13, 28]. Recently, the sepsis-3 adult definition has refined the identification of septic shock patients . However, although focusing exclusively on adults, the main advance of the sepsis-3 definition is the stratification of patients into two categories: sepsis and septic shock. In adults, the SOFA score assesses organ failures after sepsis with good correlation to mortality . This adult score is not adapted for paediatric patients. The paediatric index of mortality (PIM2)  and the PELOD-2  score are paediatric scores that can be used to compare population of patients but are not specific for sepsis. In our study, the two scores were built with arterial lactate, vaso-inotrope score and septic cardiomyopathy and were associated with poor outcome. Each of those criteria was independently associated with mortality or the need for ECLS. Blood lactate is a widely used biomarker in septic shock patients. The highest value during the first 24 h of PICU was associated with mortality in paediatric septic shock patients . Dynamic values, such as lactime  or lactate clearance , are better predictors of mortality in adult patients, but they are not used routinely. Interestingly, the best predictor of survival to ECLS therapy in children with septic shock was a low pre-ECLS arterial lactate value .
Use of vasopressors or inotropes to treat fluid-unresponsive shock is still the first line of treatment [10, 11]. The VIS is a score used to assess the responsiveness and dependency to vasopressor in cardiac ICUs , with a VIS >200 being associated with mortality or need for ECLS in post-operative paediatric cardiac patients with low cardiac output syndrome . In our study, norepinephrine was used in 77.2 % of the patients and an inotrope (dobutamine, dopamine, epinephrine or milrinone) added in 80.2 %. The cut-off chosen in the definition by the two-round survey is higher than the VIS scores found in the literature in patients under ECLS therapy for other indications [22, 35, 36] or alternative vasopressors , but similar to calculated inotrope scores in adult patients needing ECLS support for septic shock .
The presence of a septic cardiomyopathy was independently associated with increased mortality or need for ECLS in the development population [OR = 18, (95 % CI: 3.4–95.4)]. Septic cardiomyopathy is the component carrying the highest weight in the septic shock scores. Although the pathogenesis of septic cardiomyopathy is multifactorial, it is known to be reversible [36, 37, 38]. LVEF has proven to be sufficient for the diagnosis of low cardiac output with a LVEF <40 % . Diastolic dysfunction and right ventricle dysfunction in sepsis are less studied but their role in septic cardiomyopathy seems to precede systolic left ventricular dysfunction . Cardiac arrest is the ultimate evolution of septic shock. In our study, in both cohorts, 59.7 % of the deceased patients had a resuscitated cardiac arrest in the course of the septic shock. Patients in cardiac arrest due to septic shock can benefit from ECLS therapy with an overall survival of 75 % in one series , compared to 32.7 % in our cohort. Improved assessment of septic cardiomyopathy before cardiac arrest occurs is mandatory. Regular evaluation of cardiac output and function can be based on invasive and non-invasive criteria. Non-invasive quantification relies mostly on continuous oesophageal or supra-sternal Doppler or cardiac ultrasound. Importantly, it is well recognized that children with septic shock without apparent need for inotropes can subsequently develop septic cardiomyopathy and low cardiac output . This haemodynamic pattern can be explained by the unmasking of septic cardiomyopathy by restoration of vasomotor tone in resuscitated shock after norepinephrine infusions, as well as over-enthusiastic fluid resuscitation .
This study has some limitations. First, the data collection was retrospective and thus is at risk of missing data and information bias. Second, data were collected during the whole septic shock care period with selection of a unique worst value for each item. Thus, the maximal values could be at varying time points in the clinical course. Calculation of the score was based on worst values for each criterion that may not be synchronous. This could have increased the estimated risk of death for patients in the validation population as well as overestimating the AUCROC. However, most patients died shortly after admission (median time to death = 1 day) and the highest values were mostly collected within the first 24 h after admission. Similarly, a delay to death of less than 24 h was also shown in more than 13,000 paediatric sepsis admissions in the North Thames, UK, region . Other scores used similar methodology during their creation, including SOFA and PELOD [8, 30]. Evaluating the kinetics of both scores as well as its composing criteria in the course of the disease is important and needs prospective evaluation. Third, there were significant baseline differences in the characteristics of the two populations (development and validation) and between centres included in the validation cohort. These may be explained by differences in patients’ recruitment and severity. Some of them were known prognostic factors in septic shock such as PIM2 scores and immunosuppression or were part of either the definition (blood lactates) or the outcome (mortality), outlining the patients’ recruitment and severity difference between both cohorts. The high mortality rate in the development cohort ensured the high power of this study to select the criteria associated with refractory septic shock. Meanwhile, the scores have an excellent discriminative value in the validation cohort. This reinforces the quality of the scores in various patients. Finally, this definition is based on a draft definition that has been developed using fictitious clinical cases and an international PICU physician opinion survey. The risk of opinion bias has been adjusted with the modification of this definition and the constitution of the septic shock scores tailored to an actual patient population. This definition is very coherent in real life, shown by the excellent discriminative power of both scores in the validation cohort. In regard of these limitations, a prospective study is warranted for refinement and external validation of this definition.
In conclusion, we have defined refractory septic shock in children as the association of high blood lactate with high vaso-inotrope doses associated with myocardial dysfunction. This definition is based on two septic shock scores showing excellent discriminative power in a multicentre validation population. The RSS Computed Score is a powerful and potentially useful tool to compare patients in future interventional randomized multicenter studies on septic shock. The RSS Bedside Score is easy to calculate and may assist in determining patients who would be suitable for inclusion in clinical trials of rescue therapies.
We acknowledge Ella Nkanagu from the ESPNIC secretariat for her assistance. We would like to thank Anthony Slater, Tara Williams, and Katie Moynihan, Children`s Health Queensland, for their support and help in data extraction.
Compliance with ethical standards
Conflicts of interest
Dr. Javouhey reports grants from CSL Behring, personal fees from Gambro-Baxter, outside the submitted work. Dr. Tissieres reports grants from Merieux Foundation, non-financial support from Chiesi Inc, outside the submitted work. All other authors declare that they have no conflict of interest.
- 1.Liu L, Johnson HL, Cousens S, Perin J, Scott S, Lawn JE, Rudan I, Campbell H, Cibulskis R, Li M, Mathers C, Black RE (2012) Child health epidemiology reference group of WHO and UNICEF global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet 379:2151–2161CrossRefPubMedGoogle Scholar
- 2.Schlapbach LJ, Straney L, Alexander J, MacLaren G, Festa M, Schibler A, Slater A, ANZICS Paediatric Study Group (2015) Mortality related to invasive infections, sepsis, and septic shock in critically ill children in Australia and New Zealand, 2002–13: a multicentre retrospective cohort study. Lancet Infect Dis 15:46–54CrossRefPubMedGoogle Scholar
- 7.Singer M, Deutschman CS, Seymour C, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, Hotchkiss RS, Levy MM, Marshall JC, Martin GS, Opal SM, Rubenfeld GD, van der Poll T, Vincent JL, Angus DC (2016) The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 315:801–810CrossRefPubMedPubMedCentralGoogle Scholar
- 8.Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, Reinhart CK, Suter PM, Thijs LG (1996) The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 22:707–710CrossRefPubMedGoogle Scholar
- 10.Dellinger RP, Levy MM, Rhodes A, Annane D, Gerlach H, Opal SM, Sevransky JE, Sprung CL, Douglas IS, Jaeschke R, Osborn TM, Nunnally ME, Townsend SR, Reinhart K, Kleinpell RM, Angus DC, Deutschman CS, Machado FR, Rubenfeld GD, Webb S, Beale RJ, Vincent JL, Moreno R (2013) Surviving sepsis campaign guidelines committee including The pediatric subgroup surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Intensive Care Med 39:165–228Google Scholar
- 11.Brierley J, Carcillo JA, Choong K, Cornell T, Decaen A, Deymann A, Doctor A, Davis A, Duff J, Dugas MA, Duncan A, Evans B, Feldman J, Felmet K, Fisher G, Frankel L, Jeffries H, Greenwald B, Gutierrez J, Hall M, Han YY, Hanson J, Hazelzet J, Hernan L, Kiff J, Kissoon N, Kon A, Irazuzta J, Lin J, Lorts A, Mariscalco M, Mehta R, Nadel S, Nguyen T, Nicholson C, Peters M, Okhuysen-Cawley R, Poulton T, Relves M, Rodriguez A, Rozenfeld R, Schnitzler E, Shanley T, Kache S, Skippen P, Torres A, von Dessauer B, Weingarten J, Yeh T, Zaritsky A, Stojadinovic B, Zimmerman J, Zuckerberg A (2009) Clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock: 2007 update from the American College of Critical Care Medicine. Crit Care Med 37:666–688CrossRefPubMedPubMedCentralGoogle Scholar
- 12.Ranieri VM, Thompson BT, Barie PS, Dhainaut JF, Douglas IS, Finfer S, Gårdlund B, Marshall JC, Rhodes A, Artigas A, Payen D, Tenhunen J, Al-Khalidi HR, Thompson V, Janes J, Macias WL, Vangerow B, Williams MD, PROWESS-SHOCK Study Group (2012) Drotrecogin alfa (activated) in adults with septic shock. N Engl J Med 366:2055–2064CrossRefPubMedGoogle Scholar
- 13.Nadel S, Goldstein B, Williams MD, Dalton H, Peters M, Macias WL, Abd-Allah SA, Levy H, Angle R, Wang D, Sundin DP, Giroir B, REsearching severe Sepsis and Organ dysfunction in children: a gLobal perspective (RESOLVE) study group (2007) Drotrecogin alfa (activated) in children with severe sepsis: a multicentre phase III randomised controlled trial. Lancet 369:836–843CrossRefPubMedGoogle Scholar
- 15.Peake SL, Bailey M, Bellomo R, Cameron PA, Cross A, Delaney A, Finfer S, Higgins A, Jones DA, Myburgh JA, Syres GA, Webb SA, Williams P, ARISE Investigators, for the Australian and New Zealand Intensive Care Society Clinical Trials Group (2009) Australasian resuscitation of sepsis evaluation (ARISE): a multi-centre, prospective, inception cohort study. Resuscitation 80:811–818CrossRefPubMedGoogle Scholar
- 23.French Intensive Care Society, International congress- Réanimation 2016 (2016) Ann Intensive Care 6 (Suppl1):S50Google Scholar
- 24.Goldstein B, Giroir B, Randolph A, International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics (2005) International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med 6:2–8CrossRefPubMedGoogle Scholar
- 26.De Luca D, Piastra M, Chidini G, Tissieres P, Calderini E, Essouri S, Medina Villanueva A, Vivanco Allende A, Pons-Odena M, Perez-Baena L, Hermon M, Tridente A, Conti G, Antonelli M, Kneyber M, Respiratory Section of the European Society for Pediatric Neonatal Intensive Care (ESPNIC) (2013) The use of the Berlin definition for acute respiratory distress syndrome during infancy and early childhood: multicenter evaluation and expert consensus. Intensive Care Med 39:2083–2091CrossRefPubMedGoogle Scholar
- 37.Torgersen C, Dünser MW, Wenzel V, Jochberger S, Mayr V, Schmittinger CA, Lorenz I, Schmid S, Westphal M, Grander W, Luckner G (2010) Comparing two different arginine vasopressin doses in advanced vasodilatory shock: a randomized, controlled, open-label trial. Intensive Care Med 36:57–65CrossRefPubMedGoogle Scholar
- 38.Takasu O, Gaut JP, Watanabe E, To K, Fagley RE, Sato B, Jarman S, Efimov IR, Janks DL, Srivastava A, Bhayani SB, Drewry A, Swanson PE, Hotchkiss RS (2013) Mechanisms of cardiac and renal dysfunction in patients dying of sepsis. Am J Respir Crit Care Med 187:509–517CrossRefPubMedPubMedCentralGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.