1 Introduction

The thermodilution method performed at the time of right heart catheterisation is widely used for the measurement of cardiac output (CO) (Fig. 1). The concept of using the dilution of an injected indicator substance was first described by Stewart in 1897 [1], the technique of injection was refined by Henriques [2] and analysis of the resultant indicator dilution curves refined by Hamilton [3]. The accumulation of indicator dye in the circulation limited its utility leading to the development of the thermodilution method. The thermodilution method was first described by Fegler in 1954 [4], validated as being reliable in 1957 [5] and its first use described in humans by Ganz et al. in 1971 [6]. It involves the injection of 10 mL of cold saline solution into the right atrium or vena cava. The injectate temperature is measured with a proximal thermistor while a distal thermistor positioned in the pulmonary artery semi-continuously measures the change in blood temperature. The following formula is then used to calculate CO:

$${\text{CO }} = \frac{{\text{V}}_{{\text{I}}} \times {\text{ }}\left( {{\text{T}}_{{\text{B}}} {-}{\text{ T}}_{{\text{I}}} } \right){\text{ }} \times {\text{ S}}_{{\text{I}}} \times {\text{ C}}_{{\text{I}}} \times {{60}}}{{\text{S}}_{{\text{B}}} \times {\text{ C}}_{{\text{B}}} \int _{0} ^{\infty } \Delta {\text{T}}_{{\text{B}}} \left( {\text{t}} \right){\text{dt}}}$$

where VI—volume of injectate, TB—temperature of blood, TI—temperature of injectate, SI—specific gravity of injectate, CI—specific heat of injectate, SB—specific gravity of blood, CB—specific heat of blood.

Fig. 1
figure 1

A typical thermodilution curve of a patient with low cardiac output. The x axis is time. The y axis is the temperature of the thermistor in the pulmonary artery. The injection commences at the left border of the graph. The area under the curve is measured by the thermodilution computer. Once the blood temperature passing the distal thermistor has warmed to below 30% of the peak (coolest) temperature (arrow 1) the computer extrapolates the remainder of the curve to avoid measurement of re-circulating cool blood which would create an artefactually long tail (seen at arrow 2)

Full size image

The thermodilution calculation of CO presumes there is no significant recirculation of injectate, that the injectate is delivered smoothly and that the process of injecting does not cause perturbations in blood flow through the heart [5, 6]. The derived CO has been shown to correlate very well with both the Fick and dye injection methods [6], however anecdotally there can be considerable intra-operator and inter-operator variation in CO on a single patient. The original validation studies never specified whether single or multiple operators were delivering the injectate in their studies, leaving open the possibility of inter-operator variability when the technique is applied in clinical practice.

In light of the original assumptions in confirming the calculation of CO; namely that the injectate is delivered smoothly and the injecting itself does not cause perturbations of blood flow through the heart, it is likely that inter-operator variation will arise as a result of operator strength or technique. It is probable that different operator characteristics will both under and over-estimate the CO. Therefore, this study aimed to characterise the inter-operator variation of CO measurement using the thermodilution method in a previously described and validated mechanical representation of the heart and circulatory system [7]. We also aimed to determine if identifiable operator characteristics such as speciality, experience, grip strength, gender, age, height or weight are responsible for any variance in CO measurement accuracy.

2 Methods

2.1 Participants

Study participants consisted of volunteer physicians experienced in performing right heart catheterisation. An incentive in the form of a small chocolate was provided to encourage participation. The physicians included cardiologists, intensive care physicians and advanced trainees from each of these specialties. The gender, age (years), height (cm) and weight (kg) of each participant was recorded as these parameters might provide surrogates for strength and thus force of injection for thermodilution.

Grip strength (kg) for each participant’s dominant hand was directly measured using a baseline hydraulic hand dynamometer (Fabrication Enterprises, White Plains, New York, 10602, USA). A standardised technique was used [8]. The handle was adjusted to that of a comfortable grip for the participant, forearm was positioned such that the participant’s shoulder was adducted and neutrally rotated, elbow was flexed at 90°, forearm in neutral position, wrist between 0° and 30° dorsiflexion and between 0° and 15° ulnar deviation. The average of 3 measurements was recorded.

The approximate number of previous right heart catheterisations which the operator has performed was grouped as: 1–25 (RHC Group 1), 26–100 (RHC Group 2), or >100 (RHC Group 3).

2.2 Mock circulation loop

A mechanical representation of the heart and circulatory system, known as a mock circulation loop (MCL) (Fig. 2), was used for this study [7, 9]. Atrial and ventricular chambers were represented by clear, vertical polyvinyl chloride (PVC) pipes with tee sections connecting the inflow, outflow and heart chambers. Arterial and venous sections were simulated through horizontally placed PVC pipes, connected to form systemic and pulmonary circulations. Forward flow was generated through simulated ventricular systole and controlled pneumatically through a series of compressed air regulators (ITV2030-012BS5, SMC Pneumatics, Tokyo, Japan) and solenoid valves (VT325-035DLS, SMC Pneumatics) to provide passively filled heart chambers and variable contractility, heart rate and systolic time. A Starling response was implemented in both left and right ventricles which actively controlled ventricular contractility based on ventricular preload [10]. Mechanical check valves were used to simulate the mitral, aortic, tricuspid and pulmonary valves. Four independent, air-filled Windkessel chambers were employed to simulate systemic and pulmonary arterial and venous compliance. Variable resistance valves (VMP025.03X.71, AKO, Alb. Klein Ohio LLC, USA) allowed easy manipulation of systemic and pulmonary vascular resistance. The working fluid throughout this study was a water/glycerol mixture (50/50% by mass), heated via a heat exchanger (BT-ST55-B-1, Brazetek, Brooklyn, NY USA) and heater (Heater Cooler Unit HCU 20, MAQUET, Wayne, NJ USA) to 37 °C for suitable thermodilution catheter performance and fluid viscosity (approximately 3.1 mPa.s) similar to human blood.

Fig. 2
figure 2

Schematic of the MCL setup for thermodilution evaluation. LA left atrium, MV mitral valve, LV left ventricle, AoV aortic valve, AoC aortic compliance chamber, SQ systemic flow meter, SVR systemic vascular resistance valve, HEAT heat exchanger, SVC systemic venous compliance chamber, RA right atrium, TV tricuspid valve, RV right ventricle, PV pulmonary valve, PAC pulmonary arterial compliance chamber, PAcath pulmonary artery thermodilution catheter, PQ pulmonary flow meter, PVR pulmonary vascular resistance valve, PVC pulmonary venous compliance chamber. Arrows indicate direction of flow

Full size image

Haemodynamics were captured at 100 Hz using a dSPACE acquisition system (DS1103, dSPACE, Wixom, MI, USA). Systemic and pulmonary flow rates were recorded using magnetic flow meters (IFC010, KROHNE, Duisburg, Germany) while circulatory pressures were recorded using silicone-based transducers (PX181B-015C5V, Omega Engineering, Stamford, CT, USA).

2.3 Thermodilution evaluation

A 7 French Swan-Ganz thermodilution catheter (Edwards Lifesciences LLC, Irvine CA 92614 − 5686 USA) was deployed in the MCL as shown in Fig. 2. Catheter placement was achieved prior to the study commencement, with the same placement for all participants used to eliminate catheter placement as a variable in our study. The nominal CO of the simulated circuit was specified and confirmed using the flow sensors in the circuit. The specific heat and specific gravity of the solution in the simulated circuit was documented to enable application of the formula for CO as described in the background of this manuscript.

CO was randomly altered between 3.0 and 7.0 L/min through manipulation of heart rate (50–90 beats per min), pulmonary vascular resistance (50–300 dyne.s.cm−5), systemic vascular resistance (1000–1500 dyne.s.cm−5) and blood volume. Each participant performed thermodilution CO measurements at four different and random levels of CO using an injection of 10 mL cold saline maintained at a constant temperature using an ice bath. The injectate temperature was continuously measured throughout the injection by a thermistor positioned immediately distal to the syringe. Participants were blinded as to the CO and all other MCL parameters when they performed the measurements. Each participant performed sufficient injections of chilled normal saline solution to satisfy themselves of the current CO of the MCL. The recommended minimum requirement to determine CO was 3 measurements with <10% variability.

This research protocol was approved by the Ethics Committee of The Prince Charles Hospital. All data was recorded in a non-re-identifiable, de-identified manner.

2.4 Statistical analysis

The original human trials of thermodilution by Ganz demonstrated a standard deviation (SD) of only ±3.1% for the difference between the then gold standard of dye dilution technique and thermodilution [4]. Based on the original description of the thermodilution technique by Fegler [4] it has been accepted practice to regard a difference of up to 10% in CO between measurements as an acceptable degree of error. Each participant’s CO measurement was analysed in terms of how much it differed from the known CO of the MCL.

As the data constituted a repeated measures time series, they were analysed as a mixed effect, longitudinal and correlated dataset. The panel variable was the individual participant and the time variable was the number of the run (1–4). Demographic data were summarised as mean (SD or SE), median (IQR) or proportion as appropriate for the individual data.

The outcome (dependent) variable was the difference between the measured CO and the actual CO, structured such that an increase in the difference implied overestimation of the actual CO by the operator. The predictor (independent) variables were age, gender, body-mass index (BMI), grip strength and right heart catheter experience. Given a level of significance of 0.05 and a power of 0.80, a sample size of 8 independent participants was required to detect a 0.175 L min deviation from a paired mean difference between actual and measured CO of zero L/min. This equated to an absolute error of 3.5% using a standard deviation of 3.1% (0.155 L/min) for a CO of 5.0 L/min.

Because we anticipated performing multiple regression, the total sample size required was drawn from the work of Green [11] As before, using α = 0.05, β = 0.20 and assuming an estimated three predictors, a large effect size required a sample size of 31 whilst a medium effect size required a sample size of 73. These estimates roughly agree with the standard 20:1 rule of thumb which would give a sample size of 60. Eventually 15 independent participants were recruited, each performing 4 runs to give a total of 60 repeated measures.

Linear, mixed effects, multivariable regression models were built and tested and inter-operator agreement, stratified by level of right heart catheterisation experience, was assessed using the kappa statistic.

To assess bias and overall level of agreement, Tukey (Bland Altman) mean-difference plots were drawn for each level of right heart catheter experience as well as for the entire group. The difference (bias) was represented both as the absolute value (L/min) as well as the percentage error (difference/mean). The correlations between the differences and their respective means were also calculated. Results were displayed in both tabular and graphical form.

Binary comparisons were made using either a paired t-test for normally distributed data or the Wilcoxon signed-rank test for non-normal data. Data normality was checked using the Shapiro–Wilk test. STATA™ (ver 12.0) was used for all analyses. The level of significance was set at P < 0.05 throughout.

3 Results

All participant details are outlined in Table 1. In summary, there were a total of 15 participants in this study with a wide range of specialties including 6 consultant cardiologists, 5 advanced trainees in cardiology, and 4 consultants in intensive care. The majority (9 of 15) of participants had performed 26–100 previous right heart catheterisations, whilst 4 participants had previously performed more than 100 right heart catheterisations. Participants generally achieved a suitable CO measurement (i.e., 3 measurements within 10% variability) within three or four saline injections, however the number of injections required was not recorded in our study.

Table 1 Participant characteristics
Full size table

Compared to the CO recorded in the MCL, participants tended to overestimate CO using the thermodilution technique with a mean difference of +0.78 ± 0.71 L/min (Figs. 3a, 4). The overall r2 value for actual vs measured CO was 0.85. The difference between MCL and thermodilution derived CO declined significantly with increasing right heart catheterisation experience (P < 0.001) (Fig. 3b; Table 2).

Fig. 3
figure 3

Measured cardiac output using the right heart catheter (RHCQ) versus the mock circulation loop (MCLQ) for a all users and b users grouped by operator experience. A line of identity (solid black line) is included for reference

Full size image
Fig. 4
figure 4

Tukey (Bland Altman) plot for all data. On average, the measured cardiac output was 0.78 L/min (95% CI −0.61, +2.17 L/min) greater than the actual MCL cardiac output. The solid line represents the mean bias and dashed lines represent ±2 standard deviations

Full size image
Table 2 Bias, error and correlation results for each right heart catheterisation group [mean (SE)]
Full size table

Overall, the kappa value (κ) was low at + 0.17. However, when participants were grouped according to right heart catheterisation experience there was a strong trend for the inter-operator agreement to increase with increasing experience (κ = +0.01, +0.16 and +0.37 respectively for participants who had performed 1–25, 26–100 and more than 100 procedures).

Both BMI (P < 0.001) and grip strength (P = 0.033) were also significant model predictors with a tendency toward increasing measurement accuracy as BMI increased and grip strength decreased. Age, gender and clinical level of seniority were all non-significant predictors in the multivariable model. The final regression model (Table 3) was:

$${{\Delta }_{calculated-actual}}=4.98-0.87\left( RHC~experience \right)-0.10\left( BMI \right)+0.02(Grip~strength)$$
Table 3 Multivariable model development
Full size table

4 Discussion

The determination of CO at the time of right heart catheterisation is key to deriving a variety of measurements which are reflective of a subject’s haemodynamics. This study demonstrates that participants tended to overestimate CO, while operator experience is the only user characteristic evaluated in this study to be directly related to CO measurement accuracy.

The MCL was chosen rather than using the circulation of a living organism because it enabled the comparison of our participants’ CO measurements with an absolutely known CO. The MCL has been demonstrated to be an accurate simulation of a real biological circulatory system [7, 9, 10], and has been previously used to evaluate alternative CO measurement techniques [12]. In any biological circulatory system, we could at best compare the CO measurements of our participants with the CO calculated by some other method, leaving the comparison open to inaccuracies of measurement technique and changes over the time taken to perform the studies. Using the MCL enabled a standardised and known comparator and allowed the testing to occur over many hours without concern for the welfare of a living creature. The disadvantage of the MCL is that the rigid vascular chambers tended to generate mixing artefact (between the circulating fluid and the injectate) which may have created errors in readings from the distal thermistor. Operators with a more forceful injection technique had to modify their technique to prevent this turbulence which may have reduced the inter-operator variability seen. Injection duration was not recorded during this study, and should be evaluated during future research. The pulmonary artery of the MCL is not anatomically correct and does not contain a bifurcation, which justifies our decision to maintain catheter placement for all users. Clinician-defined catheter placement could be further explored, as this may influence CO measurement accuracy.

There is no true “gold standard” method for determining CO. In clinical practice two methods, the Fick method and the thermodilution method, have become the de-facto “gold standard” with each having limitations in different specific circumstances. For the thermodilution technique, respiratory variation in right ventricular temperature [13] and tricuspid regurgitation have been proposed as potential sources of error. The inaccuracy of thermodilution in tricuspid regurgitation is debatable with ex-vivo studies suggesting a small but predictable under-estimate of CO [14]. Some small in-vivo studies are consistent with this finding [15] but most in-vivo studies would suggest this discrepancy is not significant [16–18]. Low CO states may be more affected by respiratory variation because of the longer duration of the thermodilution curve [13]. The measurement of the area under the thermodilution curve assumes a constant pulmonary artery blood temperature, in a free breathing (or ventilated) subject however fluctuations are generated by respiratory pressure induced flow changes in the thorax and abdomen [19]. Although respiration was not simulated in our MCL, our results demonstrated no change in CO measurement accuracy between low, medium and high CO states. When using modern CO computers, signal processing strategies should eliminate most of the “noise” by respiratory variation [19, 20] except under extreme circumstances such as weaning from cardiopulmonary bypass or high volume intravenous infusions (where rapid inflow of cooled blood from the bypass circuit or external infusions reduces pulmonary artery temperatures rapidly) [21]. However a study comparing an automated injector to manual injections (unreported number of human operators) demonstrated that the automated injector produced more accurate results than the human injectors [22]. The automated injector measured CO at various phases of the respiratory cycle (synchronised to a mechanical ventilator). The humans in that study were instructed to perform their injections at end exhalation. It was hypothesised that injections across the respiratory cycle when averaged increased the accuracy of measurements. As there was no respiratory cycle in the MCL, this should not have had any impact on the results seen in our study. Meanwhile, the spread of participant experience was unfortunately quite narrow with most (9 of 15) participants having previously performed 26–100 previous right heart catheters, which is a limitation of our study.

We observed that participants tended to overestimate the CO, a phenomenon that was noted in the very first publication of Fegler [4], but not commented upon in the seminal in-human study of Ganz [6]. Many subsequent studies have assessed the accuracy of the thermodilution method under a variety of different clinical scenarios, comparing it to alternative techniques for measuring CO [5, 14–16, 18, 23–25]. Depending on the clinical circumstances, the thermodilution technique could both over and under estimate CO [5, 14–16, 18, 23–25]. It is possible that the overestimation seen in our study was an artefact of the blood simulant solution used upon the thermodilution calculations, or the placement of the catheter in the MCL. Therefore, future inter-operator validation studies should be completed in-vivo with properly-placed catheters, and compared with the CO recorded with calibrated perivascular flow sensors.

We showed that differing operator techniques (as taught by different specialties), age and gender did not appear to generate artefact resulting in incorrect measurements. BMI and grip strength were statistically, but likely not clinically, significant predictors as their overall effect was trivial. It is unclear from this study why operator experience impacts upon the accuracy of the technique. We hypothesise that a less consistent injection technique, due to inexperience, results in more variation between the calculated CO of individual injections. We did not collect data on individual injections, only the participants’ final conclusion as to the CO and so are unable to explore this hypothesis further. However, our conclusion that operator experience predicted the accuracy of determination of CO using the thermodilution technique is significant to clinical practice because it mandates the need for adequate exposure to, and training in, the thermodilution technique for clinicians who perform right heart catheterisation.

5 Conclusion

This is the first study to specifically assess inter-operator variability in CO determination using the thermodilution method. We confirmed that although participants tended to overestimate CO, the thermodilution method generally is an accurate method for determining CO. We identified that operator experience is the major user characteristic evaluated in this study to be directly related to CO measurement accuracy, and that additional operator experience resulted in improved CO measurement accuracy.