Introduction

Myocardial ischemia during coronary surgery may have severe consequences for the patient and may also pose a difficult diagnostic problem for the surgical team. At present there is no direct method to evaluate the effect of various therapeutic maneuvers on regional perfusion of the heart. The Doppler ultrasound technique used daily to measure graft flow velocity yields the velocity of the blood, but does not show the extent of the ischemic region in the myocardium [1]. Infrared (IR) imaging techniques were introduced to cardiac monitoring in the early 1970’s [2]. Intraoperative evaluation of graft function has been carried out since the 1990’s, and since then it has been the major application of IR imaging in this field [3, 4]. In order to take full advantage of the highly resolved spatial and temporal information available, successful attempts have been made towards extracting quantitative information from IR data. Using transient heat transfer models, myocardial perfusion has been estimated [5], and graft volume flow has been measured with good accuracy [6]. Furthermore, grading of regional ischemia was recently achieved with promising results [7].

Augmented reality (or mixed reality) denotes the fusion of digital images with the optical view of the real world, e.g. the patient undergoing surgery [8]. Using projection of color-coded IR images onto the imaged tissues in real time would give an intuitive representation of the tissue temperature. In this way, a detailed real-time temperature map of the heart would be available on the surface of the organ itself, provided that the geometry between the IR imaging and projection systems is synchronized. In this way a picture of myocardial perfusion may be seen by the surgeon in real time. Even if coronary artery surgery has become a standardized procedure, there are still questions about optimal myocardial protection and the metabolic events during the reperfusion phase. It is difficult to study these questions with the techniques available at present. The distribution of cardioplegia in the individual patient may vary with the pattern of stenotic arteries. Monitoring of adequate regional cooling of the heart could add valuable information when planning the operation. The method suggested in this paper may be useful in this setting. Such data may be combined with measurements of metabolic parameters to gain more knowledge about the state of the myocardium during cardiac surgery.

We have constructed a system consisting of an IR camera and a projector integrated in such a way that they have identical optical axes providing a simple solution to the geometrical correspondence problem. The aim of this study was to demonstrate, using augmented reality in an animal model, the feasibility of presenting IR tissue temperature images that reflect myocardial perfusion in the surgical field.

Methods

Experimental design

A series of experiments were carried out on five pigs with opened thorax. A coronary artery was occluded while an IR imaging–reprojection system projected real-time color-coded temperature maps onto the surface of the heart.

The study was approved by the Ethics Committee for Animal Experiments in Linköping (Dnr: 86-08/2008-12-04). The Swedish national guidelines and rules regarding animal research were followed. We used 5 Swedish Landrace pigs (Sus scrofa). Their mean (±SD) weight was 57.5 ± 7 (50–64) kg immediately before the experiment. The standardized protocol shown in Fig. 1 was followed.

Fig. 1
figure 1

The experimental protocol used. Low flow (grey) was approximately 50 % of the basal flow (1). 0 flow (black) denotes total occlusion. White color denotes reperfusion. Measurement points: 1 baseline, 2 at the end of low flow ischemia before reperfusion, 3 after 3 min of the first reperfusion, 4 at the end of the first 5 min total ischemia, 5 measurement after 3 min of reperfusion, 6 measurement in the middle (at 5 min) of a second total ischemia (10 min), 7 at the end of the second total ischemia of 10 min, 8 final measurement after 3 min of reperfusion

Full size image

Anesthesia

The pigs were heavily premedicated and deeply sedated with 75 mg tiletamine, 75 mg zolazepam (Zoletil®; Virbac, Carros Cedex, France) and 1.5 mg dexmedetomidinhydrochloride (Dexdomitor®, Orion Pharma, Espoo, Finland). Premedication was complemented, if necessary, with an induction dose of sodium thiopental. The trachea was then intubated under direct laryngoscopy. A Siemens Servo ventilator 900D was used to ventilate the pigs. Anesthesia was maintained with isoflurane. A thermal ceiling was used to maintain constant body temperature.

Monitoring

An arterial catheter (Arrow® 20GBD Swindon, SN35JH UK) was inserted into the right carotid artery. A Swan–Ganz catheter was inserted into the pulmonary artery (7.5 Fr CCO/SvO2/CEDV Edwards Life® Sciences LLC. Irvine, CA 92614-5686, USA). The cardiac output and mixed venous oxygen saturation were recorded continuously, and the myocardial temperature was measured with a needle thermometer inserted in the apical region of the left ventricle (Ellab, Copenhagen, Danmark).

The blood flow in the left anterior descending coronary artery (LAD) was measured directly using a 2 mm VingMed® flow measurement probe and Doppler ultrasound technique (Medistim® flow measuring device; Medistim® AS, Oslo, Norway).

Registration of data occurred at defined measuring points, whereas the augmented reality and the control IR temperature measurements were recorded continuously. The whole experiment was captured with a video camera.

Surgical preparation

We opened the thorax by median sternotomy, and after exposing the heart, an elastic vessel loop (yellow arrow) was placed around the middle of the left descending coronary artery (LAD) as shown in Fig. 2. The 2 mm ultrasound probe (black arrow) was inserted proximally around the LAD for flow measurements (Fig. 2). Ischemia was induced by occluding the LAD with vessel loops. Partial occlusion (50 % reduction of the LAD flow as measured with flowmetry) and total occlusion (0 ml/min flow) were verified by a Doppler ultrasound probe around the artery proximal to the occlusion. Effects of total occlusion and subsequent reperfusion were studied twice in each pig, making a total of 10 periods of occlusion in each area.

Fig. 2
figure 2

The surgical exposure of the heart. Yellow arrow elastic vessel loop placed around the middle of the left descending coronary artery. Black arrow ultrasound probe for flow measurements in the LAD. Blue arrow needle tissue thermometer placed intramuscularly in the left ventricle to register the myocardial tissue temperature

Full size image

IR imaging and projection system

The imaging and reprojection system consisted of an IR camera FLIR A325 (FLIR Systems AB, Danderyd, Sweden), a projector, Samsung SP-P400B (Samsung Electronics Co., Suwon City, Gyeonggi Do, Korea) and a module for co-aligning the optical axis of the IR camera and the projector. The IR camera was an IR non-cooled bolometer camera with 320 × 240 pixels. The thermal sensitivity was <60 mK. The resolution of the projector was 800 × 600 pixels, and the reprojected frame rate of the system 30 Hz. The IR camera and the projector were integrated in such a way that they had identical optical axes thereby solving the geometrical correspondence problem in a simple manner (cf. Fig. 3a).

Fig. 3
figure 3

Schematic set-up of the IR imaging–reprojection system

Full size image

The system was placed on a boom approximately 1 m above the animal. The boom was connected to a 2.1 m high stand on wheels allowing easy movement and transportation. A second IR camera FLIR SC660 (FLIR Systems AB, Danderyd, Sweden) was mounted on the boom in order to capture radiometric images usable for temperature measurement, directly. This was a non-cooled bolometer camera with 640 × 480 pixels. The thermal sensitivity was <45 mK, and the actual frame rate of the captured images was around 6 Hz. The software ThermaCAM™ Researcher Pro 2.9 from FLIR Systems was installed in a laptop PC and was used for IR image capturing and post processing. Emissivity was set at 0.98 and reflected apparent temperature was set at 20 °C. A separate visual DV camera CanonHV30 (Canon Inc., Ohta-ku, Tokyo, Japan) recorded sequences of the experiment in HD-format.

Statistics

All variables presented are given as mean ± SD. The data were compared with repeated measures ANOVA followed by Tukey’s HSD test. The limit for significance was set at p = 0.05. Temperature changes measured with different techniques were correlated with Pearson correlation. The technique of Bland and Altman was used to calculate 95 % limits of agreement.

Results

Quantitative thermographic images

Basic circulatory characteristics of the pigs during the experiment are given in Table 1, which shows that the mean temperature as measured by the intramyocardial needle thermometer decreased by 1.5–1.8° at each occlusion.

Table 1 Basic clinical variables presented as mean ± SD of the observations in 5 pigs
Full size table

The ischemia and reperfusion sequences induced were clearly visible as changes in surface temperature on the thermographic images. Heart surface temperatures were indicated by a color scheme and areas of measurement were easily defined, as illustrated in Fig. 4a–c.

Fig. 4
figure 4

a–c Thermographic images of the heart with measurement areas indicated. The heart is yellow–white with the base at the top of the picture and the apex in the middle. The black area around is the retractor. A thin rubber band is applied around the left anterior descending branch of the left coronary artery (LAD), and is seen as two dark lines from the top of the picture. AR01 is the measurement area located distally to the position of the subsequent occlusion of the LAD. AR02 is the control area, not affected by the arterial occlusion. a Is at baseline before occluding the LAD, b at occlusion and c at reperfusion. Note lowered temperature seen as darkening of the myocardial surface distal to the rubber band after occlusion, and subsequent increase in surface temperature after reperfusion

Full size image

An area of measurement on the surface distal to the occlusion was defined together with a control area not affected by the occlusion. Changes in temperature in the two areas of measurement during the procedure were recorded as shown in Fig. 5. Temperatures at baseline, after 5 min of total occlusion (immediately before reperfusion), and after 1 min of reperfusion were extracted from the graphic recordings.

Fig. 5
figure 5

Example of a temperature recording from the two measurement areas of the heart in Fig. 4 The red line is the temperature in the area distal to the occlusion (AR01) and the blue line is the temperature in the control area (AR02). The intervals on the x axis are 40 s and on the y axis 1 °C. Note the sharp rise in temperature of more than 5 °C after reperfusion at 5 min

Full size image

Temperature data from two occlusion–reperfusion sequences in each of the five pigs are shown in Fig. 6 and Table 2. There was a significant decrease in surface temperature over the experimental areas of measurement during occlusion (by 2.8 °C) and a subsequent return to slightly above baseline temperatures during reperfusion (an increase of 3.1 °C). In the control areas, however, the measured temperature remained constant. The correlation between temperature decrease measured by needle thermometry and by thermographic recordings was r = 0.86 (p < 0.001). There was a tendency for thermographic measurements to overestimate the decrease relative to the needle thermometer (mean difference 2.2 °C, 95 % limits of agreement 0.2 and 4.2 °C).

Fig. 6
figure 6

Temperature changes over the individual measurement areas during the occlusion–reperfusion procedures (10 procedures, two in each pig; separate symbols for each individual). The values over the experimental area during occlusion were significantly (p < 0.001) lower than those during baseline and reperfusion. No significant differences were found in the control areas

Full size image
Table 2 Temperatures and temperature changes in the surface areas studied
Full size table

Augmented reality

The augmented reality presentation with reprojection of the thermographic image onto the cardiac surface in real time was clear and visualized temperature changes caused by the change in coronary blood flow. The projection was dynamic and followed the movements of the heart and changes in temperature without perceivable delay. A photograph of the surgical field with the reprojected thermographic image on the heart during occlusion of the LAD is shown in Fig. 7 together with the thermographic image inserted. Good agreement was noted visually between the augmented reality image projected on the surface of the heart and the reference thermographic image (A video showing the projected images on the heart during occlusion of the LAD and subsequent reperfusion is included in the supplementary material).

Fig. 7
figure 7

The reprojected thermographic image on the operating field at occlusion as seen by the surgeon, and the original reference thermographic image as small insert on the right. The head of the pig is at the top and the heart is visible through the sternotomy between the retractor arms. Note the surgeon’s hand in the upper left corner stretching the rubber band occluding the LAD. The area with lower surface temperature is clearly visible in black near the apex of the heart, corresponding to the original thermographic image in the insert

Full size image

Discussion

The present study demonstrates that the combination of IR imaging and augmented reality (IR–AR) allows temperature changes due to ischemia to be assessed quantitatively and visualized in situ during occlusion of a coronary artery, and recovery of the myocardium after subsequent reperfusion. The decrease in temperature during coronary occlusion, as well as the subsequent increase during early reperfusion, was rapid. This is potentially useful in cardiac surgery situations where prompt action is required in cases of impaired myocardial circulation.

Thermographically measured temperature changes correlated closely to classical invasive measurements of myocardial temperature. The fact that a greater decrease was found with the IR–AR system than with an intramyocardial needle thermometer is not surprising, as the cooling effect of ischemia is expected to be more pronounced on the surface than in the deeper myocardium.

The main advantage of this novel method is that it enables an intuitive presentation of relevant information in the surgical field itself, in contrast to other IR imaging methods which require users to divert their attention from the heart to look at images on distant monitors. Surgeons are thus able to focus on the surgical field and see immediately any ischemia where- and whenever it actually occurs.

An alternative and possibly more sensitive method for detecting arterial obstruction during surgery is thermal angiography following the injection of a cold liquid [9]. In contrast to that technique, our method makes it possible to monitor the circulation continuously throughout the procedure, not just when liquid is injected. It is possible to complement the IR–AR technique with thermal angiography at selected time-points, should more sensitive readings be deemed necessary.

Compared with intraoperative angiography, the IR–AR technique avoids catheterization of the arteries and consequent endothelial damage as well as the use of ionizing radiation and potentially nephrotoxic contrast media. Since the equipment has no direct contact with the subject examined, there are no sterility problems when used during surgery. This is an advantage compared to Doppler ultrasound monitoring of the flow in a coronary artery or graft [10]. Furthermore, Doppler ultrasound does not provide any information about the size of the ischemic zone. Laser Doppler flowmetry, another method for monitoring myocardial blood flow [11], still requires the surgeon to divert his or her attention from the surgical field and cannot be used on the beating heart because of motion artifacts.

As general changes in temperature may occur during a long surgical procedure, the initially selected color scale may become less suitable. In that event, it is easy to change to a different predefined color table, without losing precise quantitative information. In that way, small changes in myocardial temperature may become more visible.

Dynamic changes in the extension of the penumbra zone is a potential monitoring mode when assessing therapies used in the salvage or prevention of ischemia in experimental or clinical cardiac surgery, and may eventually introduce new monitoring practices during cardiac anesthesia. This new technique enables fast and efficient monitoring of myocardial perfusion during cardiac surgery. It is conceivable that the concentrically arranged zones of different temperatures seen in Fig. 7—the “IR penumbra zones”—represent zones differing in severity of ischemia. However, to answer that question more studies relating temperature change to metabolic effects of ischemia are necessary, as well as studies relating the measured surface temperature to myocardial perfusion, and heat transfer within heart tissues and to the surroundings.

The most obvious limitation of this study is the small number of animals used. If results similar to this feasibility study are obtained in other series, the technique may well be considered for clinical trials. Another weakness is that no other method for ischemia detection was used in our study. However, these animal experiments do show that this technique responds rapidly to acute induction of regional ischemia. It is conceivable that a similar response may be reproduced in humans, but this has yet to be shown. The method may also be useful in analyzing regional perfusion after CABG and thereby be a useful complement to routine graft flow measurements. The experiment was designed to test the feasibility of the method and to give initial impressions about the applicability of this method in clinical practice. Pigs do not have coronary disease which is why the information gained from this experiment is useful but at the same time does not exactly represent the clinical situation and extrapolation to the same should be regarded with reservation. The fact that this study has investigated the LAD and not the other coronary arteries could be considered a limitation. The LAD was chosen because it is the largest coronary artery and the most accessible for long-term manipulation in this experimental model. The heart can be studied for a long period of time without changing its natural position. In humans other areas of the heart can be studied using modern surgical tools to hold the heart in position when working with the left circumflex and right coronary artery while concomitantly studying temperature changes in their dependent areas. The penumbra zone, which may represent not oxygenated any more but still vital tissue as can be seen after the reperfusion, could clearly be seen and defined throughout the experiments, but this was not quantitatively analyzed. It will be interesting to look at this area and how it responds to therapeutic interventions, in future clinical application studies. However, in this study the size of the ischemic tissue zone was not biochemically or histologically validated, but was assumed from the decrease in temperature in the LAD area as confirmed by high precision IR imaging.

In conclusion, the IR–AR technique described seems to be a promising tool for experimental studies on cardiac ischemia. If future studies are successful, this technique may also find a place in clinical practice.