Introduction

Congenital heart disease is the most common cause of death in infants, stimulating increased interest in developing appropriate diagnostic tools for the reliable detection of congenital heart malformations. Two-dimensional echocardiography (2DE) has been the main noninvasive technique used in the diagnosis of congenital heart disease. However, given that the heart has a complex anatomy and is in constant motion, conventional 2DE can provide only partial information about the spatial and temporal relationships between cardiac structures. In addition, most cardiologists are familiar with 2D imaging and mentally reconstruct stereoscopic images of the heart, relying primarily on experience and medical knowledge. Three-dimensional echocardiography (3DE) may provide valuable information that is not obtainable with conventional 2DE and that can enhance our understanding of the complex anatomy of cardiac structures [1–3].

Virtual endoscopy (VE) can simulate the tracks of a “conventional” endoscope using 3D image datasets, such as those from computed tomography (CT), magnetic resonance (MR), or echo scans [4]. Preliminary studies have suggested that VE can provide accurate and reproducible visualizations [5]. Virtual visualizations of the cardiovascular structures have been considered to be more realistic and useful, compared with standard endoscopic views [6]. However, almost all studies have used volumetric CT and MR data [7–11]; few have dealt with echocardiography data [12–14]. The rapidly increasing power of computer hardware and software has led to the use of virtual reality (VR) as a technique for medical visualization in the past decade. Technologically advanced 3D reconstruction software has multiple advantages over traditional imaging modalities for the evaluation of congenital heart malformations. With the ongoing revolution in the computer industry, static and dynamic VE visualization of volumetric three-dimensional echocardiography datasets is becoming possible.

This paper introduces a novel approach for the visualization of dynamic 3DE data. In close cooperation with the electronic engineering department of Fudan University, we have developed a VE system called the “three-dimensional echocardiographic intracardiac endoscopic simulation system” (3DE IESS). We have previously applied VR techniques for clinical research purposes [13]. Our approach is based on a tool for the interactive VR visualization of 3DE datasets in the form of 3D heart models displayed on VR equipment. VR can assist with the interpretation of 3D data for heart volume and facilitates the development of 3D heart models [15]. In this pilot study, we have evaluated the ability of 3DE IESS to reconstruct the inner-cardiac structures. The purpose of this study was to assess whether VE is feasible for 3DE and to determine if 3DE imaging in a virtual environment can be used effectively as a clinical diagnostic tool.

Methods

Study population

This protocol was approved by Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine (Shanghai 200092, China), and have therefore been performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments.

Datasets from 13 healthy subjects and 35 patients with congenital heart disease were collected. There were 30 males and 18 females, with a median age of 3.8 years (range: 0.2–8.5 years).

Data acquisition and processing

After conventional transthoracic echocardiography, data acquisition for 3DE was performed. The 3DE image datasets were generated using a 2- to 4-MHz matrix array transducer in a Philips Sonos 7500 echocardiographic system (Philips Medical Systems, Andover, MA, USA) equipped with a 3D data-acquisition software package. After acquisition, the echocardiographic 3D images were stored on CD-ROM and transferred to a computer for post-processing and data analysis.

The images were evaluated with respect to subjective image quality based on possible anatomical differentiation of heart tissues. All available images were graded as below: insufficient (poor quality with step artifacts and stripes throughout the image limiting evaluation of the heart), sufficient (sufficient quality with step artifacts and stripes which have minor implication on the evaluation of the heart), good (good quality with minor motion artifacts not hampering the evaluation of the heart) or optimal (excellent quality without motion artifacts).

VE reconstructions

The cardiac structures were reconstructed after pre-processing. The intracardiac anatomy was assessed using 3DE IESS, which provides a flexible framework for intuitive visualization and interactive evaluation of large medical datasets. The interior of the heart was examined using VE. To secure relevant examinations of all details related to intracardiac morphology, the heart model can be reproducibly visualized, and the virtual eye can be installed at any point inside the heart.

The software for our echocardiographic analysis system was written in Visual C++ 6.0 and employed graphics and visualization utilities available in the VTK (Visualization ToolKit, Kitware, Inc.). Color and lighting were used to enhance realism.

Visualization in a virtual reality environment

In the VR environment, the observer is within the heart, close to anatomical structures in regions of interest, and can move slowly around objects of interest. Slightly angling or rotating the virtual eye in various projections can produce specific and unique imaging. By controlling the view with the mouse, the operator can move the object of interest. Primarily head-on views of the intracardiac anatomy from the atrium and ventricle were reconstructed to determine the precise anatomy of the heart.

Three experienced pediatric cardiologists blinded to the patients’ diagnoses separately reviewed the 40 2DE datasets and 40 corresponding VE datasets. Data sheets were used to record the results of their findings. Clinical diagnoses of healthy hearts were conducted using 2D Doppler transthoracic echocardiography. Surgical findings served as the gold standard for the clinical diagnoses of congenital heart malformations. The findings were relevant to a group of patients who had undergone surgery for congenital heart disease, rather than patients who had not had a surgical procedure. The cardiologists judged whether abnormal intracardiac anatomy was present on a five-point scale (1 = definitely absent; 2 = probably absent; 3 = cannot be determined; 4 = probably present; and 5 = definitely present). Three observers independently examined each of the 2D and VE datasets to provide 162 diagnostic evaluations of each method.

Statistical analysis

The results of the area under the receiver operating characteristic (ROC) curve for VE echocardiography and 2DE were analyzed using SPSS 13.0 for Windows (SPSS, Inc. Chicago, IL). Agreement for inter-observer variability was analyzed by computing the kappa values.

Results

Of the 48 patients enrolled, the three-dimensional reconstruction analyzed by 3DE IESS was sufficient in 40 patients with good or optimal image quality, thus the feasibility was 83.3%. It was not feasible in 8 patients (16.7%) due to poor image quality failure in reconstruction. Reconstruction analyzed by 3DE IESS was sufficient for all 40 patients. An overview of the selected patients is shown in Table 1.

Table 1 Summary of study patient diagnoses
Full size table

The mean time for VE reconstruction was approximately 5 min. The time needed for laborious manual data analysis ranged from 30 to 60 min.

Images of the atrial septum were captured from either the right or left atrial sides. VE was oriented towards the atrial septal surface, and a left atrial head-on view could be observed. Similarly, the interventricular septum could be visualized head-on from either the left or right ventricles (Figs. 1, 2, 3, 4).

Fig. 1
figure 1

Virtual endoscopic view of ASD from the right atrium by 3DE IESS to image the relationships among the ASD, IVC, SVC, and CS. ASD atrial septal defect, IVC inferior vena cava, SVC superior vena cava, CS coronary sinus

Full size image
Fig. 2
figure 2

Virtual endoscopic view of tetralogy of Fallot from the ascending aorta toward the interventricular septum by 3DE IESS. LV left ventricle, RV, right ventricle; AAo ascending aorta, IVS interventricular septum, VSD ventricular septal defect; MV mitral valve, TV tricuspid valve

Full size image
Fig. 3
figure 3

Virtual endoscopic view of double outlet right ventricle viewed above the interventricular septum by 3DE IESS. LV left ventricle, RV right ventricle, IVS interventricular septum, VSD ventricular septal defect; Ao aorta, PA pulmonary artery

Full size image
Fig. 4
figure 4

Virtual endoscopic view of a normal heart from the pulmonary artery by 3DE IESS, used to image the relationship between LPA and RPA. LPA left pulmonary artery, RPA right pulmonary artery

Full size image

VE displayed dynamic, continuous images for evaluation with a realistic perspective, giving the impression of a 3D image (Fig. 5). In the dynamic mode display, the opening and closing of the cardiac valves can be easily visualized. Moreover, valve motion in any desired phase of the cardiac cycle can also be observed.

Fig. 5
figure 5

Multiphase reconstructed images of intracardiac structures (viewed from the left ventricle) in the cardiac cycle. AML anterior mitral leaflet, PML posterior mitral leaflet, Ao aorta, IVS interventricular septum

Full size image

Compared with clinical diagnosis, the accuracy of VE was 98.7% for ASD, 92.4% for VSD, 92.6% for TOF, and 94% for DORV, respectively. Diagnostic accuracy of VE was significantly higher than that of 2DE for TOF and DORV except for ASD and VSD (Table 2).

Table 2 Incremental diagnostic value of VE over 2DE
Full size table

The results of clinical evaluations of the 132 inner-cardiac abnormalities and 30 normal cases by three observers for the 2DE images are shown in Table 3. Among the 132 congenital heart abnormalities, only 89 (67.4%) studies were identified as “definitely present”, 12 (9.1%) as “probably present”, 11 (8.3%) as “cannot determine” and 4 (3.0%) as “probably absent”. However, 16 (12.1%) images were mistakenly identified as “definitely absent” of abnormalities. Twenty-six (86.7%) of the 30 normal cases were identified as “definitely absent” of congenital heart defects. Other ratings demonstrated uncertainty or confusion about the true state of the cardiac structures to various degree. Table 4 illustrates the results of the same observers for the 3D VE images. Better performance was achieved. VE demonstrated clinically significant abnormalities in 92 (69.7%) of the 132 inner-cardiac abnormalities. Only 3 (2.3%) of the 132 abnormal and 1 (3.3%) of the 30 normal cases were mistakenly identified as “definitely absent” or “definitely present”, respectively.

Table 3 Results of clinical evaluations for data presented as 2D echocardiographic images
Full size table
Table 4 Results of clinical evaluations for data presented as 3D echocardiographic virtual endoscopy images
Full size table

ROC curves for 2DE and VE were constructed, and the area under the curve for each echocardiographic image was compared. As shown in Fig. 6, the ROC curve for VE (solid line) was closer to the optimal performance point (upper left corner) than the ROC curve for 2DE (dashed line). The area under the ROC curve was 0.96 for VE and 0.93 for 2DE.

Fig. 6
figure 6

Receiver operating characteristic (ROC) curves for VE (solid) and 2DE (dashed)

Full size image

Inter-observer variability

Strong inter-observer agreement was observed in this study. Kappa values (range: 0.73–0.79) for VE and 2DE indicated substantial agreement (Table 5).

Table 5 Observer agreement for diagnosis based on echocardiography
Full size table

Discussion

Although traditional 2DE may provide important information about the spatial and temporal relationships of cardiac anatomy during the cardiac cycle, specific details may be lacking. The complexity of cardiac anatomy requires the 3D spatial orientation of images to achieve a better understanding of the heart structures. In addition, conventional echocardiography requires a mental conceptualization process by the cardiologist to reconstruct a complex 3D anatomy image of the heart based on an interpretation of multiple 2D slices.

3D echocardiography may provide more reliable information than 2D methods and may improve the comprehension of anatomical relationships, especially for complex congenital heart diseases [16]. Real-time 3DE (RT-3DE) has proven to be valuable for better evaluation of morphologic abnormalities and comprehension of the complex cardiac anatomy involved in congenital heart disease [17–19]. However, RT-3DE volume-rendered images often pose difficulties in understanding both the origin and orientation of reconstructed tomographic views. Viewing heart images is analogous to outdoor observation: one cannot observe the inside structures without removing, at least partially, the walls. In order to display the intracardiac anatomy, the heart must be cut open using a cutting plane, and the images beyond this plane must be reconstructed as if the heart were cut open via surgery.

While the 3DE reconstructions can only be presented on a flat 2D screen, VR allows for visualization of the actual 3D anatomy of the heart. Lee [20] proposed the idea of applying virtual ultrasound simulations in 3DE in 1996 and reported a study on a unique computer-aided interactive 3D heart model with virtual ultrasound simulation for fetal ultrasound training. The Fetal Imaging Workstation Demonstration created the illusion of a scanning plane coursing through a 3D ultrasound heart model via pre-rendered frame animations, special effects, and solid constructive geometry. Physicians may be able to visualize, scan, navigate around, manipulate, and measure the volume of a reconstructed 3D virtual heart through virtual ultrasound simulation. VR can effectively reconstruct a virtual heart model and provide spatial cardiac information [15]. Some congenital heart diseases, such as ventricular septal defects and atrial septal defects (VSDs and ASDs, respectively), have been diagnosed, analyzed, or assessed within a virtual reality heart using the software module TiDAS (Toolkit for the Interactive Design of Animation and Simulation) [12]. In 2005, Bosch et al. [14] described dynamic 3D echocardiography using virtual reality and showed that dynamic holographic imaging of 3DE data was feasible.

The results show that reconstruction from VE can be achieved with high accuracy and low artificial variability in patients with good even optimal echocardiographic image quality. In this study, we demonstrated that 3DE can serve as a clinical tool for VE, whereas most papers published to date have exclusively used CT and MR. For these purposes, we recently introduced a novel method called the “three-dimensional echocardiographic intracardiac endoscopic simulation system” or 3DE IESS. Employing virtual reality to capture realistic views of the cardiac anatomy, the 3DE IESS is an innovative approach for noninvasive imaging of the intracardiac structures. VE allows cardiologists and cardiac surgeons to examine the model as if they were actually entering into the 3D anatomy of the heart themselves. In the stereoscopic reconstructed image, the interior of the heart can be clearly visualized at different levels, similar to a real intracardiac endoscopic image. To secure relevant examination of all details related to cardiac morphology, the viewpoint can be rescaled and the “virtual eye” can be repositioned to any point inside the heart. In addition, the 3DE IESS can usually be installed on a computer.

VE can produce unconventional views of congenital heart disease and improve our understanding of intracardiac malformations, as an orthogonal view may be difficult to display using conventional echocardiography. The interatrial and interventricular septum can be visualized head-on, providing for better intuition concerning the relevant spatial relationships with neighboring structures. In a view from the inside of the right atrium, the spatial relationship of a defect to the superior vena cava becomes readily evident when the virtual eye is in a superior and posterior orientation. Visualization of a more unusual inferior and posterior secundum atrial septum is obtained by manipulation of the virtual eye and rotation of the 3D image inferior and posterior to the inferior vena cava. Likewise, the entrance of the coronary sinus into the atrium and its correlation to atrial structures can be similarly visualized.

Many clinical and experimental studies have demonstrated that VR is now ready for clinical application. Employing virtual reality, 3DE allows for a better understanding of the position of cardiac structures in relation to one another [12–15, 21, 22]. The unique ability of VE to depict the intracardiac anatomy from both the atrium and the ventricle, with access to head-on views, allows for better anatomical definition in relation to adjacent cardiac structures. This novel method can provide clinicians with more confidence in diagnoses of the complex anatomy of congenital heart disease. Additionally, VE can provide accurate and reproducible visualizations. Its versatility allows observers to retrieve an infinite number of different views of the intracardiac anatomy following the examination procedure.

VE offers great potential for aiding in complex diagnostic situations, assisting in planning surgical procedures, and in postoperative assessments [23]. VE is particularly helpful because it enables the heart to be visualized from the same perspective that the surgeon would see intraoperatively. Virtual reality visualization may lead to a better understanding of the intracardiac anatomy and facilitate the accuracy of preoperative planning.

Virtual reality also provides a unique resource for cardiac disease education and can add to the existing knowledge of complex pathology. Weidenbach et al. [24, 25] introduced a simulator (EchoComJ) with an intelligent training system to support trainees in adjusting to standard echocardiographic views. It may be helpful to convey or demonstrate intracardiac malformations to those unfamiliar with 2DE views—especially cardiologists, for whom detailed knowledge of the intracardiac anatomy is essential.

Although VE provides a solid overview of the completely acquired volume, not all structures of interest can be easily observed, as 3D imaging requires new knowledge of image interpretation, and thus there is a learning curve involved, particularly for less experienced operators, that should be accounted for before the full advantage of this new modality can be realized. One problem with VE is that operators must be experienced in anatomy, radiology, computer science, and clinical evaluation.

Study limitations

There are some limitations to this study. One major limitation of VE with the existing technology is the time-consuming nature of the off-line processing. VE image obtained with the commercially available equipment provided images with sufficient detail transferred to a separate workstation for off-line data analysis. The mean time for VE reconstruction was approximately 5 min. Even though, the interpretation of 3D images are time consuming and dependent on the complexity of the intracardiac structures. Laborious manual data analysis in a number of selected ideal endoscopic views, however, is time-consuming. In the majority of patients, off-line image processing and tracing required approximately 30–60 min.

Another limitation is the possible difficulty in navigating. In several instances during this study, observers lost their sense of orientation while ‘inside’ the intracardiac anatomy. Understanding the orientation of a 3D visualization of echocardiographic images is often difficult due to the lack of visible anatomical landmarks. A navigation system would be sufficient in resolving the disorientation problems [26].

There are still some limitations on visualization due to artifacts of moving heart and moving floppy heart structures. 3D reconstruction of VE depends on the quality of the original images of which judged on the presence or absence of artifacts throughout the cardiac cycle. The acquisition of multiple cardiac cycles may cause stitch artifacts. Poor acoustic window and minor patients’ movement can also result in artifacts. Additionally, only the 2D TTE images were selected for analysis. It is possible that the 2D transthoracic echocardiogram might not capture the clinical diagnosis of a congenital heart disease, whereas a transesophageal matrix array transducer probably does not incur problems with acoustic shadowing and can provide images of good quality. Even though this is a potential limitation of the study, it does not have much of an effect from our perspective because only high quality transthoracic 2DE and VE images were included in the study. Our initial research demonstrated that 3DE can serve as a clinical tool for VE. Further studies are necessary to improve image quality and “reality” of endoscopic mode images.

Moreover, the study is limited by the lack of a gold standard for the diagnosis. Although clinical findings served as the gold standard for the clinical diagnoses of congenital heart defect, clinical diagnosis of normal hearts was made using 2D Doppler transthoracic echocardiography.

Admittedly, the study is limited by the relatively small sample size, as only 40 patients were compared between the 2D and 3D image analyses. However, the conclusion of the study should not be weakened by the small number of patients. Moreover, they were not analyzed in a quantitative pattern, which prevents robust assessment.

Conclusions

We have demonstrated that three-dimensional echocardiographic virtual endoscopy can enhance our understanding of intracardiac structures and provide intuitive, reproducible, and clinically useful visualizations. Virtual reality techniques may provide important information by specifying the spatial orientation of the intracardiac anatomy, versus standard 2DE. This technique can aid cardiologists by adding many two-dimensional slices to accurately form three-dimensional objects from these data. The role of 3DE VE in clinical practice has yet to be clarified. However, this preliminary VE study allows us to speculate on situations in which this novel method may be useful.

Virtual reality expands the capabilities of noninvasive cardiology and may open new doors for the evaluation of congenital heart diseases. In order to accomplish this, prospective studies will be needed.