From Local to Global: A Holistic Lung Graph Model

Abstract

Lung image analysis is an essential part in the assessment of pulmonary diseases. Through visual inspection of CT scans, radiologists detect abnormal patterns in the lung parenchyma, aiming to establish a timely diagnosis and thus improving patient outcome. However, in a generalized disorder of the lungs, such as pulmonary hypertension, the changes in organ tissue can be elusive, requiring additional invasive studies to confirm the diagnosis. We present a graph model that quantifies lung texture in a holistic approach enhancing the analysis between pathologies with similar local changes. The approach extracts local state-of-the-art 3D texture descriptors from an automatically generated geometric parcellation of the lungs. The global texture distribution is encoded in a weighted graph that characterizes the correlations among neighboring organ regions. A data set of 125 patients with suspicion of having a pulmonary vascular pathology was used to evaluate our method. Three classes containing 47 pulmonary hypertension, 31 pulmonary embolism and 47 control cases were classified in a one vs. one setup. An area under the curve of up to 0.85 was obtained adding directionality to the edges of the graph architecture. The approach was able to identify diseased patients, and to distinguish pathologies with abnormal local and global blood perfusion defects.

1 Introduction

An important task in the radiology workflow is detecting subtle alterations in patient scans that could help to correctly identify and diagnose diseases. However, when there is a widespread distribution of the disease in an organ, i.e. the lungs, the pathological changes can be so elusive that require more (invasive) studies to establish a diagnosis [16]. Particularly in the case of pulmonary hypertension (PH), an invasive catheterization procedure is the gold standard for diagnosis and allows the differentiation of PH from pulmonary embolism (PE) [12]. A correct interpretation of the cases in their early stages is relevant since both pathologies have similar local manifestations, despite requiring different treatments [10]. Moreover, the pulmonary perfusion changes in PH and PE might not only be present in small regions, but also have a profound impact on the entire pulmonary circulatory network [11]. A holistic lung blood perfusion analysis, instead of local independent assessments of haemodynamic alterations, could improve the clinical evaluation of a patient by providing a global pathological status of the lungs.

Graph modeling is a complete framework that has been previously proposed for brain connectivity analysis, but has rarely been applied to other organs [14]. In short, graph methods divide the brain into fixed anatomical regions, and compare neural activations between different regions [15]. Based on these approaches, we propose a graph model of the lungs built from 3D local texture descriptors extracted on an atlas-based parcellation, with the purpose of encoding pulmonary blood perfusion relations between neighboring regions. An early version of this method was proposed in [8], combining simple intensity features in a 30 patients dataset of dual energy CT scans, combining 11 energy levels. The approach was also compared against 8 other methods, including deep learning approaches in the ImageCLEF tuberculosis challenge 2017 [4], obtaining the 3 top scores for the lung image analysis task in drug resistance detection [7].

In this paper we exploit the graph architecture for the analysis of the whole pulmonary circulatory network in a relatively large and heterogeneous dataset of patients with pulmonary vascular pathologies. The global analysis of the lungs was refined to detect and differentiate between PH and PE, relying only on the CT scans taken in an emergency radiology service. A novel visual interpretation of the biomedical tissue texture is presented as well, which can help radiologists to promptly interpret and localize 3D textural changes in anatomical structures.

Fig. 1.

Examples of the three classes in the dataset. Iodine maps showed that both PH and PE cases presented hypo- and hyper-perfused regions, not apparent in the CT scans.

2 Methods

2.1 Dataset

Experiments were carried out on contrast-enhanced chest CT images at 70 KeV of 125 patients: 31 with diagnosed PE, 47 with diagnosed PH and 47 control cases (CC) (see Fig. 1). The resolution of the CT slices in x- and y-directions varied from 0.5742 to 0.9766 mm, while the inter-slice distance was 1.00 mm. Since 3D rotation-invariant texture features were used in our approach, all CT images were converted into isometric volumes, with a voxel size of $1\times 1\times 1$ mm$^3$.

2.2 Holistic Graph Model of the Lungs

A pipeline composed of four steps was developed for building a distinctive graph per patient (see Fig. 2). Initially, the lung fields were automatically extracted using the method explained in [6]. Then, the lung masks were geometrically divided into a 36-region atlas [3, 5] derived from the 3D model of the human lung presented by Zrimec et al. [17]. For each region r of this atlas two texture-based feature descriptors were extracted: the Fourier histograms of oriented gradients (FHOG) [13] and the locally-oriented 3D Riesz-wavelet transform (3DRiesz) [9]. These descriptors have been successful for multiple biomedical texture analysis applications [1, 2]. FHOG was computed using 28 3D directions for the histogram, obtaining a 28-dimensional feature vector per image voxel v ($\mathbf {f}_{H}(v) \in \mathbb {R}^{28}$). For 3DRiesz we used the 3rd-order Riesz-wavelet transform, with 4 scales and 1st-order alignment (see [9]). The feature vector for a single voxel was defined as the weighted sum of the absolute Riesz response along the 4 scales, obtaining a 10-dimensional feature vector ($\mathbf {f}_\mathcal {R}(v) \in \mathbb {R}^{10}$). Finally, the average and standard deviation of these descriptors were obtained for each region r: $\varvec{\mu }_{H}(r)$, $\varvec{\sigma }_{H}(r)$, $\varvec{\mu }_{\mathcal {R}}(r)$, and $\varvec{\sigma }_{\mathcal {R}}(r)$.

Fig. 2.

Construction of the graph model: First the lungs were automatically segmented. Then they were divided using the geometric atlas with 36 regions. Finally, the graph was built based on the 3D adjacency of the regions (including left-right lung connections). The edges contained the similarities between 3D texture descriptors computed in the regions.

2.3 Undirected Weighted Graph Model of the Lung

In [4] Dicente et al. tested several graph configurations for lung modeling of tuberculosis cases using the 36-regions atlas. The graph architecture obtaining the best scores contained 84 edges, based on a region adjacency defined on the atlas. Formally, there exists an undirected edge $E_{i,j}$ between nodes $N_i$ and $N_j$ if regions $r_i$ and $r_j$ are 3D adjacent in the atlas or symmetric with respect the left-right division of the lungs (see Fig. 2). Considering $\mathbf {f}_i$ and $\mathbf {f}_j$ the feature vectors of regions $r_i$ and $r_j$ respectively, the weight $w_{i,j}$ of an edge $E_{i,j}$ was defined as: $w_{i,j} = 1- corr (\mathbf {f}_i, \mathbf {f}_j) \in \left[ 0,2\right]$. The use of rotation-invariant texture descriptors where each component corresponds to a texture-direction implies that regions with similar texture will be highly correlated, and thus, with $w_{i,j} \approx 0$. Since the edges used for this graph were undirected, then $w_{i,j} = w_{j,i}$ and this graph has 84 weights. For a patient p this graph will be referred as $G^{U}_p$.

2.4 Directed Weighted Graph-Model of the Lungs

Using the same graph architecture (36 nodes and 84 edges), we introduce the notion of directionality. let $\mathbf {f}_i$ and $\mathbf {f}_j$ be the feature vectors of regions $r_i$ and $r_j$, respectively. The weight of an edge $E_{i,j}$ between nodes $N_i$ and $N_j$ is then defined as $w_{i,j}=1- corr (\mathbf {f}_i, \mathbf {f}_j)$ if $||\mathbf {f}_i|| \le ||\mathbf {f}_j||$, and 0 otherwise. Since $w_{i,j} \ne w_{j,i}$ (except when $||\mathbf {f}_i|| = ||\mathbf {f}_j||$), this graph contains 168 weights, of which 84 are equal to 0. However, for each patient these 84 weights equal to 0 may be at different positions since the condition for being 0 relies on the properties of each patient lung tissue. The idea of this approach is to better exploit the same graph architecture adding more information about the texture in the regions (nodes), i.e. which node contained more/less texture than its neighbors For a patient p this graph is referred as $G^{D}_p$.

2.5 Graph-Based Patient Descriptor

As mention before, the undirected graph $G^{U}_p$ of a patient p only contained 84 weights. Then, we defined the graph-based patient descriptor $\varvec{w}^{U}_p$ as the vector containing the 84 weights ordered by their location in the graph. For the directed graph, $\varvec{w}^{D}_p$ contained the 168 weights ordered by their graph location.

Patient Descriptor Normalization. The descriptor vectors $\varvec{w}_p$ were normalized based on the training set, obtaining $\varvec{\hat{w}}_p$. However, since each component of $\varvec{w}_p$ corresponded to a weight in the graph, these can not be seen independently and the normalization was done for all components simultaneously.

Concatenation of Patient Descriptors. As mentioned before, four different feature vectors were computed in each atlas region r($\varvec{\mu }_{H}(r)$, $\varvec{\sigma }_{H}(r)$, $\varvec{\mu }_{\mathcal {R}}(r)$, and $\varvec{\sigma }_{\mathcal {R}}(r)$) providing complementary information about the texture and its variability. Given a patient p, a different graph was obtained from each of these feature vectors. The final patient descriptor vector $\varvec{W}_p$ used in our experiments was defined as the concatenation of the four normalized graph-based descriptors: $\varvec{W}_p = (\varvec{\hat{w}}_{\varvec{\mu }_{H}, p} || \varvec{\hat{w}}_{\varvec{\sigma }_{H},p} || \varvec{\hat{w}}_{\varvec{\mu }_{\mathcal {R}}, p} || \varvec{\hat{w}}_{\varvec{\sigma }_{\mathcal {R}}, p} )$. After this concatenation, $\varvec{W}^{U}_p \in \mathbb {R}^{336}$ when using the undirected graph and $\varvec{W}^{D}_p \in \mathbb {R}^{672}$ for the directed approach.

3 Experimental Setup

Four binary classification experiments were performed: (a) PH vs. PE, (b) PH vs. CC, (c) PE vs. CC, and (d) (PH+PE) vs. CC. In experiment (d) PH and PE patients were considered to belong to same class. The dataset was divided in each experiment in $\sim$70% for training and $\sim$30% for testing. This results in 34 PH, 22 PE and 34 CC for training and 13 PH, 9 PE and 13 CC for testing. As the classes were not balanced in the test sets, the random accuracy (when assigning the most frequent label to all the patients) was 59.09% for experiments (a) and (c), 50.00% for experiment (b) and 62.86% for experiment (d). 2-class SVM classifier with Gaussian kernel was used in each experiment. The parameter optimization was performed using 10-fold cross-validation on the training set. All experiments were performed using both the undirected and the directed graph models.

The descriptor vectors for a given patient $\varvec{W}^{U}_p$ and $\varvec{W}^{D}_p$ had a dimension significantly higher than the size of our dataset (336 and 672 vs. 125). To avoid the known overfitting problems when using large feature spaces, feature dimensionality reduction was performed in the training phase. This consisted on selecting those dimensions that correlate above the average with the training labels. This method reduced the feature space dimension approximately by two, both when using the undirected and directed graph models. Moreover, since the random split of the dataset in training and test sets may have a strong effect in the classification performance, we ran each experiment using five different random splits to ensure the robustness of our models.

4 Results

Figure 3 shows the average and standard deviation of the accuracy and the AUC over the five random splits for each of the four experiments. The accuracy and the confusion matrices in Fig. 4 are reported using the standard classification decision threshold of 0.5, and not in the optimal threshold provided by the ROC curves. The results of each experiment were above the random performance in all cases (see red line in Fig. 3). When analyzing all the performance measures, the directed graph model performed better than the undirected in the experiments (b) and (c). However, both models performed almost equivalent in experiments (a) and (b). The highest benefit of the directed graph among the undirected is shown in experiment (b), where the true positive rate increases from 69.23% and 75.38% to 76.92% and 81.54% in the PH and CC classes respectively. In experiment (d) the increment is similar, however, the directed graph is still not able to identify the CC class above random. These results suggest that PH and PE do not form a clear cluster in the feature space.

Fig. 3.

Average accuracy and AUC over the five random splits of the dataset (both training and test set). The error bars show the standard deviation. The red line indicates the random performance in each measure and experiment.

Fig. 4.

Confusion matrices with the % of TP, FP, TN and FN for the four experiments over the 5 random dataset splits.

Fig. 5.

3D visualization of the graphs obtained for the three patients shown in Fig. 1. The first row contains the subgraph corresponding to the regions in the 2D slice. All six graphs are depicted using the same color code. This example has been generated using the $\varvec{\mu }_\mathcal {R}$ feature. In this case, heterogenous edge connections can be seen in PH and PE patients.

5 Discussion

This work presents a comprehensive graph model of the lungs that connects 3D regional texture nodes enclosing the whole circulatory network of lungs. The method was evaluated in a large dataset of 125 CT thorax scans from patients with clinical symptoms commonly associated to vascular pathologies, such as pulmonary hypertension and pulmonary embolism. Even for experienced radiologists detecting and differentiating pulmonary hypertension is a challenging task diagnosed only by using a catheterization procedure. The proposed approach was able to correctly identify most of the patients with these pulmonary perfusion defects in a standard CT scan, even without any prior clinical information.

The proposed composite interpretation of the lung circulatory network was able to distinguish between local defects (PE) and elusive global pathological patterns (PH). By analyzing the confusion matrices in Fig. 4, it can be seen that the method had the worse classification scores with the control class. Nonetheless, it is important to highlight that all the patients in the dataset were obtained from an emergency department and may well have other lung malfunctions. A straightforward 3D visualization of the textural changes in the lungs was additionally generated from the obtained graph architectures. The graph depicting both lungs could be useful for radiologists to understand intrinsic parenchymal texture distributions that might not be apparent in 2D renderings of the organ.

6 Conclusions and Future Work

A precise assessment of pulmonary perfusion can lead to a fast diagnosis and optimal treatment in the presence of haemodynamic changes. A directed weighted graph model of the lungs encoding blood perfusion relations from an automatically generated geometric atlas of the lungs was evaluated. The approach was able to not only recognize diseased patients, but also to classify correctly similar abnormal local and global patterns in the pulmonary parenchymal texture. This holistic graph descriptor can be expanded to include more lung pathologies and potentially be applied to other anatomical structures as well.