Sharpening filter for false color imaging of dual-energy X-ray scans

Abstract

Dual-energy X-ray imaging has a vast range of application in security. Luggage inspection is an essential process for an airplane or court house security as well as securing mass events. An image of a content of some package may help to figure out if there is any dangerous object inside and avoid possibly threatening situation. As the raw X-ray images are not always easy to analyze and interpret, some image processing methods like an object detection, a frequency resolution increase or a pseudocoloring are being used. Since color can be a powerful tool to improve the usefulness of an information display, we propose pseudocoloring improvement by modifying material-based approach with edge detection to fill and sharpen color layers over the image making it easier to read and analyze. We demonstrate the effectiveness of the methods using real data, acquired from a professional dual-energy X-ray scanner.

1 Introduction

X-Ray scanners provide images based on different level of absorption of radiation of different materials. Scanners are used, for example, in medical tomography and security on which we focus. Dual-energy X-ray imaging is a well-known technique. It requires two measurements at different energies and allows displaying material properties (the effective atomic number and mass density). In most cases, the resulting images are displayed in gray scale where the most absorbing parts are colored white, and areas transparent to the X-rays are colored with black (see Fig. 1a). An inversion of this colorspace is also popular and frequently used. This kind of presentation, in some applications, is good enough for well-trained technicians, but the necessity of providing more details to images causes that some marking is required. A common approach is to use a linear color map (LCM) (see Fig. 1b). More advanced methods allow to classify scanned objects or their parts to certain materials on the basis of mass attenuation coefficient [1–3]. This coefficient depends on the material’s atomic number. In theory, it should allow us to classify the object on the basis of their atomic number, but tests showed that objects’ thickness has a major impact on the classification.

The proposed method—edge enhanced image pseudocoloring (EEIPC)—is based on material qualification, but also takes objects’ shapes into account. An image features such a structure and geometric description are used to correct object assignment. Our aim is to provide images that allow to easily distinguish one object from another by significantly enhancing object edges during the colorizing procedure. The presented method is capable of colorizing objects in a way that it matches its material. It is also important to remove the most of artifacts and unexpected transitions of material color ranges, but preserving sharp borders between objects. The EEIPC method is an enhancement of the material classification presented in our previous work [4]. Similar image filter is also discussed in [5]. In this paper we also are concerned about assessment of image sharpness. We examine image sharpness and color using two measures: standard deviation of image histogram and LPC-Sharpness Index introduced in [6, 7]. It is also important to know that the most important quality estimation of our method comes from the scanners operators using this method in their daily work.

Fig. 1

Approaches of false coloring: a raw image, b linear color mask, c material-based color mask (color figure online)

2 Methodology

X-ray inspection system is composed of X-ray source, X-ray detector and conveyor belt [8]. Security scanners use a very short (0.0110 nm) electromagnetic radiation waves. X-rays are emitted with photon energy ranging from 30 to 200 KeV [9]. Photons successfully passed through a scanned object are measured by detectors ordered in a L-shaped bar [10]. As the baggage goes through the scanner tunnel, the attenuated X-rays are collected by the row of detectors. Detectors’ driver assembles data from multiple detectors into one discrete grayscale image.

2.1 Hardware

Our scanner works with a single X-ray monoblock generator and a dual-energy range detector which provides simultaneously two energy images. This means that during one-time X-ray shot, we obtain two sets of data for two images generation (low and high energies, $I_{\mathrm{LE}}$ and $I_\mathrm{HE}$, respectively). A detector for dual-energy digital radiography has a complicated construction. There are two scintillators, for low energy: GOS (153 $\hbox {mg/cm}^{2}$) and for high energy: CsI(TI). The first one is 3.0 mm thick, and the second one is 4.1 mm thick. There is also a filter element (0.6 mm copper) between LE and HE scintillators. In order to produce our test images, a tube generates X-rays using 160 keV voltage with 5300 A current, which are measured by the detectors after passing through scanned objects. We are using Spellman XRB160P model/type tube with X-Scan 1,5L-1253 DE-ENET detectors.

Tested objects were moved by conveyor at 40 cm/s speed rate. Scanned objects could have maximum width of 1000 mm and unlimited length due to the possibility of moving them horizontally.

2.2 Dual energy

For our scans, we use dual-energy data collection. Dual-energy X-ray imaging brings us benefits in the form of depth resolution enhancement, because of slightly different attenuation characteristics which are depending on photons energy. In this section data acquisition from detectors and creation of raw images is discussed, see [11]. The X-ray tube emits $N_0$ photons, but only N passes through the scanned object and reaches the detector. The numbers of photons that will pass through the object depends on its attenuation coefficient ($\mu$) and object thickness (T):

$$\begin{aligned} N=N_0 \exp (-\mu T) \end{aligned}$$

(1)

The coefficient $\mu$ depends on the density of the material ($\rho$), and mass attenuation of the material ($\tau$). $\tau$ is experimentally obtained and tabularized function depending on the material effective atomic number (Z) and energy of the photons (E):

$$\begin{aligned} \mu =\rho \cdot \tau (E,Z) \end{aligned}$$

(2)

Attenuation graphs for some substances are displayed in Fig. 2.

Fig. 2

Dependence between photon energy and attenuation. Values are tabularized, see [12]

From Eq. (1) we have:

$$\begin{aligned} -\log (N/N_0)=\mu T \end{aligned}$$

(3)

and finally we get the following equation:

$$\begin{aligned} -\log (N/N_0)=T\cdot \rho \cdot \tau (E,Z)=m_E \end{aligned}$$

(4)

which means, there is a proportion between number of rays that crossed a scanned object and a coefficient of its attenuation. If our object is not a uniform, $m_E$ is replaced by summations:

$$\begin{aligned} m_E=\sum _{i=0}^{\varOmega }-T_i\cdot \rho _i\cdot \tau _i(E,Z) \end{aligned}$$

(5)

where $\varOmega$ is number of different materials. $N_0$ is a variable which depends on photon energy and it will be denoted as $N_0(E)$ from now. The X-ray spectrum is polychromatic in energy range [E1, E2]:

$$\begin{aligned} m_{E}=-\log \left( \int ^{E2}_{E1}N^\prime _0(E) \cdot \exp \left( -\sum _{i=0}^{\varOmega }\rho _i\tau _i(E,Z) \right) \mathrm{d}E \right) \end{aligned}$$

(6)

where $N^\prime _0(E)$ is number of photons emitted with energy E in range [E1, E2]:

$$\begin{aligned} N^\prime _0(E)=\frac{D(E)\cdot N_0(E)}{\int ^{E2}_{E1}D(E)\cdot N_0(E)\cdot \mathrm{d}E} \end{aligned}$$

(7)

where D(E) represents detector efficiency. The detector efficiency and X-ray tube spectrum are known.

From the Eq. (6) it could be possible to determine Z value of each material in a scanned object but in real scan we do not know how much objects and what materials we have to designate proper $\rho$ and $\tau$ values for this equation. To retrieve approximate information of an object material, we use combination of data from two measurements of different energies.

We receive number of registered photons that crossed a scanned object (N) from the detector. Knowing the number of photons emitted ($N_0$), the ratio is:

$$\begin{aligned} I(E)=N/N_0 \end{aligned}$$

(8)

As it was stated in 2.1, we gather data in two energy ranges, LE and HE, see [13]. Finally, we can use following formulas, to calculate so-called Q values [14]:

$$\begin{aligned} Q=\dfrac{m_\mathrm{LE}}{m_\mathrm{HE}}=\dfrac{\log (I_\mathrm{LE})}{\log (I_\mathrm{HE})} \end{aligned}$$

(9)

where $(I_\mathrm{LE})$ and $(I_\mathrm{HE})$ stand for low-energy image and high-energy image, respectively, and Q determines the expected material of an object registered on a pixel and will be used in the proposed algorithm.

3 Algorithm

A current section describes a coloring algorithm that uses two raw images received from the scanner to produce sharp and colorized image.

3.1 Material determination

X-ray images obtained for high energy ($I_\mathrm{HE}$) and low energy ($I_\mathrm{LE}$) are shown in Fig. 3. All image examples in this work are cropped to the region of interest from received scans.

Fig. 3

Raw images: a high energy, b low energy

In order to determine the Q(x, y) value for every pixel we use quotient of the logarithms of object’s material values (see Eq. 9) of HE and LE scan.

$$\begin{aligned} Q(x,y) = \frac{\log (I_\mathrm{LE}(x,y))}{\log (I_\mathrm{HE}(x,y))} \end{aligned}$$

(10)

The number of materials ($\varOmega$) as well as their Q ranges can be chosen arbitrary. Typically to the standards used in the professional security scanners, only three types of materials are distinguished:

• 1. metals (average $Z > 26$)—$Q^{(1)} \in (1.02, 1.15)$,

• 2. non-organic (others)—$Q^{(2)} \in (1.005, 1.02]$,

• 3. organic (carbon based)—$Q^{(3)} \in (0.972, 1.005]$,

but in general there can be any $\varOmega$ number of different materials. For all materials their Q ranges were obtained experimentally. These values are tied strictly to the scanner configuration and may vary on different machines and energy settings. In our case, pixels with values below 0.972 are treated as a background and pixels with values above 1.15 are marked as not penetrated by X-rays.

After pixel classification we create $\varOmega +2$ (number of materials, plus background and foreground) binary images for all material types based on pixel value:

$$\begin{aligned} \pi ^{(i)} (x,y) = {\left\{ \begin{array}{ll} 1 &{} \left\{ Q(x,y):Q_L^{(i)}<Q(x,y)<Q_H^{(i)} \right\} \\ 0 &{} \text {else} \end{array}\right. } \end{aligned}$$

(11)

where $i \in (0,\varOmega +1)$, Q(x, y) is a quotient of the logarithms of $I_\mathrm{LE}$ and $I_\mathrm{HE}$ images (Eq. 9), $Q_L^{i}$ and $Q_H^{i}$ are low and high limits of the Q values for ith material, respectively (see Fig. 4).

Fig. 4

Material masking: a brightness and contrast adjusted raw image (for comparison), b metal mask, c plastic mask, d organic mask

3.2 Edge sharpening

Images generated in such a way are very sharp, and they have a lot of isolated pixels and unwanted artifacts. Such problem can be fixed by using a morphological filling or blurring. We use a simple Gaussian blurring technique to prepare those images for the next processing step. Optimal sizes of the Gaussian filter masks are $((S,T)=5,5)$ and $((S,T)=7,7)$, i.e., the kernel of a Gaussian filter with a standard deviation of $\sigma \in (2/3, 1)$. A filter with smaller standard deviation is unable to cover all in-object holes, while bigger masks give no improvement in the final image while increasing computational complexity. As a result we obtain blurred ($i=\varOmega +2$) material images:

$$\begin{aligned} \pi _{\sigma }^{(i)} = \varGamma _{\sigma } ( \pi ^{(i)} ), \quad i \in (0, \varOmega +2) \end{aligned}$$

(12)

where $\varGamma _{\sigma }$ is a Gaussian blur operator. Because of the blurring procedure, the material images are overlapping each other nearby edges. We use the edge detection algorithm [15, 16] to restore sharp edges.

$$\begin{aligned} \epsilon =\text {CED}(I_\mathrm{HE}) \end{aligned}$$

(13)

where CED is the Canny edge operator, $I_\mathrm{HE}$ is the high energy image, and $\epsilon$ is an edge image. We chose the Canny edge detector due to its ability to describe edges in a vector form. It provides the information about an angle of a contrast gradient in every point classified as the edge.

$$\begin{aligned} \mathbf {g}(x,y)=\nabla \pi _{\sigma }(x_{\epsilon }, y_{\epsilon }) \end{aligned}$$

(14)

where $(x_{\epsilon }, y_{\epsilon })$ are pixels that belong to the edge $\epsilon$ determined by the Canny edge detector. A Sobel filter is used to calculate these gradient angles $(\theta )$ from horizontal $(g_x)$ and vertical $(g_y)$ direction:

$$\begin{aligned} \theta = \arctan \left( \frac{g_x}{g_y} \right) \end{aligned}$$

(15)

These angles are often called contrast gradient degree. In the Canny algorithm, angles are discretized to one of the following four values (discretized directions $\theta _\mathrm{D}$) which are equal to 0$^{\circ }$, 45$^{\circ }$, 90$^{\circ }$ and 135$^{\circ }$ covering 45$^{\circ }$ around each one. For every pixel which belongs to Canny’s edge vector, the appropriate contrast gradient degree is assigned. We used voting algorithm to determine an affiliation of a point to a certain mask. The size of the region of interest created around current pixel is equal to the size of previously used Gaussian filter mask. An example is shown in Fig. 5.

Fig. 5

Example behavior of an algorithm in case of a pixel with $45 \pm 22.5$ contrast gradient degree and $7 \times 7$ neighbor mask. a Pixels near the edge on image $\pi _{\beta }$, b scheme of the algorithm

As shown in Fig. 5, a border $\epsilon$ found by the Canny algorithm is marked with black pixels (Fig. 5b). The central pixel $\mathbf {r}(x_0, y_0)$ is a border point being processed in the current step. In this example the central pixel has the contrast gradient $\mathbf {g}(x,y)$ with angle $\theta _\mathrm{D}$ in the range of $45^{\circ } \pm 22.5^{\circ }$, so it requires comparison of two neighboring pixels from SW and NE (pixels with bold border in Fig. 5b). Values of these two pixels are read from blurred material image $\pi _{\sigma }^{(i)}$ (see Fig. 6), and a greater one is chosen.

$$\begin{aligned} r_f(x,y) = {\left\{ \begin{array}{ll} \mathbf {r} + \mathbf {g} &{} \text {if } (\pi _{\sigma }(\mathbf {r}+\mathbf {g}) > \pi _{\sigma }(\mathbf {r}-\mathbf {g}) ) \\ \mathbf {r} - \mathbf {g} &{} \text {else} \end{array}\right. } \end{aligned}$$

(16)

A line perpendicular to the proper discretized gradient $\kappa (\mathbf {g_{\bot }}(x,y))$ splits the neighbor into two triangles (or two rectangles depending on the $\theta _\mathrm{D}$). This line is marked as diagonal line in our example image in Fig. 5b. The triangle (or rectangle) that contains the pixel with a higher value of a pixel $\mathbf {r_f}(x,y)$ is being set as the foreground (dark background pixels in Fig. 5b), and the second part of the square is marked as the background (light background pixels in Fig. 5). These values are used to create so-called affiliation matrices

$$\begin{aligned} \alpha ^{(i)}(x,y)=0, \quad i \in (0, \varOmega + 1) \end{aligned}$$

(17)

which contains information whether the pixel is included in the material layer or not. Every affiliation matrix has the same size as the acquired image and is initially filled with zeros. Only the pixels that were effectively blurred are being processed this way (see Fig. 6c).

Fig. 6

Construction of affiliation matrix: a raw mask, b mask blurred using $7 \times 7$ Gaussian matrix, c affiliation matrix containing non-extreme points on b

Next, the algorithm fills the affiliation matrices with the proper values. Every pixel with a value equal to one (1) on the blurred image $\pi _{\sigma }^{(i)}$ (pure black in Fig. 6b) will be mapped into maximum value on the affiliation matrix. Every pixel with a zero value on the same image will be mapped into minimal negative value on the affiliation matrix. The values of the other pixels are set to 0 (shown in a gray in Fig. 6b).

$$\begin{aligned} \alpha ^{(i)}(x,y) = {\left\{ \begin{array}{ll} \mathrm{MAX} &{} \text {if } \left( \pi _{\sigma }^{(i)}(x,y) = 1 \right) \\ -\mathrm{MAX} &{} \text {if } \left( \pi _{\sigma }^{(i)}(x,y) = 0 \right) \\ 0 &{} \text {else} \end{array}\right. } \end{aligned}$$

(18)

At this step, for each pixel in Canny edge image $\epsilon$, the region of interest of the affiliation mask must be determined. This region is centered on the position of the current Canny edge vector pixel $\mathbf {r}(x_0,y_0)$. The sizes of this region and Gaussian mask used in previous step are the same. In the region of interest for each foreground pixel, a positive value is added. This value is proportional to the distance between the point and the borderline $\kappa (\mathbf {g_{\bot }})$ separating the foreground and the background parts. For each background pixel a negative value is added in the same manner.

$$\begin{aligned} \alpha ^{(i)}(x,y)= || \mathbf {r}(x,y), \kappa (\mathbf {g_{\bot }}) ||_{\widehat{r_f}} \end{aligned}$$

(19)

where $|| \mathbf {a}, \kappa ||_{\widehat{b}}$ is a signed distance between the vector $\mathbf {a}$ and the line $\kappa$ depending on the direction of the unit vector $\widehat{b}$.

After the inspection of each edge pixel, all negative pixels are removed from the affiliation mask:

$$\begin{aligned} \alpha ^{(i)}(x,y)= \max \left( 0, \,\, \alpha ^{(i)}(x,y)\right) \end{aligned}$$

(20)

It is the final step of the construction of the affiliation matrix for the single layer. Now the affiliation matrix must be determined for each material layers taking all the affiliation matrices into account.

At this point of analysis there are some pixels that belong to several material layers. We have to determine to which single layer the analyzed pixel should be assigned. We use the voting algorithm which compares affiliation masks pixels from the same coordinates on different material layers. If pixel has a maximum value on some mask, its value is set to 1, while on the other masks the pixels on the same coordinates are set to zero.

$$\begin{aligned} \alpha ^{(i)} (x,y) = {\left\{ \begin{array}{ll} 1 &{} \text {if } \alpha ^{(i)} (x,y) \equiv \max \limits _{k \in (0,\varOmega +1)}\left\{ \alpha ^{(k)}(x,y)\right\} \\ 0 &{} \text {else } \end{array}\right. } \end{aligned}$$

(21)

Fig. 7

Sample images and their colorized version as well as close-ups to some of their details. a, d, g, j Original grayscale images, b, e, h, k images colorized with LCM method, c, f, i, l images colorized with our EEIPC method metal elements are clearly visible and distinguishable (color figure online)

3.3 Image colorizing

On the basis of the affiliation matrices $\alpha ^{(i)}$ and the corresponding low-energy image $I_\mathrm{LE}$ and high-energy image $I_\mathrm{HE}$, we construct final color image. We have decided to use CIE L*a*b* color space because it separates achromatic channel from chromatic channels and due to its ability to compare colors as a simple Euclidean distance of its coordinates. With S-CIELAB extension it allows for measuring color reproduction errors of digital images [17].

To properly colorize scans a specific color maps are predefined for all material types, i.e., for all $Q^{(i)}$ material ranges a specific lightness $L^{*(i)}$ value as well as $a^{*(i)}$ and $b^{*(i)}$ chromatic color values are predefined. In this step we map high-energy image $I_\mathrm{HE}$ into an $L^*$ channel

$$\begin{aligned} L^*(x,y)= \bigcup _{i=0}^{\varOmega +1} \left\{ \varPsi _L\left( I_\mathrm{HE}(x,y) \cdot \alpha ^{(i)}(x,y), \,\, L^{*{(i)}}\right) \right\} \end{aligned}$$

(22)

and chromatic $a^*$, $b^*$ channels are calculated using the following formulas

$$\begin{aligned} a^*(x,y)= & {} \bigcup _{i=0}^{\varOmega +1} \left\{ \varPsi _L\left( I_\mathrm{LE}(x,y) \cdot \alpha ^{(i)}(x,y), \,\, a^{*{(i)}}\right) \right\} \end{aligned}$$

(23)

$$\begin{aligned} b^*(x,y)= & {} \bigcup _{i=0}^{\varOmega +1} \left\{ \varPsi _L\left( I_\mathrm{LE}(x,y) \cdot \alpha ^{(i)}(x,y), \,\, b^{*{(i)}}\right) \right\} \end{aligned}$$

(24)

where $i \in (0, \varOmega +1)$, and $\varPsi$ are mapping operations defined as:

$$\begin{aligned} \varPsi _L(I, c^{(i)})= & {} 100 \cdot \left( I \cdot \left( c^{(i+1)} - c^{(i)}\right) + c^{(i)} \right) \end{aligned}$$

(25)

$$\begin{aligned} \varPsi _{ab}(I, c^{(i)})= & {} 255 \cdot \left( I \cdot \left( c^{(i+1)} - c^{(i)}\right) + c^{(i)} \right) - 128\nonumber \\ \end{aligned}$$

(26)

After coloring procedure, the final image is converted into sRGB color space and displayed to the scanner operator. Examples of original images and pseudocolored images as well as some edge details are shown in Figs. 7, 8, and 9.

Fig. 8

Close-up to details of colorized images. a LCM method, b EEIPC method (color figure online)

Fig. 9

Close-up to details of colorized images. a LCM method, b EEIPC method (color figure online)

4 Tests and results

For testing purposes we have prepared a set of 12 images scanned in low and high energies. For every pair of LE and HE images we calculate contrast and sharpness indices for original grayscale images, images with linear color map (LCM) without sharpening filter and images colorized with the presented method (EEIPC). As a contrast measure we chose a standard deviation of an image histogram

$$\begin{aligned} \sigma ^2 = \sum _{k=0}^{L-1}(i_k - \overline{h})^2 \cdot h(i_k) \end{aligned}$$

(27)

where $i_k$ is a kth image value, L is a maximum image value, $h(i_k)$ is $i_k$th value of a normalized histogram of the image, and $\overline{h}$ is an average value of the histogram. As an image sharpness measure we chose Local Phase Coherence Sharpness Index (LPC-SI) proposed by Hassen [7] which describes the alignment of local phase patterns in the scalespace in the vicinity of distinctive sharp image features and as a result gives a single number determining the image sharpness. These measures are applicable only to single-channel images. To compare tri-chromatic images, $\sigma$ and LPC-SI were calculated for each color channel separately in an sRGB color space and then the arithmetic average value from three channels was taken as a result. The quantitative evaluation results are given in Table 1 for original grayscale images, for images with linear color map (LCM) and for our colorizing method with sharpness filter (EEIPC). For every image we highlight the better evaluation measure value.

Table 1 Quality evaluation of colorizing methods

The quantitative results show clearly that our EEIPC method gives more sharp edges and cleaner boundaries between objects at about 30 % compared to LCM method and almost 60 % compared to original grayscale images. It also does not change low-frequency parts of an image and preserves overall mean and contrast $\sigma$ of the original image.

Figure 7 shows a few examples of scans for original grayscale images, LCM colorized images and images colorized with our EEIPC method. There are also some close-ups to edge details and material boundaries shown in Fig. 8 for both colorizing methods. Figure 9 shows a close to a part of a scan in chromatic channels of CIELAB color model. There are sharp metal edges clearly visible in Fig. 9b, d enhanced by our EEIPC method, whereas these edges are not visible at all when only a LCM method was used.

The most important benefit of presented algorithm is an ability to distinguish between material layers regardless of whether the edges of scanned object are smooth or sharp. The described algorithm can analyze unrestricted number of different layers. We have to remember that in practical scanning baggage procedure, inspectors can be confused seeing more than four different color ranges.

Algorithm’s time computational complexity depends linearly on the number of layers’ edge pixels. For each layer the complexity is similar to two-dimensional convolution filter and is thus in the order of O(PMf) where P is the number of pixels, M is number of material masks, and f is the area of the Gaussian mask. Presented algorithm runs almost real time on our testing machine with Intel i5-3470 3.2 GHz. Processing time for the testing images with resolution $900 \times 540$ pixels varies between 1 and 3 s for typical scanned object and three material layers. Such processing time is shorter than time needed for the scanner user to complete the operating task.

5 Conclusions and future

The discussed false coloring algorithm EEIPC has already been introduced into professional usage as a part of a software driver for scanners produced under the name of Arida. It is being used among others image adjusting and object detection algorithms. A colored X-ray image is a basic tool for baggage inspectors. This algorithm is also used for potential risk detection filter where areas of attenuation coefficient are close to known drugs and explosives. Currently our team works on an algorithm for risk detection based on both material properties and on geometric information like, for example, the shape of a weapon, see [18–20]. There is also important to evaluate the quality of the colorized images by a subjective quality tests in which a number of human subjects are asked to judge the quality of the sequence images. The subjective tests will be based on the recommendations given by the International Telecommunication Union (ITU) [21] that define how to carry out subjective quality tests.