Joint Image Extraction Algorithm and Super-Resolution Algorithm for Rheumatoid Arthritis Medical Examinations

  • Tomio Goto
  • Yoshiki Sano
  • Takuma Mori
  • Masato Shimizu
  • Koji Funahashi
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 98)


Super-resolution techniques have been widely used in fields such as television, aerospace imaging, and medical imaging. In medical imaging, X-rays commonly have low resolution and a significant amount of noise, because radiation levels are minimized to maintain patient safety. So, we proposed a novel super-resolution method for X-ray images, and a novel measurement algorithm for treatment of rheumatoid arthritis (RA) using X-ray images generated by our proposed super-resolution method. However, in our proposed system, there are several operations to do by doctors manually, and it is hard for them. By utilizing image recognition technology, it is possible to extract joint images from X-ray images automatically. In this paper, we will discuss an algorithm to extract joint images from X-ray images automatically. Experimental results show that correct joint images will be obtained for our proposed method. Therefore, our proposed measurement algorithm is effective for RA medical examinations.


Image extraction Joint space distance Rheumatoid arthritis Medical examinations 

1 Introduction

X-ray images are widely used to diagnose a variety of diseases. However, to reduce the patient’s exposure to radiation, X-ray dosage is minimized as much as possible. As a result, X-ray images contain a significant amount of noise and resolution is compromised. Thus, it is necessary to increase image resolution and reduce noise. We proposed a novel super-resolution system for X-ray images that consists of total variation (TV) regularization, a shock filter, and a median filter. In addition, we proposed a novel measurement algorithm for the treatment of RA, using X-ray images generated by our proposed super-resolution system [1]. In this paper, we improve measurement accuracy to optimize parameters for super-resolution.

2 Super-Resolution System

Super-resolution is a technique for increasing the resolution of an enlarged image by generating new high-frequency components. This technique estimates and generates such components from the characteristics of the original signals. In recent years, various super-resolution techniques have been proposed, and most are classified as either reconstruction-based super-resolution [2] or learning-based super-resolution [3].

In resolution for X-ray images, which is used at medical operation, the resolution of one pixel will be 0.15 mm\(^{2}\), and joint space distance will be about from 1.0 mm to 1.5 mm. Therefore, when the joint space distance is measured with one pixel error, the error will be from 10% to 15%, improving measurement accuracy is required.

It is possible to solve this problem by utilizing a super-resolution technique. By magnifying segmented images by a multiple of 4 \(\times \) 4, a resolution of a pixel will be 0.0375 mm, thus the error can be suppressed from 2.5% to 3.75%. Also by utilizing the shock filter, clearer edges in images will be obtained, thus it is easy to select edges. However, the error still remains and it is not small to measure for RA medical examinations. In this paper, we improve measurement accuracy to set bigger magnification rate and optimal parameters for super-resolution.

A block diagram of our proposed super-resolution system is shown in Fig. 1. Each of the non-linear filters is explained in the following sections.
Fig. 1.

Block diagram of our proposed super-resolution system.

2.1 Total Variation Regularization

As shown in Fig. 1, the TV regularization decomposition [4, 5, 6, 7, 8] is performed as follows. The structure component u is calculated to minimize the evaluation function F(u) as shown in Eq. (1):
$$\begin{aligned} F(u) = \varSigma _{i,j} |\nabla u_{i,j}| + \lambda \varSigma _{i,j} |f_{i,j} - u_{i,j}|^2 . \end{aligned}$$
where f is a pixel value of the input image. The Chambolle’s projection algorithm [9] is used to solve the minimization problem as shown in Eq. (2).
$$\begin{aligned} P_{i,j}^{(n+1)} = \frac{P_{i,j}^{(n)}+(\frac{\tau }{\lambda }) \nabla (f+\lambda div P_{i,j}^{(n)})}{max \{1,|P_{i,j}^{(n)} + (\frac{\tau }{\lambda }) \nabla (f+\lambda div P_{i,j}^{(n)})|\}}. \end{aligned}$$
where P is a pixel value. The texture component v and the structure component u are obtained by using the equations in (3).
$$\begin{aligned} v=\lambda div P, \quad \quad u=f-v. \end{aligned}$$
Figure 2 shows an example of the TV decomposition for an X-ray image, (a) is an original image, (b) is a structure component and (c) is a texture component.
Fig. 2.

Example of total variation decomposition for X-ray image.

2.2 Shock Filter

The shock filter is a nonlinear edge enhancement filter, which was proposed by Osher and Rudin [10] and Alvarez and Mazorra [11]. The process is achieved by utilizing Eq. (4):
$$\begin{aligned} u_{i,j}^{(n+1)} = u_{i,j}^{(n)} - sign \left( \varDelta \left( K_\sigma * \varDelta u_{i,j}^{(n)}\right) \right) \left| \nabla u_{i,j}^{(n)} \right| dt. \end{aligned}$$
where u is a structure component, K is a smoothing filter. It is possible to reconstruct steep edges by calculating a simple operation; thus, this filter is suitable for high-speed processing. In addition, several artifacts generated during edge enhancement processing can be controlled successfully, i.e., ringing noise and jaggy noise. Figure 3 shows an input image and output images obtained by utilizing the shock filter.
Fig. 3.

Example of shock filter for X-ray image.

2.3 Median Filter

Most noise is classified as a texture component by utilizing TV regularization. As mentioned previously, X-ray images contain significant noise. Therefore, we propose applying the median filter to the texture components of X-ray images. The median filter sorts nine pixel values in 3 \(\times \) 3 pixels around the pixel of interest. Next, the filter replaces the fifth pixel value with a new pixel value of interest. This process is applied to all of the texture components.

3 Measurement Algorithm About Joint Space Distance

Rheumatoid arthritis (RA) is a disease that causes joint inflammation, and most commonly afflicts women between 30 and 50 years of age. As symptoms progress, the patient’s joint space distance (JSD) will narrow. This change can be observed with X-ray images; however, at present, an accurate measurement method has not been established. Therefore, a more accurate JSD measurement technique is required. We proposed two JSD measurement algorithms [1]. Figure 4 shows an example of an output image. In our proposed method, we use an input image, which is magnified by utilizing a super-resolution method. Our algorithm for measurement is shown as follows:
  1. 1.

    Select several points on an edge of an upper bone by clicking mouse button, manually.

  2. 2.

    Set axes corresponding to a joint from selected points automatically.

  3. 3.

    Calculate coordinate values of selected points based on the axes.

  4. 4.

    Calculate a quadratic function by using the least squares method from the calculated fitting function.

  5. 5.

    Calculate a quadratic function by selecting several points on an edge at a lower bone, similarly.

  6. 6.

    Measure the joint space distance from normal lines and integral operation.

We also propose another JSD measurement algorithm, which is calculated by using an area of JSD, and its algorithm for measurement is shown as follows:
  1. 1.

    An integral calculus range in a range of curve p is set, and to calculate the integral calculus \(S_R\) of differences between curve p and curve q.

  2. 2.

    An integral calculus range in a range of curve q is set, and to calculate the integral calculus \(S_B\) of differences between curve p and curve q.

  3. 3.

    The value W is defined as the distance between 2 points, which is automatically detected when the coordinate axis are set.

  4. 4.

    The JSD value D is calculated from two values: \(S_R\) and \(S_B\) and averages of the distances of the top and bottom as shown in Eq. (5).

$$\begin{aligned} D = \frac{S_R + S_B}{2W} \end{aligned}$$
Fig. 4.

Application for JSD calculation of X-ray image.

4 Proposed Extraction Method

By utilizing image recognition technology, we will extract parts of images around the third finger joints from many images, automatically. And by learning these images for correct images, it is possible to recognize joint space distance. We propose a novel method for extracting the third finger joints for correct images and the other images for incorrect images, which are used in the learning method.

Figure 6 shows our proposed extracting algorithm. First, to input the center coordinates in the third finger joints, then several points in the radius of the circle are selected, randomly. Then, \(128 \sqrt{2}\) square pixels are extracted because rotation will be processed in the post-processing and it is possible to prevent the purchase of noise by the rotation process. And those are turned in the range of \(\pm 20\) degrees, then 128 square pixels in center are output as a correct image. Next, the Gaussian filter is processed to the X-ray image including the whole both hands, those are binarized, the isolated points are removed to do closing processing based on morphology operation, and bone regions are extracted as shown in Fig. 5.
Fig. 5.

Extracted results for bone area

Finally, some points in the whole image are selected, randomly, and when the point is \(128 \sqrt{2}\) square pixels in bone regions and far from the center coordinates, 128 square pixels in center are output as an incorrect image. To select center image points randomly in the circle, in which the center coordinates draw, it is possible to control the gaps between the center point of joints and extracted images. Also, bone parts will be included in the center of the extracted images, which are far from the center of joints, and they are output as incorrect images, so that it is possible to output bone parts without the third joints of correct images.
Fig. 6.

Proposed extraction algorithm

5 Experimental Results

Learning images are made by using our proposed method, and machine learning with the Haar-Like feature by utilizing those images is performed.

5.1 Image Extraction for Machine Learning by Our Proposed Method

Figures 7 and 8 show the experimental results for correct and images, and Fig. 9 shows the experimental results for incorrect images. As a result, by utilizing the center points of joints, it is confirmed to extract correct images without greatly deviating from the center of those images. Also, to extract bone parts, it is effective method for features machine learning. And it is also confirmed to extract incorrect images with bone parts.
Fig. 7.

Output results for left hands as correct images

Fig. 8.

Output results for right hands as correct images

Fig. 9.

Output results as incorrect images

5.2 Feature Machine Learning Using Extracted Images

Machine learning is performed by using extracted images, which was made from 24 X-Ray images by using our proposed method, while changing the radius of the search circle with 0, 16 and 32 pixels. As a result, when the radius of the search circle was 0 pixel, it is possible to succeed machine learning and to make a detector. This means that our proposed method is effective for feature machine learning of the Haar-Like features.

6 Conclusion

In this paper, an automation of measurement application for joint space distance is subject, and an algorithm for extracting images, which are used for feature learning at finger joints, has been proposed and its application has been implemented. By extracting images, which are used to the center of the third finger joints and bone parts as correct and incorrect images, respectively, the Haar-Like feature learning succeeds and it is possible to make its detector.

For further research, we intend to improve the detective accuracy to consider an enhancement of image features based on image processing and to consider the other features and effectiveness of our proposed method by using Deep-Learning.


  1. 1.
    Goto, T., Mori, T., Kariya, H., Shimizu, M., Sakurai, M., Funahashi, K.: Super-resolution technology for X-ray images and its application for rheumatoid arthritis medical examinations. In: International KES Conference on Innovation in Medicine and Healthcare (InMed), pp. 217–226, June 2016CrossRefGoogle Scholar
  2. 2.
    Singh, A., Ahuja, N.: Single image super-resolution using adaptive domain transformation. In: IEEE International Conference on Image Processing (ICIP), pp. 947–951, September 2013Google Scholar
  3. 3.
    Cho, C., Jeon, J., Paik, J.: Example-based super-resolution using selfpatches and approximated constrained least squares filter. In: IEEE International Conference on Image Processing (ICIP), pp. 2140–2144, October 2014Google Scholar
  4. 4.
    Rudin, L., Ohser, S., Fetami, E.: Nonlinear total variation based noise removal algorithm. Phys. D 60, 259–268 (1992)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Meyer, Y.: Oscillating patterns in image processing and non-linear evolution equation. In: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures. University Lecture Series, vol. 22. American Mathematical Society (1992)Google Scholar
  6. 6.
    Vese, L.A., Osher, S.J.: Modeling textures with total variation minimization and oscillating patterns in image processing. J. Sci. Comput. 19, 553–572 (2003)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Aujol, J.-F., Aubert, G., Blanc-Feraud, L., Chambolle, A.: Image decomposition into a bounded variation component and an oscillating component. J. Math. Imaging Vis. 22, 71–88 (2005)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Goto, K., Nagashima, F., Goto, T., Hirano, S., Sakurai, M.: Super-resolution for high resolution displays. In: IEEE Global Conference on Consumer Electronics (GCCE), pp. 309–310, October 2014Google Scholar
  9. 9.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1), 89–97 (2004)MathSciNetMATHGoogle Scholar
  10. 10.
    Osher, S.J., Rudin, L.I.: Feature-oriented Image enhancement using shock filters. SIAM J. Numer. Anal. 27, 910–940 (1990)CrossRefGoogle Scholar
  11. 11.
    Alvarez, L., Mazorra, L.: Signal and image restoration using shock filters and anisotropic diffusion. SIAM J. Numer. Anal. 31(2), 590–605 (1994)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  • Tomio Goto
    • 1
  • Yoshiki Sano
    • 1
  • Takuma Mori
    • 1
  • Masato Shimizu
    • 1
  • Koji Funahashi
    • 2
  1. 1.Department of Computer Science and EngineeringNagoya Institute of TechnologyNagoyaJapan
  2. 2.Orthopaedic SurgeryKariya Toyota General HospitalKariyaJapan

Personalised recommendations