2-Stage classification of knee joint thermograms for rheumatoid arthritis prediction in subclinical inflammation

  • Shawli Bardhan
  • Mrinal Kanti BhowmikEmail author
Technical Paper


Presence of inflammation in knee joint is the early indication of arthritis. In this paper, we performed the inflamed region segmentation from knee joint thermograms for structural feature extraction based knee abnormality prediction. Existing four popular segmentation techniques are investigated, namely K-means, Fuzzy C-means, Otsu, Single seeded region growing. We proposed modified multi-seeded region growing method that generates 98.6% accurate segmentation rate compared to ground truth of inflammation. Based on the spread of the inflammation oriented structural feature analysis, in the first stage of classification we classified arthritis affected knee joint thermograms, and all other types of thermograms (non-arthritis) with 91% accuracy. Among different types of arthritis, the most damaging type that causes disability of joints in long run is known as rheumatoid arthritis (RA). Early diagnosis of RA in subclinical stage enormously helps clinicians to decrease the disease affect. In second stage of classification, we integrated the RA and non-RA categorization by extracting texture, shape and frequency level features. Experiment shows that the combination of all features decreases the accurate detection rate of RA classification. To increase the classification rate, we incorporated the accuracy based feature selection procedure. The RA classification rate obtained with accuracy based feature selection is 73% whereas existing support vector machine-recursive feature elimination (SVM-RFE) and RELIEF methods provide 67% and 71% correct classification rate respectively. The area under the curve (AUC) of accuracy based feature selection, SVM-RFE, and RELIEF for RA classification are 0.72, 0.65 and 0.67, respectively and it indicates better classification outcome of the accuracy based feature selection method.


Arthritis RA Segmentation Feature extraction Feature selection Classification 



This work was supported by the Indian Council of Medical Research (ICMR), Government of India, under Grant Number. 5/7/1516/2016-RCH, Dated: 20/06/2017. The second author would like to thank Dr. S. B. Nath, Assistant Professor, Agartala Government Medical College (AGMC), Agartala for his kind support during the collection and inflammation oriented ground truth generation of knee thermogram dataset. Also authors would like to thank Prof. Dipti Prasad Mukherjee, India Statistical Institute, Kolkata for his valuable suggestions regarding feature selection during the classification stage of this work and Prof. Debotosh Bhattacharjee, Jadavpur University, Kolkata for his guidance during the revision stage of the manuscript.

Compliance with ethical standards

Conflict of interest

The author declares no potential conflict of interest with respect to the authorship and/or publication of this article.

Ethical approval

The study was approved by the Research cell and Ethics committee of Agartala Government Medical College, Tripura, India with approval Ref. No. 4(6-11)-AGMC/Medical Education/Ethics Com./2018/15136, Dated, 31st December, 2018.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Silman AJ, Hochberg MC (2001) Epidemiology of the rheumatic diseases. 2nd edn. Oxford University Press, New YorkGoogle Scholar
  2. 2.
    Milind P, Sushila K (2012) How to live with rheumatoid arthritis? Int Res J Pharm 3(3):115–121Google Scholar
  3. 3.
    Bhowmik MK et al (2016) Pain related inflammation analysis using infrared images. SPIE Commercial + Scientific Sensing and Imaging. International Society for Optics and PhotonicsGoogle Scholar
  4. 4.
    Sacks JJ, Helmick CG, Langmaid G (2004) Deaths from arthritis and other rheumatic conditions, United States, 1979–1998. J Rheumatol 31:1823–1828Google Scholar
  5. 5.
    (2018) Arthritis by the numbers/book of trusted facts and figures (vol 2, 4100.17.10445). Accessed 29 Oct 2018
  6. 6.
    Schäfer VS et al (2016) Arthritis of the knee joint in rheumatoid arthritis—evaluation of treatment response by ultrasound in daily clinical practice. Open Rheumatol J 10:81–87 (PMC Web) CrossRefGoogle Scholar
  7. 7.
    Ropes MW (1959) Diagnostic criteria for rheumatoid arthritis. Anti Rheum Dis 18:49–53CrossRefGoogle Scholar
  8. 8.
    Neelima AM (2012) Textbook of oral and maxillofacial surgery, Chapter-2 (art of diagnosis), 3rd edn. Jaypee Brothers Medical Publisher Ltd., Daryaganj, p 15CrossRefGoogle Scholar
  9. 9.
    Snekhalatha U, Anburajan M, Teena T, Venkatraman B, Menaka M, Raj B (2012) Thermal image analysis and segmentation of hand in evaluation of rheumatoid arthritis. In: Computer communication and informatics (ICCCI), 2012 international conference, IEEE, pp 1–6Google Scholar
  10. 10.
    Borojević N, Kolarić D, Grazio S, Grubišić F, Antonini S, Nola IA, Herceg Ž (2011) Thermography of rheumatoid arthritis and osteoarthritis. In: ELMAR, 2011 proceedings, pp 293–295, IEEEGoogle Scholar
  11. 11.
    Frize M, Adéa C, Payeur P, Di Primio G, Karsh J, Ogungbemile A (2011) Detection of rheumatoid arthritis using infrared imaging. In: Medical imaging 2011: image processing, vol 7962. International Society for Optics and Photonics, p 79620MGoogle Scholar
  12. 12.
    Acharya UR, Ng EYK, Tan JH, Sree SV (2012) Thermography based breast cancer detection using texture features and support vector machine. J Med Syst 36(3):1503–1510CrossRefGoogle Scholar
  13. 13.
    Francis SV, Sasikala M, Saranya S (2014) Detection of breast abnormality from thermograms using curvelet transform based feature extraction. J Med Syst 38(4):23CrossRefGoogle Scholar
  14. 14.
    Milosevic M, Jankovic D, Peulic A (2014) Thermography based breast cancer detection using texture features and minimum variance quantization. EXCLI J 13:1204Google Scholar
  15. 15.
    EtehadTavakol M, Chandran V, Ng EYK, Kafieh R (2013) Breast cancer detection from thermal images using bispectral invariant features. Int J Thermal Sci 69:21–36CrossRefGoogle Scholar
  16. 16.
    Wood AM, Brock TM, Heil K, Holmes R, Weusten A (2013) A review on the management of hip and knee osteoarthritis. Int J Chronic Dis. Google Scholar
  17. 17.
    Informed Health Online [Internet] (2006) Rheumatoid arthritis: Overview. 2013 Oct 23 [Updated 2016 Aug 11]. Institute for Quality and Efficiency in Health Care (IQWiG), Cologne, Germany. Accessed 18 Jan 2018
  18. 18.
    “What is AiArthritis?” (n.d.) Retrieved 18 January 2018, from
  19. 19.
  20. 20.
    Araki S, Nomura H, Wakami N (1993) Segmentation of thermal images using the fuzzy c-means algorithm. In: Fuzzy systems, second IEEE international conference on, IEEE, pp 719–724Google Scholar
  21. 21.
    Golestani N, Tavakol ME, Ng EYK (2014) Level set method for segmentation of infrared breast thermograms. EXCLI J 13:241Google Scholar
  22. 22.
    Snekhalatha U, Anburajan M, Sowmiya V, Venkatraman B, Menaka M (2015) Automated hand thermal image segmentation and feature extraction in the evaluation of rheumatoid arthritis. Proc Inst Mech Eng Part H J Eng Med 229(4):319–331CrossRefGoogle Scholar
  23. 23.
    Shahari S, Wakankar A (2015) Color analysis of thermograms for breast cancer detection. In: Industrial instrumentation and control (ICIC), 2015 international conference on, IEEE, pp 1577–1581Google Scholar
  24. 24.
    Jadin MS, Taib S (2012) Infrared image enhancement and segmentation for extracting the thermal anomalies in electrical equipment. Electron Electr Eng 120(4):107–112Google Scholar
  25. 25.
    Font-Aragonés X, Faúndez-Zanuy M, Mekyska J (2013) Thermal hand image segmentation for biometric recognition. IEEE Aerosp Electron Syst Mag 28(6):4–14CrossRefGoogle Scholar
  26. 26.
    Dutta T, Sil J, Chottopadhyay P (2016) Condition monitoring of electrical equipment using thermal image processing. In: 2016 IEEE first international conference on control, measurement and instrumentation (CMI), IEEE, pp 311–315Google Scholar
  27. 27.
    Selvarasu N, Vivek S, Nandhitha NM (2007) Performance evaluation of image processing algorithms for automatic detection and quantification of abnormality in medical thermograms. In: Conference on computational intelligence and multimedia applications, 2007. international conference on, IEEE, vol 3, pp 388–393Google Scholar
  28. 28.
    Nandhitha NM, Sheela Rani B, Kalyana Sundaram P, Venkataraman B, Raj B (2007) Performance evaluation of hot spot extraction and quantification algorithms for online weld monitoring from weld thermographs. 24th Int Symp Autom Robot Construct 3:461–466 (ISBN: 0-7695-3050-8, 13 Dec 2007) Google Scholar
  29. 29.
    George YM et al (2014) Remote computer-aided breast cancer detection and diagnosis system based on cytological images. IEEE Syst J 8(3):949–964. CrossRefGoogle Scholar
  30. 30.
    Frize M et al (2009) Preliminary results of severity of illness measures of rheumatoid arthritis using infrared imaging. In: 2009 IEEE international workshop on medical measurements and applications.
  31. 31.
    Bhowmik MK et al (2017) Designing of ground truth annotated DBT-TU-JU breast thermogram database towards early abnormality prediction. IEEE J Biomed Health Inf. Google Scholar
  32. 32.
    Haralick RM et al (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC-3 6:610–621. CrossRefGoogle Scholar
  33. 33.
    Weszka JS et al (1976) A comparative study of texture measures for terrain classification. IEEE Trans Syst Man Cybern SMC-6 4:269–285. CrossRefGoogle Scholar
  34. 34.
    Amadasun M, King R (1989) Textural features corresponding to textural properties. IEEE Trans Syst Man Cybern 19(5):1264–1274. CrossRefGoogle Scholar
  35. 35.
    Wu C-M, Chen Y-C (1992) Statistical feature matrix for texture analysis. CVGIP Graph Models Image Process 54(5):407–419. CrossRefGoogle Scholar
  36. 36.
    Laws KI (1979) Texture energy measures. In: DARPA image understanding workshop. DARPA, Los Altos, CA, pp 47–51Google Scholar
  37. 37.
    Wu C-M et al (1992) Texture features for classification of ultrasonic liver images. IEEE Trans Med Imaging 11(2):141–152. CrossRefGoogle Scholar
  38. 38.
    Guyon I, Weston J, Barnhill S, Bapnik V (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46(1–3):389–422CrossRefGoogle Scholar
  39. 39.
    Sun Y (2007) Iterative RELIEF for feature weighting: algorithms, theories, and applications. IEEE Trans Pattern Anal Mach Intell 29(6):1035–1051. CrossRefGoogle Scholar
  40. 40.
    Shafreen Banu K, Hari Ganesh S (2015) A study of feature selection approaches for classification. In: 2015 international conference on innovations in information, embedded and communication systems (ICIIECS), Coimbatore, pp 1–4Google Scholar
  41. 41.
    De Silva DVSX et al (2010) Adaptive sharpening of depth maps for 3D-TV. Electron Lett 46(23):1546–1548CrossRefGoogle Scholar
  42. 42.
    Sadri A, Reza et al (2013) Segmentation of dermoscopy images using wavelet networks. IEEE Trans Biomed Eng 60(4):1134–1141CrossRefGoogle Scholar
  43. 43.
    Bardhan S, Bhowmik MK, Nath S, Bhattacharjee D (2015) A review on inflammatory pain detection in human body through infrared image analysis. In: 2015 international symposium on advanced computing and communication (ISACC) 2015.
  44. 44.
    Ghosh S, Dubey SK (2013) Comparative analysis of K-means and fuzzy c-means algorithms. Int J Adv Comput Sci Appl 4:4Google Scholar
  45. 45.
    Sthitpattanapongsa P, Srinark T (2011) An equivalent 3D Otsu’s thresholding method. Pacific-Rim symposium on image and video technology. Springer, BerlinGoogle Scholar

Copyright information

© Australasian College of Physical Scientists and Engineers in Medicine 2019

Authors and Affiliations

  1. 1.Computer Science and EngineeringTripura UniversitySuryamaninagarIndia

Personalised recommendations