Multimedia Tools and Applications

, Volume 78, Issue 6, pp 6847–6868 | Cite as

Natural tongue physique identification using hybrid deep learning methods

  • Huihui Li
  • Guihua WenEmail author
  • Haibin Zeng


Traditional Chinese Medicine (TCM) illustrates that the physique determines the susceptibility of human to certain diseases and treatment programs for illness. Tongue diagnosis is an important way to identify the physique, but now it is performed by the doctor’s professional experience and the design of a questionnaire. Consequently, accurate physique identification cannot be obtained easily. In this paper, we propose a new method to identify the physique through wild tongue images using hybrid deep learning methods. It begins with constructing a large number of tongue images that are taken in natural conditions, instead of in a controlled environment. Based on the resulting database, a new method of tongue coating detection is put forward that applies a rapid deep learning method to complete the initial tongue coating detection, and then utilizes another deep learning method, a calibration neural network, to further improve the accuracy of tongue detection. Finally, an effective deep learning method is applied to identify the tongue physique. Experiments validate the proposed method, illustrating that physique identification can be performed well using hybrid deep learning methods.


Deep learning Tongue coating Physique identification Traditional Chinese Medicine (TCM) 



This study was supported by China National Science Foundation (Grant Nos. 60973083 and 61273363), Science and Technology Planning Project of Guangdong Province (Grant Nos. 2014A010103009 and 2015A020217002), and Guangzhou Science and Technology Planning Project (Grant No. 201504291154480,201803010088). We also thank LetPub ( for its linguistic assistance during the preparation of this manuscript.


  1. 1.
    Deng T (2002) Treatment based on syndrome differentiation is the soul of clinical medicine of traditional Chinese medicine. Chin Arch Tradit Chin Med 20 (4):394–395. Google Scholar
  2. 2.
    Du JQ, Lu YS, Zhang K, Zhu MF, Ding CH (2008) A novel approach of tongue body and tongue coating separation based on FCM. In: 2008 2nd international conference on bioinformatics and biomedical engineering, Shanghai, 16–18 May, 2008, pp 2499–2503.
  3. 3.
    He K, Zhang X, Ren S, Sun J (2014) Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer vision – ECCV 2014. Lecture notes in computer science, vol 8691. Springer, Cham, pp 346–361. Google Scholar
  4. 4.
    Hu MC, Cheng MH, Lan KC (2016) Color correction parameter estimation on the smartphone and its application to automatic tongue diagnosis. J Med Syst Jan 40(1):18. CrossRefGoogle Scholar
  5. 5.
    Huang B, Wu J, Zhang D, Li N (2010) Tongue shape classification by geometric features. Inf Sci (Ny) 180(2):312–324. CrossRefGoogle Scholar
  6. 6.
    Ini R, Siio I (2013) TongueDx: a tongue diagnosis system for personal health care on smartphone. Human Interface (Japan).
  7. 7.
    Institute of Chinese Medicine (2009) ZYYXH/T157-2009 Chinese medicine constitution classification and decision. Chinese Traditional Chinese Medicine Press, Beijing (In Chinese)Google Scholar
  8. 8.
    Kamarudin ND, Ooi CY, Kawanabe T, Mi X (2016) Tongue’s substance and coating recognition analysis using HSV color threshold in tongue diagnosis. In: Proceedings of SPIE 10011, first international workshop on pattern recognition, 100110J (11 July 2016).
  9. 9.
    Kanawong R (2012) Computer-aided tongue image diagnosis and analysis. Dissertation, University of Missouri at ColumbiaGoogle Scholar
  10. 10.
    Kanawong R, Xu W, Xu D, Li S, Ma T, Duan Y (2012) An automatic tongue detection and segmentation framework for computer-aided tongue imag+e analysis. Int J Funct Inform Personal Med 4(1):56–68. Google Scholar
  11. 11.
    Kim KH, Do JH, Ryu H, Kim JY (2008) Tongue diagnosis method for extraction of effective region and classification of tongue coating. In: 2008 first workshops on image processing theory, tools and applications, Sousse, 23–26 Nov. 2008, pp 1–7.
  12. 12.
    Ko MM, Park TY, Lee JA, Kang BK, Lee J, Lee MS (2013) A study of tongue and pulse diagnosis in traditional Korean medicine for stroke patients based on quantification theory type II. Evid Based Complement Alternat Med 2013:508918. Google Scholar
  13. 13.
    Lee J, Ko M, Park T, Kang B, Moon T, Lee M (2012) P02.79. Study on the basic tongue diagnosis indicators for pattern identifications in stroke using a decision tree method. BMC Complement Altern Med 12(S1):P135. CrossRefGoogle Scholar
  14. 14.
    Li Q, Liu Z (2009) Tongue color analysis and discrimination based on hyperspectral images. Comput Med Imaging Graph 33(3):217–221. CrossRefGoogle Scholar
  15. 15.
    Li FF, Zhao C, Xia Z et al (2012) Computer-assisted lip diagnosis on traditional Chinese medicine using multi-class support vector machines. BMC Complementary Alternative Med 12:127. CrossRefGoogle Scholar
  16. 16.
    Li H, Lin Z, Shen X, Brandt J, Hua G (2015) A convolutional neural network cascade for face detection. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, 7–12 June, 2015, pp 5325–5334.
  17. 17.
    Lin F, Xiahou J, Xu Z (2016) TCM Clinic records data mining approaches based on weighted-LDA and multi-relationship LDA model. Multimed Tools Appl 75 (22):14203–14232. CrossRefGoogle Scholar
  18. 18.
    Liu B, Zhang Z (2004) The ancient method of syndrome differentiation of thinking. Chin J Basic Med Tradit Chin Med 10(5):325–331. Google Scholar
  19. 19.
    Lo LC, Chen YF, Chen WJ, Cheng TL, Chiang JY (2012) The study on the agreement between automatic tongue diagnosis system and Traditional Chinese Medicine practitioners. Evid Based Complement Alternat Med 2012:505063. Google Scholar
  20. 20.
    Luo D, Wen G, Li D, Hu Y, Huan E (2018) Iterative regression based Face Detection using deep learning. Multimed Tools Appl 2018(6):1–18Google Scholar
  21. 21.
    Ning J, Zhang L, Zhang D, Wu C (2010) Interactive image segmentation by maximal similarity based region merging. Pattern Recognit 43 (2):445–456. CrossRefGoogle Scholar
  22. 22.
    Pang B, Zhang D, Li N, Wang K (2004) Computerized tongue diagnosis based on bayesian networks. IEEE Trans Biomed Eng 51(10):1803–1810. CrossRefGoogle Scholar
  23. 23.
    Pang B, Zhang D, Wang K (2005) The bi-elliptical deformable contour and its application to automated tongue segmentation in Chinese medicine. IEEE Trans Med Imaging 24(8):946–956. CrossRefGoogle Scholar
  24. 24.
    Ren S, He K, Girshick R, Sun J (2017) Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. CrossRefGoogle Scholar
  25. 25.
    Shi M, Li G, Li F (2013) C2g2FSnake: automatic tongue image segmentation utilizing prior knowledge. Sci China Inform Sci 56(9):1–14. MathSciNetGoogle Scholar
  26. 26.
    Shi MJ, Li GZ, Li FF, Xu C (2014) Computerized tongue image segmentation via the double geo-vector flow. Chin Med 9:7. CrossRefGoogle Scholar
  27. 27.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv: [cs.CV]
  28. 28.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, 7–12 June 2015., pp 1–9
  29. 29.
    Wang X, Zhang D (2010) An optimized tongue image color correction scheme. IEEE Trans Inf Technol Biomed 14(6):1355–1364. CrossRefGoogle Scholar
  30. 30.
    Wang X, Zhang D (2013) A high quality color imaging system for computerized tongue image analysis. Expert Syst Appl 40(15):5854–5866. CrossRefGoogle Scholar
  31. 31.
    Wang X, Zhang B, Yang Z, Wang H, Zhang D (2013) Statistical analysis of tongue images for feature extraction and diagnostics. IEEE Trans Image Process 22(12):5336–5347. CrossRefGoogle Scholar
  32. 32.
    Wu K, Zhang D (2015) Robust tongue segmentation by fusing region-based and edge-based approaches. Expert Syst Appl 42(21):8027–8038. CrossRefGoogle Scholar
  33. 33.
    You M, Ge L, Li GZ, Xu L, Huang S (2009) A TCM platform for maters’ experience sharing. In: International joint conference on bioinformatics, systems biology and intelligent computing (IJCBS09). IEEE Computer Society, Washington, DC, pp 388–391Google Scholar
  34. 34.
    Zhang J, Hu G, Zhang X (2015) Extraction of tongue feature related to TCM physique based on image processing. In: 2015 12th international computer conference on wavelet active media technology and information processing (ICCWAMTIP), Chengdu, China, 18–20 December 2015, pp 251–255.
  35. 35.
    Zhang B, Kumar BV, Zhang D (2014) Detecting diabetes mellitus and nonproliferative diabetic retinopathy using tongue color, texture, and geometry features. IEEE Trans Biomed Eng 61(2):491–501. CrossRefGoogle Scholar
  36. 36.
    Zhang Q, Shang HL, Zhu JJ, Jin MM, Wang WX, Kong QS (2013) A new tongue diagnosis application on Android platform. In: 2013 IEEE international conference on bioinformatics and biomedicine (BIBM), 18–21 December 2013, Shanghai, China, pp 334–327.
  37. 37.
    Zhuo L, Zhang J, Dong P, Zhao Y, Peng B (2014) An SA–GA–BP neural network-based color correction algorithm for TCM tongue images. Neurocomputing 134:111–116. CrossRefGoogle Scholar
  38. 38.
    Zhuo L, Zhang P, Qu P, Peng Y, Zhang J, Li X (2016) A k-PLSR-based color correction method for TCM tongue images under different illumination conditions. Neurocomputing 174:815–821. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringSouth China University of TechnologyGuangzhouChina
  2. 2.Guangdong Artificial Intelligence Engineering Research Center for Traditional Chinese MedicineGuangzhouChina
  3. 3.Guangdong Artificial Intelligence Engineering Research Center for Traditional Chinese MedicineShenzhenChina

Personalised recommendations