Assessing General Well-Being Using Facial Expressions

  • John Vong
  • Insu Song
Part of the Topics in Intelligent Engineering and Informatics book series (TIEI, volume 11)


Global cell-phone ownership has surpassed over 5 billion. The proliferation of cell phones offers an unprecedented opportunity for aid organizations and governments in developing countries for providing affordable medical services for everyone. The available standardized interfaces of low-cost cell-phones allow us to create powerful medical diagnostics systems. For instance, digital cameras of cell phones now provide easy to use interfaces for capturing useful information of various medical conditions. However, photographic images often contain private and sensitive personal information in its raw form thus considered unsuitable for many available online services. Therefore, there is a need for a computational algorithm for extracting anonymous, de-identified, digital features from captured images for assessing medical conditions and general personal wellbeing. We present a de-identified feature generation method, called Gaussian Hamming Distance (GHD). We show that GHD features are significantly correlated with personal wellbeing. Its low computational complexity makes it ideal to be used with low-cost mobile devices. Its prediction power is suitable for providing a variety of online services including recommending useful health information for improving general wellbeing.


Facial Palsy SVM Face SOM Health informatics eHealth Medical data analysis 



This work was supported by JCU Singapore Research Grant JCUS/003/2011/IS and a grant from the Bill & Melinda Gates Foundation through the Grand Challenges Explorations Initiative (Grant Number: OPP1032125).


  1. AD Tilaka, J Diederich, I Song, Teoh A (2013) Automated method for diagnosing speech and language dysfunction in schizophrenia. In: Margaret Lech IS, Peter Yellowlees, and Joachim Diederich (ed) Mental Health Informatics. Studies in Computational Intelligence. Springer, BerlinGoogle Scholar
  2. Angaran DM (1999) Telemedicine and telepharmacy: current status and future implications. Am J Health-Syst Pharm 56(14):1405–1426Google Scholar
  3. Chen EYH, Chan WSC, Chan SSM, Liu KY, Chan CLW, Wong PWC, Law YW, Yip PSF (2007) A cluster analysis of the circumstances of death in suicides in Hong Kong. Suicide Life Threat Behav 37(5):576–584CrossRefGoogle Scholar
  4. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297MATHGoogle Scholar
  5. Cowie R, Douglas-Cowie E, Tsapatsoulis N, Votsis G, Kollias S, Fellenz W, Taylor JG (2001) Emotion recognition in human-computer interaction. IEEE Signal Process Mag 18(1):32–80CrossRefGoogle Scholar
  6. Ekman P, Rosenberg EL, Heller M (1998) What the face reveals. Basic and applied studies of spontaneous expression using the facial action coding system (FACS). Psychotherapies 18(3):179–180Google Scholar
  7. Elvevag B, Foltz PW, Rosenstein M, DeLisi LE (2009) An automated method to analyze language use in patients with schizophrenia and their first-degree relatives. J Neuro Linguist 23(3):270–284Google Scholar
  8. Frantzidis CA, Bratsas C, Klados MA, Konstantinidis E, Lithari CD, Vivas AB, Papadelis CL, Kaldoudi E, Pappas C, Bamidis PD (2010) On the classification of emotional biosignals evoked while viewing affective pictures: an integrated data-mining-based approach for healthcare applications. IEEE Trans Inf Technol Biomed 14(2):309–318CrossRefGoogle Scholar
  9. Hadfield P, Shah B, Glover G (1995) Facial palsy due to tuberculosis: the value of CT. The J Laryngol Otology 109(10):1010–1012CrossRefGoogle Scholar
  10. Hamming WR (1950) Error detecting and error correcting codes. Bell Syst Tech J 29(2):147–160CrossRefMathSciNetGoogle Scholar
  11. Harrison R, Clayton W, Wallace P (1996) Can telemedicine be used to improve communication between primary and secondary care? Br Med J 313(7069):1377–1381CrossRefGoogle Scholar
  12. Hilty DM, Luo JS, Morache C, Marcelo DA, Nesbitt TS (2002) Telepsychiatry—an overview for psychiatrists. CNS Drugs 16(8):527–548. doi: 10.2165/00023210-200216080-00003 CrossRefGoogle Scholar
  13. Hilty DM, Marks SL, Urness D, Yellowlees PM, Nesbitt TS (2004) Clinical and educational telepsychiatry applications: a review. Can J Psychiatry Rev Can Psychiatry 49(1):12–23Google Scholar
  14. House JWaB DE (1985) Facial nerve grading system. Otolaryngol Head Neck Surg 93:146–147Google Scholar
  15. Joachims T (1999) Making large-scale support vector machine learning practical. MIT Press, CambridgeGoogle Scholar
  16. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480CrossRefGoogle Scholar
  17. Lei B, Song I, Rahman SA (2012) Optimal watermarking scheme for breath sound. In: The 2012 international joint conference on Neural Networks (IJCNN), 2012. IEEE, pp 1–6Google Scholar
  18. Lei B, Song I, Rahman SA (2013) Robust and secure watermarking scheme for breath sound. J Syst Softw 86:1638Google Scholar
  19. Low LSA, Maddage NC, Lech M, Sheeber LB, Allen NB (2011) Detection of clinical depression in adolescents’ speech during family interactions. IEEE Trans Biomed Eng 58(3 part 1):574–586Google Scholar
  20. Murakami S, Mizobuchi M, Nakashiro Y, Doi T, Hato N, Yanagihara N (1996) Bell palsy and herpes simplex virus: identification of viral DNA in endoneurial fluid and muscle. Ann Int Med 124 (1-Part-1):27–30Google Scholar
  21. Nambu M, Nakajima K, Noshiro M, Tamura T (2005) An algorithm for the automatic detection of health conditions. Eng Med Biol Mag IEEE 24(4):38–42CrossRefGoogle Scholar
  22. Otsuka T, Ohya J (1996) Recognition of facial expressions using HMM with continuous output probabilities, pp 323–328Google Scholar
  23. Pantic M, Rothkrantz LÜM (2000) Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 22(12):1424–1445CrossRefGoogle Scholar
  24. Song I, Dillon D, Goh TJ, Sung M (2011) A health social network recommender system. In: Agents in principle, agents in practice—14th international conference, PRIMA 2011. Lecture notes in computer science, Springer, Berlin pp 361–372Google Scholar
  25. Song I, Marsh NV (2012) Anonymous indexing of health conditions for a similarity measure. IEEE Trans Inf Technol Biomed 16(4):737–744CrossRefGoogle Scholar
  26. Song I, Yen NY, Vong J, Diederich J, Yellowlees P (2013) Profiling bell’s palsy based on house-brackmann score. In: IEEE symposium on computational intelligence in healthcare and e-health (CICARE) 2013, IEEE, pp 1–6Google Scholar
  27. Stone RT, Wei CS (2011) Exploring the linkage between facial expression and mental workload for arithmetic tasks, pp 616–619Google Scholar
  28. Tang L, Zhou X, Yu Z, Liang Y, Zhang D, Ni H (2011) MHS: a multimedia system for improving medication adherence in elderly care. IEEE Syst J 5(4):506–517CrossRefGoogle Scholar
  29. Terzis JK, Konofaos P (2008) Nerve transfers in facial palsy. Facial Plast Surg 24(02):177–193CrossRefGoogle Scholar
  30. Trad SGJ, Dormont D, Stankoff B, Bricaire F, Caumes E (2005) Nuclear bilateral Bell’s palsy and ageusia associated with Mycoplasma pneumoniae pulmonary infection. J Med Microbiol 54:417–419CrossRefGoogle Scholar
  31. Ultsch A (1993) Self-organizing neural networks for visualization and classification, Springer, BerlinGoogle Scholar
  32. Ultsch A (2003) Maps for the visualization of high-dimensional data spaces. In: Proceedings of workshop on self organizing maps, 2003 pp 225–230Google Scholar
  33. Ultsch A (2005) Clustering with SOM: U*C. Paper presented at the WSOMGoogle Scholar
  34. Valstar MF, Pantic M (2011) Fully automatic recognition of the temporal phases of facial actions. IEEE Trans Syst Man and Cybern Part B: Cybern 42:28Google Scholar
  35. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on Computer vision and pattern recognition (CVPR) 2001, vol. 511, pp I-511–I-518 IEEEGoogle Scholar
  36. Völter C, Helms J, Weissbrich B, Rieckmann P, Abele-Horn M (2004) Frequent detection of Mycoplasma pneumoniae in Bell’s palsy. Eur Arch Oto-Rhino-Laryngol Head Neck 261(7):400–404Google Scholar
  37. Vong J, Fang J, Song I (2012) Delivering financial services through mobile phone technology: a pilot study on impact of mobile money service on micro–entrepreneurs in rural Cambodia. Int J Inf Syst Change Manage 6(2):177–186Google Scholar
  38. Yacoob Y, Devis LS (1996) Recognizing human facial expressions from long image sequences using optical flow. IEEE Trans Pattern Anal Mach Intell 18(6):636–642CrossRefGoogle Scholar
  39. Yellowlees P, Burke MM, Marks SL, Hilty DM, Shore JH (2008) Emergency telepsychiatry. J Telemed Telecare 14(6):277–281. doi: 10.1258/jtt.2008.080419 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Singapore 2015

Authors and Affiliations

  1. 1.Financial IT AcademySingapore Management UniversitySingaporeSingapore
  2. 2.School of Business (IT)James Cook UniversitySingaporeSingapore

Personalised recommendations