Abstract
As the landscape around Big data continues to exponentially evolve, the « big » facet of Big data is no more number one priority of researchers and IT professionals. The race has recently become more about how to sift through torrents of data to find the hidden diamond and engineer a better, smarter and healthier world. The ease with which our mobile captures daily data about ourselves makes it an exceptionally suitable means for ultimately improving the quality of our lives and gaining valuable insights into our affective, mental and physical state. This paper takes the first exploratory step into this direction by using the mobile to process and analyze the “digital exhaust” it collects to automatically recognize our emotional states and accordingly respond to them in the most effective and “human” way possible. To achieve this we treat all technical, psycho-somatic, and cognitive aspects of emotion observation and prediction, and repackage all these elements into a mobile multimodal emotion recognition system that can be used on any mobile device.
Chapter PDF
Similar content being viewed by others
Keywords
References
Agger, B.: Everyday Life in Our Wired World. In: The Virtual Self: A Contemporary Sociology. Blackwell Publishing Ltd., Oxford (2008), doi:10.1002/9780470773376.ch1
Salmeron, J.L.: Fuzzy cognitive maps for artificial emotions forecasting. Applied Soft Computing 12, 3704–3710 (2012)
Calvo, R.A., D’Mello, S.: Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications. IEEE Transactions on Affective Computing 1(1), 18–37 (2010)
Alepis, E., Virvou, M., Kabassi, K.: Mobile education: Towards affective bi-modal interaction for adaptivity. In: Third International Conference on Digital Information Management, ICDIM 2008, pp. 51–56 (2008)
Klasnja, P., Pratt, W.: Healthcare in the Pocket: Mapping the Space of Mobile-Phone Health Interventions. J. Biomed. Inform. 45(1), 184–198 (2012)
Carneiro, D., Castillo, J.C., Novais, P., Fernández-Caballero, A., Neves, J.: Multimodal behavioral analysis for non-invasive stress detection. Expert Systems with Applications 39, 13376–13389 (2012)
Cai, H., Lin, Y.: Modeling of operators’ emotion and task performance in a virtual driving environment. Int. J. Human-Computer Studies 69, 571–586 (2011)
Kang, S.-H., Watt, J.H.: The impact of avatar realism and anonymity on effective communication via mobile devices. Computers in Human Behavior 29, 1169–1181 (2013)
Busso, C., Bulut, M., Narayanan, S.: Social emotions in nature and artifact: emotions in human and human-computer interaction. In: Marsella, S., Gratch, J. (eds.). Oxford University Press, New York (2012) (Press)
Quintana, D.S., Guastella, A.J., Outhred, T., Hickie, I.B., Kemp, A.H.: Heart rate variability is associated with emotion recognition: Direct evidence for a relationship between the autonomic nervous system and social cognition. International Journal of Psychophysiology 86, 168–172 (2012)
El Ayadi, M., Kamel, M.S., Karray, F.: Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recogn. 44(3), 572–587 (2010), doi:10.1016/j.patcog.2010.09.020
Yang, B., Lugger, M.: Emotion recognition from speech signals using new harmony features. Signal Processing 90, 1415–1423 (2010)
Busso, C., Lee, S., Narayanan, S.: Analysis of emotionally salient aspects of fundamental frequency for emotion detection. IEEE Trans. Audio Speech Lang. Proc. 17, 582–596 (2009)
Pierre-Yves, O.: The production and recognition of emotions in speech: features and algorithms. Int. J. Human-Computer Studies 59, 157–183 (2003)
Albornoz, E.M., Milone, D.H., Rufiner, H.L.: Spoken emotion recognition using hierarchical classifier. Computer Speech and Language 25, 556–570 (2011)
Lee, C.-C., Mower, E., Busso, C., Lee, S., Narayanan, S.: Emotion recognition using a hierarchical binary decision tree approach. Speech Communication 53, 1162–1171 (2011)
Koolagudi, S.G., et al.: Real Life Emotion Classification using Spectral Features and Gaussian Mixture Models. Procedia Engineering 38, 3892–3899 (2012)
Ververidis, D., Kotropoulos, C.: A Review of Emotional Speech Databases, http://citeseerx.ist.psu.edu/viewdoc/summary?doi:10.1.1.98.9202
Yoon, W.-J., Cho, Y.-H., Park, K.-S.: A Study of Speech Emotion Recognition and Its Application to Mobile Services. In: Indulska, J., Ma, J., Yang, L.T., Ungerer, T., Cao, J. (eds.) UIC 2007. LNCS, vol. 4611, pp. 758–766. Springer, Heidelberg (2007)
Tarng, W., Chen, Y.-Y., Li, C.-L., Hsie, K.-R., Chen, M.: Applications of Support Vector Machines on SmartPhone Systems for Emotional Speech Recognition. World Academy of Science, Engineering and Technology 48 (2010)
Rachuri, K.K., Musolesi, M., Mascolo, C., Rentfrow, P.J., Longworth, C., Aucinas, A.: EmotionSense: A Mobile Phones based Adaptive Platform for Experimental Social Psychology Research. In: UbiComp 2010, Copenhagen, Denmark, September 26-29 (2010)
Tian, Y., Kanade, T., Cohn, J.F.: Facial Expression Analysis. In: Handbook of Face Recognition, pp. 487–519. Springer, London (2011)
Mawafo, J.C.T., Clarke, W.A., Robinson, P.E.: Identification of Facial Features on Android Platforms. In: Industrial Technology (ICIT), pp. 1872–1876 (2013)
Niforatos, E., Karapanos, E.: EmoSnaps: A Mobile Application for Emotion Recall from Facial Expressions. In: CHI 2013, Paris, France, April 27-May 2 (2013)
Swinton, R., El Kaliouby, R.: Measuring emotions through a mobile device across borders, ages, genders and more. In: ESOMAR 2012 (2012)
Affdex, http://www.affectiva.com/affdex/#pane_overview (accessed on May 9, 2013)
Barrett, L.F., Kensinger, E.A.: Context is routinely encoded during emotion perception. Psychol. Sci. 21, 595–599 (2010)
Oh, K., Park, H.-S., Cho, S.-B.: A Mobile Context Sharing System using Activity and Emotion Recognition with Bayesian Networks. In: 2010 Symposia and Workshops on Ubiquitous, Autonomic and Trusted Computing, pp. 244–249 (2010)
Yoon, J.-W., Cho, S.-B.: An intelligent synthetic character for smartphone with Bayesian networks and behavior selection networks. Expert Systems with Applications 39, 11284–11292 (2012)
Lee, H., Choi, Y.S., Lee, S., Park, I.P.: Towards Unobtrusive Emotion Recognition for Affective Social Communication. In: The 9th Annual IEEE Consumer Communications and Networking Conference-Special Session Affective Computing for Future Consumer Electronics, pp. 260–264.
Tetteroo, D.: Communicating emotions in instant messaging, an overview. In: The 9th Twente Student Conference on IT, Enschede (June 23, 2008)
Fragopanagos, N., Taylor, J.G.: Emotion recognition in human–computer interaction, Department of Mathematics, King’s College, Strand, London WC2 R2LS. UK Neural Networks 18, 389–405 (2005)
Kao, E.C.-C., Liu, C.-C., Yang, T.-H., Hsieh, C.-T., Soo, V.-W.: Towards Text-based Emotion Detection A Survey and Possible Improvements. In: International Conference on Information Management and Engineering, ICIME 2009, April 3-5, pp. 70–74 (2009), doi:10.1109/ICIME.2009.113
Sebe, N., Cohen, I., Gevers, T., Huang, T.S.: Multimodal approaches for emotion recognition: a survey. In: Proc. SPIE, vol. 5670, pp. 56–67 (2005)
Hussain, S.S., Peter, C., Bieber, G.: Emotion Recognition on the Go: Providing Personalized Services Based on Emotional States. In: Proc. of the 2009 Workshop: Measuring Mobile Emotions: Measuring the Impossible?, Bonn, Germany (September 15, 2009)
Lee, S., Hong, C-S., Lee, Y. K., Shin, H.-S.: Experimental Emotion Recognition System and Services for Mobile Network Environments. In: Proc. of IEEE SENSORS 2010 Conference, pp. 136–139 (2010)
Kim, H.-J., Choi, Y.S.: Exploring Emotional Preference for Smartphone Applications. In: The 9th Annual IEEE Consumer Communications and Networking Conference - Special Session Affective Computing for Future Consumer Electronics, pp. 245–249 (2012)
Mousannif, H., Khalil, I., Kotsis, G.: The cloud is not “there”, we are the cloud! International Journal of Web and Grid Services 9(1), 1–17 (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 IFIP International Federation for Information Processing
About this paper
Cite this paper
Mousannif, H., Khalil, I. (2014). The Human Face of Mobile. In: Linawati, Mahendra, M.S., Neuhold, E.J., Tjoa, A.M., You, I. (eds) Information and Communication Technology. ICT-EurAsia 2014. Lecture Notes in Computer Science, vol 8407. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-55032-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-55032-4_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-55031-7
Online ISBN: 978-3-642-55032-4
eBook Packages: Computer ScienceComputer Science (R0)